Decoder-Side Neural Network Fine-Tuning for Adaptive Video Coding

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing video coding technologies lack efficient methods for decoder-side fine-tuning of neural networks to improve video quality and compression efficiency.

Innovation Solution

A method and apparatus for decoder-side fine-tuning of neural networks through iterative processes involving finetuning driving content, computing outputs, losses, and updates using approximated ground truth data, and applying these updates to decoder-side neural networks, utilizing techniques like backpropagation and stochastic gradient descent.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If video content is encoded in high quality, then video quality is improved, but compression efficiency deteriorates (larger file size)

Engineering Contradiction:
Improvevideo qualityVSAvoidcompression efficiency
Core Design Contradiction:
Manufacturing precisionVSQuantity of substance

Solution Approach 1:

The patent applies different quality levels to different regions of the video content. Important regions (e.g., regions containing objects of interest) are encoded at high quality, while less important regions are encoded at lower quality. This resolves the contradiction by making video quality subjective and region-dependent rather than uniformly high, thereby improving compression efficiency while maintaining perceived quality.

Inventive Principle:
Principle #3Local quality

Solution Approach 2:

The patent dynamically adjusts quality allocation based on content analysis. The system identifies important regions in each frame and adapts the encoding quality accordingly, rather than using a static quality setting for the entire video. This dynamic approach optimizes the balance between video quality and compression efficiency for each specific region and frame.

Inventive Principle:
Principle #15Dynamics

2Ease of manufacture

If neural networks are trained offline, then training flexibility is improved, but adaptability to specific video content deteriorates

Engineering Contradiction:
Improvetraining flexibilityVSAvoidadaptability to video content
Core Design Contradiction:
Ease of manufactureVSAdaptability or versatility

Solution Approach 1:

The patent performs preliminary offline training to establish a base neural network model with general capabilities. This preliminary action provides training flexibility and establishes a foundation that can be quickly adapted to specific video content through fine-tuning or parameter adjustment during encoding, resolving the contradiction between offline training benefits and content-specific adaptability.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The system enables the neural network to adapt to specific video content through self-service mechanisms during the encoding process. The network automatically adjusts its parameters or selects appropriate pre-trained models based on the characteristics of the input video content, eliminating the need for manual retraining while maintaining adaptability.

Inventive Principle:
Principle #25Self-service

Data Source

PatentUS12536711B2Decoder-side fine-tuning of neural networks for video coding for machines
Publication Date: 2026.01.27 NOKIA TECHNOLOGIES OY
  • US12536711B2 patent drawing
  • US12536711B2 patent drawing
  • US12536711B2 patent drawing

AI summary

Various embodiments provide an apparatus, a method, and a computer program product. An example apparatus includes at least one processor; and at least one non-transitory memory comprising computer program code; wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to iteratively perform following until a stopping criterion is met: provide a finetuning driving content (FDC) or a content derived from FDC to a decoder side neural network (DSNN); compute an output of the DSNN as a processed FDC; compute a loss based on the processed FDC and an approximated ground truth data (AGT) associated with the FDC; compute an update to the DSNN; and apply the computed update to the DSNN.