Substitutional Neural Residual Compression for Video Encoding

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current video compression methods, such as H.264/AVC, HEVC, and VVC, rely on handcrafted coding tools and lack the ability to fully leverage nonlinearity and nonlocal spatiotemporal correlations for improved compression quality.

Innovation Solution

A method and apparatus for substitutional neural residual compression using Deep Neural Networks (DNNs) to estimate motion vectors, generate predicted image frames, and encode residuals, allowing for flexible bitrate control without the need for multiple model instances.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If handcrafted coding tools are used in video compression, then device complexity is reduced, but compression performance is limited due to inability to leverage nonlinearity and nonlocal spatiotemporal correlations

Engineering Contradiction:
Improvecompression performanceVSAvoidmodel complexity
Core Design Contradiction:
Manufacturing precisionVSDevice complexity

Solution Approach 1:

The patent changes the fundamental parameters of video compression by replacing handcrafted linear tools with deep neural networks that can model nonlinear relationships. The DNNs learn optimal transformation parameters from data, enabling capture of nonlocal spatiotemporal correlations that traditional methods miss, thus improving compression performance despite increased model complexity.

Inventive Principle:
Principle #35Parameter changes

Solution Approach 2:

The patent substitutes mechanical handcrafted coding tools with intelligent deep neural network systems. Instead of manually designed linear transforms and hand-tuned parameters, the system uses trained DNNs that automatically learn optimal compression representations, replacing the mechanical approach with an adaptive intelligent system.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

2Adaptability or versatility

If multiple model instances are used to achieve flexible bitrate control, then bitrate flexibility is improved, but device complexity increases

Engineering Contradiction:
Improvebitrate flexibilityVSAvoidnumber of model instances
Core Design Contradiction:
Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent introduces dynamic adaptability through a single DNN model that can adjust its behavior based on target bitrate parameters. Instead of using multiple static model instances for different bitrates, the system uses one dynamic model that adapts its compression strategy based on the desired bitrate, reducing complexity while maintaining flexibility.

Inventive Principle:
Principle #15Dynamics

Solution Approach 2:

The patent creates a universal DNN model that can handle multiple bitrate requirements through a single instance. The model is designed to be multi-functional, capable of producing compressed representations at various bitrates by adjusting its internal parameters and transformation strategies, eliminating the need for separate specialized models for each bitrate target.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Data Source

PatentUS11900640B2Method and apparatus for substitutional neural residual compression
Publication Date: 2024.02.13 TENCENT AMERICA LLC
  • US11900640B2 patent drawing
  • US11900640B2 patent drawing
  • US11900640B2 patent drawing

AI summary

A method of substitutional neural residual compression is performed by at least one processor and includes estimating motion vectors, based on a current image frame and a previous reconstructed image frame, obtaining a predicted image frame, based on the estimated motion vectors and the previous reconstructed image frame, and subtracting the obtained predicted image frame from the current image frame to obtain a substitutional residual. The method further includes encoding the obtained substitutional residual, using a first neural network, to obtain an encoded representation, and compressing the encoded representation.