Conditional Convolutional Neural Networks for Video Compression

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing video compression methods using artificial neural networks struggle to efficiently handle complex Group of Pictures (GOP) structures with varying distances between reference frames, leading to sub-optimal motion flow construction and limited coding efficiency.

Innovation Solution

The proposed solution involves using conditional convolution layers in the neural network that depend on the inter-dependencies in the GOP structure, allowing the model to handle complex GOP structures and varying distances between reference frames, thereby improving motion flow estimation and compression.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If existing video compression methods use artificial neural networks with standard convolutional layers, then the model can process video data, but it cannot efficiently handle complex GOP structures with varying distances between reference frames, leading to sub-optimal motion flow construction

Engineering Contradiction:

Improveability to handle complex GOP structuresVSAvoidmotion flow construction quality

Core Design Contradiction:

Adaptability or versatilityVSManufacturing precision

Solution Approach 1:

The patent applies dynamics by making the convolutional layers adaptive through conditional convolution. The convolution kernels are dynamically adjusted based on the GOP structure and reference frame distances, allowing the model to adapt its filtering behavior to different temporal configurations rather than using fixed kernels throughout

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent changes parameters by introducing conditional convolution where the convolution operation parameters (kernels) are modified based on input conditions representing GOP structure and reference frame distances. This allows the model to adjust its computational parameters to match the specific temporal structure being processed

Inventive Principle:

Principle #35Parameter changes

2Productivity

If a generic neural network is used for video compression, then the network can process various video data, but it cannot specialize motion flow estimation for different temporal structures, limiting coding efficiency

Engineering Contradiction:

Improvecompression coding efficiencyVSAvoidnetwork structure complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent achieves universality by creating a single conditional convolutional network that can handle multiple GOP structures and temporal configurations through its adaptive convolution mechanism, eliminating the need for separate specialized networks for different video types or structures

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The network maintains low structural complexity while achieving high productivity through dynamic conditional convolution. The same network architecture adapts its convolutional parameters based on input conditions, allowing one network to efficiently process various temporal structures without requiring complex architecture changes

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS20240187640A1Temporal structure-based conditional convolutional neural networks for video compression

Publication Date: 2024.06.06 INTERDIGITAL VC HOLDINGS INC

AI summary

Video encoding and decoding is implemented with auto encoders using luminance information to derive motion information for chrominance prediction. In one embodiment conditional convolutions are used to encode motion flow information. A current condition, for example, GOP structure, is used as input to a succession of fully connected layers to implement the conditional convolution. In a related embodiment, more than one reference frame is used to encode motion flow information.