Hybrid Neural Codec Proxy Training

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing neural coding systems for streaming data are inefficient as pre-process and post-process neural networks are typically independent, limiting their effectiveness in converting data types not supported by standard codecs.

Innovation Solution

A hybrid codec system is introduced, comprising a real codec for encoding and decoding, and a proxy codec that is differentiable, allowing for back-propagation of loss functions to train neural networks effectively, even with data types not initially supported.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of manufacture

If pre-process and post-process neural networks are trained independently, then the system is simpler to implement and train, but the effectiveness of data conversion and reconstruction is reduced

Engineering Contradiction:
Improveease of trainingVSAvoideffectiveness of data conversion
Core Design Contradiction:
Ease of manufactureVSReliability

Solution Approach 1:

The patent combines pre-process and post-process neural networks into a unified training framework where both networks are trained simultaneously using a shared loss function. This merging allows the networks to learn complementary representations and improve their collaborative effectiveness in converting and reconstructing data, resolving the contradiction between independent training simplicity and joint training effectiveness.

Inventive Principle:
Principle #5Merging (Combining)

2Adaptability or versatility

If a real codec is used for encoding and decoding, then the system can handle data types not supported by standard codecs, but the training process becomes more complex due to non-differentiability

Engineering Contradiction:
Improvesupport for data typesVSAvoidcomplexity of training process
Core Design Contradiction:
Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent introduces a proxy codec as an intermediary that differentiates the training process from the actual encoding/decoding process. The proxy codec provides a differentiable approximation of the real codec during training, enabling gradient-based optimization. The real codec is then used for actual data processing. This intermediary resolves the contradiction by allowing versatile data type support through the real codec while maintaining training simplicity through the differentiable proxy.

Inventive Principle:
Principle #24Intermediary (Mediator)

3Adaptability or versatility

If neural networks are used to convert data types, then more data formats can be supported, but the encoding efficiency may be reduced without proper training

Engineering Contradiction:
Improvedata format supportVSAvoidencoding efficiency
Core Design Contradiction:
Adaptability or versatilityVSProductivity

Solution Approach 1:

The patent implements a feedback mechanism where the loss function continuously monitors the reconstruction quality and guides the training of neural networks. The loss function compares the original data with the reconstructed data and provides gradients that feed back into the training process. This feedback ensures that neural networks learn to maintain encoding efficiency while supporting versatile data formats, resolving the contradiction between adaptability and productivity.

Inventive Principle:
Principle #23Feedback

Data Source

PatentUS20250047943A1Devices, systems, codecs and methods for neural coding
Publication Date: 2025.02.06 SONY INTERACTIVE ENTERTAINMENT LLC
  • US20250047943A1 patent drawing
  • US20250047943A1 patent drawing
  • US20250047943A1 patent drawing

AI summary

A hybrid codec for training a neural coding system, the hybrid codec including a real codec for encoding data pre-processed by a first neural network, and for decoding the encoded data for post-processing by a second neural network; and a proxy codec that is a differentiable representation of the real codec, and is for back-propagating a loss function, representative of a coding bit-rate and a coding distortion between the post-processed data and the original data, for training the weights of at least the first neural network.