Video Codec Connectors for Human and Machine Vision Outputs

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing video compression methods optimized for human viewing fail to meet the accuracy requirements of machine tasks, leading to poor performance when used for tasks such as object detection and classification due to differing quality metrics between human and machine vision.

Innovation Solution

Introduce a connector at the decoder output to adapt decoded video content for secondary tasks by using filtering functions like average pooling and convolutional layers, optimizing the codec parameters for specific machine tasks while maintaining compatibility with human viewing.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Illumination intensity

If video compression is optimized for human viewing quality, then visual quality is improved, but accuracy for machine tasks deteriorates

Engineering Contradiction:
Improvevisual qualityVSAvoidmachine task accuracy
Core Design Contradiction:
Illumination intensityVSMeasurement precision

Solution Approach 1:

The patent segments the decoding process into two distinct paths: a first decoding path optimized for human viewing quality and a second decoding path optimized for machine task accuracy. This segmentation allows each path to use different processing parameters and algorithms tailored to its specific purpose, resolving the contradiction between visual quality and machine task performance

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent creates a universal decoding system that can serve multiple functions: it can output decoded video optimized for human viewing, optimized for machine tasks, or both simultaneously. The multi-functional decoder includes multiple output interfaces that can be selectively activated based on the intended use case, allowing the same compression system to satisfy both human visual quality requirements and machine task accuracy requirements

Inventive Principle:
Principle #6Universality (Multi-functionality)

2Productivity

If compression bitrate is reduced, then transmission efficiency is improved, but task accuracy deteriorates

Engineering Contradiction:
Improvetransmission efficiencyVSAvoidtask accuracy
Core Design Contradiction:
ProductivityVSMeasurement precision

Solution Approach 1:

The patent implements dynamic parameter adjustment in the second decoding path, where processing parameters such as filtering strength, block sizes, and algorithm selection are dynamically adapted based on the specific machine task requirements and available bitrate. This dynamic adjustment allows the system to maintain task accuracy across varying compression levels by optimizing the decoding process for each specific scenario

Inventive Principle:
Principle #15Dynamics

Solution Approach 2:

The patent changes key decoding parameters in the second path compared to the first path, including using different filter types, block partitioning schemes, and reconstruction algorithms. These parameter changes are specifically designed to preserve features important for machine tasks even at low bitrates, while the first path uses parameters optimized for human visual perception

Inventive Principle:
Principle #35Parameter changes

3Productivity

If template matching prediction is used, then compression efficiency is improved, but compatibility with existing standards deteriorates

Engineering Contradiction:
Improvecompression efficiencyVSAvoidstandard compatibility
Core Design Contradiction:
ProductivityVSAdaptability or versatility

Solution Approach 1:

The patent introduces an intermediary processing layer between the standard compliant decoder and the template matching prediction algorithm. This intermediary layer performs format conversion and feature extraction that bridges the gap between standard video formats and the requirements of template matching, allowing the algorithm to be applied to standard-compliant decoded video without requiring changes to the underlying compression standard

Inventive Principle:
Principle #24Intermediary (Mediator)

Data Source

PatentUS12556720B2Learned video compression and connectors for multiple machine tasks
Publication Date: 2026.02.17 INTERDIGITAL VC HOLDINGS INC
  • US12556720B2 patent drawing
  • US12556720B2 patent drawing
  • US12556720B2 patent drawing

AI summary

A processing module, or connector, adapts an output of a codec, or a decoded output, to a form suitable for an alternate task. In one embodiment, the output of a codec is used for a machine task and the connector adapts this output to a form suitable for a video display. In another embodiment, metadata accompanies the codec output, which can instruct the connector how to adapt the codec output for an alternate task. In other embodiments, the processing module performs averaging over a N×M window, or convolution.