Multi-Scale Neural Network Filters for Machine-Oriented Video Coding

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing video coding technologies face challenges in effectively utilizing neural networks for end-to-end compression and decompression, particularly in adapting to machine-specific analysis tasks, leading to inefficiencies in bitrate and performance for machine-based video processing.

Innovation Solution

Implementing multi-scale blocks within neural network filters that adapt to machine-specific tasks by incorporating overfitting processes at both the encoder and decoder sides, utilizing neural networks to enhance filtering and processing for machine consumption.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If neural networks are used for end-to-end compression and decompression, then video coding efficiency is improved, but adaptability to machine-specific analysis tasks deteriorates

Engineering Contradiction:

Improvevideo coding efficiencyVSAvoidadaptability to machine-specific analysis tasks

Core Design Contradiction:

ProductivityVSAdaptability or versatility

Solution Approach 1:

The neural network filter is divided into multiple independent scale processing branches (first scale, second scale, third scale), each handling different spatial resolutions. This segmentation allows the system to maintain general efficiency while adapting to specific machine analysis tasks by selectively processing features at appropriate scales.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces multi-scale dimensionality by processing input data at three different spatial resolutions simultaneously. This adds a scale dimension to the neural network, enabling it to capture features at multiple levels and adapt to diverse machine-specific analysis requirements while maintaining overall coding efficiency.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Adaptability or versatility

If multi-scale blocks are implemented with overfitting processes, then adaptability to machine-specific tasks is improved, but device complexity increases

Engineering Contradiction:

Improveadaptability to machine-specific tasksVSAvoiddevice complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

Different scale processing branches are applied to different regions of the feature space based on local requirements. Each branch processes data at its optimal scale, with local overfitting applied selectively to enhance adaptability for specific machine tasks without uniformly increasing complexity across the entire system.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The multi-scale blocks are nested within the neural network filter structure, with smaller scale processing units embedded within larger ones. This nested architecture allows progressive refinement of features at different scales while sharing computational resources, reducing overall device complexity compared to fully independent processing paths.

Inventive Principle:

Principle #7Nested doll (Nesting)

Data Source

PatentUS20260010765A1Multi-scale blocks for neural network based filters

Publication Date: 2026.01.08 NOKIA TECHNOLOGIES OY

AI summary

An example apparatus includes: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: receive at least one input tensor; and process the at least one input tensor using a group of layers of at least one neural network to obtain a processed tensor.