Multi-view Video Artifact Reduction via Bilateral Flow Fusion

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing multi-view video coding technologies degrade the image quality of auxiliary and intermediate viewpoints due to quantization issues and synthesis problems, leading to artifacts that affect the rendering quality of 3D spaces during view reconstruction.

Innovation Solution

A method that derives bilateral and refined flows in addition to unidirectional flows between viewpoints and times, using these flows to select and warp reference frames, generate intermediate frames, and fuse them with the current frame to enhance image quality, thereby reducing coding artifacts.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If multi-view video is compressed using existing coding technologies, then transmission efficiency is improved, but image quality of auxiliary and intermediate viewpoints degrades due to quantization parameters and synthesis problems

Engineering Contradiction:

Improvetransmission efficiencyVSAvoidimage quality

Core Design Contradiction:

ProductivityVSManufacturing precision

Solution Approach 1:

The patent segments the video coding process into main viewpoint coding and auxiliary viewpoint synthesis. By separately handling the main viewpoint with full coding resources and generating auxiliary viewpoints through synthesis using bilateral flow, the system maintains transmission efficiency while improving auxiliary viewpoint quality through dedicated flow derivation and reference frame selection.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces bilateral flow as an intermediary mechanism between main and auxiliary viewpoints. This bilateral flow acts as a mediator that captures temporal and spatial relationships, enabling higher quality synthesis of auxiliary viewpoints without increasing transmission bitrate, thus resolving the contradiction between compression efficiency and image quality.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Productivity

If quantization parameters are increased to improve compression, then transmission efficiency is improved, but coding artifacts increase and degrade reconstruction quality

Engineering Contradiction:

Improvecompression efficiencyVSAvoidcoding artifacts

Core Design Contradiction:

ProductivityVSObject-generated harmful factors

Solution Approach 1:

The patent implements feedback through bilateral flow derivation that uses information from both past and future frames. This feedback mechanism allows the system to compensate for quantization artifacts by incorporating temporal context from multiple directions, reducing artifact propagation while maintaining compression efficiency through intelligent reference frame selection.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The patent applies beforehand cushioning by deriving bilateral flow in advance using high-quality reference frames before the actual synthesis process. This pre-computation cushions against the harmful effects of quantization by establishing accurate motion compensation patterns that reduce artifact generation during compression.

Inventive Principle:

Principle #11Beforehand cushioning (Prior cushioning)

3Measurement precision

If optical flow technology is used to find reference information, then feature information extraction is improved, but accuracy decreases when video contains coding artifacts

Engineering Contradiction:

Improvefeature information extractionVSAvoidaccuracy with artifacts

Core Design Contradiction:

Measurement precisionVSReliability

Solution Approach 1:

The patent inverts the traditional optical flow approach by using bilateral flow that incorporates information from both forward and backward temporal directions. Instead of unidirectional flow that is vulnerable to artifacts, the bidirectional approach inverts the problem-solving strategy, using future and past frames equally to compensate for artifact-induced inaccuracies in feature extraction.

Inventive Principle:

Principle #13The other way round (Inversion)

Solution Approach 2:

The patent creates a composite flow field by combining multiple flow vectors from different temporal directions (forward and backward). This composite bilateral flow integrates information from multiple sources, making the feature extraction more robust to coding artifacts present in any single reference frame, thereby improving reliability without sacrificing measurement precision.

Inventive Principle:

Principle #40Composite materials

Data Source

PatentUS20250106432A1Multi-view video coding artifact reduction method and system

Publication Date: 2025.03.27 KOREA ELECTRONICS TECH INST

AI summary

There is provided a multi-view video coding artifact reduction method and system. According to an embodiment, the multi-view video coding artifact reduction method selects reference fames in neighbors of a current frame constituting a multi-view video, derives unidirectional flows and bilateral flows between the current frame and the reference frames, warps the reference frames based on the derived flows, generates intermediate frames by fusing the current frame, the reference frames, and the warped reference frames, and generates a final frame by fusing the current frame and the intermediate frames. Accordingly, image quality of the multi-view video can be more enhanced.