Segmentation-Guided Video Restoration for Conferencing Compression Artifacts

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Video codecs introduce noticeable compression artifacts during encoding and decoding, and conferencing tools often suffer from quality degradation due to network congestion or bandwidth limitations, leading to suboptimal video quality.

Innovation Solution

Adaptive post-processing of decoded video using machine learning models that leverage scenario detection, segmentation, and video quality analysis to selectively apply post-processing operations, enhancing video quality without significantly increasing network bandwidth or reducing quality.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If lossy compression is used to reduce bandwidth consumption, then network bandwidth usage is reduced, but video quality deteriorates due to compression artifacts

Engineering Contradiction:

Improvenetwork bandwidth consumptionVSAvoidvideo quality

Core Design Contradiction:

Quantity of substanceVSManufacturing precision

Solution Approach 1:

The patent applies machine learning models to convert the harmful compression artifacts (blocking, ringing, blurring, banding, mosquito noise) into beneficial visual quality. The models learn to reconstruct and enhance degraded video regions by identifying patterns in artifacts and generating plausible original content, effectively transforming the damaged output of lossy compression into high-quality visual output without requiring additional bandwidth

Inventive Principle:

Principle #22Blessing in disguise (Convert harm into benefit)

Solution Approach 2:

The system dynamically adjusts post-processing parameters based on detected artifact types and severity. Different machine learning models are selected and configured based on the specific compression artifacts present (e.g., blocking vs. ringing vs. blurring), allowing optimal parameter settings for each artifact type while maintaining low bandwidth consumption

Inventive Principle:

Principle #35Parameter changes

2Reliability

If aggressive lossy compression is applied to ensure timely delivery, then packet delivery reliability is improved, but video quality deteriorates due to extensive compression artifacts

Engineering Contradiction:

Improvepacket delivery reliabilityVSAvoidvideo quality

Core Design Contradiction:

ReliabilityVSManufacturing precision

Solution Approach 1:

The machine learning-based post-processing system converts the harmful effects of aggressive compression into beneficial visual quality. By training models specifically on heavily compressed video with various artifact types, the system learns to reconstruct and enhance degraded regions, transforming the low-quality output of aggressive compression into acceptable or high-quality visual output while maintaining the compression-induced reliability benefits

Inventive Principle:

Principle #22Blessing in disguise (Convert harm into benefit)

3Manufacturing precision

If post-processing models are applied to enhance video quality, then video quality is improved, but computational resources are consumed

Engineering Contradiction:

Improvevideo qualityVSAvoidcomputational resource consumption

Core Design Contradiction:

Manufacturing precisionVSUse of energy by moving object

Solution Approach 1:

The system applies machine learning post-processing selectively to specific regions of video frames rather than uniformly processing entire frames. Foreground regions containing important visual content receive intensive post-processing, while background regions receive reduced or no processing. This local quality approach maintains overall video quality while significantly reducing computational resource consumption compared to full-frame processing

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The system applies post-processing at selective stages in the video processing pipeline and only when necessary based on detected artifact levels. Rather than applying heavy processing to all video content, the system uses lightweight processing as a baseline and applies more intensive processing only to regions or frames where compression artifacts exceed threshold levels, optimizing the balance between quality improvement and computational cost

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS20260006155A1Machine learning models for adaptive post-processing using results of segmentation in conferencing tools

Publication Date: 2026.01.01 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

Innovations in machine learning (“ML”) models used in adaptive post-processing of decoded video in a conferencing tool are described. For example, as part of post-processing of decoded video, a super-resolution/video restoration model increases spatial resolution (e.g., by interpolation between sample values), mitigates compression artifacts, and mitigates upscaling artifacts introduced when increasing spatial resolution. Or, as another example, as part of post-processing of decoded video, a video restoration model mitigates compression artifacts, without increasing spatial resolution. For adaptive post-processing, a post-processing model can be selectively applied depending on results of scenario detection, results of segmentation, and/or results of video quality analysis. With the innovations, a conferencing tool can in effect provide video at higher quality without significantly increasing the network bandwidth consumed by the video or, alternatively, provide video using less network bandwidth without significantly hurting the quality of the video.