Temporal Trilinear Filtering for Video Rendering

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing digital image processing techniques, such as trilinear filtering, consume excessive power and require multiple mipmap levels to mitigate aliasing artifacts in anisotropically-warped images, especially when the scale between the image and display changes significantly, and often necessitate buffering previous frames.

Innovation Solution

A temporal trilinear filtering method that uses per-frame mipmap levels and frame-specific bias values to select the appropriate mipmap level for each frame without requiring buffering of previous frames, employing bilinear texture filtering and computing a mipmap-level determining factor based on the texture's scale and frame position within a sequence of N frames.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If trilinear filtering is used to smooth aliasing artifacts, then the quality of the rendered image is improved, but the power consumption increases significantly due to twice as many filter computations

Engineering Contradiction:
Improveimage qualityVSAvoidpower consumption
Core Design Contradiction:
Manufacturing precisionVSUse of energy by moving object

Solution Approach 1:

The patent applies partial action by performing bilinear filtering instead of full trilinear filtering. It selectively applies filtering operations only when necessary (when scale changes are detected) and only at the current mipmap level, rather than always performing the complete trilinear filtering across multiple mipmap levels. This reduces the number of filter computations while still addressing aliasing artifacts in critical scenarios.

Inventive Principle:
Principle #16Partial or excessive action

Solution Approach 2:

The patent changes the filtering parameter by switching between different mipmap levels based on detected scale changes. Instead of consistently applying trilinear filtering, it dynamically adjusts which mipmap level to use, changing the filtering intensity and type based on the specific rendering conditions. This allows the system to maintain image quality when needed while reducing computational overhead when scale changes are minimal.

Inventive Principle:
Principle #35Parameter changes

2Manufacturing precision

If multiple mipmap levels are used to filter surfaces without aliasing artifacts when scale changes significantly, then the rendering quality is improved, but the device complexity and computational requirements increase

Engineering Contradiction:
Improverendering qualityVSAvoidfilter computation complexity
Core Design Contradiction:
Manufacturing precisionVSDevice complexity

Solution Approach 1:

The patent introduces dynamic adaptation by detecting scale changes between frames and dynamically selecting appropriate mipmap levels. The system monitors the scale of rendered pixels relative to the digital display and adjusts the filtering approach in real-time. When scale changes are detected, it switches to using different mipmap levels; when stable, it maintains the current level, avoiding unnecessary computational complexity.

Inventive Principle:
Principle #15Dynamics

Solution Approach 2:

The patent segments the filtering process by separating it into distinct stages: scale change detection, mipmap level selection, and filtering application. This segmentation allows the system to handle only the necessary portions of the filtering pipeline based on current rendering conditions, reducing overall complexity while maintaining quality where needed.

Inventive Principle:
Principle #1Segmentation

3Manufacturing precision

If temporal anti-aliasing techniques such as TXAA are used to reduce aliasing artifacts, then the image quality is improved, but the requirement to buffer previous frames increases memory usage and processing overhead

Engineering Contradiction:
Improvealiasing artifact reductionVSAvoidframe buffering requirement
Core Design Contradiction:
Manufacturing precisionVSDevice complexity

Solution Approach 1:

The patent extracts and removes the frame buffering requirement from the temporal anti-aliasing process. Instead of relying on buffered previous frames like TXAA, it uses real-time scale change detection between the current frame and the digital display to determine filtering needs. This extraction eliminates the memory overhead and processing complexity associated with maintaining and managing frame buffers while still achieving temporal anti-aliasing effects.

Inventive Principle:
Principle #2Taking out (Extraction)

Solution Approach 2:

The system performs self-service by using its own scale change detection capability to determine when filtering is needed, rather than relying on externally buffered frame data. The scale detection mechanism inherently provides the temporal comparison information needed for anti-aliasing decisions, making the separate frame buffering infrastructure unnecessary.

Inventive Principle:
Principle #25Self-service

Data Source

PatentUS11640699B2Temporal approximation of trilinear filtering
Publication Date: 2023.05.02 META PLATFORMS TECHNOLOGIES LLC
  • US11640699B2 patent drawing
  • US11640699B2 patent drawing
  • US11640699B2 patent drawing

AI summary

In one embodiment, a method includes receiving instructions to render a snapshot of a scene for a video, where the snapshot is to be displayed using a sequence of N frames, computing a mipmap-level determining factor for a texture appearing in the scene based on a scale of the texture on a pixel grid, selecting a mipmap level of the texture for each of the N frames based on the mipmap-level determining factor, where the mipmap levels selected for the N frames are non-uniform and temporally approximate the mipmap-level determining factor, rendering each of the N frames by sampling the mipmap level of the texture selected for that frame, and displaying the rendered N frames sequentially to represent the snapshot of the scene.