Rasterized Triangle ID Buffers for Dense Optical Flow
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current graphics processing systems face challenges in efficiently processing screen space data for machine learning models, as this data is more complex and unordered compared to user-comprehended data, hindering effective training and operation of frame generation and super resolution models.
Innovation Solution
The implementation of a methodology to enable optical flow generation using rasterized triangle identifier buffers within graphics processors, leveraging parallel processing techniques and SIMT architectures to optimize data processing and improve the comprehension of screen space data for machine learning models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If screen space data is used for machine learning models, then the data is readily comprehended by users, but the data becomes more difficult for machine learning models to comprehend and process
Solution Approach 1:
The patent segments screen space data into distinct triangular primitive components, each with identifiable vertices and attributes. By dividing the continuous screen space into discrete triangular elements, the data becomes structured and organized, making it more suitable for machine learning processing while preserving user-comprehensible visual representation.
Solution Approach 2:
The patent introduces an intermediary processing stage that transforms screen space data into a structured format suitable for machine learning. This intermediary representation maintains the visual integrity for user comprehension while organizing data into a form that machine learning models can effectively process, bridging the gap between human-readable and machine-processable data.
2Device complexity
If screen space data is used directly for training, then the data representation is simple, but the processing efficiency for frame generation and super resolution models is reduced
Solution Approach 1:
The patent performs preliminary organization and structuring of screen space data before it enters the machine learning training pipeline. By pre-processing the data into structured triangular primitive representations with defined vertices and attributes, the system prepares the data in advance, which improves processing efficiency during actual training and inference operations without complicating the overall data representation.
3Device complexity
If complex screen space data is processed without structured representation, then the processing pipeline is simple, but the training effectiveness of machine learning models is hindered
Solution Approach 1:
The patent applies local quality by assigning specific attributes and properties to different parts of the screen space data structure. Each triangular primitive and its vertices have defined local characteristics that are preserved and utilized during processing. This localized structuring maintains overall pipeline simplicity while enhancing training effectiveness through meaningful local data organization that machine learning models can leverage.
Data Source
AI summary
Described herein, in one embodiment, is computer-implemented method of estimating dense optical flow using primitive ID data comprising generating a primitive ID buffer (e.g., triangle ID buffer) for a frame during rendering, rendering the frame with a unique color for each primitive ID, providing a rendered primitive ID buffer for the current and previous frame along with a confidence buffer and sparse optical flow between frames to a machine learning model, and generating an estimated dense optical flow via the machine learning model based on the input.