Neural Network Image Upsampling with Jitter Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for upsampling images or video frames often result in artifacts due to limited or unreliable information from rendering engines, leading to suboptimal quality, especially when upscaling from lower to higher resolutions, which can cause issues like ghosting and lag.
Innovation Solution
A deep learning-based super-resolution process that utilizes a neural network to infer higher quality upscaled images by considering sub-pixel jitter and temporal reconstruction, combining anti-aliasing and super-resolution techniques, and applying appropriate filtering to align jitter offsets with historical data, thereby enhancing image quality and reducing processing resources needed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If traditional upsampling methods are used to increase resolution, then processing speed is maintained, but image quality deteriorates with artifacts like ghosting and lag
Solution Approach 1:
The patent replaces traditional mechanical upsampling algorithms with a neural network-based system that uses learned patterns from training data to generate high-resolution images. The neural network substitutes conventional interpolation methods, achieving superior image quality without proportionally increasing processing time through optimized inference mechanisms.
Solution Approach 2:
The neural network is trained in advance on large datasets of high-resolution images to learn complex patterns and relationships. This preliminary training phase enables the network to perform rapid inference during actual upsampling operations, resolving the contradiction between quality and speed by preparing the system beforehand.
2Manufacturing precision
If native resolution rendering is used to achieve high image quality, then image quality is improved, but processing resources and time increase significantly
Solution Approach 1:
Instead of rendering images at native high resolution, the system renders at lower resolution and uses a neural network to generate a copy that appears visually indistinguishable from native resolution. This copying approach consumes significantly fewer processing resources while maintaining perceived image quality.
Solution Approach 2:
The patent changes the resolution parameter during rendering, using lower resolution as input to the neural network. The network then transforms these parameters to produce high-resolution output, avoiding the computational cost of native resolution rendering while achieving comparable visual quality.
3Productivity
If information from rendering engines is used for upsampling, then processing efficiency is maintained, but image quality deteriorates due to limited or unreliable information
Solution Approach 1:
The neural network acts as an intermediary between the low-resolution rendering output and the final high-resolution image. It processes the limited information from the rendering engine and supplements it with learned patterns from training data, improving image quality without sacrificing processing efficiency.
Solution Approach 2:
The patent substitutes traditional upsampling algorithms that directly process rendering engine output with a neural network system. This replacement enables the system to overcome limitations of limited rendering information by leveraging learned patterns, achieving both efficiency and quality.
Data Source
AI summary
Apparatuses, systems, and techniques are presented to reconstruct one or more images. In at least one embodiment, one or more neural networks are used to upsample one or more images based, at least in part, on one or more brightness values.