Depth-Guided Motion-Weighted Upsampling for Low-Latency Imaging

The patent segments the upsampling process into multiple stages: first applying a computationally efficient upsampling method (e.g., bilinear or bicubic interpolation) to produce an intermediate high-resolution image, then applying a lightweight neural network refinement stage. This segmentation allows the system to achieve good image quality while reducing overall processing time by avoiding the need for a single heavy neural network to perform all upsampling operations.

The patent applies preliminary upsampling using traditional interpolation methods before the neural network refinement stage. This preliminary action handles the bulk of the computational workload efficiently, and the neural network then only needs to perform refinement operations, significantly reducing the compute requirements compared to using a neural network for the entire upsampling process.

The patent divides the upsampling task between traditional interpolation methods and a lightweight neural network. The interpolation method handles the computationally intensive pixel value calculation, while the neural network performs only refinement operations. This segmentation reduces the total computational burden and power consumption compared to using a neural network for complete upsampling.

The patent uses simple, lightweight interpolation kernels (such as bilinear or bicubic interpolation) as temporary computational tools to perform the bulk of the upsampling work. These simple methods are computationally inexpensive and can be implemented with minimal hardware resources, serving as an efficient placeholder before the final neural network refinement stage.

The patent merges traditional interpolation methods with neural network refinement in a hybrid approach. The interpolation method provides fast, computationally efficient upsampling that maintains processing speed, while the neural network refinement layer enhances image quality by correcting artifacts and improving detail. This combination achieves both speed and quality that neither method could achieve alone.

The patent introduces an intermediate high-resolution image as a mediator between the low-resolution input and the final output. The interpolation method first creates this intermediate image, which then serves as input to the neural network refinement stage. This intermediary allows the system to benefit from both the speed of traditional methods and the quality enhancement of neural networks.