Video Resolution Enhancement via Spatial Temporal Correlation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current super-resolution techniques for video processing are computationally expensive, especially for larger low-resolution images, and struggle to efficiently enhance resolution in real-time or near real-time video processing, particularly when objects are stationary.
Innovation Solution
A method and system that predict interpolated pixels based on spatial correlation and generate motion vectors to reconstruct enhanced image frames, utilizing a combination of graphics processing units (GPUs) and central processing units (CPUs) to perform spatial and temporal correlation analysis, with correction factors applied to improve resolution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If spatial interpolation and motion compensation algorithms are applied to extract pixel information from low-resolution images, then enhanced resolution can be achieved, but computational cost increases significantly
Solution Approach 1:
The patent segments the image processing task into distinct phases: motion estimation operates on low-resolution frames first, then motion compensation is applied to high-resolution frames. This segmentation allows computationally intensive operations to be performed on smaller data sets, reducing overall computational cost while maintaining enhanced resolution output.
Solution Approach 2:
The patent performs motion estimation and generates motion vectors from low-resolution frames before upsampling to high resolution. By completing the motion analysis preliminary at lower resolution, the system avoids performing these computationally expensive operations on full-resolution data, thereby reducing computational cost while preserving the ability to generate high-quality enhanced frames.
2Manufacturing precision
If traditional super-resolution techniques are used for video processing, then resolution enhancement is achieved, but real-time processing capability is lost
Solution Approach 1:
The patent separates motion estimation from the resolution enhancement process, performing motion analysis on low-resolution frames and applying the results to high-resolution frames. This segmentation enables parallel processing where motion vectors are computed once and then reused across multiple resolution levels, significantly improving processing speed for real-time video applications.
Solution Approach 2:
By performing motion estimation preliminarily on low-resolution frames before the resolution enhancement step, the system prepares motion compensation data that can be efficiently applied during the upsampling process. This preliminary action reduces the computational burden during real-time processing, enabling frame rates suitable for video applications.
3Measurement precision
If motion compensation is performed on high-resolution frames, then accurate temporal information is extracted, but processing time increases significantly
Solution Approach 1:
The patent segments the temporal analysis task by performing motion estimation on low-resolution frames to capture temporal dynamics, then applying these motion vectors to guide the extraction of temporal information from high-resolution frames. This segmentation allows accurate temporal information to be obtained from high-resolution data while the computationally intensive motion analysis is performed on smaller low-resolution data.
Solution Approach 2:
The patent performs motion estimation preliminarily on low-resolution frames to establish motion vectors that guide subsequent temporal information extraction from high-resolution frames. This preliminary motion analysis reduces the search space and computational requirements when extracting temporal information at high resolution, thereby reducing processing time while maintaining accuracy.
Data Source
AI summary
A method and computer program product are provided for resolution enhancement of a video stream based on spatial and temporal correlation. For instance, the method can include predicting interpolated pixels for an image frame of the video stream based on a spatial correlation of pixels in the image frame. The method can also include generating one or more motion vectors for the image frame. Based on the spatially-correlated pixels and the one or more motion vectors, an enhanced image can be reconstructed. Further, the method can include providing a correction factor to one or more pixels in the enhanced image frame.