Neural Frame Reconstruction via Displacement Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current frame reconstruction methods in encoders and decoders face challenges in accurately predicting and reconstructing target frames using motion data, often resulting in blurry or poorly distinguishable images due to independent coding of motion data and subsequent motion compensation.
Innovation Solution
The method involves generating displacement maps between reference and target frames using an optical flow model, training a conditioned reconstruction model, and applying blending coefficients to warped reference frames to reconstruct the target frame, thereby improving rate distortion performance and image quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If independent coding of motion data and subsequent motion compensation is used, then device complexity is reduced, but manufacturing precision (reconstruction quality) deteriorates resulting in blurry images
Solution Approach 1:
The patent merges motion data coding and image reconstruction into a unified neural network-based joint coding framework. The optical flow model and reconstruction model work together to simultaneously code motion information and reconstruct images, eliminating the separation between motion compensation and reconstruction stages, thereby improving reconstruction quality without proportionally increasing complexity.
Solution Approach 2:
The patent introduces displacement maps as an intermediary representation that captures motion information between reference frames and target frames. These displacement maps serve as a bridge between the optical flow model and the reconstruction model, enabling accurate motion representation while maintaining computational efficiency through the neural network's optimized processing.
2Speed
If traditional motion compensation methods are used, then processing speed is maintained, but measurement precision (motion data accuracy) deteriorates
Solution Approach 1:
The patent replaces traditional mechanical motion compensation algorithms with a neural network-based optical flow model. This substitution enables the system to learn complex motion patterns from training data and generate more accurate displacement maps, improving motion data precision while maintaining processing speed through the neural network's efficient parallel computation.
3Ease of operation
If all pixels are processed equally in reconstruction, then ease of operation is maintained, but manufacturing precision (pixel selection accuracy) deteriorates
Solution Approach 1:
The patent applies local quality by using blending coefficients to selectively weight different pixels from reference frames during reconstruction. The neural network learns to assign different importance levels to different spatial regions and pixels based on their relevance to the target frame, allowing important pixels to contribute more to the reconstruction while maintaining computational efficiency through the unified model.
Data Source
AI summary
Systems and methods are disclosed for reconstructing a frame. A computer-implemented method may use a computer system that includes non-transient electronic storage, a graphical user interface, and one or more physical computer processors. The computer-implemented method may include: obtaining one or more reference frames from non-transient electronic storage, generating one or more displacement maps based on the one or more reference frames and a target frame with the physical computer processor, generating one or more warped frames based on the one or more reference frames and the one or more displacement maps with the physical computer processor, obtaining a conditioned reconstruction model from the non-transient electronic storage, and generating one or more blending coefficients and one or more reconstructed displacement maps by applying the one or more displacement maps, the one or more warped frames, and a target frame to the conditioned reconstruction model with the physical computer processor.