Light field image super-resolution reconstruction method based on frequency domain analysis and deep learning

Abstract

The invention discloses a light field image super-resolution reconstruction method based on frequency domain analysis and deep learning, which first uses optical flow-based rendering and bicubic interpolation to generate an initial high spatial and angular resolution light field image; then adopts Discrete cosine transform is used to perform frequency domain conversion on the generated initial light field image to obtain a frequency component image that characterizes the characteristics of the light field image; based on this, super-resolution reconstruction can be modeled as a frequency restoration problem, so multiple 2D convolutions are constructed neural network to model the initial restoration of each frequency component, and integrate semantic information into the network to enhance the restoration effect; then combine the frequency components after the initial restoration and build a 3D convolutional neural network for fine restoration; finally, use The inverse discrete cosine transform is used to reconstruct all the restored frequency components into the required light field image; the advantage is that it can effectively improve the spatial and angular resolution of the light field image, and can restore texture information and retain angular consistency.

Description

technical field [0001] The present invention relates to an image super-resolution reconstruction technology, in particular to a light field image super-resolution reconstruction method based on frequency domain analysis and deep learning. Background technique [0002] As an emerging computational imaging technology, light field imaging can simultaneously record the intensity (that is, spatial information) and direction (that is, angle information) of light in a scene, and is being extensively researched and concerned by academia and industry. Recently, optical instruments based on light field imaging, such as light field cameras, have been developed to obtain more scene information. At the same time, many light field applications have also emerged, such as 3D reconstruction, depth estimation, post-capture refocusing, etc. By inserting optical components such as microlens arrays between the main lens and the imaging sensor, the light field camera can collect spatial and angu...

AI Technical Summary

Problems solved by technology

[0007] In summary, although the current related research has achieved good super-resolution results of light field images, there are still some deficiencies in dealing with large reconstruction scales and challenging scenes, such as occlusion scenes, especially in restoring There is still room for improvement in reconstructing the texture information of light field images, avoiding visual artifacts such as ghosting, and preserving angular consistency

Images