Three-dimensional high dynamic range imaging method based on full convolutional neural network

Abstract

The invention discloses a three-dimensional high dynamic range imaging method based on a full convolutional neural network. The method comprises the following steps: converting a left viewpoint underexposure image and a right viewpoint overexposure image into the same exposure by using an exposure calibration network; based on the estimated disparity map, respectively performing forward drawing onthe left viewpoint underexposure image and the right viewpoint overexposure image to generate a drawn right viewpoint underexposure image and a drawn left viewpoint overexposure image; introducing extra exposure information to guide hole filling in an image generated by drawing; extracting fusion features by utilizing an HDR image fusion network, fusing the left viewpoint overexposure image afterhole filling and the original left viewpoint overexposure image into a left viewpoint HDR image, and fusing the right viewpoint overexposure image after hole filling and the original right viewpointoverexposure image into a right viewpoint HDR image; and then obtaining a three-dimensional HDR image. The method has the advantages that the dynamic range of the original three-dimensional LDR imagecan be improved, and detail information of the original three-dimensional LDR image in the areas with insufficient exposure and excessive exposure can be reconstructed.

Description

technical field [0001] The invention relates to a stereoscopic high dynamic range imaging method, in particular to a stereoscopic high dynamic range imaging method based on a fully convolutional neural network. Background technique [0002] With the rapid development of three-dimensional (Three-dimensional, 3D) imaging technology and display devices, more and more stereoscopic images have entered people's lives, including 3D movies, 3D games and so on. However, most of the existing stereo cameras (stereo LDR cameras) can only capture the limited dynamic range of natural scenes (such as common 8-bit images, whose dynamic range is less than 3 orders of magnitude), which inevitably causes the captured stereo images to vary. Partial underexposure or overexposure occurs. A high dynamic range (HDR) imaging technique that reconstructs the dynamic range of a natural scene by fusing a series of low dynamic range (LDR) images captured according to different exposures can effectively ...

AI Technical Summary

Problems solved by technology

For the above methods, due to the saturation of the sensor in the stereo camera, the CRF-based exposure calibration will introduce quantization distortion and cause color artifacts in the calibrated stereo image, especially in the underexposed and overexposed regions of the stereo image, which In turn, the performance of subsequent stereo matching will be reduced; since the image drawing in SHDRI is one-way drawing, a large number of holes will be generated on the left border of the image (the right viewpoint image is drawn to the left viewpoint) or the right border (the left viewpoint image is drawn to the right viewpoint) , and these holes are difficult to effectively fill by traditional methods

also, et al. and Park et al.’s methods only set the brightness threshold to construct the fusion weight map when generating stereoscopic HDR images, which may easily lead to the loss of details and color distortion in the generated stereoscopic HDR images.

Chen et al. observed that the essence of SHDRI is to use stereoscopic multi-exposure images to generate left-view multi-exposure images and right-view multi-exposure images respectively, so they proposed a viewpoint exposure transfer method based on generative adversarial networks (GAN) , achieved good results, but this method does not consider the parallax of stereo images in essence, so for some challenging scenes, blurring and ghosting artifacts will be introduced

[0004] Although the current related research has achieved a good SHDRI effect, there are still some shortcomings in improving the details of the generated image.

Images