Multi-scale pyramid type jump connection method for image completion

Abstract

The invention discloses a multi-scale pyramid type jump connection method for image completion, and is used for any image completion network taking an encoder-decoder as a framework, an encoder feature set and a decoder feature set are taken as input and a weighted composite feature set is taken as output, the weighted composite feature set replaces a decoder feature set to be inputted into a subsequent output layer, and the method comprises the following steps: S1, inputting an encoder feature set into a gating, and filtering local white noise mapped by a cavity region to obtain a weak noise feature set; s2, integrating the weak noise feature set into a composite feature of a calibrated size by using a feature integration module; and S3, inputting the composite feature into a channel spatial domain weight adaptive fusion module, and adaptively fusing the channel weight and the spatial domain weight with the decoder feature set to obtain the weighted composite feature set. According to the method, the problems of artifacts and fuzziness caused by local white noise are effectively relieved through noise filtering of each scale feature of the encoder and adaptive integration processing of channel and airspace two-dimensional weights.

Description

technical field [0001] The invention relates to the technical field of image completion based on deep learning, in particular to a multi-scale pyramid-type skip connection method for image completion. Background technique [0002] Image completion algorithms aim at synthesizing the structure and texture of the hollow regions of the damaged image, restoring the image into a realistic image. Image completion algorithms are essential in many image editing tasks. It can be used to repair damaged photos or fill in empty areas left after removing distracting objects from images. It can also be extended to object removal, scratch repair, etc. , Completion of lost content in image coding and transmission, super-resolution image reconstruction and other applications. [0003] The content generated by the image completion algorithm should be similar to the original image, or match the context semantics of the hollow area, so that the completed image is visually natural and real. The...

AI Technical Summary

Problems solved by technology

[0006] The purpose of the present invention is to overcome the problem of encoder features carrying noise caused by the hole region of the image to be completed, fully integrate the multi-scale encoder features after denoising, and realize the adaptive fusion of the features of each layer of the encoder and decoder. A multi-scale pyramidal skip connection method for image completion is proposed. This method can effectively overcome the artifacts caused by local white noise by filtering the noise of each scale feature of the decoder and adaptively integrating the two-dimensional weights of channels and spaces. , fuzzy problem while making full use of the context information of the image, it can be applied to any image completion based on the encoder-decoder model framework

Images