Image fusion method based on joint convolutional self-coding network

Abstract

The invention discloses an image fusion method based on a joint convolutional self-coding network, and belongs to the field of image fusion. The method mainly solves the problems that during image fusion, a data set and labels are insufficient, and an end-to-end fusion result of an image is obtained through a joint convolutional self-coding network. The method comprises the following steps: firstly, training a to-be-fused image set by using a joint convolutional self-coding network model in a training process, and designing a multi-task loss function suitable for image fusion for training. Inthe testing process, two to-be-fused images are input into the network model, public features and private features are obtained through the network coding layer, a fusion rule is designed according toredundancy and complementary features of the features, fusion of the feature layers is achieved, and fused feature mapping is decoded and reconstructed to obtain a fused image. According to the method, the characteristics of the self-coding neural network can be fully utilized to integrate complementation and redundant information of the to-be-fused image to formulate a fusion strategy, image details are effectively protected, and compared with a traditional fusion method, the quality of the fused image is greatly improved.

Description

technical field [0001] The invention belongs to the field of image fusion, and relates to an image fusion method based on a joint convolutional self-encoding network, which is widely used in the fields of scene monitoring, battlefield reconnaissance and the like. Background technique [0002] Image fusion is an image enhancement technology, and it is also a research branch and research focus in the field of information fusion. The fused image is generated by fusing images acquired by different sensors. The fused image is robust and contains rich information of the source image, which is beneficial to subsequent image processing. Therefore, the field of image fusion involves a wide range of research and the fusion process is complex. And it is diverse, so it is difficult to have a mature and general-purpose image fusion algorithm suitable for the field of image fusion. Usually, our research objects include: multi-focus image fusion, infrared and visible light image fusion, an...

AI Technical Summary

Problems solved by technology

[0008] The purpose of the present invention is to address the deficiencies of the above-mentioned prior art, and propose an image fusion method based on a joint convolutional autoencoder network, to solve the problem of insufficient training data of fusion images obtained by existing multi-sensor image fusion methods based on neural networks or training Label information cannot be obtained, and fully combined with the proposed joint convolutional autoencoder network's ability to reconstruct images, the image fusion evaluation index is introduced into the loss function of the training network, which can not only reconstruct the original input image, but also effectively protect image details , enhance the image contrast and edge contour, improve its visual effect, and improve the quality of the fusion image

Images