Image fusion method based on joint convolutional self-coding network

Abstract

The invention discloses an image fusion method based on a joint convolutional self-coding network, and belongs to the field of image fusion. The method mainly solves the problems that during image fusion, a data set and labels are insufficient, and an end-to-end fusion result of an image is obtained through a joint convolutional self-coding network. The method comprises the following steps: firstly, training a to-be-fused image set by using a joint convolutional self-coding network model in a training process, and designing a multi-task loss function suitable for image fusion for training. Inthe testing process, two to-be-fused images are input into the network model, public features and private features are obtained through the network coding layer, a fusion rule is designed according toredundancy and complementary features of the features, fusion of the feature layers is achieved, and fused feature mapping is decoded and reconstructed to obtain a fused image. According to the method, the characteristics of the self-coding neural network can be fully utilized to integrate complementation and redundant information of the to-be-fused image to formulate a fusion strategy, image details are effectively protected, and compared with a traditional fusion method, the quality of the fused image is greatly improved.

Description

technical field

[0001] The invention belongs to the field of image fusion, and relates to an image fusion method based on a joint convolutional self-encoding network, which is widely used in the fields of scene monitoring, battlefield reconnaissance and the like. Background technique

[0002] Image fusion is an image enhancement technology, and it is also a research branch and research focus in the field of information fusion. The fused image is generated by fusing images acquired by different sensors. The fused image is robust and contains rich information of the source image, which is beneficial to subsequent image processing. Therefore, the field of image fusion involves a wide range of research and the fusion process is complex. And it is diverse, so it is difficult to have a mature and general-purpose image fusion algorithm suitable for the field of image fusion. Usually, our research objects include: multi-focus image fusion, infrared and visible light image fusion, an...

Examples

Embodiment Construction

[0055] An embodiment of the present invention ("street" infrared and visible light image) will be described in detail below in conjunction with the accompanying drawings. This embodiment is carried out under the premise of the technical solution of the present invention, as figure 1 As shown, the detailed implementation and specific operation steps are as follows:

[0056] Step 1. During the training process, the image to be fused passes through the private feature branch and the public feature branch of the encoding layer to obtain private features and public features respectively. In order to improve the ability of the joint convolutional self-encoding network for image fusion, we introduced the image fusion evaluation indicators MSE, SSIM, entropy and gradient into the loss function, designed a multi-task loss function for network training, and improved the joint convolutional self-encoding. The feature extraction ability of the network.

[0057] Step 2, during the test pr...

technical field

[0001] The invention belongs to the field of image fusion, and relates to an image fusion method based on a joint convolutional self-encoding network, which is widely used in the fields of scene monitoring, battlefield reconnaissance and the like. Background technique

[0002] Image fusion is an image enhancement technology, and it is also a research branch and research focus in the field of information fusion. The fused image is generated by fusing images acquired by different sensors. The fused image is robust and contains rich information of the source image, which is beneficial to subsequent image processing. Therefore, the field of image fusion involves a wide range of research and the fusion process is complex. And it is diverse, so it is difficult to have a mature and general-purpose image fusion algorithm suitable for the field of image fusion. Usually, our research objects include: multi-focus image fusion, infrared and visible light image fusion, an...

Examples

Embodiment Construction

[0055] An embodiment of the present invention ("street" infrared and visible light image) will be described in detail below in conjunction with the accompanying drawings. This embodiment is carried out under the premise of the technical solution of the present invention, as figure 1 As shown, the detailed implementation and specific operation steps are as follows:

[0056] Step 1. During the training process, the image to be fused passes through the private feature branch and the public feature branch of the encoding layer to obtain private features and public features respectively. In order to improve the ability of the joint convolutional self-encoding network for image fusion, we introduced the image fusion evaluation indicators MSE, SSIM, entropy and gradient into the loss function, designed a multi-task loss function for network training, and improved the joint convolutional self-encoding. The feature extraction ability of the network.

[0057] Step 2, during the test pr...

Images