Infrared and visible light image fusion method based on self-attention mechanism

Abstract

The invention discloses an infrared and visible light image fusion method based on a self-attention mechanism, and belongs to the field of image fusion. The method mainly solves the importance problemthat subjective feelings of human eyes are ignored during image fusion. The method comprises the following steps: 1) establishing a deep self-coding network structure based on a self-attention mechanism, extracting features in an end-to-end manner, and simultaneously learning a fusion rule and a reconstruction rule; 2) inputting the feature maps of different branches of the coding layer into a self-attention mechanism to obtain an attention map, and adopting a mean value fusion strategy to obtain an output feature map; 3) designing a content loss function and a detail loss function which arerespectively used for highlighting infrared target information, sharpening edges and better utilizing texture details in the source image; 4) training a neural network, and visualizing a self-attention mechanism to adjust a network structure and a loss function. According to the method, attention can be distributed in an optimal mode by learning an attention graph, image key information is collected, the visual effect is improved, and the quality of a fused image is improved.

Description

technical field [0001] The invention belongs to the field of image fusion, relates to an infrared and visible light image fusion method based on a self-attention mechanism, and is widely used in the fields of military monitoring, video monitoring, computer vision and the like. Background technique [0002] The fusion of infrared and visible light images is of great significance in the fields of video surveillance, object detection and target recognition. The infrared sensor can capture the thermal information of the scene, and has strong anti-interference ability and target recognition ability for the external environment. However, in terms of imaging effect, signal-to-noise ratio, etc., the performance is average, and background details are easily lost, and the resolution is low. The visible light sensor can obtain the geometric and texture details and color information of the image, and use the reflectivity of light to form an image. Therefore, it has the characteristics ...

AI Technical Summary

Problems solved by technology

[0005] The purpose of the present invention is to address the deficiencies of the above-mentioned prior art, and propose a fusion method of infrared and visible light images based on a self-attention mechanism to solve the problem of ignoring the subjective feelings of the human eye, and drive the self-attention unit through content loss and detail loss Obtain key information, enhance image clarity, improve visual effects, and improve the quality of fused images

Images