Multi-modal image fusion method, system and equipment based on information bottleneck and medium

Abstract

The invention belongs to the field of image fusion, and provides a multi-modal image fusion method and system based on information bottleneck, equipment and a medium. The fusion method comprises the following steps: acquiring an infrared and visible light multi-modal image; preprocessing each pair of infrared and visible light multi-mode images; extracting features of each pair of infrared and visible light multi-mode images after preprocessing; screening the features of each pair of infrared and visible light multi-mode images by using an information bottleneck layer; fusing the screened infrared and visible light multi-modal image features to obtain a fused feature map; and reconstructing the fused feature map to obtain a fused image.

Description

technical field [0001] The invention belongs to the field of image fusion, and in particular relates to an information bottleneck-based multi-modal image fusion method, system, device and medium. Background technique [0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art. [0003] Image fusion is essentially an enhancement technique that fuses different types of data from different sensors to generate an image that contains rich information or is more conducive to subsequent applications. The information obtained by the same type of sensors is very limited, and the information from different sensors needs to be fused. Fusion technology plays an important role in video surveillance, modern military, and medical pathology research. Infrared and visible light image fusion is an important part of image fusion technology, and visible light images are generated by visible light se...

AI Technical Summary

Problems solved by technology

Although existing methods have achieved better results in most cases, the inventors have found that there are also some negative factors

First, in most traditional methods, image transformation and fusion rules need to be manually designed, which may limit its application in some tasks

Second, due to the diversity of source images, the amount of calculation, and the limitations of many factors, it is quite difficult for traditional methods to propose a design method that considers the ideal fusion task.

Third, although deep learning has shown strong capabilities in feature extraction and data representation, and has avoided the complexity of traditional manual design in image fusion tasks, it is not clear whether it is useful information or redundant information in the feature extraction part. There is no comprehensive theoretical understanding that can adequately explain

Fourth, the existing deep learning methods are difficult to achieve a balance in multi-modal information

In summary, in the multi-modal image fusion problem based on deep learning, there is still a lack of effective solutions for comprehensively utilizing its available feature information for fusion.

Images