Explanatable thermal infrared visible light image registration method and system

Abstract

The invention discloses an interpretable thermal infrared visible light image registration method and system. The method comprises the following basic steps: 1) simulating the process of extracting descriptors, matching and estimating transformation parameters and transforming images by using a neural network through a traditional registration algorithm; 2) adopting a coarse-to-fine registration strategy of firstly carrying out global transformation and then carrying out local transformation; 3) constructing a loss function to train the network; and 4) processing the thermal infrared image-visible light image registration problem under different internal and external parameters by using the trained network. According to the explainable registration deep neural network (ERDNN) provided by the invention, the pixel-level registration of the thermal infrared camera and the visible light camera which are not overlapped with the optical center can be realized. The invention further provides a method for realizing the pixel-level registration of the thermal infrared camera and the visible light camera which are not overlapped with the optical center. The trained descriptor sub-network can be used as a general descriptor extractor to extract thermal infrared-visible light cross-modal descriptors. The method has wide use value and application prospect in the fields of computer vision, automatic driving, monitoring security and protection and the like.

Description

technical field [0001] The invention relates to an interpretable thermal infrared visible light image registration method and system, and belongs to the field of computer vision. It has broad application prospects in the fields of computer vision, multimodal data fusion, monitoring and security, and autonomous driving. Background technique [0002] Infrared thermal imaging technology has many advantages over visible light imaging. Visible light cameras require auxiliary light to image clearly and stably; in contrast, thermal infrared cameras mainly perform imaging based on the infrared rays generated by the thermal radiation of objects, and can be used in extreme weather conditions such as night, poor visibility, and haze. In addition, compared with radar detection technology, infrared imaging is passive radiation imaging, which does not actively emit electromagnetic waves, which is not easy to be detected by the enemy. Moreover, thermal infrared cameras also have a certai...

AI Technical Summary

Problems solved by technology

[0006] However, the above methods are all based on unlabeled data sets, and the evaluation indicators are mostly subjective evaluations.

In addition, the feature point descriptors extracted by traditional feature-based registration methods have not been tested on large-scale data sets, and it is difficult to measure the correlation between thermal infrared images and visible light images in different scenarios, resulting in low universality of the algorithm ; while the unsupervised deep learning method does not consider the thermal infrared and visible light image registration process, which belongs to end-to-end training, resulting in low algorithm stability

Images