Multi-modal image registration method based on deep learning

Abstract

The invention belongs to the field of image processing, discloses a multi-modal image registration method based on deep learning, and is used for solving the problem that the error is large when a traditional image registration method is used for registering complex multi-modal images shot by cameras in different imaging modes. The image registration method comprises the following steps: firstly, putting original images in an image set A into a semantic segmentation network to obtain a segmented image set B; multiplying the corresponding images in the image set B and the image set A pixel by pixel to obtain an image set C; and then selecting one image from the image set C as a fixed image, calculating geometric transformation by using gray value information of the image to obtain a deformed image, calculating the similarity between the deformed image and the reference image, iterating the optimal transformation T with the maximum similarity of the two images through an optimization algorithm, and applying the transformation T to the image set A to complete registration of the images. According to the method, the complex multi-modal image can be registered, and the method has the characteristics of high registration precision and real-time performance.

Description

technical field [0001] The invention relates to an image registration method, in particular to a multi-modal image registration method based on deep learning. Background technique [0002] Cameras with different imaging modes are developed on the basis of ordinary aerial cameras. The shooting of different imaging modes refers to the expansion to multiple directions such as infrared light and ultraviolet light on the basis of visible light, and through the combination of various filters or beam splitters and various photosensitive films, it can receive the same target at the same time. The information radiated or reflected on different narrow spectral bands can obtain several photos of the target in different spectral bands. The imaging technology of different imaging modes combines the functions of the spatial imaging system and the spectral detection system, and can obtain the information of the measured target from different dimensions such as the spectral dimension and t...

AI Technical Summary

Problems solved by technology

[0005] However, in the face of complex multi-modal images, due to the large difference in the gray value of the photos taken by different imaging modes, the images taken in different wavelength bands will not only highlight different objects, such as some Irrelevant background objects, and even for the same target, the gray value in the multi-modal image will be very different, resulting in the existing gray-based image registration algorithm due to the redundant background in the multi-modal image Gray value information, resulting in the addition of background gray value information that has nothing to do with the registration subject when calculating the gray value information of the image, which cannot correctly calculate the similarity between the two images and calculate the optimal transformation T through the optimization algorithm. As a result, the alignment of the two images cannot be completed, and finally the result of image registration cannot meet the expected requirements

Images