Multi-modal medical image fusion based on multi-CNN combination and fuzzy neural network

Abstract

The invention belongs to the field of medical image fusion, and particularly relates to multi-modal medical image fusion based on multi-CNN combination and a fuzzy neural network. In order to enable texture details of a focus part in a multi-modal medical image to be more sufficient in representation and clearer in edge, the method provided by the invention mainly comprises two parts: 1) constructing a G-CNN group (G-CNNs); and 2) G-CNNs fusion based on the fuzzy neural network. In the first part, different Gabor expression pairs of CT and MR are obtained through a group of Gabor filters with different proportions and directions, and then each pair of different Gabor expressions is used to train a corresponding CNN so as to generate G-CNNs; and in the second part, a fuzzy neural network is utilized to fuse a plurality of outputs of the G-CNNs to obtain a final fused image.

Description

technical field [0001] The invention belongs to the field of medical image fusion, in particular to multimodal medical image fusion based on multi-CNN combination and fuzzy neural network. Background technique [0002] Image fusion has a wide range of applications such as medical imaging, remote sensing, machine vision, biometrics and military applications. The purpose of fusion is to achieve better contrast and perceptual experience. In recent years, with the increasing demand for clinical applications, research on multimodal medical image fusion has attracted much attention. The purpose of multimodal medical image fusion is to provide a better medical image to help doctors perform surgical intervention. [0003] Nowadays, there are many modalities in medical images, such as magnetic resonance (MR) images, computed tomography (CT) images, positron emission tomography (PET) images and X-ray images, etc., and images of different modalities have their own advantages and lim...

AI Technical Summary

Problems solved by technology

Combine the decision map with the source image to generate a fusion image, but the CNN model proposed by liu, the final output feature dimension of convolution and downsampling is low, resulting in the loss of fusion image information

Zhang et al. proposed a general image fusion framework FCNN based on a fully convolutional neural network. This framework uses a fully convolutional neural network to solve the problem of information loss, but the convolution kernel setting of the convolutional layer is too simple, and the extracted The feature cannot represent the texture information of the lesion well, making it difficult to directly use the fused image for clinical diagnosis

Images