Multimodal medical image fusion method based on global information fusion

Abstract

The invention discloses a multi-modal medical image fusion method based on global information fusion. The method comprises the following steps: 1, carrying out preprocessing of color space conversion and image cutting on original multi-modal medical images; 2, establishing a modal branch network which interacts through a fusion module at multiple scales, and establishing the fusion module formed by Transform to merge multi-modal feature information; 3, establishing a reconstruction module, and synthesizing a fusion image from the multi-scale multi-modal features; 4, training and evaluating the model on a public data set; and 4, realizing a medical image fusion task by using the trained model. According to the method, multi-modal semantic information can be fully fused through the Transform fusion module and the interactive modal branch network, the fine-grained fusion effect is achieved, the structure and texture information of an original image are well reserved, and the mosaic phenomenon caused by a low-resolution medical image is improved.

Description

technical field [0001] The invention relates to the technical field of image fusion, in particular to a medical image fusion technology based on deep learning. Background technique [0002] Medical images can assist doctors to better understand the structure and organization of the human body, and are widely used in clinical applications such as disease diagnosis, treatment planning, and surgical guidance. Due to the different imaging mechanisms, medical images of different modalities pay different attention to human organs and tissues. Medical images of a single modality often cannot provide comprehensive and sufficient information. Doctors often need to observe multiple images at the same time to make an accurate judgment on the condition, which will inevitably bring certain difficulties to diagnosis. Due to the limitations of single-modal medical images, multi-modal medical image fusion is a very necessary research field. Multimodal medical image fusion refers to the sy...

AI Technical Summary

Problems solved by technology

Traditional medical image fusion methods have achieved good fusion results, but there are also some deficiencies that limit the further improvement of fusion performance

First, the fusion performance of traditional methods relies heavily on artificially defined features, which limits the generalization of the method to other fusion tasks

Second, different features may require different fusion strategies to function

Third, for the fusion method based on sparse representation, its dictionary learning is relatively time-consuming, so it takes more time to synthesize a fusion image

However, these fusion strategies cannot effectively extract the global semantic information of multimodal images

In addition, the current medical image fusion method based on deep learning has insufficient and imprecise problems in the utilization of multi-modal image information.

Images