Semi-supervised multi-mode nuclear magnetic resonance image synthesis method based on coarse-to-fine learning

Abstract

The invention provides a semi-supervised multi-mode nuclear magnetic resonance image synthesis method based on coarse-to-fine learning. The method comprises the following steps: constructing most of unpaired data and a small part of paired data by using a multi-modal data set of all cases, constructing an MRI synthesis model comprising a generative network and an enhanced network, training the generative network by using all the data, training the enhanced network by using the small part of paired data, and constructing an MRI synthesis model comprising the generative network and the enhanced network; inputting an image of a source mode into the trained generation network, mapping the image of the source mode into a coarse synthesis image of a corresponding target mode by using the learned cross-mode distribution mapping relation, inputting the coarse synthesis image into the trained enhancement network, and performing refined enhancement on the coarse synthesis image of the target mode by the enhancement network to obtain a refined enhancement image of the target mode; and obtaining a cross-modal synthesis MR image of the target modal. The method can be applied to cross-modal medical image synthesis, the existing source modal image of the patient is used for synthesizing the target modal image, and assistance is provided for assisting doctors in disease diagnosis.

Description

technical field [0001] The invention relates to the technical field of image processing, in particular to a semi-supervised multi-mode nuclear magnetic resonance image synthesis method based on coarse-to-fine learning. Background technique [0002] In recent years, with the rapid development of digital information construction such as images, medical care, sensor networks, and multi-sensory devices, the generation and collection of multi-modal data has become more convenient and easier. The so-called multi-modal data refers to the same sample semantic object, there are multiple sources or forms of information, which effectively describe the sample from different angles. For example, when we browse the web, the description of an object on the web page may contain pictures, text, and hyperlinks. Compared with single-modal data, multi-modal data contains more information. Comprehensive consideration of the information contained in multi-modal data allows us to have a deeper un...

AI Technical Summary

Problems solved by technology

The disadvantages of this method are: due to various reasons in reality, it is very difficult to obtain paired multimodal data, and there is often a large proportion of unpaired data in the data set. If the unpaired data is directly discarded, it will cause serious problems Waste of useful information, and only using a small amount of paired data for training will lead to the failure of supervised learning models that require a large amount of paired data to achieve the desired effect

Images