Nuclear magnetic resonance image prediction method, training method, device, equipment, medium and product

Abstract

The invention provides a nuclear magnetic resonance image prediction method, a training method, a device, equipment, a medium and a product, which mainly utilize overturning distribution alignment constraint to enable input and potential distribution of a target to converge to standard normal distribution from two directions symmetrical about the standard normal distribution, namely, mean values are opposite, and variances are the same, so that in the middle and later stages of training, the target potential distribution can be predicted. As the two distributions converge towards the standard normal distribution, the two distributions can maintain the non-overlapping part to the greatest extent on the premise of ensuring partial overlapping, namely, maintain the common features and independent features of the input and target images, so that the requirement of the spatial structure of the potential feature space is met, the feature learning ability of the model is improved, and the feature learning efficiency of the model is improved. And meanwhile, the feature decoupling capability is improved, so that the image generation quality stability is ensured.

Description

Technical Field

[0001] The present application relates to the technical field of medical image processing, and in particular, to a nuclear magnetic resonance image prediction method, training method, device, equipment, medium and product. Background Technique

[0002] Medical image synthesis technology is a key application of deep learning image generation models in medical image data analysis. This technology is mainly used to synthesize missing parts or higher-quality images based on existing images or low-quality images, so as to accelerate imaging, reduce the cost of contrast agents, and improve the use efficiency of equipment.

[0003] Multi-phase contrast-enhanced (CE)-magnetic resonance (MRI) technology has important value in the clinical diagnosis of tumors, organ lesions and vascular abnormalities. However, this technology has problems such as long scanning time, possible nephrotoxicity caused by gadolinium-based contrast agents, and expensive MRI equipment, which limit its large-scale clinical application. Different from the significant differences in contrast and texture in multi-modal imaging (such as CT, MRI, ultrasound), the imaging results of different phases of CE-MRI are less different, mainly depending on the distribution of contrast agents in blood vessels, and are usually divided into the pre-phase (Pre), arterial phase (C+A), venous phase (C+V) and delayed phase (Delay). In order to more effectively separate and synthesize these subtle image differences, a new synthesis model needs to be constructed.

[0004] Currently, the mainstream medical image synthesis methods include those based on autoencoders (AE) and diffusion models. Autoencoders are widely used because of their simple structure and small computational cost. However, when existing AE methods are used to process MRI images, the following problems mainly exist:

[0005] 1. The degree of structuring in the latent feature space is insufficient, affecting the feature learning ability;

[0006] 2. The feature decoupling ability is weak, resulting in unstable quality of the generated images;

[0007] 3. Excessive parameters lead to large consumption of computing resources;

[0008] These problems seriously affect the feasibility of this technology in clinical applications. Summary of the Invention

[0009] The present application proposes a nuclear magnetic resonance image prediction method, training method, device, equipment, medium and product, which can solve one of the problems existing in the background technique.

[0010] To achieve the above object, the present application adopts the following technical solutions:

[0011] In a first aspect, a method for training a nuclear magnetic resonance (MRI) image prediction model is provided. The training method includes:

[0012] Obtaining training data, where the training data includes paired multi-phase MRI images; and

[0013] Using the training data to train the MRI image prediction model,

[0014] The MRI image prediction model is used for:

[0015] Encoding the input first-phase MRI image using a shared variational autoencoder to obtain a first initial latent distribution, and encoding the input second-phase MRI image to obtain a second initial latent distribution. The first initial latent distribution is defined by a first initial mean and a first initial variance, and the second initial latent distribution is defined by a second initial mean and a second initial variance;

[0016] Using a flipped distribution alignment constraint to process the first initial latent distribution to obtain a first aligned latent distribution, and processing the second initial latent distribution to obtain a second aligned latent distribution. The first aligned latent distribution is defined by a first aligned mean and a first aligned variance, and the second aligned latent distribution is defined by a second aligned mean and a second aligned variance. The flipped distribution alignment constraint makes the first aligned mean have the opposite sign and the same value as the second initial mean, makes the second aligned mean have the opposite sign and the same value as the first initial mean, and makes the first aligned variance the same as the second initial variance, and makes the second aligned variance the same as the first initial variance; and

[0017] Performing decoding processing on the first aligned latent distribution and the second aligned latent distribution.

[0018] Based on the above technical solution, mainly using the flipped distribution alignment constraint, the latent distributions of the input and the target start to converge to the standard normal distribution from two directions symmetric about the standard normal distribution, that is, the means are opposite and the variances are the same. Then, in the middle and late stages of training, as the two distributions converge to the standard normal distribution, the two distributions can, on the premise of ensuring partial overlap, retain the non-overlapping part to the greatest extent, that is, retain the common features and independent features of the input and target images. In this way, the requirements for the structural degree of the latent feature space are met, the feature learning ability of the model is improved, and at the same time, the feature decoupling ability is improved, ensuring the stability of the image generation quality.

[0019] In a possible design of the first aspect, the loss function of the flipped distribution alignment constraint is

[0020]

[0021] wherein, μ A is the first initial mean value, μ B is the second initial mean value, σ A is the first initial variance, σ B is the second initial variance, and ||*||1 represents the L1 norm.

[0022] In a possible design of the first aspect, the MRI image prediction model is further configured to:

[0023] Process the first initial latent distribution to obtain a first self-reconstruction distribution, and process the second initial latent distribution to obtain a second self-reconstruction distribution. The first self-reconstruction distribution is defined by a first self-reconstruction mean value and a first self-reconstruction variance, and the second self-reconstruction distribution is defined by a second self-reconstruction mean value and a second self-reconstruction variance; and

[0024] Perform decoding processing on the first self-reconstruction distribution and the second self-reconstruction distribution.

[0025] In a possible design of the first aspect, in the MRI image prediction model, for the self-reconstruction part, pixel loss is used for supervision, and for the alignment part, pixel loss, GAN loss, and perceptual loss are used for collaborative supervision.

[0026] In a second aspect, a nuclear magnetic resonance MRI image prediction method is provided. The prediction method includes:

[0027] Obtain a certain-phase MRI image; and

[0028] Process the certain-phase MRI image by using the trained MRI image prediction model as described above to obtain another-phase MRI prediction image.

[0029] In a third aspect, a training device for a nuclear magnetic resonance MRI image prediction model is provided. The training device includes:

[0030] A first acquisition unit, configured to acquire training data, where the training data includes paired multi-phase MRI images; and

[0031] A training unit, configured to use the training data to train the MRI image prediction model,

[0032] The MRI image prediction model is configured to:

[0033] Using a shared variational autoencoder, the input first-phase MRI image is encoded to obtain a first initial latent distribution, and the input second-phase MRI image is encoded to obtain a second initial latent distribution. The first initial latent distribution is defined by a first initial mean and a first initial variance, and the second initial latent distribution is defined by a second initial mean and a second initial variance;

[0034] Using the flip distribution alignment constraint, the first initial latent distribution is processed to obtain a first aligned latent distribution, and the second initial latent distribution is processed to obtain a second aligned latent distribution. The first aligned latent distribution is defined by a first aligned mean and a first aligned variance, and the second aligned latent distribution is defined by a second aligned mean and a second aligned variance. The flip distribution alignment constraint makes the first aligned mean have the opposite sign and the same value as the second initial mean, makes the second aligned mean have the opposite sign and the same value as the first initial mean, and makes the first aligned variance the same as the second initial variance, and makes the second aligned variance the same as the first initial variance; and

[0035] Performing decoding processing on the first aligned latent distribution and the second aligned latent distribution.

[0036] In a fourth aspect, a nuclear magnetic resonance (MRI) image prediction device is provided. The prediction device includes:

[0037] A second acquisition unit configured to acquire a certain-phase MRI image; and

[0038] A prediction unit configured to process the certain-phase MRI image by using the trained MRI image prediction model as described above to obtain another-phase MRI prediction image.

[0039] In a fifth aspect, an electronic device is provided. The electronic device includes: a processor, and a memory coupled to the processor. The memory is configured to store a computer program; the processor is configured to execute the computer program stored in the memory so that the electronic device executes the training method according to any possible implementation manner in the first aspect, or executes the prediction method in the second aspect.

[0040] In the present application, the electronic device described in the fifth aspect may be a terminal device or a network device, or a chip (system) or other components or assemblies disposed in a terminal device or a network device.

[0041] In a sixth aspect, a computer-readable storage medium is provided, including a computer program or instruction. When the computer program or instruction runs on a computer, the computer is caused to execute the training method according to any possible implementation manner in the first aspect, or execute the prediction method in the second aspect.

[0042] In a seventh aspect, a computer program product is provided, including: a computer program or instructions, which, when running on a computer, cause the computer to execute the training method according to any possible implementation manner of the first aspect, or execute the prediction method according to the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the related art descriptions will be briefly introduced below. Obviously, the drawings in the following descriptions are only some embodiments of the embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0044] Figure 1 is a schematic comparison diagram of the existing autoencoder image generation framework (a)-(d) provided by the embodiments of the present application and the method (e);

[0045] Figure 2 is a schematic structural diagram of the flip distribution alignment variational autoencoder provided by the embodiments of the present application;

[0046] Figure 3 is a schematic diagram of the convergence of the input and target image latent distributions provided by the embodiments of the present application: (a) only using KL divergence constraint, (b) using KL divergence + flip distribution alignment (FDA) constraint;

[0047] Figure 4 is a schematic diagram of the visualization results of feature decoupling provided by the embodiments of the present application: (a) at the pixel level (b) at the latent space level;

[0048] Figure 5 is a schematic diagram of the visualization of the synthesis results and corresponding errors of multiple methods provided by the embodiments of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0049] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] It should be noted that although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from the module division in the device or the order in the flowchart. Terms such as "first" and "second" in the description and claims of the specification and the above drawings are used to distinguish similar objects and do not necessarily need to describe a specific order or sequence.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs. The terms used herein are for the purpose of describing embodiments of this application only and are not intended to limit this application.

[0052] An embodiment of this application proposes a multi-phase MRI image synthesis method based on Flip Distribution Alignment - Variational Autoencoder (FDA-VAE), or a prediction method, or a training method of the model. This method realizes efficient multi-phase image synthesis and interpretable feature decoupling through a new type of lightweight feature decoupling VAE model.

[0053] Using the multi-phase MRI image synthesis method, it is ultimately possible to use any one-phase MRI image to predict any subsequent phase MRI image. For example, it is possible to use the pre-phase MRI image to predict the C+V phase MRI image, or use the pre-phase MRI image to predict the C+A phase MRI image, etc.

[0054] 1. Symbol Explanation and Background Introduction

[0055] Variational Autoencoder (VAE) is a variant of Autoencoder (AE) and is commonly used in image generation tasks. It also includes an encoder and a decoder. For the original input x, the encoder encodes the input into a mean tensor μ and a variance tensor σ, and this mean and variance tensor can encode the input into a latent distribution feature. Then, a random sample is taken from this distribution as the latent feature, and the sampling process is achieved through the reparameterization trick:

[0056]

[0057] where ε is a point randomly sampled from the standard normal distribution. Finally, the decoder decodes the sampled latent feature into the output image. For the output synthesized image Generally, multiple losses such as pixel loss, GAN loss, and perceptual loss are used for joint supervision. For the distribution encoded by the encoder To prevent the model from collapsing into an ordinary autoencoder, that is, the case where the variance σ is 0, generally and the standard normal distribution calculate the Kullback–Leibler (KL) divergence:

[0058]

[0059] The encoder-decoder of an autoencoder is usually composed of multiple feature extraction modules. Early classical autoencoders were generally composed of multi-layer convolutional neural networks (CNNs). However, due to the problem of limited receptive fields in the pure CNN architecture, it is difficult to capture long-range dependencies. The Vision Transformer (ViT) model solves this problem, but the multi-head attention mechanism brings a large amount of computational overhead. Recently, the Mamba model based on state-space models (SSMs) achieves global feature capture ability while saving computational overhead. In addition, some methods build hybrid architecture autoencoders by integrating multiple feature extraction modules to integrate the advantages of different methods, thereby constructing a high-performance autoencoder backbone model.

[0060] On the basis of combining the advantages of multiple feature extraction modules, it is also very important to construct a flexible autoencoder training architecture. As Figure 1 (a)-(d) shown in are the existing common autoencoder training methods. In addition to the classical autoencoder (a) and variational autoencoder (d) architectures introduced in the previous text, there are also methods (b) that improve the quality of target image synthesis by introducing multi-dimensional inputs (using multiple different images as inputs) and methods (c) that strengthen the common features between the input and target images at the latent feature level through contrastive learning.

[0061] 2. Method

[0062] The embodiment of this application proposes a lightweight feature decoupled VAE model for contrast-enhanced MRI image synthesis. By constructing a lightweight VAE generator backbone and introducing explicit latent space feature decoupling alignment constraints, the independent and common features of the input and target images are separated, so as to achieve efficient and high-quality medical image generation.

[0063] The model mainly consists of a shared encoder, a flipped distribution alignment constraint layer, and two independent decoders. In the training stage, a pair of different-phase MRI images are input into the model respectively to obtain two self-reconstruction and cross-phase synthesis results (a total of four outputs). In the validation and inference stage, to save computational overhead, it can be set to only retain the required target decoder. The input image is input into the encoder to obtain the latent distribution features, then the mean of the latent distribution features is flipped, and the target features are sampled from the flipped distribution. Finally, the corresponding decoder is used to decode the target features to obtain the predicted target image.

[0064] 2.1 Lightweight Variational Autoencoder Framework

[0065] In high-resolution image generation methods, a pre-trained variational autoencoder model is often used as a data compressor to improve image generation performance. Specifically, advanced image generation methods such as the Latent Diffusion Model (LDM) and the Vision Autoregressive Model (VAR) usually pre-train a variational autoencoder or a vector quantization autoencoder on a large number of natural images. Through the pre-training of self-reconstruction of the input images, they compress the input pictures into smaller latent representations and use them for subsequent latent representation generation training. In similar methods, the pre-trained VAE models of LDM and VAR generally use about 1.2 million natural image data for training, and the model size is about 100 million parameters. In a similar LDM-based medical image synthesis method, about 38,000 medical image slices are generally used to train a VAE model with a volume of about 12 million parameters. In contrast, the training data pair of the existing end-to-end synthesis model based on autoencoder for medical images is about 2,500 pairs, while the corresponding autoencoder parameters exceed 100 million. Although the image translation task requires more complex non-linear mapping ability than the self-reconstruction task, in multi-phase contrast-enhanced magnetic resonance imaging data, there is often a highly consistent structural correlation between different-phase images. Therefore, there may be a problem of insufficient efficiency in the existing models from the perspective of knowledge representation.

[0066] Based on the volume of training data, this method constructs a lightweight VAE model with a smaller number of parameters as the backbone network for image synthesis. Through an efficient and interpretable feature decoupling constraint and a bidirectional synthesis strategy, the quality of the synthesized images is improved, thus achieving better results than the existing large-volume models. Among them, the encoder and decoder of the hybrid architecture VAE backbone model are both composed of three residual convolutional layers and a non-local attention block, so as to achieve efficient local and global feature extraction.

[0067] 2.2 Flip Distribution Alignment Constraint

[0068] Based on the VAE backbone model, we encode the input image A and the target image B into two different latent distributions and where μ A , μ B and σ A 2 σ B 2 represent the mean A, B and variance A, B obtained from input A and B respectively. As Figure 3As shown, two distributions are visualized as blue and red circles, where the center represents the distribution mean and the radius represents the variance. When only the KL divergence is used to constrain the two distributions, as shown in Figure 3 (a) in Figure 3 (left of (a)), in the early stage of training ( Figure 3 (left of (a)), the two distributions converge from random directions to the standard normal distribution (i.e., the origin of coordinates in the figure). At this time, there is an unknown gap between the two latent distributions. When sampling features from two different distributions to generate the target image B, the unknown differences between the latent distributions will bring errors in the synthesis results. In the later stage of training ( (right of (a)), due to the lack of constraints on the relative positions of the two distributions, they may coincide prematurely, that is,

[0069] the situation of . At this time, the encoder can only extract the common features of the input and target images and ignore their unique parts. Figure 3 (b)), we require that the latent distributions of the input and the target converge to the standard normal distribution from two directions symmetric about the standard normal distribution, that is, the means are opposite and the variances are the same, as shown by the arrows of "μ Figure 1 to -μ A ", "σ B to σ A ", and "μ B to -μ B " in A :

[0070]

[0071] This constraint realizes a structured latent space modeling. In the early stage of training ( Figure 3 (left of (b)), distributions A and B converge to the standard normal distribution simultaneously from two symmetric directions, so that there is a predictable relative relationship between the input and target features, and a fast conversion can be achieved by flipping the mean of the input distribution, that is:

[0072]

[0073] In the middle and later stages of training ( Figure 3 (right of (b)), as the two distributions converge to the standard normal distribution, the two distributions can retain the non - overlapping parts to the greatest extent while ensuring partial overlap, that is, retain the common features and independent features of the input and target images.

[0074] 2.3 Y - type Bidirectional Generation Strategy

[0075] To further enhance feature decoupling, we designed a Y-shaped bidirectional generation strategy. The complete method includes a shared encoder that encodes the input and output images into a symmetric latent distribution, and two independent decoders for synthesizing different images respectively. In each training iteration, a pair of MRI image slices A and B from different periods are obtained from the dataset. Taking the input A as an example (the input B is the same): For the given input image x A , the process includes encoding, flipping the distribution, and decoding to obtain two outputs and where represents the self-reconstruction result of the given input, represents the converted synthesis result (cross-period synthesis result) of the given output:

[0076] μ A ,σ A =Encoder(x A ),μ B ,σ B =Encoder(x B )

[0077]

[0078] For the self-reconstruction result, pixel loss is used for supervision, and for the converted synthesis result, pixel loss GAN loss and perceptual loss are used for collaborative supervision. The loss function of the complete model is summarized as:

[0079]

[0080] The Y-shaped bidirectional generation strategy ensures that the latent distribution features obtained after the input and output images enter the encoder can most represent the original input, making the flipping distribution alignment constraint and the corresponding synthesis result have stronger interpretability.

[0081] Specifically, in the training stage, taking the MRI image A as the input, the target image is obtained. Then, the real MRI image B is used for supervision reference. Similarly, taking the MRI image B as the input, the target image is obtained. Then, the real MRI image A is used for supervision reference.

[0082] 3. Method Advantages

[0083] (1)An interpretable feature decoupling medical image generation model is constructed. Compared with most existing methods that learn the non-linear mapping relationship between input and output images through deep autoencoders with a large number of parameters, our method explicitly separates the independent and common features of input and output images at the latent feature level, and realizes feature transformation through an efficient and simple distribution flipping mechanism. At the same time, the bidirectional synthesis strategy we designed also ensures the interpretability and training stability of this mechanism. In Figure 4 we show the visualization results of feature decoupling at the pixel level and the latent feature level

[0084] (2) The evaluation values of the synthesis results designed for multiple groups of different input and output images on the publicly available multi-phase CE-MRI dataset are better than existing methods, and at the same time, it is the method with the fewest synthesis inference parameters among all current methods. We use a lightweight hybrid architecture variational autoencoder that performs well in the self-reconstruction task but has weak performance in the conversion synthesis task as the backbone model for image synthesis. By constructing a structured latent space modeling, the backbone model achieves higher synthesis quality in the medical image conversion synthesis task while requiring far fewer parameters than current existing high-performance methods. In Tables 1 and 2, we respectively show the quantitative evaluation values of the synthesis results of multiple existing methods and our method, as well as the number of parameters required for the corresponding models and the inference speed (the inference speed results are calculated on a single Nvidia RTX 4090 24G GPU, and the specific values measured on devices with different performances may vary). Among them, Table 1 shows the quantitative evaluation results of the comparative experiment and ablation experiment. We use Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), and Learned Perceptual Image Patch Similarity (LPIPS) as evaluation metrics to evaluate the similarity between the synthesized image and the corresponding real image. Generally speaking, compared with existing advanced methods based on autoencoders (such as Pix2Pix, ResVit, TransUnet, PTNet, I2I-Mamba), our method (FDA-VAE) has obtained better quantitative evaluation results in most cross-phase synthesis tasks. At the same time, to verify the effectiveness of the core innovation points in this method, we separately trained the lightweight variational autoencoder backbone model (VAE(backbone)) used and the method that only uses the backbone network plus the flipped distribution alignment constraint (VAE(KL+FDA)). Combining the evaluation data in Tables 1 and 2, the lightweight VAE backbone model has a slightly lower evaluation result than existing advanced methods because its number of parameters is much smaller than that of existing models. However, by introducing the flipped distribution alignment constraint and the Y-shaped bidirectional synthesis strategy respectively, better synthesis results are obtained without increasing the number of inference parameters.

[0085] (3) Figure 5 Shows the error visualization heat map of the synthesis results of this method and other image synthesis methods and the corresponding real results. It can be seen that the synthesis results of this method have the smallest error at the pixel level.

[0086] Table 1 Performance comparison with representative medical image synthesis methods (bold represents the best, underline represents the second best, the same as Table 2)

[0087]

[0088] Table 2 Comparison of the Parameter Quantity and Inference Speed of Representative Medical Image Synthesis Methods

[0089]

[0090] The embodiment of the present application also provides a training device for a nuclear magnetic resonance (MRI) image prediction model, and the training device includes:

[0091] A first acquisition unit, configured to acquire training data, where the training data includes paired multi-phase MRI images; and

[0092] A training unit, configured to use the training data to train the MRI image prediction model,

[0093] The MRI image prediction model is used for:

[0094] Using a shared variational autoencoder to encode the input first-phase MRI image to obtain a first initial latent distribution, and encoding the input second-phase MRI image to obtain a second initial latent distribution, where the first initial latent distribution is defined by a first initial mean and a first initial variance, and the second initial latent distribution is defined by a second initial mean and a second initial variance;

[0095] Using a flipped distribution alignment constraint to process the first initial latent distribution to obtain a first aligned latent distribution, and processing the second initial latent distribution to obtain a second aligned latent distribution, where the first aligned latent distribution is defined by a first aligned mean and a first aligned variance, and the second aligned latent distribution is defined by a second aligned mean and a second aligned variance, and the flipped distribution alignment constraint makes the first aligned mean have the opposite sign and the same value as the second initial mean, makes the second aligned mean have the opposite sign and the same value as the first initial mean, and makes the first aligned variance the same as the second initial variance, and makes the second aligned variance the same as the first initial variance; and

[0096] Performing decoding processing on the first aligned latent distribution and the second aligned latent distribution.

[0097] The embodiment of the present application also provides a nuclear magnetic resonance (MRI) image prediction device, and the prediction device includes:

[0098] A second acquisition unit, configured to acquire a certain-phase MRI image; and

[0099] A prediction unit, configured to use the trained MRI image prediction model as described above to process the certain-phase MRI image to obtain another-phase MRI prediction image.

[0100] An embodiment of the present application further provides an electronic device, including: a processor, and a memory coupled to the processor, where the memory is used to store a computer program; the processor is used to execute the computer program stored in the memory so that the electronic device executes the method described in any one of the above embodiments.

[0101] The electronic device may be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The electronic device may include, but is not limited to, a processor and a memory.

[0102] The so-called processor may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The processor is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire device.

[0103] The memory may be used to store the computer program. The processor realizes various functions of the electronic device by running or executing the computer program stored in the memory and calling the data stored in the memory.

[0104] The memory may mainly include a program storage area and a data storage area. Among them, the program storage area may store an operating system, application programs required for at least one function, etc.; the data storage area may store data created according to the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0105] The embodiment of the present application further provides a storage medium, which is a computer-readable storage medium. The computer program is stored in the computer-readable storage medium. When the computer program is executed by a processor, the steps of the above various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium, etc.

[0106] The embodiment of the present application further provides a computer program product, including: a computer program or instruction. When the computer program or instruction runs on a computer, the computer is enabled to execute the method of any one of the above possible implementation manners.

[0107] The above is the preferred embodiment of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements are also regarded as the protection scope of the present application.

Claims

1. A training method for a nuclear magnetic resonance (MRI) image prediction model, characterized in that The training method includes: Obtaining training data, where the training data includes paired multi-phase MRI images; and Using the training data to train an MRI image prediction model, The MRI image prediction model is used for: Using a shared variational autoencoder to encode the input first-phase MRI image to obtain a first initial latent distribution, and encoding the input second-phase MRI image to obtain a second initial latent distribution, where the first initial latent distribution is defined by a first initial mean and a first initial variance, and the second initial latent distribution is defined by a second initial mean and a second initial variance; Using a flipped distribution alignment constraint to process the first initial latent distribution to obtain a first aligned latent distribution, and processing the second initial latent distribution to obtain a second aligned latent distribution, where the first aligned latent distribution is defined by a first aligned mean and a first aligned variance, and the second aligned latent distribution is defined by a second aligned mean and a second aligned variance, and the flipped distribution alignment constraint makes the first aligned mean have the opposite sign but the same value as the second initial mean, makes the second aligned mean have the opposite sign but the same value as the first initial mean, and makes the first aligned variance the same as the second initial variance, and makes the second aligned variance the same as the first initial variance; and Performing a decoding process on the first aligned latent distribution and the second aligned latent distribution.

2. The training method according to claim 1, wherein The loss function of the flipping distribution alignment constraint is where, μ A is the first initial mean, μ B is the second initial mean, σ A is the first initial variance, σ B is the second initial variance, ||*||1 represents the L1 norm.

3. The training method according to claim 1, wherein The MRI image prediction model is further used for: Processing the first initial latent distribution to obtain a first self-reconstruction distribution, and processing the second initial latent distribution to obtain a second self-reconstruction distribution, where the first self-reconstruction distribution is defined by a first self-reconstruction mean and a first self-reconstruction variance, and the second self-reconstruction distribution is defined by a second self-reconstruction mean and a second self-reconstruction variance; and Performing a decoding process on the first self-reconstruction distribution and the second self-reconstruction distribution.

4. The training method according to claim 2, wherein In the MRI image prediction model, for the self-reconstruction part, pixel loss is used for supervision, and for the alignment part, pixel loss, GAN loss, and perceptual loss are used for collaborative supervision.

5. A method for predicting nuclear magnetic resonance (MRI) images, characterized in that, The prediction method includes: Obtaining a certain-phase MRI image; and Using the trained MRI image prediction model as described in any one of claims 1-4 to process the certain-phase MRI image to obtain another-phase MRI prediction image.

6. A training device for a nuclear magnetic resonance (MRI) image prediction model, characterized in that, The training device includes: A first acquisition unit for obtaining training data, where the training data includes paired multi-phase MRI images; and A training unit for using the training data to train an MRI image prediction model, The MRI image prediction model is used for: Using a shared variational autoencoder to encode the input first-phase MRI image to obtain a first initial latent distribution, and encoding the input second-phase MRI image to obtain a second initial latent distribution, where the first initial latent distribution is defined by a first initial mean and a first initial variance, and the second initial latent distribution is defined by a second initial mean and a second initial variance; Using the flipped distribution alignment constraint, process the first initial latent distribution to obtain a first aligned latent distribution, and process the second initial latent distribution to obtain a second aligned latent distribution. The first aligned latent distribution is defined by a first aligned mean and a first aligned variance, and the second aligned latent distribution is defined by a second aligned mean and a second aligned variance. The flipped distribution alignment constraint makes the first aligned mean have the opposite sign and the same value as the second initial mean, makes the second aligned mean have the opposite sign and the same value as the first initial mean, and makes the first aligned variance the same as the second initial variance, and makes the second aligned variance the same as the first initial variance; and Perform decoding processing on the first aligned latent distribution and the second aligned latent distribution.

7. An MRI image prediction device for nuclear magnetic resonance, characterized in that, The prediction device includes: A second acquisition unit for acquiring a certain-phase MRI image; and A prediction unit for processing the certain-phase MRI image by using the MRI image prediction model trained according to any one of claims 1-4 to obtain another-phase MRI prediction image.

8. An electronic device, characterized in that, The electronic device includes: a processor, and a memory coupled to the processor, The memory for storing a computer program; and The processor for executing the computer program stored in the memory so that the electronic device executes the training method according to any one of claims 1-4, or executes the prediction method according to claim 5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a computer program or instruction, which when running on a computer, causes the computer to execute the training method according to any one of claims 1-4, or execute the prediction method according to claim 5.

10. A computer program product, characterized in that, The computer program product includes: a computer program or instruction, which when running on a computer, causes the computer to execute the training method according to any one of claims 1-4, or execute the prediction method according to claim 5.

Images