Longitudinal magnetic resonance image generation model training method, generation method and equipment

Abstract

The invention provides a longitudinal magnetic resonance image generation model training method, a generation method and equipment, and relates to the technical field of image data processing, and the training method comprises the steps: gradually adding noise to a longitudinal magnetic resonance prediction image in a forward diffusion process, so as to obtain a noise image of each forward step, according to the last noise image and baseline image conforming to Gaussian distribution, gradually obtaining a composite image of each reverse step based on a learnable condition encoder and a generator in a reverse generation process, and after obtaining the composite image of one reverse step each time, obtaining the composite image of the next reverse step; performing posterior sampling on the composite image of the reverse step to obtain a composite noise image; and gradually inputting the noise image of each reverse step and the synthesized noise image into a discriminator for discrimination. The training efficiency and effectiveness of the longitudinal magnetic resonance image generation model can be improved, the diversity of the generated composite image can be ensured, and the generated composite image and the magnetic resonance baseline image can be ensured to belong to the same testee.

Description

Technical Field

[0001] The present application relates to the technical field of image data processing, and particularly to a method, a generation method and a device for training a longitudinal magnetic resonance image generation model. Background Art

[0002] Longitudinal magnetic resonance image generation is a very important research direction in medical image analysis, aiming to simulate the evolution process of an individual's brain at different time points by using advanced generation models (such as generative adversarial networks GANs, diffusion models or variational autoencoders VAE), and generate very realistic longitudinal magnetic resonance synthetic images for predicting the changes of an individual's brain at future times. The predicted longitudinal magnetic resonance synthetic images have important application values in fields such as disease progression prediction, treatment effect evaluation, data augmentation and personalized medicine. For example, in disease progression simulation, the generation model can predict the brain changes at future time points based on the magnetic resonance baseline images of patients, thereby helping doctors formulate intervention strategies in advance.

[0003] Currently, the above-mentioned commonly used generation models generate high-quality longitudinal magnetic resonance synthetic images by learning the spatial and temporal features of magnetic resonance images. However, generative adversarial networks are prone to mode collapse, resulting in low diversity of the generated images, while diffusion models are prone to slow sampling speed, and the above-mentioned generation models all have the problem that they cannot ensure that the predicted and generated longitudinal magnetic resonance synthetic images belong to the same subject as the magnetic resonance baseline images, and thus cannot ensure the application reliability and effectiveness of the predicted and generated longitudinal magnetic resonance synthetic images.

[0004] Therefore, there is an urgent need to design a way to improve the training efficiency of the longitudinal magnetic resonance image generation model, ensure the diversity of the generated synthetic images, and ensure that the generated synthetic images and the magnetic resonance baseline images belong to the same subject. Summary of the Invention

[0005] In view of this, embodiments of the present application provide a method, a generation method and a device for training a longitudinal magnetic resonance image generation model to eliminate or improve one or more defects existing in the prior art.

[0006] One aspect of the present application provides a method for training a longitudinal magnetic resonance image generation model, including: Perform iterative training on the diffusion generative adversarial network for at least one round, and perform model training steps separately in each round. Among them, the model training steps include: gradually adding noise to the longitudinal magnetic resonance prediction image of the subject in each forward step of the forward diffusion process of the diffusion generative adversarial network to obtain a noise image corresponding to each forward step, and stop adding noise after obtaining a noise image that conforms to the Gaussian distribution; according to the noise image that conforms to the Gaussian distribution and the magnetic resonance baseline image of the subject, gradually obtain the longitudinal magnetic resonance synthesis images corresponding to each reverse step based on the learnable conditional encoder and the generator in the reverse generation process of the diffusion generative adversarial network, and after obtaining the longitudinal magnetic resonance synthesis image corresponding to each reverse step each time, perform posterior sampling on the longitudinal magnetic resonance synthesis image corresponding to this reverse step to obtain the synthetic noise image corresponding to this reverse step; gradually input the synthetic noise image of each reverse step and the noise image of the forward step corresponding to this reverse step into the discriminator, so that the discriminator outputs the discriminant result data corresponding to each reverse step respectively to optimize the discriminator and the generator; Take the learnable conditional encoder and the generator in the diffusion generative adversarial network after iterative training as the longitudinal magnetic resonance image generation model.

[0007] In some embodiments of the present application, the step of gradually obtaining the longitudinal magnetic resonance synthesis images corresponding to each reverse step based on the learnable conditional encoder and the generator in the reverse generation process of the diffusion generative adversarial network according to the noise image that conforms to the Gaussian distribution and the magnetic resonance baseline image of the subject, and after obtaining the longitudinal magnetic resonance synthesis image corresponding to each reverse step each time, performing posterior sampling on the longitudinal magnetic resonance synthesis image corresponding to this reverse step to obtain the synthetic noise image corresponding to this reverse step includes: In each reverse step of the reverse generation process of the diffusion generative adversarial network, perform an image synthesis training sub-step separately; Among them, the image synthesis training sub-step includes: In the current reverse step, input the noise image that conforms to the Gaussian distribution, the magnetic resonance baseline image of the subject, and the auxiliary data of the subject into the learnable conditional encoder, so that the learnable conditional encoder outputs the corresponding encoded data; among them, the auxiliary data includes: the age, gender, and diagnosis label of the subject; Input the encoded data into the generator, so that the generator outputs the longitudinal magnetic resonance synthesis image corresponding to the current reverse step; Perform posterior sampling on the longitudinal magnetic resonance synthesis image corresponding to the current reverse step to obtain the synthetic noise image corresponding to this reverse step; Calculate the generator loss corresponding to the current backward step according to the longitudinal magnetic resonance synthesis image corresponding to the current backward step and the longitudinal magnetic resonance prediction image of the subject. Update the model parameters of the generator based on the generator loss corresponding to the current backward step.

[0008] In some embodiments of the present application, the generator loss includes: mean absolute error loss and the training loss of the generator. Correspondingly, the calculating the generator loss corresponding to the current backward step according to the longitudinal magnetic resonance synthesis image corresponding to the current backward step and the longitudinal magnetic resonance prediction image of the subject includes: Obtain the mean absolute error between the longitudinal magnetic resonance synthesis image of the current backward step and the longitudinal magnetic resonance prediction image of the current round as the mean absolute error loss corresponding to the current backward step. And, obtain the training loss of the generator corresponding to the current backward step according to the longitudinal magnetic resonance prediction image of the current round and the noise image of the current backward step.

[0009] In some embodiments of the present application, the generator loss further includes: the correlation loss between the time interval and the image difference. Correspondingly, the calculating the generator loss corresponding to the current backward step according to the longitudinal magnetic resonance synthesis image corresponding to the current backward step and the longitudinal magnetic resonance prediction image of the subject further includes: Obtain the correlation loss between the time interval and the image difference corresponding to the current backward step according to the longitudinal magnetic resonance synthesis image of the current backward step, the longitudinal magnetic resonance prediction image of the current round, the age of the subject corresponding to the acquisition of the magnetic resonance baseline image, the age of the subject corresponding to the longitudinal magnetic resonance prediction image of the current round, the preset maximum age and the minimum age.

[0010] In some embodiments of the present application, updating the model parameters of the generator based on the generator loss corresponding to the current backward step includes: Determine the generator loss corresponding to the current backward step according to the product of the training loss of the generator corresponding to the current backward step, the mean absolute error loss and a preset first hyperparameter, and the product of the correlation loss between the time interval and the image difference and a preset second hyperparameter, and update the model parameters of the generator based on the generator loss corresponding to the current backward step.

[0011] In some embodiments of the present application, before performing at least one round of iterative training on the diffusion generative adversarial network, it further includes: Obtain the original magnetic resonance images collected for each subject at historical time points and the original longitudinal magnetic resonance prediction images collected for each subject at target time points after the historical time points; Perform image preprocessing, data cleaning, and normalization processing on the original magnetic resonance images and the original longitudinal magnetic resonance prediction images of each subject; Respectively use the original magnetic resonance images of each subject corresponding to each subject after the image preprocessing, data cleaning, and normalization processing as the magnetic resonance reference images corresponding to each subject, and respectively use the original longitudinal magnetic resonance prediction images of each subject corresponding to each subject after the image preprocessing, data cleaning, and normalization processing as the longitudinal magnetic resonance prediction images corresponding to each subject.

[0012] The second aspect of the present application provides a method for generating longitudinal magnetic resonance images, including: Input a noise image conforming to a Gaussian distribution, the magnetic resonance baseline image of the current target subject, and the auxiliary data of the target subject into the longitudinal magnetic resonance image generation model, so that based on the learning conditional encoder and generator in the longitudinal magnetic resonance image generation model, gradually obtain the longitudinal magnetic resonance synthesis images corresponding to each step, and after obtaining the longitudinal magnetic resonance synthesis image corresponding to each reverse step each time, perform posterior sampling on the longitudinal magnetic resonance synthesis image corresponding to the reverse step to obtain the synthetic noise image corresponding to the reverse step; wherein, the longitudinal magnetic resonance image generation model is pre-trained based on the longitudinal magnetic resonance image generation model training method; the auxiliary data includes: the age, gender, and diagnosis label of the target subject; Use the longitudinal magnetic resonance synthesis image output by the generator in the last reverse step as the longitudinal magnetic resonance image prediction result data of the target subject at the target time point.

[0013] The third aspect of the present application provides a longitudinal magnetic resonance image generation model training device, including: An iterative training module for performing at least one round of iterative training on a diffusion generative adversarial network and separately executing a model training step in each round, wherein the model training step includes: gradually adding noise to the longitudinal magnetic resonance prediction image of the subject in each forward step of the forward diffusion process of the diffusion generative adversarial network to obtain a noise image corresponding to each forward step, and stopping adding noise after obtaining a noise image conforming to a Gaussian distribution; based on the noise image conforming to a Gaussian distribution and the magnetic resonance baseline image of the subject, gradually obtaining a longitudinal magnetic resonance synthetic image corresponding to each backward step based on a learnable conditional encoder and a generator in the backward generation process of the diffusion generative adversarial network, and after obtaining the longitudinal magnetic resonance synthetic image corresponding to a backward step each time, performing posterior sampling on the longitudinal magnetic resonance synthetic image corresponding to the backward step to obtain a synthetic noise image corresponding to the backward step; gradually inputting the synthetic noise image of each backward step and the noise image of the forward step corresponding to the backward step into a discriminator, so that the discriminator outputs discriminant result data corresponding to each backward step respectively to optimize the discriminator and the generator; A model determination module for using the learnable conditional encoder and the generator in the diffusion generative adversarial network after iterative training as a longitudinal magnetic resonance image generation model.

[0014] The fourth aspect of the present application provides a longitudinal magnetic resonance image generation device, including: A model prediction module for inputting a noise image conforming to a Gaussian distribution, a magnetic resonance baseline image of a current target subject, and auxiliary data of the target subject into a longitudinal magnetic resonance image generation model, so as to gradually obtain a longitudinal magnetic resonance synthetic image corresponding to each step based on the learnable conditional encoder and the generator in the longitudinal magnetic resonance image generation model, and after obtaining the longitudinal magnetic resonance synthetic image corresponding to a backward step each time, performing posterior sampling on the longitudinal magnetic resonance synthetic image corresponding to the backward step to obtain a synthetic noise image corresponding to the backward step; wherein, the longitudinal magnetic resonance image generation model is pre-trained based on the longitudinal magnetic resonance image generation model training method; the auxiliary data includes: the age, gender, and diagnostic label of the target subject; A result output module for using the longitudinal magnetic resonance synthetic image output by the generator in the last backward step as the longitudinal magnetic resonance image prediction result data of the target subject at the target time point.

[0015] The fifth aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the longitudinal magnetic resonance image generation model training method described above is implemented, and / or the longitudinal magnetic resonance image generation method described above is implemented.

[0016] The sixth aspect of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the longitudinal magnetic resonance image generation model training method described above is implemented, and / or the longitudinal magnetic resonance image generation method described above is implemented.

[0017] The seventh aspect of the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, the longitudinal magnetic resonance image generation model training method described above is implemented, and / or the longitudinal magnetic resonance image generation method described above is implemented.

[0018] The longitudinal magnetic resonance image generation model training method provided by the present application performs at least one round of iterative training on a diffusion generative adversarial network, and respectively executes model training steps in each round. Among them, the model training steps include: gradually adding noise to the longitudinal magnetic resonance prediction image of the subject in each forward step of the forward diffusion process of the diffusion generative adversarial network to obtain the noise image corresponding to each forward step, and stopping adding noise after obtaining a noise image that conforms to a Gaussian distribution; according to the noise image that conforms to a Gaussian distribution and the magnetic resonance baseline image of the subject, gradually obtain the longitudinal magnetic resonance synthetic image corresponding to each backward step based on a learnable conditional encoder and a generator in the backward generation process of the diffusion generative adversarial network, and after each time obtaining the longitudinal magnetic resonance synthetic image corresponding to a backward step, perform posterior sampling on the longitudinal magnetic resonance synthetic image corresponding to this backward step to obtain the synthetic noise image corresponding to this backward step; gradually input the synthetic noise image of each backward step and the noise image of the forward step corresponding to this backward step into a discriminator, so that the discriminator respectively outputs the discriminant result data corresponding to each backward step to optimize the discriminator and the generator; using the learnable conditional encoder and the generator in the diffusion generative adversarial network after iterative training as a longitudinal magnetic resonance image generation model; it can improve the training efficiency and effectiveness of the longitudinal magnetic resonance image generation model, can ensure the diversity of the generated synthetic images, and can ensure that the generated synthetic images and the magnetic resonance baseline images belong to the same subject.

[0019] Additional advantages, objects, and features of the present application will be partly set forth in the description which follows, and in part will become obvious to those having ordinary skill in the art upon examination of the following, or may be learned by practice of the present application. The objects and other advantages of the present application may be realized and attained by the structure particularly pointed out in the specification and the drawings.

[0020] Those skilled in the art will understand that the objects and advantages that can be achieved with the present application are not limited to those specifically described above, and the above and other objects that the present application can achieve will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present application, and constitute a part of the present application, but do not limit the present application. The components in the drawings are not drawn to scale, but are only for showing the principles of the present application. For the convenience of showing and describing some parts of the present application, the corresponding parts in the drawings may be enlarged, that is, may become larger relative to other components in the exemplary device actually manufactured according to the present application. In the drawings: Figure 1 FIG. 1 is a first flowchart of a method for training a longitudinal magnetic resonance image generation model in an embodiment of the present application.

[0022] Figure 2 FIG. 2 is a first flowchart of the model training steps within step 100 in an embodiment of the present application.

[0023] Figure 3

[0024] Figure 4

[0025] Figure 5

[0026] Figure 6

[0027] Figure 7

[0028] Figure 8

[0029]

[0028] Figure 8 FIG. 3 is a flowchart of a method for training a longitudinal magnetic resonance image generation model in an application example of the present application.

[0029] Figure 9 Schematic diagram of the comparison results of the longitudinal data (GT(3 years)) provided by this application three years after synthesis.

[0030] Figure 10 Schematic diagram of the comparison results of the longitudinal data (GT(6 years)) provided by this application six years after synthesis.

[0031] Figure 11 Schematic flowchart of the longitudinal magnetic resonance image generation method in an embodiment of this application.

[0032] Figure 12 Schematic diagram of the structure of the longitudinal magnetic resonance image generation model training device in an embodiment of this application.

[0033] Figure 13 Schematic diagram of the structure of the longitudinal magnetic resonance image generation device in an embodiment of this application. Detailed implementation manners

[0034] To make the objectives, technical solutions, and advantages of this application clearer and more understandable, the following further elaborates on this application in conjunction with the implementation manners and the accompanying drawings. Herein, the illustrative implementation manners of this application and their descriptions are used to explain this application, but do not limit this application.

[0035] Herein, it should also be noted that in order to avoid obscuring this application with unnecessary details, only the structures and / or processing steps closely related to the solution of this application are shown in the drawings, while other details less relevant to this application are omitted.

[0036] It should be emphasized that the term "including / containing" when used herein refers to the presence of features, elements, steps, or components, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0037] Herein, it should also be noted that if not otherwise specified, the term "connection" in this text can refer not only to direct connection but also to indirect connection with an intermediate.

[0038] In the following, embodiments of this application will be described with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0039] Commonly used methods for generating longitudinal magnetic resonance images include generative adversarial networks (such as conditional GANs, CycleGAN, and 3DGANs), diffusion models (such as the denoising diffusion probabilistic model DDPM), and variational autoencoders (such as time series VAE and conditional VAE). These methods generate high-quality longitudinal images by learning the spatial and temporal features of magnetic resonance images. In conditional GANs (i.e., conditional generative adversarial networks) among GANs, by introducing conditional information such as baseline magnetic resonance images and time points, images at future time points that match the conditions are generated; CycleGAN is a variant model of generative adversarial networks, which is designed to learn image-to-image conversion between two different domains without paired training data. It does not require corresponding images of the same scene or object in two different domains. CycleGAN learns the mapping between images at different time points through cyclic consistency loss without paired data; 3D GANs is a three-dimensional generative adversarial network for generating three-dimensional geometric data; in the diffusion model, the denoising diffusion probabilistic model DDPM generates high-quality images by gradually adding and removing noise, and can generate magnetic resonance images with rich details. The conditional diffusion model further introduces conditional information to generate future images consistent with the baseline image and time point. In the variational autoencoder VAE, the time series combines time series modeling (such as recurrent neural network RNN, long short-term memory artificial neural network LSTM) and VAE to generate a continuous sequence of longitudinal magnetic resonance images; the conditional variational autoencoder VAE generates future images that match the baseline image and time point by introducing conditional information.

[0040] Traditionally, studying the changes of the brain over time relied on the acquisition and analysis of longitudinal magnetic resonance image data. However, data at some time points may be missing or incomplete, which limits the accuracy of characterizing the changes of the brain over time. To solve these problems, generative models are used to simulate the changes of an individual's brain at different time points to generate realistic magnetic resonance images. In addition, synthetic longitudinal magnetic resonance images are used for disease progression prediction, treatment effect evaluation, and data augmentation, etc. However, generative adversarial networks are prone to mode collapse and generate images with low diversity; diffusion models have slow sampling speed; and none of the above generative models can guarantee that the predicted and generated longitudinal magnetic resonance synthetic images belong to the same subject as the magnetic resonance baseline image.

[0041] Based on this, in order to design a method that can improve the training efficiency of the longitudinal magnetic resonance image generation model, ensure the diversity of the generated synthetic images, and ensure that the generated synthetic images and the magnetic resonance baseline images belong to the same subject, the embodiments of the present application respectively provide a method for training a longitudinal magnetic resonance image generation model, a method for generating longitudinal magnetic resonance images, a longitudinal magnetic resonance image generation model training device for executing the method for training a longitudinal magnetic resonance image generation model, a longitudinal magnetic resonance image generation device for executing the method for generating longitudinal magnetic resonance images, an entity device, a computer-readable storage medium, and a computer program product, which can improve the training efficiency and effectiveness of the longitudinal magnetic resonance image generation model, ensure the diversity of the generated synthetic images, and ensure that the generated synthetic images and the magnetic resonance baseline images belong to the same subject.

[0042] Specifically, it is described in detail through the following embodiments.

[0043] Based on this, the embodiments of the present application provide a method for training a longitudinal magnetic resonance image generation model that can be implemented by a longitudinal magnetic resonance image generation model training device. Refer to Figure 1 , and the method for training a longitudinal magnetic resonance image generation model specifically includes the following content: Step 100: Perform iterative training on the diffusion generative adversarial network for at least one round, and respectively execute the model training step in each round. Among them, the model training step includes: gradually adding noise to the longitudinal magnetic resonance prediction image of the subject in each forward step of the forward diffusion process of the diffusion generative adversarial network to obtain the noise image corresponding to each forward step, and stop adding noise after obtaining a noise image that conforms to the Gaussian distribution; according to the noise image that conforms to the Gaussian distribution and the magnetic resonance baseline image of the subject, gradually obtain the longitudinal magnetic resonance synthetic image corresponding to each backward step based on the learnable conditional encoder and the generator in the reverse generation process of the diffusion generative adversarial network, and after each time obtaining the longitudinal magnetic resonance synthetic image corresponding to a backward step, perform posterior sampling on the longitudinal magnetic resonance synthetic image corresponding to this backward step to obtain the synthetic noise image corresponding to this backward step; gradually input the synthetic noise image of each backward step and the noise image of the forward step corresponding to this backward step into the discriminator, so that the discriminator respectively outputs the discriminant result data corresponding to each backward step to optimize the discriminator and the generator.

[0044] First of all, it should be noted that the method for training a longitudinal magnetic resonance image generation model provided in the embodiments of the present application refers to a model training method based on the diffusion generative adversarial network. That is to say, the model training framework of the longitudinal magnetic resonance image generation model is the diffusion generative adversarial network.

[0045] Diffusion Generative Adversarial Network (Diffusion GAN) is a novel generative model that combines diffusion models and generative adversarial networks, aiming to overcome the limitations of traditional adversarial generative networks in problems such as training instability and mode collapse, as well as the slow data generation problem of diffusion models. The core idea of this model is to gradually generate data by introducing a diffusion process, thereby introducing more controllability and stability in the generation process. Specifically, the Diffusion Generative Adversarial Network can be divided into two main stages: the forward diffusion process and the reverse generation process. In the forward diffusion process, real data samples are transformed into a noise distribution as labels through a series of steps of gradually adding Gaussian noise, which is similar to the noise addition mechanism in diffusion models. In the reverse generation process, the generator network reconstructs real data samples from the noise distribution, then generates noisy samples through posterior sampling, and finally the discriminator conducts adversarial training on each step of the forward-noisy samples and the posterior-sampled noisy samples to ensure the authenticity of each generated sample. The Diffusion Generative Adversarial Network can generate high-quality samples, maintain the stability and diversity of training, and can accelerate the sampling speed.

[0046] That is to say, for the longitudinal magnetic resonance image generation method using an adversarial generative network, since the adversarial generative network directly synthesizes an image at once, its generator only learns to generate a few image patterns and ignores the diversity of the entire data distribution, resulting in the problem of low diversity of the generated images. In the embodiments of the present application, a Diffusion Generative Adversarial Network is adopted. The diffusion model in it gradually denoises and restores the data distribution, modeling the details of the real distribution at each step and hardly falling into a local optimum of only generating a few patterns. Therefore, it can effectively ensure the diversity of the generated synthetic images.

[0047] Moreover, since the noise addition at each step in the forward propagation process of the diffusion model follows a Gaussian normal distribution, the diffusion is very small at each step and there are many steps, so it takes a long inference time to generate data. Based on this, the problem of slow sampling speed exists in the longitudinal magnetic resonance image generation method using the diffusion model. In the embodiments of the present application, a Diffusion Generative Adversarial Network is adopted. According to the situation that more data in real applications follows a multimodal distribution, in the embodiments of the present application, the diffusion process at each step of the Diffusion Generative Adversarial Network does not have to follow a Gaussian normal distribution. A conditional generator is used to fit the diffusion process at each step (step). Since each step is a non-Gaussian normal distribution, the number of diffusion steps can be reduced, thereby achieving fast data generation and further improving the training efficiency of the longitudinal magnetic resonance image generation model.

[0048] In step 100, refer to Figure 2 The model training steps in step 100 specifically include the following content: Step 110: Gradually add noise to the longitudinal magnetic resonance prediction image of the subject in each forward step of the forward diffusion process of the diffusion generative adversarial network to obtain the noise image corresponding to each forward step, and stop adding noise after obtaining a noise image that conforms to the Gaussian distribution.

[0049] It can be understood that the forward step refers to the step in the forward diffusion process. In step 100, noise is gradually added to the longitudinal magnetic resonance prediction image of the subject in each forward step of the forward diffusion process of the diffusion generative adversarial network until the latest added noise image conforms to the Gaussian distribution and then stops. At this time, the total number of each forward step can be denoted as T.

[0050] Step 120: According to the noise image that conforms to the Gaussian distribution and the magnetic resonance baseline image of the subject, gradually obtain the longitudinal magnetic resonance synthesis image corresponding to each backward step based on the learnable conditional encoder and the generator in the backward generation process of the diffusion generative adversarial network, and after obtaining the longitudinal magnetic resonance synthesis image corresponding to each backward step each time, perform posterior sampling on the longitudinal magnetic resonance synthesis image corresponding to this backward step to obtain the synthesized noise image corresponding to this backward step.

[0051] It can be understood that the backward step refers to each step in the backward generation process, and the number of backward steps is the same as and corresponds one-to-one with the number of forward steps.

[0052] Step 130: Gradually input the synthesized noise image of each backward step and the noise image of the forward step corresponding to this backward step into the discriminator, so that the discriminator outputs the discriminant result data corresponding to each backward step respectively to optimize the discriminator and the generator.

[0053] In an example, see Figure 3 , each forward step is successively A 1 、A 2 、A 3 and A 4 ; in forward step A 1 , through the forward diffusion process 1 corresponding to forward step A of step 110, the noise image x 0 corresponding to the longitudinal magnetic resonance prediction image x 1 is obtained; in forward step A 2 , through the forward diffusion process 2 corresponding to forward step A of step 120, based on the noise image x 1 , the noise image x 0 corresponding to the longitudinal magnetic resonance prediction image x 2; In the forward step A 3 Among them, in the forward step A of step 120 3 The corresponding forward diffusion process , based on the noise image x 2 , the longitudinal magnetic resonance prediction image x 0 The corresponding noise image x 3 ; In the forward step A 4 Among them, in the forward step A of step 120 3 The corresponding forward diffusion process , based on the noise image x 3 , the longitudinal magnetic resonance prediction image x 0 The corresponding noise image x 4 , if the noise image x 4 At this time, if the noise image x is the first noise image that conforms to the Gaussian distribution in the current round, stop executing the forward expansion process. At this time, the forward step has 4 steps, that is, T is equal to 4.

[0054] Correspondingly, in this example, the reverse step is also 4 steps, and each reverse step is B 1 , B 2 , B 3 And B 4 , in the reverse step B 1 Among them, through step 120, the longitudinal magnetic resonance synthesis image corresponding to the longitudinal magnetic resonance prediction image x 0 , because the reverse step B 1 Corresponds to the forward step A 4 , and the forward step A 4 Obtains the noise image x 4 , therefore, in the reverse step B 1 Among them, for the longitudinal magnetic resonance synthesis image corresponding to the reverse step B 1 Perform posterior sampling to obtain the synthetic noise image corresponding to this reverse step B 1 ; At this time, the synthetic noise image And the noise image x 4 Constitute a group of data to be discriminated, and input this group of data to be discriminated into the discriminator through step 130, so that the discriminator outputs the discriminant result data of whether the synthetic noise image Is true or false.

[0055] In the reverse step B 2 Among them, through step 120, the longitudinal magnetic resonance synthesis image corresponding to the longitudinal magnetic resonance prediction image x 0 , because the reverse step B 2 Corresponds to the forward step A 3 , and the forward step A 3 ​​​​Obtain the noisy image x 3 , therefore, in the backward step B 2 , for the longitudinal magnetic resonance synthesis image corresponding to the backward step B 2 perform posterior sampling to obtain the synthesized noisy image corresponding to this backward step B 2 ; at this time, the synthesized noisy image and the noisy image x 3 constitute a set of data to be discriminated, and this set of data to be discriminated is input into the discriminator through step 130 , so that the discriminator outputs the discrimination result data indicating whether the synthesized noisy image is true or false.

[0056] In the backward step B 2 , through step 120, obtain the longitudinal magnetic resonance synthesis image corresponding to the longitudinal magnetic resonance prediction image x 0 , because the backward step B 2 corresponds to the forward step A 3 , and the forward step A 3 obtains the noisy image x 3 , therefore, in the backward step B 2 , for the longitudinal magnetic resonance synthesis image corresponding to the backward step B 2 perform posterior sampling to obtain the synthesized noisy image corresponding to this backward step B 2 ; at this time, the synthesized noisy image and the noisy image x 3 constitute a set of data to be discriminated, and this set of data to be discriminated is input into the discriminator through step 130, so that the discriminator outputs the discrimination result data indicating whether the synthesized noisy image is true or false.

[0057] In the backward step B 3 , through step 120, obtain the longitudinal magnetic resonance synthesis image corresponding to the longitudinal magnetic resonance prediction image x 0 , because the backward step B 3 corresponds to the forward step A 2 , and the forward step A 2 obtains the noisy image x 2 , therefore, in the backward step B 3 , for the longitudinal magnetic resonance synthesis image corresponding to the backward step B 3 perform posterior sampling to obtain the synthesized noisy image corresponding to this backward step B 3 ; at this time, the synthesized noisy image and the noisy image x​​​​​​​​2 Form a set of data to be discriminated, and input the set of data to be discriminated into the discriminator through step 130, so that the discriminator outputs a synthetic noise image Discrimination result data that is true or false.

[0058] In the reverse step B 4 Through step 120, the longitudinal magnetic resonance prediction image x 0 Corresponding longitudinal magnetic resonance synthetic image Since the reverse step B 4 Corresponds to the forward step A 1 And the forward step A 1 Obtains the noise image x 1 Therefore, in the reverse step B 4 Perform posterior sampling on the longitudinal magnetic resonance synthetic image corresponding to the reverse step B 4 To obtain the synthetic noise image corresponding to this reverse step B ; At this time, the synthetic noise image 4 And the noise image x Form a set of data to be discriminated, and input the set of data to be discriminated into the discriminator through step 130, so that the discriminator outputs a synthetic noise image Discrimination result data that is true or false. 1 Form a set of data to be discriminated, and input the set of data to be discriminated into the discriminator through step 130, so that the discriminator outputs a synthetic noise image Discrimination result data that is true or false.

[0059] Step 200: Use the learnable conditional encoder and the generator in the diffusion generative adversarial network after iterative training as the longitudinal magnetic resonance image generation model.

[0060] It can be understood that the longitudinal magnetic resonance image generation model is a model used to generate and output the longitudinal magnetic resonance synthetic image of the subject according to the input magnetic resonance baseline image of the subject.

[0061] In one or more embodiments of the present application, the magnetic resonance baseline image can be abbreviated as the baseline image, and the model of the longitudinal magnetic resonance synthetic image can be abbreviated as the synthetic image; the longitudinal magnetic resonance prediction image is actually a real longitudinal magnetic resonance image used as the label of the longitudinal magnetic resonance synthetic image during model training, so it can be abbreviated as the real image.

[0062] Among them, the magnetic resonance baseline image is the longitudinal magnetic resonance image of the subject at a certain historical time, and the longitudinal magnetic resonance prediction image is the longitudinal magnetic resonance image collected by the subject at the target time after this historical time. The time interval between the historical time and the target time can be set according to actual application requirements.

[0063] As can be seen from the above description, the method for training a longitudinal magnetic resonance image generation model provided by the embodiments of the present application can improve the training efficiency and effectiveness of the longitudinal magnetic resonance image generation model, and can ensure the diversity of the generated synthetic images.

[0064] At the same time, for the problem that the diffusion generative adversarial network only considers synthesizing natural images and does not consider the effectiveness of the synthesized images and the identity of the subjects, in the process of synthesizing medical images, not only the authenticity of the synthesized images needs to be ensured, but also the synthesized images need to come from the same subject. Therefore, in the method for training a longitudinal magnetic resonance image generation model provided by the embodiments, by using a learnable conditional encoder and adding information such as the baseline image of the subject for encoding constraints, the generator can generate synthetic images of the same subject, so as to maintain the consistency of the subject, that is, it can ensure that the generated synthetic images and the magnetic resonance baseline images belong to the same subject, and further improve the application effectiveness and reliability of the longitudinal magnetic resonance synthetic images generated by the longitudinal magnetic resonance image generation model obtained through training.

[0065] To further ensure that the generated synthetic images and the magnetic resonance baseline images belong to the same subject, in a method for training a longitudinal magnetic resonance image generation model provided by the embodiments of the present application, refer to Figure 4 , step 120 specifically includes the following content: Step 121: In each reverse step during the reverse generation process of the diffusion generative adversarial network, perform an image synthesis training sub-step respectively; wherein, the image synthesis training sub-step includes: in the current reverse step, input the noise image conforming to the Gaussian distribution, the magnetic resonance baseline image of the subject, and the auxiliary data of the subject into the learnable conditional encoder, so that the learnable conditional encoder outputs corresponding encoded data; wherein, the auxiliary data includes: the age, gender, and diagnosis label of the subject; input the encoded data into the generator, so that the generator outputs the longitudinal magnetic resonance synthetic image corresponding to the current reverse step; perform posterior sampling on the longitudinal magnetic resonance synthetic image corresponding to the current reverse step to obtain the synthetic noise image corresponding to this reverse step; calculate the generator loss corresponding to the current reverse step according to the longitudinal magnetic resonance synthetic image corresponding to the current reverse step and the longitudinal magnetic resonance prediction image of the subject; update the model parameters of the generator based on the generator loss corresponding to the current reverse step.

[0066] Specifically, refer to Figure 5 , the image synthesis training sub-step in step 121 specifically includes the following content: Step 1211: In the current reverse step, input the noise image conforming to the Gaussian distribution, the magnetic resonance baseline image of the subject, and the auxiliary data of the subject into the learnable conditional encoder, so that the learnable conditional encoder outputs corresponding encoded data; wherein, the auxiliary data includes: the age, gender, and diagnostic label of the subject.

[0067] Step 1212: Input the encoded data into the generator, so that the generator outputs a longitudinal magnetic resonance synthetic image corresponding to the current reverse step.

[0068] Step 1213: Perform posterior sampling on the longitudinal magnetic resonance synthetic image corresponding to the current reverse step to obtain a synthetic noise image corresponding to this reverse step.

[0069] Step 1214: Calculate the generator loss corresponding to the current reverse step according to the longitudinal magnetic resonance synthetic image corresponding to the current reverse step and the longitudinal magnetic resonance prediction image of the subject.

[0070] Step 1215: Update the model parameters of the generator based on the generator loss corresponding to the current reverse step.

[0071] That is to say, in the generator network of the adversarial generation network, the embodiments of the present application propose to add a learnable conditional encoder, and add the baseline image, age, gender, and diagnostic label of the subject to encode and constrain the model to generate images of the same person, so as to maintain the consistency of the subject.

[0072] In order to further ensure that the generated synthetic image and the magnetic resonance baseline image belong to the same subject, and to ensure the continuity of adjacent slice images in the synthetic image, in a method for training a longitudinal magnetic resonance image generation model provided by the embodiments of the present application, the generator loss includes: mean absolute error loss and the training loss of the generator ; Correspondingly, referring to Figure 6 , the specific content of step 1214 is as follows: Step 1214-1: Obtain the mean absolute error between the longitudinal magnetic resonance synthetic image of the current reverse step and the longitudinal magnetic resonance prediction image of the current round as the mean absolute error loss corresponding to the current reverse step.

[0073] Specifically, the loss function corresponding to the mean absolute error loss is shown in the following formula (1): (1) Wherein, represents the longitudinal magnetic resonance synthetic image corresponding to the current reverse step, The longitudinal magnetic resonance prediction image of the current round is represented.

[0074] In each step of the reverse generation process, the image synthesized based on the learnable conditional encoder is constrained to be as similar as possible to the original input. This can constrain the entire training process to converge quickly while ensuring that the synthesized image is similar to the original image.

[0075] That is to say, since the initial noise of the diffusion model is random, if the noise evolution paths at different positions are unstable or uncoordinated, it will also lead to discontinuity between slice images in the synthesized 3D brain image. Therefore, in view of the problem that the prior art easily causes discontinuity between adjacent slice images in the synthesized 3D brain image, the embodiment of the present application adopts step 1214-1 to first directly generate a synthetic image using a generator, and then adds a noise generator in each diffusion step. The loss function ensures the reliability of each step, and uses the discriminator to ensure that the generator's results are more realistic.

[0076] And, step 1214-2: according to the longitudinal magnetic resonance prediction image of the current round and the noise image of the current reverse step, obtain the training loss of the generator of the current reverse step.

[0077] The loss function corresponding to the training loss of the generator is shown in the following formula (2): (2) in, represents the noise image corresponding to the t-1th step; represents the noise image corresponding to the tth step; represents the parameters of the generator network; Represents the real noisy image Medium sampling; Represents the image obtained by posterior sampling from the image predicted by the generator sampling; represents the discriminator.

[0078] In order to further improve the rationality and effectiveness of the synthesized image, the difference between the synthesized image and the baseline image needs to increase as the time interval increases. In a longitudinal magnetic resonance image generation model training method provided in an embodiment of the present application, the generator loss also includes: a correlation relationship loss between the time interval and the image difference; For the corresponding Figure 6 , the step 1214 further specifically includes the following contents: Step 1214-3: Obtain the loss of the correlation relationship between the time interval and the image difference corresponding to the current reverse step according to the longitudinal magnetic resonance synthesis image of the current reverse step, the longitudinal magnetic resonance prediction image of the current round, the age of the subject corresponding to the acquisition of the magnetic resonance baseline image, the age of the subject corresponding to the longitudinal magnetic resonance prediction image of the current round, the preset maximum age and minimum age.

[0079] Among them, the loss of the correlation relationship between the time interval and the image difference The corresponding loss function is shown in the following formula (3): (3) Among them, is the age of the subject when the baseline image is acquired, and a is the age of the longitudinal magnetic resonance prediction image. are the maximum age and minimum age preset in all longitudinal magnetic resonance prediction images and baseline images respectively.

[0080] To further improve the application effectiveness and reliability of the longitudinal magnetic resonance image generation model, in a longitudinal magnetic resonance image generation model training method provided in an embodiment of the present application, see Figure 6 , the specific content of step 1215 is as follows: Step 1215-1: Determine the generator loss corresponding to the current reverse step according to the training loss of the generator corresponding to the current reverse step, the product of the absolute mean error loss and a preset first hyperparameter, and the product of the loss of the correlation relationship between the time interval and the image difference and a preset second hyperparameter, and update the model parameters of the generator based on the generator loss corresponding to the current reverse step.

[0081] Among them, the generator loss The corresponding loss function is shown in the following formula (4): (4) is the first hyperparameter preset during the training process; is the second hyperparameter preset during the training process.

[0082] To further improve the effectiveness and reliability of the longitudinal magnetic resonance image generation model training, in a longitudinal magnetic resonance image generation model training method provided in an embodiment of the present application, see Figure 7 , before step 100 of the longitudinal magnetic resonance image generation model training method, the following specific content is further included: Step 010: Obtain the original magnetic resonance images collected for each subject at a historical time point and the original longitudinal magnetic resonance prediction images collected for each subject at a target time point after the historical time point.

[0083] Step 020: Perform image preprocessing, data cleaning, and normalization on the original magnetic resonance images and the original longitudinal magnetic resonance prediction images of each subject.

[0084] Among them, image preprocessing can be divided into three parts: preprocessing of structural magnetic resonance imaging, preprocessing of functional magnetic resonance imaging, and preprocessing of diffusion tensor imaging. The preprocessing steps of structural magnetic resonance images include: head: motion correction to correct image artifacts caused by subject movement; tissue extraction to remove parts unrelated to the brain, such as the skull, from the structural magnetic resonance images; spatial normalization to standardize the images to the MNI standard space for comparison between different subjects.

[0085] Data cleaning can be: truncating the data to find the 99.9% threshold of the used data, and assigning numbers greater than the threshold to the threshold. Normalize the data to [-1, 1].

[0086] Step 030: Respectively use the original magnetic resonance images of each subject corresponding to the image preprocessing, data cleaning, and normalization as the magnetic resonance reference images of each subject, and respectively use the original longitudinal magnetic resonance prediction images of each subject corresponding to the image preprocessing, data cleaning, and normalization as the longitudinal magnetic resonance prediction images of each subject.

[0087] To further improve the effectiveness and reliability of the training of the longitudinal magnetic resonance image generation model, in a method for training a longitudinal magnetic resonance image generation model provided in an embodiment of the present application, see Figure 4 , the specific content of step 130 is as follows: Step 131: In each reverse step during the reverse generation process of the diffusion generative adversarial network, respectively execute the discriminant sub-step; wherein, the discriminant sub-step includes: in the current reverse step, use the synthetic noise image corresponding to the current reverse step and the noise image of the forward step corresponding to the current reverse step as a group of data to be discriminated, and input the data to be discriminated into the discriminator respectively, so that the discriminator outputs the discriminant result data corresponding to the current reverse step; update the model parameters of the generator according to the discriminant result data corresponding to the current reverse step; and calculate the discriminator loss corresponding to the current reverse step to update the model parameters of the discriminator based on the discriminator loss.

[0088] Among them, the loss function corresponding to the discriminator loss is shown in the following formula (5): (5) Among them, represents the forward diffusion process, represents the reverse generation process; represents the real noisy image sampling in; represents from the forward diffusion process sampling; represents the image obtained by posterior sampling of the image predicted by the generator sampling; that is to say, refers to calculating the loss with the input being the longitudinal magnetic resonance predicted image, refers to calculating the loss for the longitudinal magnetic resonance synthesized image.

[0089] To further illustrate the embodiments of the above longitudinal magnetic resonance image generation model training method, the present application also provides a specific application example of the longitudinal magnetic resonance image generation model training method, which can also be a longitudinal magnetic resonance image generation model training method based on a diffusion generative adversarial network. This application example specifically includes the following content: S1: Obtain the original magnetic resonance images collected by each subject at historical time points and the original longitudinal magnetic resonance predicted images collected by each subject at target time points after the historical time points; The file formats of the original magnetic resonance images and the original longitudinal magnetic resonance predicted images are both nii. Among them, the nii format (an extension of the NIFTI format) serves multi-dimensional neuroimaging, can truly reflect metadata, and contains orientation information.

[0090] S2: Perform image preprocessing on the original magnetic resonance images and the original longitudinal magnetic resonance predicted images of each subject; Image preprocessing is divided into three parts: preprocessing of structural magnetic resonance imaging, preprocessing of functional magnetic resonance imaging, and preprocessing of diffusion tensor imaging. The preprocessing steps of structural magnetic resonance images include: Head: Motion correction to correct image artifacts caused by subject movement; Tissue extraction to remove parts of the structural magnetic resonance image that are irrelevant to the brain, such as the skull, etc.; Spatial normalization to normalize the image to the MNI standard space for comparison between different subjects.

[0091] S3: Perform data cleaning and normalization processing on the original magnetic resonance images and the original longitudinal magnetic resonance predicted images of each subject; Data cleaning can truncate the data to find the 99.9% threshold of the data used, and assign numbers greater than the threshold to the threshold. Normalize the data to [-1, 1].

[0092] S4: Construct a diffusion generative adversarial network; Specifically, the generator in the diffusion generative adversarial network can adopt the architecture of U-NET. U-Net is an end-to-end image segmentation model based on convolutional neural network (CNN), which can include multiple residual (ResNet) modules and attention mechanism (attention) modules. In addition, a z-conditioning is added for style transfer. This z represents noise and can be transformed through a fully connected network (called the mapping network). Then the obtained embedding vector (which can be represented by w) will be sent to each adaptive group normalization layer. Each adaptive group normalization layer contains a fully connected layer, which takes w as input and outputs the offset and scaling parameters of the normalized groups for each channel. The discriminator can be constructed using a convolutional network with residual (ResNet) blocks.

[0093] Among them, the architecture of the generator includes the following 25 layers connected in sequence: 1) "3*3*3conv3D(stride1), 64", which means: a three-dimensional convolutional layer with 64 dimensions, a size of 3*3*3, and a stride of 1; 2) "ResNetBlock, 64", which means: a residual layer with 64 dimensions; 3) "ResNetBlock, 64", which means: a residual layer with 64 dimensions; 4) "Convdown(stride2), 64", which means: a convolutional downsampling layer with 64 dimensions and a stride of 2; 5) "ResNetBlock, 128", which means: a residual layer with 128 dimensions; 6) "ResNetBlock, 128", which means: a residual layer with 128 dimensions; 7) "Convdown(stride2), 128", which means: a convolutional downsampling layer with 128 dimensions and a stride of 2; 8) "ResNetBlock, 256", which means: a residual layer with 256 dimensions; 9) "ResNetBlock, 256", which means: a residual layer with 256 dimensions; 10) "Convdown(stride2), 256", which means: a convolutional downsampling layer with 256 dimensions and a stride of 2; 11) "ResNetBlock, 512", which means: a residual layer with 512 dimensions; 12) "ResNetBlock, 512", which means: a residual layer with 512 dimensions; 13) "Convdown(stride2), 512", which means: a convolutional downsampling layer with 512 dimensions and a stride of 2; 14) "ResNetBlock, 512", which means: a residual layer with 512 dimensions; 15) "ResNetBlock, 512", which means: a residual layer with 512 dimensions; 16) "Convtranspose3D, upsampling(stride2), 256", which means: a neural network layer that performs transposed convolution on 3D data and an upsampling layer with 256 dimensions and a stride of 2; 17) "ResNetBlock, 256", which means: a residual layer with 256 dimensions; 18) "ResNetBlock, 256", which means: a residual layer with 256 dimensions; 19) "Convtranspose3D, upsampling(stride2), 128" which means: a neural network layer that performs transposed convolution on 3D data and an upsampling layer with 128 dimensions and a stride of 2; 20) "ResNetBlock, 128", which means: a residual layer with 128 dimensions; 21) "ResNetBlock, 128", which means: a residual layer with 128 dimensions; 22) "Convtranspose3D, upsampling(stride2), 64", which means: a neural network layer that performs transposed convolution on 3D data and an upsampling layer with 64 dimensions and a stride of 2; 23) "ResNetBlock, 64", which means: a residual layer with 64 dimensions; 24) "ResNetBlock, 64", which means: a residual layer with 64 dimensions; 25) "Convtranspose3D, upsampling(stride2), 1", which means: a neural network layer that performs transposed convolution on 3D data and an upsampling layer with 1 dimension and a stride of 2.

[0094] Among them, the architecture of the discriminator includes the following 8 layers connected in sequence: 1) "1*1*1conv3D, 64", which means: a three-dimensional convolutional layer with 64 dimensions and a size of 1*1*1; 2) "ResNetBlockdown, 64", which means: a residual downsampling layer with 64 dimensions; 3) "ResNetBlock down, 128", which means: a 128-dimensional residual downsampling layer; 4) "ResNetBlock down, 256", which means: a 256-dimensional residual downsampling layer; 5) "ResNetBlock down, 512", which means: a 512-dimensional residual downsampling layer; 6) "minibatch std layer", which means the mini-batch standard deviation layer; 7) "Global Sum Pooling", which means the global sum pooling layer; 8) "FC layer", which means: the fully connected layer.

[0095] S5: Training the diffusion generative adversarial network The training process includes the forward diffusion process and the reverse generation process. See Figure 8 .

[0096] Specifically, first, based on the forward noise addition process of the diffusion model, the real longitudinal magnetic resonance prediction image is gradually noisy until its final noise image conforms to the Gaussian distribution. The process of gradually adding noise lasts for a total of T steps. In the reverse generation process, the noise image generates a longitudinal magnetic resonance synthetic image through a generator, and then for the generated longitudinal magnetic resonance synthetic image and the real longitudinal magnetic resonance prediction image calculate the absolute mean error loss , where the absolute mean error loss is shown by the above formula (1).

[0097] Based on the generated longitudinal magnetic resonance synthetic image posterior sampling is used to obtain the synthetic noise image , and the noise image in each step and the synthetic noise image are fed into the discriminator for confrontation to ensure that the distribution of the synthetic noise image obtained by posterior sampling in each step is consistent with that of the real noise image. After T steps in the reverse generation process, the final synthetic longitudinal magnetic resonance synthetic image is obtained. In the generator, a learnable conditional encoder is proposed to encode the baseline image, age, gender, and diagnosis label of the subject to constrain the model to generate images of the same person, so as to maintain the consistency of the subject. Through the above and the learnable conditional encoder, it is jointly ensured that the synthetic image is an image of the same subject.

[0098] Based on the synthetic longitudinal magnetic resonance synthetic image synthesized in each step and the real longitudinal magnetic resonance prediction image , set the difference between the constrained synthetic image and the baseline image to increase with the growth of the time interval. By increasing the time interval, ensure that the change of the brain image with a large age interval is greater than that of the baseline image, and vice versa, the change of the brain image with a small interval is smaller. The loss of the association relationship between the time interval and the image difference , as shown in formula (3).

[0099] The loss function for training the conditional generator (i.e., the aforementioned generator) is as shown in formula (2). The overall loss function of the generator part is as shown in formula (4). The loss function of the discriminator is as shown in formula (5).

[0100] That is to say, in step S5, first, based on the forward noise addition process of the diffusion model, gradually add noise to the real longitudinal magnetic resonance prediction image until its final noise image conforms to the Gaussian distribution. The process of gradually adding noise lasts for a total of T steps, and is obtained. In the reverse generation process, the noise image generates an initial baseline image through a conditional generator, and then for the generated longitudinal magnetic resonance synthetic image and the real longitudinal magnetic resonance prediction image calculate the absolute mean error, and based on the generated longitudinal magnetic resonance synthetic image use posterior sampling to obtain , and based on the original and send them into the discriminator for confrontation to ensure that the distribution of the synthetic noise image and the real noise image obtained by each step of posterior sampling is consistent. After T steps of the reverse generation process, the final synthesized longitudinal magnetic resonance synthetic image is obtained.

[0101] In addition, in the application example of this application, a Unet structure can also be used to construct an original brain image generation model. The Unet structure consists of an encoder and a decoder; the application example of this application uses structural magnetic resonance data. If gray matter volume images or T2 images are used, they can also be regarded as alternative solutions.

[0102] On this basis, the application example of this application also provides the experimental result data between the longitudinal magnetic resonance image generation model (IP-DDGAN) and the conditional generative adversarial network (CGAN), the self-attention generative adversarial network (SAGAN), the cycle generative adversarial network (CycleGAN), the diffusion model (DDPM), and the latent diffusion model (LDM). See Figure 9 for the comparison results of the synthesized longitudinal data (GT (3 years)) after 3 years; see Figure 10Comparison results of the longitudinal data (GT(6years)) six years after synthesis; among them, "CN-CN" refers to normal person to normal person; "MCI-MCI" refers to mild cognitive impairment to mild cognitive impairment; "AD-AD" refers to Alzheimer's disease to Alzheimer's disease; "CN-MCI" refers to normal person to mild cognitive impairment patient; "MCI–AD" refers to mild cognitive impairment patient to Alzheimer's disease patient, in Figure 9 and Figure 10 In, red represents an error greater than 0, and blue represents an error less than 0. However, due to the use of absolute values, there is no blue color in the image. The specific comparison values are shown in Tables 1 to 4: Table 1 - Comparison results of the longitudinal data synthesized three and six years later by each model for "CN-CN" Table 2 - Comparison results of the longitudinal data synthesized three and six years later by each model for "MCI to MCI" Table 3 - Comparison results of the longitudinal data synthesized three and six years later by each model for "AD to AD" Table 4 - Comparison results of the longitudinal data of each model for "CN to MCI" and "MCI to AD" Among them, PSNR refers to peak signal-to-noise ratio, and the larger the value, the better; SSIM refers to structural similarity, and the closer it is to 1, the better; MMD refers to maximum mean discrepancy, and the closer it is to 0, the better.

[0103] That is to say, the longitudinal magnetic resonance image generation model training method provided by the application example of the present application synthesizes longitudinal structural magnetic resonance images of the same subject based on the diffusion generative adversarial network model, and ensures that the synthesized longitudinal images come from the same subject. Currently, the data structure used is MRI images, but not limited to T1 images, and gray matter density images, etc. can be input. Among them, the T1 image refers to the T1-weighted image (T1WI) in magnetic resonance imaging (MRI). The absolute mean error loss function is used to constrain the error at each step in the diffusion process, thereby accelerating the rapid convergence of the model. In the diffusion generative adversarial network, it is assumed to follow a multimodal distribution, so the number of diffusion steps is small, which speeds up sampling. It should be effective at different diffusion steps.

[0104] Based on this, the longitudinal magnetic resonance image generation model training method provided by the application example of the present application has the following beneficial effects: 1) Regarding problem 1: Generative adversarial networks are prone to mode collapse and the diversity of generated images is low; The path adopted by the application example of this application to solve the above problem 1 is: GAN directly synthesizes images at one time, and the generator only learns to generate a few image patterns, ignoring the diversity of the entire data distribution. However, the diffusion model gradually "denoises and restores" the data distribution. Each step is modeling the details of the real distribution, making it more difficult to fall into the local optimum of only generating a few patterns.

[0105] 2) Regarding problem 2: the diffusion model has slow sampling speed; The path adopted by the application example of this application to solve the above-mentioned problem 2 is: in the forward propagation process of the diffusion model, the noise added in each step obeys the Gaussian normal distribution, which results in each diffusion step being very small and the number of steps being large, so it takes a long time to infer to generate data. However, in real applications, more data obey a multimodal distribution. The diffusion process of each step of the model proposed in this study does not have to obey the Gaussian normal distribution. A conditional generator is used to fit the diffusion process of each step. Since each step is a non-Gaussian normal distribution, the number of diffusion steps can be reduced, thereby achieving fast data generation.

[0106] 3) Regarding question 3: The current diffusion generative adversarial network only considers the synthesis of natural images, and does not consider the validity of the synthesized images and the identity of the subjects. In the process of synthesizing medical images, it is necessary not only to ensure the authenticity of the synthesized images, but also to ensure that the synthesized images come from the same subject. In addition, the current diffusion generative adversarial network cannot ensure that the volume of each brain region in the synthesized medical image is very close. The images may look similar, but the residual is large compared to the actual longitudinal direction, so the authenticity of the image cannot be guaranteed. And discontinuity between adjacent slices is prone to occur in the synthesized 3D brain images; The path adopted by the application example of this application to solve the above problem 3 is: the initial noise of the diffusion model is random. If the noise evolution path at different positions is unstable or uncoordinated, it will also lead to discontinuity between slices; in this application, the generator is first used to directly generate the synthetic image, and the absolute mean error loss is added in each diffusion step. , ensuring the reliability of each step of the process, and using the discriminator to ensure that the results of the generator are more realistic.

[0107] That is to say, the application example of this application uses a diffusion generative adversarial network to synthesize the longitudinal images of each subject, and ensures that the synthesized images are from the same subject; based on the constraint of absolute average error in each step of the reverse generation process, the similarity of the microstructure of the medical image is achieved, such as the consistency of gray matter volume, white matter volume, etc., and the continuity of adjacent slice images can be guaranteed; using L IDThe loss function ensures an increased time interval, guaranteeing that the synthetic brain image changes more significantly with a larger age interval compared to the baseline image, and conversely, the synthetic brain image changes less with a smaller interval.

[0108] Based on the above embodiments of the training method for the longitudinal magnetic resonance image generation model, the present application also provides a method for generating longitudinal magnetic resonance images. Refer to Figure 11 The method for generating longitudinal magnetic resonance images specifically includes the following content: Step 300: Input a noise image conforming to a Gaussian distribution, the magnetic resonance baseline image of the current target subject to be measured, and the auxiliary data of the target subject into the longitudinal magnetic resonance image generation model, so that the learning conditional encoder and generator in the longitudinal magnetic resonance image generation model gradually obtain the longitudinal magnetic resonance synthetic images corresponding to each step respectively. After obtaining the longitudinal magnetic resonance synthetic image corresponding to each reverse step, posterior sampling is performed on the longitudinal magnetic resonance synthetic image corresponding to the reverse step to obtain the synthetic noise image corresponding to the reverse step; wherein, the longitudinal magnetic resonance image generation model is pre-trained based on the above-mentioned training method for the longitudinal magnetic resonance image generation model; the auxiliary data includes: the age, gender, and diagnosis label of the target subject. Step 400: Use the longitudinal magnetic resonance synthetic image output by the generator in the last reverse step as the longitudinal magnetic resonance image prediction result data of the target subject at the target time point.

[0109] It can be understood that the processing flow of the training method for the longitudinal magnetic resonance image generation model mentioned in the embodiments of the method for generating longitudinal magnetic resonance images provided by the present application can specifically adopt the processing flow of the training method for the longitudinal magnetic resonance image generation model in the above embodiments, and its functions will not be elaborated here. Reference can be made to the detailed description of the embodiments of the training method for the longitudinal magnetic resonance image generation model.

[0110] At the software level, the present application also provides a longitudinal magnetic resonance image generation model training device for executing all or part of the content in the above-mentioned training method for the longitudinal magnetic resonance image generation model. Refer to Figure 12 The longitudinal magnetic resonance image generation model training device specifically includes the following content: The iterative training module 10 is used to perform at least one round of iterative training on the diffusion generative adversarial network, and respectively execute the model training steps in each round. Wherein, the model training steps include: gradually adding noise to the longitudinal magnetic resonance prediction image of the subject in each forward step of the forward diffusion process of the diffusion generative adversarial network to obtain the noise image corresponding to each forward step, and stopping adding noise after obtaining a noise image conforming to the Gaussian distribution; according to the noise image conforming to the Gaussian distribution and the magnetic resonance baseline image of the subject, gradually obtain the longitudinal magnetic resonance synthesis image corresponding to each step based on the learnable conditional encoder and the generator in the reverse diffusion process of the diffusion generative adversarial network, and after obtaining the longitudinal magnetic resonance synthesis image corresponding to each reverse step each time, perform posterior sampling on the longitudinal magnetic resonance synthesis image corresponding to this reverse step to obtain the synthesized noise image corresponding to this reverse step; gradually input the synthesized noise image of each reverse step and the noise image of the forward step corresponding to this reverse step into the discriminator, so that the discriminator respectively outputs the discriminant result data corresponding to each reverse step to optimize the discriminator and the generator.

[0111] The model determination module 20 is used to use the learnable conditional encoder and the generator in the diffusion generative adversarial network after iterative training as the longitudinal magnetic resonance image generation model.

[0112] The embodiment of the longitudinal magnetic resonance image generation model training device provided in this application can specifically be used to execute the processing flow of the embodiment of the longitudinal magnetic resonance image generation model training method in the above embodiment, and its functions will not be elaborated here, and reference can be made to the detailed description of the above longitudinal magnetic resonance image generation model training method embodiment.

[0113] The part of the longitudinal magnetic resonance image generation model training by the longitudinal magnetic resonance image generation model training device can be completed on the server or the client device. Specifically, it can be selected according to the processing capacity of the client device and the limitations of the user usage scenario, etc. This application does not make a limitation in this regard. If all operations are completed on the client device, the client device may further include a processor for specific processing of the longitudinal magnetic resonance image generation model training.

[0114] The above-mentioned client device may have a communication module (i.e., a communication unit), and can be communicatively connected to a remote server to realize data transmission with the server. The server may include a server on the task scheduling center side, and may also include a server on the intermediate platform in other implementation scenarios, such as a server on a third-party server platform communicatively linked to the task scheduling center server. The server may include a single computer device, or may include a server cluster composed of multiple servers, or a server structure of a distributed device.

[0115] Any suitable network protocol can be used for communication between the server and the client device, including network protocols that have not been developed as of the filing date of this application. The network protocol can include, for example, TCP / IP protocol, UDP / IP protocol, HTTP protocol, HTTPS protocol, etc. Of course, the network protocol can also include, for example, the RPC protocol (Remote Procedure Call Protocol) and the REST protocol (Representational State Transfer) used on top of the above protocols.

[0116] As can be seen from the above description, the longitudinal magnetic resonance image generation model training device provided by the embodiments of this application can improve the training efficiency and effectiveness of the longitudinal magnetic resonance image generation model, ensure the diversity of the generated synthetic images, and ensure that the generated synthetic images and the magnetic resonance baseline images belong to the same subject.

[0117] At the software level, this application also provides a longitudinal magnetic resonance image generation device for executing all or part of the longitudinal magnetic resonance image generation method. Refer to Figure 13 , and the longitudinal magnetic resonance image generation device specifically includes the following: The model prediction module 30 is configured to input a noise image conforming to a Gaussian distribution, the magnetic resonance baseline image of the current target subject to be measured, and the auxiliary data of the target subject into the longitudinal magnetic resonance image generation model, so that the learning conditional encoder and the generator in the longitudinal magnetic resonance image generation model gradually obtain the longitudinal magnetic resonance synthetic images corresponding to each step respectively, and after each longitudinal magnetic resonance synthetic image corresponding to a reverse step is obtained, posterior sampling is performed on the longitudinal magnetic resonance synthetic image corresponding to the reverse step to obtain the synthetic noise image corresponding to the reverse step; wherein, the longitudinal magnetic resonance image generation model is pre-trained based on the longitudinal magnetic resonance image generation model training method; the auxiliary data includes: the age, gender, and diagnosis label of the target subject. The result output module 40 is configured to use the longitudinal magnetic resonance synthetic image output by the generator at the last reverse step as the longitudinal magnetic resonance image prediction result data of the target subject at the target time point.

[0118] The embodiments of the longitudinal magnetic resonance image generation device provided by this application can specifically be used to execute the processing flow of the embodiments of the longitudinal magnetic resonance image generation method in the above embodiments, and its functions will not be elaborated here. Reference can be made to the detailed description of the embodiments of the longitudinal magnetic resonance image generation method above.

[0119] The part of the longitudinal magnetic resonance image generation device for generating longitudinal magnetic resonance images can be completed in a server or a client device.

[0120] An embodiment of this application also provides an electronic device, which may include a processor, a memory, a receiver, and a transmitter. The processor is used to execute the longitudinal magnetic resonance image generation model training method and / or the longitudinal magnetic resonance image generation method mentioned in the above embodiments. The processor and the memory can be connected through a bus or other means. Taking the bus connection as an example, the receiver can be connected to the processor and the memory in a wired or wireless manner.

[0121] The processor can be a Central Processing Unit (CPU). The processor can also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., or a combination of the above types of chips.

[0122] As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the longitudinal magnetic resonance image generation model training method and / or the longitudinal magnetic resonance image generation method in the embodiments of this application. The processor runs the non-transitory software programs, instructions, and modules stored in the memory, thereby executing various functional applications and data processing of the processor, that is, implementing the longitudinal magnetic resonance image generation model training method and / or the longitudinal magnetic resonance image generation method in the above method embodiments.

[0123] The memory can include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created by the processor, etc. In addition, the memory can include high-speed random access memory, and can also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory can optionally include a memory remotely set relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above networks include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0124] The one or more modules are stored in the memory and, when executed by the processor, implement the longitudinal magnetic resonance image generation model training method and / or the longitudinal magnetic resonance image generation method in the embodiments.

[0125] In some embodiments of the present application, a user device may include a processor, a memory, and a transceiver unit. The transceiver unit may include a receiver and a transmitter. The processor, the memory, the receiver, and the transmitter may be connected through a bus system. The memory is used to store computer instructions, and the processor is used to execute the computer instructions stored in the memory to control the transceiver unit to transmit and receive signals.

[0126] As an implementation manner, the functions of the receiver and the transmitter in the present application may be implemented by considering a transceiver circuit or a dedicated chip for transceiver. The processor may be implemented by considering a dedicated processing chip, a processing circuit, or a general-purpose chip.

[0127] As another implementation manner, a general computer may be considered to implement the server provided in the embodiments of the present application. That is, the program codes for implementing the functions of the processor, the receiver, and the transmitter are stored in the memory, and the general processor implements the functions of the processor, the receiver, and the transmitter by executing the codes in the memory.

[0128] The embodiments of the present application further provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the foregoing longitudinal magnetic resonance image generation model training method and / or longitudinal magnetic resonance image generation method are implemented. The computer-readable storage medium may be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0129] The embodiments of the present application further provide a computer program product, including a computer program. When the computer program is executed by a processor, the steps of the foregoing longitudinal magnetic resonance image generation model training method and / or longitudinal magnetic resonance image generation method are implemented.

[0130] Those of ordinary skill in the art should understand that the various exemplary components, systems, and methods described in connection with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Specifically, whether to implement in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium or transmitted through a data signal carried in a carrier wave on a transmission medium or a communication link.

[0131] It should be clear that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order between steps after understanding the spirit of this application.

[0132] In this application, the features described and / or illustrated for one embodiment can be used in the same or a similar manner in one or more other embodiments, and / or combined with the features of other embodiments or replace the features of other embodiments.

[0133] The above are only the preferred embodiments of this application and are not used to limit this application. For those skilled in the art, various changes and modifications can be made to the embodiments of this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included within the protection scope of this application.

Claims

1. A longitudinal magnetic resonance image generation model training method, characterized in that: include: The diffusion generative adversarial network is iteratively trained for at least one round, and the model training steps are respectively performed in each round, wherein the model training steps include: gradually adding noise to the longitudinal magnetic resonance prediction image of the subject in each forward step of the forward diffusion process of the diffusion generative adversarial network to obtain noise images corresponding to each forward step, and stopping the noise addition after obtaining a noise image that conforms to the Gaussian distribution; according to the noise image that conforms to the Gaussian distribution and the magnetic resonance baseline image of the subject, in the reverse generation process of the diffusion generative adversarial network, gradually obtaining the longitudinal magnetic resonance synthetic images corresponding to each reverse step based on the learnable conditional encoder and the generator, and after each acquisition of the longitudinal magnetic resonance synthetic image corresponding to a reverse step, the longitudinal magnetic resonance synthetic image corresponding to the reverse step is posteriorly sampled to obtain the synthetic noise image corresponding to the reverse step; gradually inputting the synthetic noise image of each reverse step and the noise image of the forward step corresponding to the reverse step into the discriminator, so that the discriminator outputs the discrimination result data corresponding to each reverse step to optimize the discriminator and the generator; The learnable conditional encoder and the generator in the iteratively trained diffusion generative adversarial network are used as a longitudinal magnetic resonance image generation model.

2. The longitudinal magnetic resonance image generation model training method according to claim 1, characterized in that: The method comprises: stepwise acquiring longitudinal magnetic resonance synthetic images corresponding to respective reverse steps based on a learnable conditional encoder and a generator according to the noise image conforming to the Gaussian distribution and the magnetic resonance baseline image of the subject in the reverse generation process of the diffusion generative adversarial network, and performing a posteriori sampling on the longitudinal magnetic resonance synthetic image corresponding to the reverse step after each acquisition of the longitudinal magnetic resonance synthetic image corresponding to a reverse step to obtain a synthetic noise image corresponding to the reverse step, including: In each reverse step of the reverse generation process of the diffusion generative adversarial network, an image synthesis training sub-step is respectively performed; Wherein, the image synthesis training sub-step includes: In the current backward step, the noise image conforming to the Gaussian distribution, the magnetic resonance baseline image of the subject and the auxiliary data of the subject are input into the learnable conditional encoder, so that the learnable conditional encoder outputs corresponding encoded data; wherein the auxiliary data includes: the age, gender and diagnosis label of the subject; Inputting the coded data into the generator so that the generator outputs a longitudinal magnetic resonance synthesis image corresponding to the current reverse step; Performing a posteriori sampling on the longitudinal magnetic resonance synthetic image corresponding to the current reverse step to obtain a synthetic noise image corresponding to the reverse step; Calculating the generator loss corresponding to the current reverse step according to the longitudinal magnetic resonance synthetic image corresponding to the current reverse step and the longitudinal magnetic resonance predicted image of the subject; Update the model parameters of the generator based on the generator loss corresponding to the current backward step.

3. The longitudinal magnetic resonance image generation model training method according to claim 2, characterized in that: The generator loss includes: absolute mean error loss and the training loss of the generator; Correspondingly, the generator loss corresponding to the current reverse step is calculated according to the longitudinal magnetic resonance synthetic image corresponding to the current reverse step and the longitudinal magnetic resonance predicted image of the subject, including: Obtaining an absolute average error between the longitudinal magnetic resonance synthesis image of the current reverse step and the longitudinal magnetic resonance prediction image of the current round as the absolute average error loss corresponding to the current reverse step; And, according to the longitudinal magnetic resonance prediction image of the current round and the noise image of the current reverse step, the training loss of the generator of the current reverse step is obtained.

4. The longitudinal magnetic resonance image generation model training method according to claim 3, characterized in that: The generator loss also includes: a correlation loss between the time interval and the image difference; Correspondingly, the calculating the generator loss corresponding to the current reverse step according to the longitudinal magnetic resonance synthetic image corresponding to the current reverse step and the longitudinal magnetic resonance predicted image of the subject also includes: According to the longitudinal magnetic resonance synthesis image of the current reverse step, the longitudinal magnetic resonance prediction image of the current round, the age of the subject corresponding to the magnetic resonance baseline image when the magnetic resonance baseline image is acquired, the age of the subject corresponding to the longitudinal magnetic resonance prediction image of the current round, and the preset maximum age and minimum age, the correlation loss between the time interval corresponding to the current reverse step and the image difference is obtained.

5. The longitudinal magnetic resonance image generation model training method according to claim 4, characterized in that: Updating the model parameters of the generator based on the generator loss corresponding to the current backward step includes: According to the training loss of the generator corresponding to the current backward step, the product of the absolute mean error loss and a preset first hyperparameter, and the product of the correlation relationship loss between the time interval and the image difference and a preset second hyperparameter, the generator loss corresponding to the current backward step is determined, and the model parameters of the generator are updated based on the generator loss corresponding to the current backward step.

6. The longitudinal magnetic resonance image generation model training method according to any one of claims 1 to 5, characterized in that: Before performing at least one round of iterative training on the diffusion generative adversarial network, the method further includes: Acquire the original magnetic resonance images of each subject acquired at a historical time point and the original longitudinal magnetic resonance prediction images of each subject acquired at a target time point after the historical time point; Performing image preprocessing, data cleaning and normalization processing on the original magnetic resonance images and the original longitudinal magnetic resonance prediction images of each subject; The original magnetic resonance images corresponding to each subject after the image preprocessing, data cleaning and normalization processing are respectively used as the magnetic resonance reference images corresponding to each subject, and the original longitudinal magnetic resonance prediction images corresponding to each subject after the image preprocessing, data cleaning and normalization processing are respectively used as the longitudinal magnetic resonance prediction images corresponding to each subject.

7. A method for generating a longitudinal magnetic resonance image, characterized in that: include: A noise image conforming to a Gaussian distribution, a magnetic resonance baseline image of the current target subject, and auxiliary data of the target subject are input into a longitudinal magnetic resonance image generation model, so that the learning conditional encoder and generator in the longitudinal magnetic resonance image generation model gradually obtain the longitudinal magnetic resonance synthetic images corresponding to each step, and after each acquisition of the longitudinal magnetic resonance synthetic image corresponding to a reverse step, the longitudinal magnetic resonance synthetic image corresponding to the reverse step is subjected to posterior sampling to obtain the synthetic noise image corresponding to the reverse step; wherein the longitudinal magnetic resonance image generation model is pre-trained based on the longitudinal magnetic resonance image generation model training method according to any one of claims 1 to 6; the auxiliary data include: the age, gender and diagnosis label of the target subject; The longitudinal magnetic resonance synthetic image output by the generator in the last backward step is used as the longitudinal magnetic resonance image prediction result data of the target subject at the target time point.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the processor implements the longitudinal magnetic resonance image generation model training method as described in any one of claims 1 to 6, and / or implements the longitudinal magnetic resonance image generation method as described in claim 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the longitudinal magnetic resonance image generation model training method as described in any one of claims 1 to 6, and / or implements the longitudinal magnetic resonance image generation method as described in claim 7.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, it implements the longitudinal magnetic resonance image generation model training method as described in any one of claims 1 to 6, and / or implements the longitudinal magnetic resonance image generation method as described in claim 7.

Images