Four-dimensional dynamic imaging method and device of conditional latent diffusion model

Abstract

The present application relates to the field of medical imaging technology, and discloses a four-dimensional dynamic imaging method and device of a conditional latent diffusion model, wherein a conditional latent diffusion model is established, three-dimensional spatial convolution and one-dimensional time convolution are combined into a denoising four-dimensional convolutional neural network in the conditional latent diffusion model, and spatial attention mechanism and time attention mechanism are added accordingly to obtain a target conditional latent diffusion model capable of generating four-dimensional dynamic magnetic resonance images. Specifically, the four-dimensional dynamic magnetic resonance images are input into the target conditional latent diffusion model after forward diffusion, three-dimensional magnetic resonance images with a preset frame number are used as condition variables of the target conditional latent diffusion model, the target conditional latent diffusion model is trained, and a four-dimensional dynamic magnetic resonance generation model is obtained, thereby reducing the dependence on a large amount of dynamic image data and realizing efficient generation of complete and high-quality four-dimensional dynamic magnetic resonance images by using only a small amount of static three-dimensional magnetic resonance images.

Description

Technical Field

[0001] The present invention relates to the field of medical imaging technology, and in particular to a four-dimensional dynamic imaging method and device of a conditional latent diffusion model. Background Art

[0002] Four-dimensional magnetic resonance imaging (4D MRI) is a dynamic magnetic resonance imaging technology that incorporates a temporal dimension and is widely used in research and clinical diagnosis, particularly for the dynamic assessment of the heart, blood vessels, and motor systems. However, traditional 4D MRI technology typically requires the continuous acquisition of large amounts of dynamic image data over a long period of time, followed by reconstruction algorithms to generate a complete 4D dynamic MRI. This results in high time and computational costs. Furthermore, the dynamic scanning process inevitably suffers from motion artifacts, noise interference, and data sparsity, which severely impacts the accuracy and consistency of 4D dynamic MRI imaging, reducing its reliability in clinical diagnosis. Summary of the Invention

[0003] In view of this, the present invention provides a four-dimensional dynamic imaging method and device based on a conditional latent diffusion model to solve the problems of motion artifacts, noise interference and data sparsity in existing 4D MRI generation methods, which seriously affect the accuracy and consistency of 4D dynamic MRI imaging and reduce its reliability in clinical diagnosis.

[0004] In a first aspect, the present invention provides a four-dimensional dynamic imaging method of a conditional latent diffusion model, the method comprising: establishing a preset conditional latent diffusion model, and obtaining a four-dimensional dynamic magnetic resonance image training set of the current part, the four-dimensional dynamic magnetic resonance image training set comprising a four-dimensional dynamic magnetic resonance image and a three-dimensional magnetic resonance image of a preset number of frames in the four-dimensional dynamic magnetic resonance image; combining three-dimensional spatial convolution and one-dimensional temporal convolution into a denoising four-dimensional convolutional neural network in the conditional latent diffusion model, and adding a spatial attention mechanism and a temporal attention mechanism accordingly to obtain a target conditional latent diffusion model that can generate a four-dimensional dynamic magnetic resonance image; in the training stage, the four-dimensional dynamic The state magnetic resonance image is used as the input of the target conditional latent diffusion model after forward diffusion, and the three-dimensional magnetic resonance image of a preset number of frames is used as the conditional variable of the target conditional latent diffusion model. The target conditional latent diffusion model is trained to obtain a four-dimensional dynamic magnetic resonance generation model; in the prediction stage, a preset four-dimensional Gaussian noise and a three-dimensional magnetic resonance image of a preset number of frames of the current part are obtained; the preset four-dimensional Gaussian noise distribution is input into the denoising four-dimensional convolutional neural network in the four-dimensional dynamic magnetic resonance generation model, and the three-dimensional magnetic resonance image of the preset number of frames of the current part is used as the conditional variable for denoising, and reverse diffusion is performed to obtain a four-dimensional dynamic magnetic resonance image.

[0005] The four-dimensional dynamic imaging method of the conditional latent diffusion model provided in this embodiment establishes a conditional latent diffusion model, combines three-dimensional spatial convolution and one-dimensional temporal convolution into a denoising four-dimensional convolutional neural network in the conditional latent diffusion model, and adds a spatial attention mechanism and a temporal attention mechanism accordingly to obtain a target conditional latent diffusion model that can generate four-dimensional dynamic magnetic resonance images. The four-dimensional dynamic magnetic resonance images are then used as the input of the target conditional latent diffusion model after forward diffusion, and three-dimensional magnetic resonance images of a preset number of frames are used as conditional variables of the target conditional latent diffusion model. The target conditional latent diffusion model is trained to obtain a four-dimensional dynamic magnetic resonance generation model, which reduces the dependence on a large amount of dynamic image data and realizes the efficient generation of complete and high-quality four-dimensional dynamic magnetic resonance images using only a small number of static three-dimensional magnetic resonance images, thereby improving the accuracy and spatiotemporal resolution of imaging. In addition, the training process of the conditional latent diffusion model is more stable, ensuring the diversity and robustness of the generated images, and providing an effective imaging solution for a wide range of clinical needs.

[0006] In an optional embodiment, the method further includes: traversing a training set of four-dimensional dynamic magnetic resonance images of multiple parts, executing the step of using the four-dimensional dynamic magnetic resonance images after forward diffusion as the input of the target conditional latent diffusion model, and using three-dimensional magnetic resonance images of a preset number of frames as conditional variables of the target conditional latent diffusion model, and training the target conditional latent diffusion model to obtain a four-dimensional dynamic magnetic resonance generation model that can generate four-dimensional dynamic magnetic resonance images of multiple parts.

[0007] The present invention trains the target conditional latent diffusion model by acquiring a training set of four-dimensional dynamic magnetic resonance images of different parts, and obtains a four-dimensional dynamic magnetic resonance generation model that can generate four-dimensional dynamic magnetic resonance images of multiple parts, laying the foundation for the subsequent generation of high-quality 4D dynamic MRI using a small number of static three-dimensional magnetic resonance images of different parts.

[0008] In an optional embodiment, the method further includes: obtaining a three-dimensional magnetic resonance image with a preset number of frames including multiple different parts; inputting a preset four-dimensional Gaussian noise distribution into a denoising four-dimensional convolutional neural network in a four-dimensional dynamic magnetic resonance generation model, and performing denoising processing using the three-dimensional magnetic resonance image with a preset number of frames including multiple different parts as a conditional variable, and performing reverse diffusion to obtain a four-dimensional dynamic magnetic resonance image.

[0009] The present invention utilizes a small amount of static 3D images of different parts to generate high-quality 4D dynamic MRI, thereby reducing dependence on dynamic data and improving the generation quality and efficiency of 4D dynamic MRI.

[0010] In an optional embodiment, the four-dimensional dynamic magnetic resonance image is used as the input of the target conditional latent diffusion model after forward diffusion, and the three-dimensional magnetic resonance image of a preset number of frames is used as the conditional variable of the target conditional latent diffusion model. The target conditional latent diffusion model is trained to obtain a four-dimensional dynamic magnetic resonance generation model, including: inputting the four-dimensional dynamic magnetic resonance image into the target conditional latent diffusion model, encoding it with a pre-trained encoder to obtain a latent variable, and then performing forward diffusion to obtain a Gaussian noise distribution; inputting the three-dimensional magnetic resonance image of the preset number of frames as the conditional variable and the Gaussian noise distribution into a denoising four-dimensional convolutional neural network for denoising, and performing reverse diffusion. Obtain denoised four-dimensional dynamic magnetic resonance image features; decode the denoised four-dimensional dynamic magnetic resonance image features through a pre-trained decoder to obtain a four-dimensional dynamic magnetic resonance image to be tested; based on the loss function of the target conditional latent diffusion model training, calculate the loss function values ​​of the four-dimensional dynamic magnetic resonance image and the four-dimensional dynamic magnetic resonance image to be tested and the sum of the loss function values ​​of the noise distribution removed by the convolutional neural network and the Gaussian noise distribution to obtain the loss function value of the target conditional latent diffusion model training; optimize the target conditional latent diffusion model based on the loss function value of the target conditional latent diffusion model training until the loss function value is no more than the preset loss value, and obtain a four-dimensional dynamic magnetic resonance generation model.

[0011] The present invention combines the loss of the tested four-dimensional dynamic magnetic resonance image and the real four-dimensional dynamic magnetic resonance image with the denoising loss of the convolutional neural network to constrain the noise distribution of the conditional latent diffusion model, improve the generalization ability of the conditional latent diffusion model, and ensure the accuracy and consistency of the generated image.

[0012] In an optional embodiment, the loss function based on the target conditional potential diffusion model training calculates the loss function values ​​of the four-dimensional dynamic magnetic resonance image and the four-dimensional dynamic magnetic resonance image to be tested, and the sum of the loss function values ​​of the noise distribution removed by the convolutional neural network and the Gaussian noise distribution, to obtain the loss function value of the target conditional potential diffusion model training, including: adding a spatial smoothing regularization term and a temporal continuity regularization term to the loss function of the target conditional potential diffusion model training to obtain a constrained loss function; based on the constrained loss function, calculating the loss function values ​​of the four-dimensional dynamic magnetic resonance image and the four-dimensional dynamic magnetic resonance image to be tested, and the sum of the loss function values ​​of the noise distribution removed by the convolutional neural network and the Gaussian noise distribution, to obtain the loss function value of the target conditional potential diffusion model training.

[0013] The present invention adds spatial smoothing regularization terms and temporal continuity regularization terms to the loss function of conditional latent diffusion model training to obtain a constrained loss function. The model is then adjusted based on the constrained loss function, which can constrain the complexity of the conditional latent diffusion model and thus improve the generalization ability of the model.

[0014] In an optional embodiment, the constrained loss function is as follows:

[0015]

[0016] in, represents the loss function after constraint; Represents the loss function for target-conditional latent diffusion model training:

[0017]

[0018] in, represents the expectation of noise and latent variables, ∈ θ represents the denoising convolutional neural network, ∈∈(0,I) represents the noise data obeying the Gaussian distribution, z t represents the latent variable, t represents the uniformly sampled time step in [0, u], Represents the characteristic information of the conditional input, which is usually used to control the conditional variables in the process of generating four-dimensional dynamic magnetic resonance images. represents the expectation of the real image and the generated image, x represents the four-dimensional dynamic magnetic resonance image, represents the four-dimensional dynamic magnetic resonance image to be measured; λ space represents the hyperparameter that controls the weight of the spatial regularization term; represents the spatial smoothing regularization term:

[0019]

[0020] in, and Represent the gradients in the spatial dimensions (height, width, depth), i, j, k represent the spatial coordinate indexes of the image respectively;

[0021] λ time represents the hyperparameter that controls the weight of the temporal regularization term; represents the time-continuous regularization term;

[0022]

[0023] in, Represents the gradient in the time dimension.

[0024] In a second aspect, the present invention provides a four-dimensional dynamic imaging device of a conditional latent diffusion model, the device comprising: an image training data acquisition module for establishing a preset conditional latent diffusion model and acquiring a four-dimensional dynamic magnetic resonance image training set of the current part, the four-dimensional dynamic magnetic resonance image training set comprising a four-dimensional dynamic magnetic resonance image and a three-dimensional magnetic resonance image of a preset number of frames in the four-dimensional dynamic magnetic resonance image; a potential diffusion model establishment module for combining three-dimensional spatial convolution and one-dimensional temporal convolution into a denoising four-dimensional convolutional neural network in the conditional latent diffusion model, and adding a spatial attention mechanism and a temporal attention mechanism accordingly to obtain a target conditional latent diffusion model that can generate a four-dimensional dynamic magnetic resonance image; a model training module for, during the training phase, combining The four-dimensional dynamic magnetic resonance image is used as the input of the target conditional latent diffusion model after forward diffusion, and the three-dimensional magnetic resonance image of a preset number of frames is used as the conditional variable of the target conditional latent diffusion model. The target conditional latent diffusion model is trained to obtain a four-dimensional dynamic magnetic resonance generation model; a three-dimensional magnetic resonance image acquisition module is used to obtain a preset four-dimensional Gaussian noise distribution and a three-dimensional magnetic resonance image of a preset number of frames of the current part in the prediction stage; a four-dimensional image generation module is used to input the preset four-dimensional Gaussian noise distribution into a denoising four-dimensional convolutional neural network in the four-dimensional dynamic magnetic resonance generation model, and use the three-dimensional magnetic resonance image of the preset number of frames of the current part as the conditional variable for denoising, and perform reverse diffusion to obtain a four-dimensional dynamic magnetic resonance image.

[0025] In a third aspect, the present invention provides a computer device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to thereby execute the four-dimensional dynamic imaging method of the conditional latent diffusion model of the above-mentioned first aspect or any corresponding embodiment thereof.

[0026] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the four-dimensional dynamic imaging method of the conditional latent diffusion model of the above-mentioned first aspect or any corresponding embodiment thereof.

[0027] In a fifth aspect, the present invention provides a computer program product comprising computer instructions for causing a computer to execute the four-dimensional dynamic imaging method of the conditional latent diffusion model of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the specific embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or prior art description. Obviously, the drawings described below are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0029] Figure 1 is a flowchart of a four-dimensional dynamic imaging method of a conditional latent diffusion model according to an embodiment of the present application;

[0030] Figure 2 is a structural example diagram of a conditional latent diffusion model according to an embodiment of the present application;

[0031] Figure 3 is a composition example diagram of a convolutional neural network and a self-attention mechanism according to an embodiment of the present application;

[0032] Figure 4 is a flowchart of a four-dimensional dynamic imaging method of another conditional latent diffusion model according to an embodiment of the present application;

[0033] Figure 5 is a flowchart of a four-dimensional dynamic imaging method of a conditional latent diffusion model according to an embodiment of the present application;

[0034] Figure 6 is a structural block diagram of a four-dimensional dynamic imaging device of a conditional latent diffusion model according to an embodiment of the present application;

[0035] Figure 7 is a hardware structure schematic diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0037] 4D MRI, a dynamic magnetic resonance imaging technique that incorporates a temporal dimension, is widely used in research and clinical diagnosis, particularly for the dynamic assessment of the heart, blood vessels, and motor systems. Existing technologies typically rely on continuously acquiring large amounts of dynamic image data over an extended period of time, followed by reconstruction algorithms to generate a complete 4D dynamic MRI. However, the main drawbacks of these methods are their high data volume requirements and computational complexity, resulting in cumbersome acquisition and processing. Furthermore, these methods are susceptible to motion artifacts and image noise, which reduces the quality and reliability of the generated 4D dynamic MRI.

[0038] In recent years, deep learning-based generative adversarial networks (GANs) have shown great potential in the field of medical image processing and generation. The core of GANs is to use adversarial training to pit the generator and discriminator against each other, thereby generating high-quality images that are close to the real data distribution. For example, in 2020, Cole et al. proposed a GAN-based MRI reconstruction training framework to achieve high-quality reconstruction of undersampled abdominal dynamic contrast-enhanced (DCE) MRI; others later used GANs to improve the generation speed and quality of 4D flow MRI; at the same time, Xu et al. proposed a paired conditional generative adversarial network called Re-Con-GAN for highly accelerated liver 4D MRI generation, which not only shortened the generation time of 4D MRI but also maintained the generation quality.

[0039] While generative adversarial networks (GANs) have made some progress in improving 4D medical imaging, this approach has several significant limitations: First, training GANs is difficult, with mode collapse being a common issue. Second, even without mode collapse, the diversity of images generated by these networks is limited. In contrast, diffusion models have achieved significant success in non-medical fields by generating diverse images and linking image and non-image data. Despite their significant performance improvement, diffusion models have yet to be effectively developed for 4D dynamic MRI imaging in the medical field.

[0040] According to an embodiment of the present invention, an embodiment of a four-dimensional dynamic imaging method of a conditional latent diffusion model is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0041] In this embodiment, a four-dimensional dynamic imaging method of a conditional latent diffusion model is provided, which can be used in the above-mentioned computer device. Figure 1 is a flowchart of the four-dimensional dynamic imaging method of the conditional latent diffusion model, as shown in Figure 1As shown, the process includes the following steps:

[0042] Step S101: establishing a preset conditional latent diffusion model and obtaining a four-dimensional dynamic magnetic resonance image training set of the current part.

[0043] The four-dimensional dynamic magnetic resonance image training set includes four-dimensional dynamic magnetic resonance images and three-dimensional magnetic resonance images with a preset number of frames in the four-dimensional dynamic magnetic resonance images.

[0044] The embodiment of the present invention does not limit the field or type of the acquired four-dimensional dynamic training data set. As long as the four-dimensional dynamic image is generated, the present invention can be used. The embodiment of the present invention takes the four-dimensional magnetic resonance imaging (4D MRI) in the medical field as an example, obtains the motion 4D MRI of the current part through high-quality scanning, and uses the data as the basis for generating high-quality 4D dynamic MRI. The embodiment of the present invention can first establish a preset conditional latent diffusion model. The conditional latent diffusion model is a generative model that introduces conditional variables, such as introducing a preset number of frames of three-dimensional magnetic resonance images in the 4D dynamic MRI as conditional variables. The preset number of frames is not limited, and the three-dimensional magnetic resonance images of the first three frames can be used as conditional variables. This is just an example, so that the model can generate data according to specific conditions. The specific model framework is as follows Figure 2 As shown in the figure, it includes variational autoencoders, latent space and convolutional neural network architectures, among which the variational autoencoder (VAE) can learn the potential representation of data, which is helpful for the training and generation process of the subsequent conditional latent diffusion model. It is mainly composed of an encoder and a decoder. The encoder outputs the parameters of the potential distribution, and the decoder samples and reconstructs the input data from the latent space. In the conditional latent diffusion model, VAE is mainly used for self-reconstruction of the input 4D MRI, so that the conditional latent diffusion model gradually learns the potential feature z of the data, where the encoding process is expressed as z = G E (x), z represents a point in the latent space, G E represents the encoder network, and the decoding process is defined as x=G D (z), G D represents the decoder network; the forward diffusion and backward diffusion processes of the conditional latent diffusion model are not performed in the original data space, but in the feature space, which is called 4D latent space. After mapping the data to the latent feature representation through the pre-trained variational autoencoder, forward diffusion and noise can be performed in the latent space. In this way, the established conditional latent diffusion model can better capture the intrinsic structure and distribution characteristics of the data, while effectively reducing training consumption and accelerating inference efficiency.

[0045] In step S102, the three-dimensional spatial convolution and the one-dimensional temporal convolution are combined into a denoising four-dimensional convolutional neural network in the conditional latent diffusion model, and the spatial attention mechanism and the temporal attention mechanism are added accordingly to obtain a target conditional latent diffusion model that can generate four-dimensional dynamic magnetic resonance images.

[0046] The embodiment of the present invention designs a denoising 4D convolutional neural network architecture (4D U-Net architecture) specifically for 4D generation tasks, which can not only process complex data in the temporal and spatial dimensions, but also effectively realize specific high-quality 4D dynamic image generation based on conditional variables, such as Figure 3 As shown, the 4D convolution block (4D Conv) designed by the present invention can simulate a four-dimensional convolution process by combining two convolution operations, specifically combining a spatial convolution (a three-dimensional convolution of size 1×3×3×3) and a temporal convolution (a one-dimensional convolution of size 3×1×1×1), thereby forming a four-dimensional convolution effect similar to 3×3×3×3. This 4D convolutional neural network architecture can better process the spatial and temporal dimensions of 4D data and capture more delicate spatiotemporal changes. Secondly, the embodiment of the present invention also designs a spatiotemporal attention module (Attention) based on the self-attention mechanism of Transformer. This module applies the spatial attention mechanism (3DTemporal attention) and the temporal attention mechanism (3D spatial attention) in the 3D space and time dimensions, respectively. By identifying important areas and key time points in the data, the conditional diffusion model's ability to perceive dynamic changes is enhanced. The spatiotemporal self-attention mechanism enables the model to pay attention to local and global change trends in the spatiotemporal dimensions at the same time, thereby more accurately restoring the dynamic motion trajectory and ensuring that the generated results better reproduce the complex dynamic features in the data while maintaining high spatial resolution, thereby obtaining a target conditional potential diffusion model that can generate four-dimensional dynamic magnetic resonance images.

[0047] Step S103, in the training phase, the four-dimensional dynamic magnetic resonance image is used as the input of the target conditional latent diffusion model after forward diffusion, and the three-dimensional magnetic resonance image of a preset number of frames is used as the conditional variable of the target conditional latent diffusion model. The target conditional latent diffusion model is trained to obtain a four-dimensional dynamic magnetic resonance generation model.

[0048] During the training phase, an embodiment of the present invention can input the collected high-quality four-dimensional dynamic magnetic resonance image into the target conditional latent diffusion model. After encoding the latent variable through the pre-trained encoder, it is denoised by forward diffusion through the latent space to obtain a Gaussian noise distribution close to pure noise. The Gaussian noise distribution is input into a convolutional neural network for denoising, and reversely diffused with the three-dimensional magnetic resonance image of a preset number of frames as the conditional variable to obtain a denoised four-dimensional dynamic magnetic resonance image. Based on the comparison between the denoised four-dimensional dynamic magnetic resonance image and the high-quality actual four-dimensional dynamic magnetic resonance image, the target conditional latent diffusion model is optimized and adjusted to obtain the final four-dimensional dynamic magnetic resonance generation model.

[0049] Step S104 , in the prediction stage, a preset four-dimensional Gaussian noise and a preset number of frames of a three-dimensional magnetic resonance image of the current part are obtained.

[0050] The four-dimensional dynamic magnetic resonance image generation method of the embodiment of the present invention is applicable to, but not limited to, dynamic assessment sites such as the heart, blood vessels, and musculoskeletal system. The four-dimensional dynamic magnetic resonance image generation method can also be used to generate 4D dynamic images in other fields. In response to generating a 4D MRI image of a current site, a preset random four-dimensional Gaussian noise distribution and a preset number of frames of three-dimensional magnetic resonance images of the current site to be imaged can be first obtained. The preset number of frames is not limited and can be set according to actual needs. Generally, a small number of static 3D images can be used as input conditions.

[0051] In step S105, the preset four-dimensional Gaussian noise distribution is input into the denoising four-dimensional convolutional neural network in the four-dimensional dynamic magnetic resonance generation model, and the three-dimensional magnetic resonance image of the preset number of frames of the current part is used as a conditional variable for denoising, and reverse diffusion is performed to obtain a four-dimensional dynamic magnetic resonance image.

[0052] In this embodiment of the present invention, a preset four-dimensional Gaussian noise distribution is input into a denoising four-dimensional convolutional neural network (CNN) within a four-dimensional dynamic MRI generation model. During the inference phase, the denoising four-dimensional convolutional neural network first samples an initial noise state z from the Gaussian noise distribution. This state, along with the conditional variables of the current three-dimensional MRI image, is then gradually introduced into the structural features of the three-dimensional MRI image through a reverse diffusion process, ultimately yielding a four-dimensional dynamic MRI image. This process can be viewed as temporal reverse denoising, where each step depends on the output of the previous step until a sample consistent with the noise distribution is generated. In this way, the conditional latent diffusion model not only reproduces the key features of the training data during inference but also generates accurate and high-temporal-resolution 4D dynamic MRI images, guided by the given conditional variables.

[0053] During the testing / application stage, the present invention only needs to obtain a small amount of three-dimensional magnetic resonance images of the current part as conditional variables, and input a random Gaussian noise distribution. After denoising the four-dimensional convolutional network and inverse diffusion, the four-dimensional dynamic magnetic resonance image features are obtained, and then the generated four-dimensional dynamic magnetic resonance image can be obtained through the VAE decoder, as an example only.

[0054] The four-dimensional dynamic imaging method of the conditional latent diffusion model provided in this embodiment establishes a conditional latent diffusion model, combines three-dimensional spatial convolution and one-dimensional temporal convolution into a denoising four-dimensional convolutional neural network in the conditional latent diffusion model, and adds a spatial attention mechanism and a temporal attention mechanism accordingly to obtain a target conditional latent diffusion model that can generate four-dimensional dynamic magnetic resonance images. The four-dimensional dynamic magnetic resonance images are then used as the input of the target conditional latent diffusion model after forward diffusion, and three-dimensional magnetic resonance images of a preset number of frames are used as conditional variables of the target conditional latent diffusion model. The target conditional latent diffusion model is trained to obtain a four-dimensional dynamic magnetic resonance generation model, which reduces the dependence on a large amount of dynamic image data and realizes the efficient generation of complete and high-quality four-dimensional dynamic magnetic resonance images using only a small number of static three-dimensional magnetic resonance images, thereby improving the accuracy and spatiotemporal resolution of imaging. In addition, the training process of the conditional latent diffusion model is more stable, ensuring the diversity and robustness of the generated images, and providing an effective imaging solution for a wide range of clinical needs.

[0055] In this embodiment, a four-dimensional dynamic imaging method of a conditional latent diffusion model is provided, which can be used in the above-mentioned computer device. Figure 4 is a flow chart of a four-dimensional dynamic imaging method according to a conditional potential diffusion model according to an embodiment of the present invention. Figure 4 As shown, the process includes the following steps:

[0056] Step S401: Establish a preset conditional latent diffusion model and obtain a 4D dynamic MRI image training set of the current part. The 4D dynamic MRI image training set includes 4D dynamic MRI images and 3D MRI images of a preset number of frames in the 4D dynamic MRI images. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0057] Step S402: Combine the three-dimensional spatial convolution and the one-dimensional temporal convolution into a denoising four-dimensional convolutional neural network in the conditional latent diffusion model, and add the spatial attention mechanism and the temporal attention mechanism accordingly to obtain the target conditional latent diffusion model that can generate four-dimensional dynamic magnetic resonance images. For detailed description, please refer to Figure 1 The step S102 shown is not repeated here.

[0058] Step S403: During the training phase, the four-dimensional dynamic magnetic resonance image is used as the input of the target conditional latent diffusion model after forward diffusion, and the three-dimensional magnetic resonance image of a preset number of frames is used as the conditional variable of the target conditional latent diffusion model. The target conditional latent diffusion model is trained to obtain a four-dimensional dynamic magnetic resonance generation model.

[0059] Specifically, the above step S403 includes:

[0060] In step S4031, the four-dimensional dynamic magnetic resonance image is input into the target conditional latent diffusion model and encoded by the pre-trained encoder to obtain latent variables, and then forward diffusion is performed to obtain Gaussian noise distribution.

[0061] Step S4032: The three-dimensional magnetic resonance image of the preset number of frames is input as a conditional variable and the Gaussian noise distribution into a denoising four-dimensional convolutional neural network for denoising, and reverse diffusion is performed to obtain the denoised four-dimensional dynamic magnetic resonance image features.

[0062] Step S4033 : The denoised four-dimensional dynamic magnetic resonance image features are decoded by a pre-trained decoder to obtain a four-dimensional dynamic magnetic resonance image to be tested.

[0063] Step S4034, based on the loss function of the target conditional latent diffusion model training, calculate the loss function values ​​of the four-dimensional dynamic magnetic resonance image and the four-dimensional dynamic magnetic resonance image to be tested, and the sum of the loss function values ​​of the noise distribution removed by the convolutional neural network and the Gaussian noise distribution to obtain the loss function value of the target conditional latent diffusion model training.

[0064] Step S4035 , optimizing the target conditional latent diffusion model based on the loss function value of the target conditional latent diffusion model training until the loss function value is no greater than a preset loss value, thereby obtaining a four-dimensional dynamic magnetic resonance generation model.

[0065] The essence of the target conditional latent diffusion model in the embodiment of the present invention is to learn the Markov chain process to convert the Gaussian distribution into a data distribution. The model is divided into two stages: forward diffusion denoising and backward diffusion denoising. The four-dimensional dynamic magnetic resonance image is input into the target conditional latent diffusion model. After being encoded by the encoder to obtain the latent variable, it is placed in the latent space for forward diffusion denoising. The forward diffusion denoising process is to add noise to the target 4D MRI data latent feature z. Therefore, the data distribution of z at t∈[0,u] can be expressed as:

[0066]

[0067] Among them, β t is the noise variance in the range (0,1); z t represents the latent variable; q(z t∣z t-1 ) represents the noise distribution after noise addition.

[0068] The forward process is the process of denoising the data, and the backward process is a denoising process. That is, a Gaussian noise is randomly sampled at time T, and then denoised step by step, and finally a generated image with the same distribution as the real image is obtained. In the reverse denoising stage, the model learns the following parameterized Gaussian transform:

[0069]

[0070] Among them, μ θ (z t ,t) is the learned mean, σ t is a fixed variance; p θ (z t-1 |z t ) represents a denoised 4D dynamic MRI image. Therefore, the generative process of sampling the data is performed by random steps: t-1 =μ θ (z t ,t)+σ t ∈, where ∈~(0,I). In this study, 4D dynamic MRI is used as x0, and the initial image is sampled with noise perturbation:

[0071]

[0072] Among them, ∈~(0,I) can be learned through the denoising autoencoder network, because in Then, the conditional image encoding features are used Noise target z t The time step t is input into the 4D U-Net network, and the noise prediction is assisted according to the guidance of the conditional variable, so as to gradually realize denoising and restore the image. Among them, the role of the conditional variable Conditioning is: due to the "noise convergence" feature, when the noise is added more, z T It is close to a "pure noise", but the training process requires comparing the input image x and the output image In order to restore a “pure noise” image to an image similar to the input image, we must provide the model with additional information guidance.

[0073] The loss function of the target-conditioned latent diffusion model mainly consists of two parts: the constrained generated noise distribution and the generated 4D MRI, which are used to guide the denoising diffusion probability network to learn how to gradually remove noise and restore the original structure of the data. Its goal is to minimize the prediction error of the denoising process, that is, the difference between the denoised result predicted by the network and the actual data, which is usually defined based on the mean square error (MSE):

[0074]

[0075] in, represents the loss function for training the target-conditional latent diffusion model; represents the expectation of noise and latent variables; ∈ θ represents the denoising convolutional neural network; ∈∈(0,I) represents the noise data obeying the Gaussian distribution; z t represents the latent variable; t represents the uniformly sampled time step in [0, u]; Characteristic information representing conditional input, usually used to control conditional variables in the process of generating four-dimensional dynamic magnetic resonance images; represents the expectation of the real image and the generated image, x represents the four-dimensional dynamic magnetic resonance image; Represents the four-dimensional dynamic magnetic resonance image to be tested.

[0076] Based on the loss function of the above-mentioned target conditional latent diffusion model, the loss function values ​​of the four-dimensional dynamic magnetic resonance image and the four-dimensional dynamic magnetic resonance image to be measured, as well as the sum of the loss function values ​​of the noise distribution removed by the convolutional neural network and the Gaussian noise distribution, are calculated to obtain the loss function value of the target conditional latent diffusion model training. Then, based on the loss function value of the target conditional latent diffusion model training, the target conditional latent diffusion model is optimized until the loss function value is no greater than the preset loss value, thereby obtaining the optimized four-dimensional dynamic magnetic resonance generation model.

[0077] The present invention combines the loss of the tested four-dimensional dynamic magnetic resonance image and the real four-dimensional dynamic magnetic resonance image with the denoising loss of the convolutional neural network to constrain the noise distribution of the conditional latent diffusion model, improve the generalization ability of the conditional latent diffusion model, and ensure the accuracy and consistency of the generated image.

[0078] Furthermore, spatial smoothing regularization terms and temporal continuity regularization terms are added to the loss function of the target conditional latent diffusion model training to obtain the constrained loss function; based on the constrained loss function, the loss function values ​​of the four-dimensional dynamic magnetic resonance image and the four-dimensional dynamic magnetic resonance image to be tested, as well as the sum of the loss function values ​​of the noise distribution removed by the convolutional neural network and the Gaussian noise distribution, are calculated to obtain the loss function value of the target conditional latent diffusion model training.

[0079] In this embodiment of the present invention, two regularization terms are added to the loss function of the target conditional latent diffusion model training to constrain the generated image in the spatial and temporal dimensions respectively. The spatial smoothing regularization term is used to encourage the generated image to have continuity and consistency in the spatial dimension and reduce noise interference. The spatial smoothing regularization term is defined as follows:

[0080]

[0081] in, and They represent the gradients in the spatial dimensions (height, width, depth), respectively. i, j, and k represent the spatial coordinate indexes of the image, respectively, to ensure a smooth transition of the image in space.

[0082] The temporal continuity regularization term in this embodiment of the present invention is used to ensure the continuity of the generated 4D MRI in the temporal dimension and prevent inter-frame content jumps. It is defined as follows:

[0083]

[0084] in, Represents the gradient in the time dimension, ensuring that the generated sequence is smooth and continuous in the time dimension and avoiding abrupt changes.

[0085] Combining the loss function and regularization term of the target conditional latent diffusion model training, the final constrained loss function is as follows:

[0086]

[0087] in, represents the loss function after constraint; space represents the hyperparameter that controls the weight of the spatial regularization term; time represents the hyperparameter that controls the weight of the temporal regularization term, where the hyperparameters that control the weights of the spatial and temporal regularization terms can be adjusted according to the actual task.

[0088] An embodiment of the present invention can calculate the loss function values ​​of the four-dimensional dynamic magnetic resonance image and the four-dimensional dynamic magnetic resonance image to be measured, as well as the loss function values ​​and regularization terms of the noise distribution removed by the convolutional neural network and the Gaussian noise distribution based on the constrained loss function formula obtained above, to obtain the loss function value of the target conditional latent diffusion model training, and adjust and optimize the target conditional latent diffusion model based on the loss function value until the loss function value is no greater than the preset loss value, to obtain a four-dimensional dynamic magnetic resonance generation model, wherein the preset loss value can be adjusted according to actual task requirements.

[0089] The present invention adds spatial smoothing regularization terms and temporal continuity regularization terms to the loss function of conditional latent diffusion model training to obtain a constrained loss function. The model is then adjusted based on the constrained loss function, which can constrain the complexity of the conditional latent diffusion model and thus improve the generalization ability of the model.

[0090] Step S404: In the prediction phase, a preset four-dimensional Gaussian noise and a preset number of frames of a three-dimensional magnetic resonance image of the current part are obtained. Figure 1 Step S104 of the illustrated embodiment will not be described in detail here.

[0091] Step S405: Input the preset four-dimensional Gaussian noise distribution into the denoising four-dimensional convolutional neural network in the four-dimensional dynamic MRI generation model, and perform denoising processing using the three-dimensional MRI images of the current part with the preset number of frames as conditional variables, and perform reverse diffusion to obtain a four-dimensional dynamic MRI image. Figure 1 Step S105 of the illustrated embodiment will not be described in detail here.

[0092] In an optional embodiment, a training set of four-dimensional dynamic magnetic resonance images of multiple parts is traversed, and the four-dimensional dynamic magnetic resonance images are used as input of a target conditional latent diffusion model after forward diffusion, and three-dimensional magnetic resonance images of a preset number of frames are used as conditional variables of the target conditional latent diffusion model. The target conditional latent diffusion model is trained to obtain a four-dimensional dynamic magnetic resonance generation model that can generate four-dimensional dynamic magnetic resonance images of multiple parts.

[0093] The embodiment of the present invention can obtain a four-dimensional dynamic magnetic resonance image training set of different parts, traverse the four-dimensional dynamic magnetic resonance image training set of different parts in sequence to train the target conditional latent diffusion model, and obtain a four-dimensional dynamic magnetic resonance generation model that can generate four-dimensional dynamic magnetic resonance images of multiple parts.

[0094] The present invention trains the target conditional latent diffusion model by acquiring a training set of four-dimensional dynamic magnetic resonance images of different parts, and obtains a four-dimensional dynamic magnetic resonance generation model that can generate four-dimensional dynamic magnetic resonance images of multiple parts, laying the foundation for the subsequent generation of high-quality 4D dynamic MRI using a small number of static three-dimensional magnetic resonance images of different parts.

[0095] In an optional embodiment, a three-dimensional magnetic resonance image with a preset number of frames containing multiple different parts is obtained; a preset four-dimensional Gaussian noise distribution is input into a denoising four-dimensional convolutional neural network in a four-dimensional dynamic magnetic resonance generation model, and the three-dimensional magnetic resonance image with a preset number of frames containing multiple different parts is used as a conditional variable for denoising, and reverse diffusion is performed to obtain a four-dimensional dynamic magnetic resonance image.

[0096] The embodiment of the present application can obtain three-dimensional magnetic resonance images of a preset frame number containing multiple different parts in response to generating a four-dimensional dynamic magnetic resonance image containing multiple parts, input random four-dimensional Gaussian noise distribution into a convolutional neural network in a four-dimensional dynamic magnetic resonance generation model for denoising processing, and perform reverse diffusion with four-dimensional images of a preset frame number containing multiple different parts, to obtain a four-dimensional dynamic magnetic resonance image containing different parts.

[0097] The present application generates high-quality 4D dynamic MRI using a small amount of static 3D images of different parts, reduces the dependence on dynamic data, and improves the generation quality and efficiency of 4D dynamic MRI.

[0098] In specific embodiments, as shown in Figure 5 different parts of motion 4D MRI are obtained by high-quality scanning, after multi-part 4D dynamic MRI data acquisition, the MRI data acquired is pre-trained using a variational autoencoder VAE, then a conditional latent diffusion model is constructed and trained, the model generates accurate and coherent 4D dynamic MRI sequences according to the latent features obtained by VAE pre-training, plus the conditional variable through the conditional latent diffusion model, then the conditional latent diffusion model is adjusted and optimized by adding a loss function of spatial smoothing regularization term and time continuity regularization term, to obtain a four-dimensional dynamic magnetic resonance generation model. In the inference stage, a small amount of 3D MRI data frames are collected, and the trained four-dimensional dynamic magnetic resonance generation model is used to obtain 4D dynamic MRI based on a small amount of 3D MRI data frames and four-dimensional Gaussian noise distribution, and the 4D dynamic MRI imaging result is evaluated and applied. For details, please refer to the above embodiments, which will not be repeated here.

[0099] In the present embodiment, a four-dimensional dynamic imaging device of a conditional latent diffusion model is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and will not be repeated here since it has been described. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware, or a combination of software and hardware is also possible and is contemplated.

[0100] The present embodiment provides a four-dimensional dynamic imaging device of a conditional latent diffusion model, as Figure 6As shown, it includes: an image training data acquisition module 601, which is used to establish a preset conditional latent diffusion model and obtain a four-dimensional dynamic magnetic resonance image training set of the current part, where the four-dimensional dynamic magnetic resonance image training set includes a four-dimensional dynamic magnetic resonance image and a three-dimensional magnetic resonance image of a preset number of frames in the four-dimensional dynamic magnetic resonance image; a potential diffusion model establishment module 602, which is used to combine the three-dimensional spatial convolution and the one-dimensional temporal convolution into a denoising four-dimensional convolutional neural network in the conditional latent diffusion model, and add a spatial attention mechanism and a temporal attention mechanism accordingly to obtain a target conditional latent diffusion model that can generate a four-dimensional dynamic magnetic resonance image; a model training module 603, which is used to, during the training phase, transform the four-dimensional dynamic magnetic resonance image through forward diffusion As the input of the target conditional latent diffusion model, the three-dimensional magnetic resonance image of the preset number of frames is used as the conditional variable of the target conditional latent diffusion model, and the target conditional latent diffusion model is trained to obtain a four-dimensional dynamic magnetic resonance generation model; the three-dimensional magnetic resonance image acquisition module 604 is used to obtain a preset four-dimensional Gaussian noise distribution and a three-dimensional magnetic resonance image of the preset number of frames of the current part in the prediction stage; the four-dimensional image generation module 605 is used to input the preset four-dimensional Gaussian noise distribution into the denoising four-dimensional convolutional neural network in the four-dimensional dynamic magnetic resonance generation model, and use the three-dimensional magnetic resonance image of the preset number of frames of the current part as the conditional variable for denoising processing, and perform reverse diffusion to obtain a four-dimensional dynamic magnetic resonance image.

[0101] In some optional embodiments, the four-dimensional dynamic imaging device of the conditional latent diffusion model also includes: a multi-site image traversal module, which is used to traverse the four-dimensional dynamic magnetic resonance image training set of multiple sites, execute the four-dimensional dynamic magnetic resonance image after forward diffusion as the input of the target conditional latent diffusion model, and use the three-dimensional magnetic resonance image of a preset frame number as the conditional variable of the target conditional latent diffusion model, and train the target conditional latent diffusion model to obtain a four-dimensional dynamic magnetic resonance generation model that can generate four-dimensional dynamic magnetic resonance images of multiple sites.

[0102] In some optional embodiments, the four-dimensional dynamic imaging device of the conditional latent diffusion model also includes: a multi-part image acquisition module, used to acquire a three-dimensional magnetic resonance image of a preset number of frames containing multiple different parts; a four-dimensional image generation module, used to input a preset four-dimensional Gaussian noise distribution into a denoising four-dimensional convolutional neural network in the four-dimensional dynamic magnetic resonance generation model, and perform denoising processing using the three-dimensional magnetic resonance image of a preset number of frames containing multiple different parts as a conditional variable, and perform reverse diffusion to obtain a four-dimensional dynamic magnetic resonance image.

[0103] In some optional embodiments, the model training module 603 comprises: an image noise adding unit configured to input the four-dimensional dynamic magnetic resonance image into a pre-trained encoder of the target conditional latent diffusion model to obtain a latent variable, and then perform forward diffusion to obtain a Gaussian noise distribution; a noise denoising unit configured to input a three-dimensional magnetic resonance image of a preset frame number as a conditional variable and the Gaussian noise distribution into a denoising four-dimensional convolutional neural network to perform denoising processing, and perform inverse diffusion to obtain a denoised four-dimensional dynamic magnetic resonance image feature; an image decoding unit configured to decode the denoised four-dimensional dynamic magnetic resonance image feature through a pre-trained decoder to obtain a to-be-tested four-dimensional dynamic magnetic resonance image; a loss function value calculation unit configured to calculate a sum of a loss function value of the four-dimensional dynamic magnetic resonance image and the to-be-tested four-dimensional dynamic magnetic resonance image and a loss function value of a noise distribution removed by the convolutional neural network and the Gaussian noise distribution based on a loss function of the target conditional latent diffusion model training to obtain a loss function value of the target conditional latent diffusion model training; and a model adjusting unit configured to optimize the target conditional latent diffusion model based on the loss function value of the target conditional latent diffusion model training until the loss function value is not greater than a preset loss value, and obtain a four-dimensional dynamic magnetic resonance generation model.

[0104] In some optional embodiments, the loss function value calculation unit comprises: a regularization term adding sub-unit configured to add a spatial smoothing regularization term and a time continuity regularization term in the loss function of the target conditional latent diffusion model training to obtain a constrained loss function; and a loss function value calculation sub-unit configured to calculate a sum of a loss function value of the four-dimensional dynamic magnetic resonance image and the to-be-tested four-dimensional dynamic magnetic resonance image and a loss function value of a noise distribution removed by the convolutional neural network and the Gaussian noise distribution based on the constrained loss function to obtain a loss function value of the target conditional latent diffusion model training.

[0105] In some optional embodiments, the constrained loss function is the following formula:

[0106]

[0107] wherein, denotes the constrained loss function; denotes the loss function of the target conditional latent diffusion model training:

[0108]

[0109] wherein, denotes an expectation of noise and a latent variable, ∈ θ denotes a denoising convolutional neural network, ∈ ∈(0, I) denotes noise data subject to a Gaussian distribution, z t denotes a latent variable, and t denotes a uniform sampling time step in [0, u], characteristic information representing condition input, usually used to control condition variables in a four-dimensional dynamic magnetic resonance image generation process, represents the expectation of the real image and the generated image, x represents the four-dimensional dynamic magnetic resonance image, represents the four-dimensional dynamic magnetic resonance image to be measured; λ space represents a hyperparameter for controlling the weight of the spatial regularization term; represents a spatial smoothing regularization term:

[0110]

[0111] wherein, and respectively represent the gradient in the spatial dimension (height, width, depth), i, j, k respectively represent the spatial coordinate index of the image;

[0112] λ time represents a hyperparameter for controlling the weight of the temporal regularization term; represents a temporal continuity regularization term;

[0113]

[0114] wherein, represents the gradient in the temporal dimension.

[0115] Further function descriptions of the above-mentioned various modules and units are the same as those of the above-mentioned corresponding embodiments, and will not be repeated here.

[0116] The four-dimensional dynamic imaging device of the conditional latent diffusion model in the embodiment is presented in the form of functional units, wherein the unit refers to an ASIC (Application Specific Integrated Circuit, Application Specific Integrated Circuit) circuit, a processor and a memory executing one or more software or fixed programs, and / or other devices that can provide the above functions.

[0117] The embodiment of the present application also provides a computer device having the above-mentioned Figure 6 four-dimensional dynamic imaging device of the conditional latent diffusion model.

[0118] Please refer to Figure 7 , Figure 7 is a structural schematic diagram of a computer device provided by an optional embodiment of the present application, as shown in Figure 7As shown, the computer device includes: one or more processors 10, memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components utilize different buses to communicate with each other and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in the memory or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Equally, multiple computer devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 A processor 10 is taken as an example.

[0119] The processor 10 may be a central processing unit, a network processor, or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0120] The memory 20 stores instructions that can be executed by at least one processor 10, so as to enable at least one processor 10 to execute the method shown in the above embodiment.

[0121] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0122] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0123] The computer device further includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 7 The bus connection is taken as an example.

[0124] The input device 30 can receive input digital or character information and generate key signal input related to user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a trackpad, a touch pad, an indicator stick, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 can include a display device, an auxiliary lighting device (e.g., an LED), and a tactile feedback device (e.g., a vibration motor). The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display, and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0125] The embodiment of the present invention also provides a computer-readable storage medium. The above-mentioned method according to the embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0126] A portion of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the form in which the computer program instruction exists in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc. Accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium that can be accessed by the computer.

[0127] While embodiments of the application have been described in connection with the preferred embodiments of the various figures, those of ordinary skill in the art will appreciate that various modifications and changes can be made without departing from the spirit and scope of the application, and that such modifications and changes fall within the scope of the appended claims.

Claims

1. A four-dimensional dynamic imaging method for a conditional latent diffusion model, characterized in that: The method comprises: Establishing a preset conditional latent diffusion model and obtaining a four-dimensional dynamic magnetic resonance image training set of the current part, wherein the four-dimensional dynamic magnetic resonance image training set includes the four-dimensional dynamic magnetic resonance image and a preset number of three-dimensional magnetic resonance images of the four-dimensional dynamic magnetic resonance image; Combining three-dimensional spatial convolution and one-dimensional temporal convolution into a denoising four-dimensional convolutional neural network in the conditional latent diffusion model, and adding spatial attention mechanism and temporal attention mechanism accordingly to obtain a target conditional latent diffusion model that can generate four-dimensional dynamic magnetic resonance images; In the training phase, the four-dimensional dynamic magnetic resonance image is used as the input of the target conditional latent diffusion model after forward diffusion, and the three-dimensional magnetic resonance image of a preset number of frames is used as the conditional variable of the target conditional latent diffusion model. The target conditional latent diffusion model is trained to obtain a four-dimensional dynamic magnetic resonance generation model; In the prediction stage, a preset four-dimensional Gaussian noise and a preset number of frames of a three-dimensional magnetic resonance image of the current part are obtained; The preset four-dimensional Gaussian noise distribution is input into the denoising four-dimensional convolutional neural network in the four-dimensional dynamic magnetic resonance generation model, and the three-dimensional magnetic resonance image of the preset number of frames of the current part is used as a conditional variable for denoising, and reverse diffusion is performed to obtain a four-dimensional dynamic magnetic resonance image.

2. The method according to claim 1, characterized in that The method further comprises: A training set of four-dimensional dynamic magnetic resonance images of multiple parts is traversed, and the steps of using the four-dimensional dynamic magnetic resonance images after forward diffusion as input of a target conditional latent diffusion model, using three-dimensional magnetic resonance images of a preset number of frames as conditional variables of the target conditional latent diffusion model, and training the target conditional latent diffusion model are performed to obtain a four-dimensional dynamic magnetic resonance generation model that can generate four-dimensional dynamic magnetic resonance images of multiple parts.

3. The method according to claim 2, characterized in that The method further comprises: Acquiring a three-dimensional magnetic resonance image of a preset number of frames including a plurality of different parts; The preset four-dimensional Gaussian noise distribution is input into the denoising four-dimensional convolutional neural network in the four-dimensional dynamic magnetic resonance generation model, and the three-dimensional magnetic resonance image with a preset number of frames containing multiple different parts is used as the conditional variable for denoising, and reverse diffusion is performed to obtain the four-dimensional dynamic magnetic resonance image.

4. The method according to claim 1, wherein The method uses the four-dimensional dynamic magnetic resonance image as the input of the target conditional latent diffusion model after forward diffusion, uses the three-dimensional magnetic resonance image of a preset number of frames as the conditional variable of the target conditional latent diffusion model, and trains the target conditional latent diffusion model to obtain a four-dimensional dynamic magnetic resonance generation model, including: Inputting the four-dimensional dynamic magnetic resonance image into a target conditional latent diffusion model, encoding it with a pre-trained encoder to obtain a latent variable, and then performing forward diffusion to obtain a Gaussian noise distribution; Inputting the three-dimensional magnetic resonance image of the preset number of frames as a conditional variable and the Gaussian noise distribution into a denoising four-dimensional convolutional neural network for denoising, and performing reverse diffusion to obtain denoised four-dimensional dynamic magnetic resonance image features; Decoding the denoised four-dimensional dynamic magnetic resonance image features through a pre-trained decoder to obtain a four-dimensional dynamic magnetic resonance image to be tested; Based on the loss function of the target conditional latent diffusion model training, calculating the loss function values ​​of the four-dimensional dynamic magnetic resonance image and the four-dimensional dynamic magnetic resonance image to be tested, and the sum of the loss function values ​​of the noise distribution removed by the convolutional neural network and the Gaussian noise distribution, to obtain the loss function value of the target conditional latent diffusion model training; The target conditional potential diffusion model is optimized based on the loss function value of the target conditional potential diffusion model training until the loss function value is no greater than a preset loss value, thereby obtaining a four-dimensional dynamic magnetic resonance generation model.

5. The method according to claim 4, characterized in that The loss function of the target conditional latent diffusion model training is calculated based on the loss function of the four-dimensional dynamic magnetic resonance image and the four-dimensional dynamic magnetic resonance image to be measured, and the sum of the loss function values ​​of the noise distribution removed by the convolutional neural network and the loss function value of the Gaussian noise distribution to obtain the loss function value of the target conditional latent diffusion model training, including: Adding a spatial smoothing regularization term and a temporal continuity regularization term to the loss function of the target conditional latent diffusion model training to obtain a constrained loss function; Based on the constrained loss function, the loss function values ​​of the four-dimensional dynamic magnetic resonance image and the four-dimensional dynamic magnetic resonance image to be measured, as well as the sum of the loss function values ​​of the noise distribution removed by the convolutional neural network and the Gaussian noise distribution are calculated to obtain the loss function value of the target conditional latent diffusion model training.

6. The method according to claim 5, characterized in that The loss function after the constraint is as follows: in, represents the loss function after constraint; Represents the loss function for target-conditional latent diffusion model training: in, represents the expectation of noise and latent variables, ∈ θ represents the denoising convolutional neural network, ∈∈(0,I) represents the noise data obeying the Gaussian distribution, represents the latent variable, t represents the uniformly sampled time step in [0, u], Represents the characteristic information of the conditional input, which is usually used to control the conditional variables in the process of generating four-dimensional dynamic magnetic resonance images. represents the expectation of the real image and the generated image, x represents the four-dimensional dynamic magnetic resonance image, represents the four-dimensional dynamic magnetic resonance image to be measured; λ space represents the hyperparameter that controls the weight of the spatial regularization term; represents the spatial smoothing regularization term: in, and Represent the gradients in the spatial dimensions height, width, and depth respectively, and i, j, and k represent the spatial coordinate indexes of the image respectively; λ time represents the hyperparameter that controls the weight of the temporal regularization term; represents the time-continuous regularization term; in, Represents the gradient in the time dimension.

7. A four-dimensional dynamic imaging device for a conditional latent diffusion model, characterized in that: The device comprises: an image training data acquisition module, configured to establish a preset conditional latent diffusion model and acquire a four-dimensional dynamic magnetic resonance image training set of the current location, the four-dimensional dynamic magnetic resonance image training set comprising a four-dimensional dynamic magnetic resonance image and a preset number of three-dimensional magnetic resonance images within the four-dimensional dynamic magnetic resonance image; A latent diffusion model building module is used to combine three-dimensional spatial convolution and one-dimensional temporal convolution into a denoising four-dimensional convolutional neural network in the conditional latent diffusion model, and to add spatial attention mechanism and temporal attention mechanism accordingly to obtain a target conditional latent diffusion model that can generate four-dimensional dynamic magnetic resonance images; a model training module for, during a training phase, using the four-dimensional dynamic magnetic resonance image as input to a target conditional latent diffusion model after forward diffusion, and using a preset number of three-dimensional magnetic resonance images as conditional variables of the target conditional latent diffusion model, training the target conditional latent diffusion model to obtain a four-dimensional dynamic magnetic resonance generation model; A three-dimensional magnetic resonance image acquisition module is used to acquire a three-dimensional magnetic resonance image of a preset four-dimensional Gaussian noise distribution and a preset number of frames of the current part during the prediction stage; The four-dimensional image generation module is used to input the preset four-dimensional Gaussian noise distribution into the denoising four-dimensional convolutional neural network in the four-dimensional dynamic magnetic resonance generation model, and use the three-dimensional magnetic resonance image of the preset number of frames of the current part as the conditional variable for denoising, and perform reverse diffusion to obtain a four-dimensional dynamic magnetic resonance image.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the four-dimensional dynamic imaging method of the conditional latent diffusion model according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the four-dimensional dynamic imaging method of the conditional latent diffusion model according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the four-dimensional dynamic imaging method of the conditional latent diffusion model according to any one of claims 1 to 6.

Images