Method and system for synthesizing brain metastasis tumor CT image into MRI image based on residual density network

Abstract

The invention discloses a method and a system for synthesizing a brain metastasis tumor CT image into an MRI image based on a residual density network, and relates to the field of medical images. The method comprises the following steps: firstly, acquiring MRI image data and CT image data of the same-position skull examination of a brain metastasis tumor patient within a preset time range; registering and re-sampling the MRI image data and the CT image data to obtain a registration data set of pixel-level corresponding pairing; then, a CT-MRI registration data set is utilized to construct a Dense-GAN model based on a residual density network, and the Dense-GAN model is used for synthesizing an MRI image from the CT image; and finally, by using the trained Dense-GAN, obtaining a synthetic image from the brain metastatic tumor CT to the MRI. According to the method, more comprehensive anatomical information is provided for a brain metastasis patient which is not suitable for MRI simulation positioning in radiotherapy, more information is provided for delineation of a brain metastasis tumor target region, and the accuracy of delineation of the target region and the tumor local control rate are improved.

Description

Technical Field

[0001] The present invention relates to the field of medical imaging, and more particularly to a method and system for synthesizing MRI images from CT images of brain metastases based on a residual density network. Background Art

[0002] Brain metastases are the most common secondary intracranial malignant tumors, and stereotactic radiotherapy (high-dose hypofractionated radiotherapy) is an important local treatment method. Radiation needs to be accurately irradiated to the tumor target area to eliminate the tumor, while minimizing the irradiation of surrounding normal tissues and organs at risk, so accurate target area delineation is crucial. Currently, radiotherapy mainly uses CT simulation positioning to acquire images for tumor target area delineation and dose calculation. However, positioning CT images cannot display the volume and shape of intracranial tumors with high clarity, which affects the accuracy of brain metastasis target area delineation and limits the application of stereotactic radiotherapy technology in the treatment of brain metastases, thereby reducing the local control rate of the tumor and shortening the patient's survival. Therefore, the accuracy of target area delineation has become a clinical medical problem that urgently needs to be solved in the current precision radiotherapy of brain metastases.

[0003] Compared with CT images, magnetic resonance imaging (MRI) images have higher spatial resolution and soft tissue contrast, and no additional radiation damage. Therefore, MRI simulation positioning systems are widely used in clinical practice. Currently, MRI simulation positioning is mainly used in combination with CT simulation positioning in clinical practice, and MRI and CT images are correlated and registered to guide the delineation of tumor target areas. However, there are still key issues that need to be solved in the MRI simulation positioning systems used in clinical practice: (1) MRI images do not have electron density information: MRI image signal intensity is a function of proton density and tissue relaxation time. Unlike CT, it is impossible to convert the corresponding tissue density based on the electron density-CT value conversion curve to accurately calculate the tissue dose distribution and directly apply it to radiotherapy planning; (2) MRI images have geometric distortion: MRI geometric distortion leads to increased MRI-CT image registration errors, affecting the accuracy of radiotherapy; (3) MRI image scanning has contraindications: MRI uses an ultra-strong constant magnetic field, and some patients are not suitable for MRI simulation positioning, such as those with pacemakers, neurostimulators, metal prostheses and joints. Therefore, further exploration of new multimodal image processing technologies is needed to solve the problems of MRI simulation positioning systems, which will help to improve the accuracy of brain metastasis target delineation.

[0004] In recent years, artificial intelligence (AI) technologies, represented by deep learning, have demonstrated high performance in image segmentation, denoising, reconstruction, transformation, and image synthesis, attracting widespread attention from domestic and international scholars in the field of radiotherapy. Some research teams have used AI algorithms to generate CT images from MRI images to address the problems of MRI-CT image fusion technology, but there are currently few reports on algorithms for synthesizing MRI images from CT images. Some researchers have attempted to learn and generate features similar to MRI images by training neural networks. However, current algorithms have poor robustness, resulting in insufficient clarity and severe distortion of the generated images. In addition, the data sets collected in these studies are small and are only suitable for feasibility studies, but are insufficient for accurate target delineation and dose calculation in clinical radiotherapy. Summary of the Invention

[0005] In view of this, the present invention provides a method and system for synthesizing MRI images from CT images of brain metastases based on a residual density network to solve the problems existing in the background technology.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for synthesizing MRI images from CT images of brain metastases based on a residual density network comprises the following steps:

[0008] Obtain CT-MRI paired datasets of patients with brain metastases;

[0009] Perform multimodal registration and resampling of data in paired datasets;

[0010] The paired dataset was used to complete the training of a residual density network-based model for synthesizing MRI images from CT images of brain metastases (Dense-GAN).

[0011] The CT image is input into Dense-GAN to obtain a synthetic MRI image.

[0012] Furthermore, registering and resampling the data in the paired dataset specifically includes the following steps: using a multimodal registration algorithm and interpolation technology to register and resample the CT-MRI paired dataset, and verifying the registration effect of the registered data to exclude data with poor registration effect.

[0013] The resampled data were unified into 512 pixels × 512 pixels, 5 mm slice thickness CT-MRI paired data;

[0014] Furthermore, after the paired data sets are aligned, the pixel levels of the CT image data and the MRI image data of the CT-MRI paired data are in a one-to-one correspondence.

[0015] Furthermore, the residual density-based brain metastasis CT image synthesis MRI image model is a Dense-GAN model, which includes: a generation model and a discrimination model, the generation model includes a convolutional layer, a residual density block and upsampling, the discrimination model includes a convolutional layer, an activation function, a normalization layer and a density network; the residual density block includes a density network, a cascade network and a local residual network, and the density network is composed of a convolutional layer and an activation function;

[0016] Among them, the loss function of the Dense-GAN is:

[0017]

[0018] Among them, x f is the output of the generative model, x f =G(x i ), where x i is the CT image input; D = σ1(C), where σ1 is the sigmoid activation function and C is the discrimination model output; x r This is a real MRI image.

[0019] Furthermore, the registration dataset is input into a pre-built Dense-GAN for training to obtain an MRI image synthesis model, including:

[0020] Performing data expansion on the registration dataset, and normalizing the expanded registration dataset into an image set of 256×256 pixels;

[0021] Then the normalized image set is divided into training set, validation set and test set;

[0022] Furthermore, it also includes optimizing the cross entropy loss function using a validation set based on the output of the synthetic MRI image, and obtaining a validation model after optimization; testing the validation model using the test set, and if the test result does not meet the prediction probability threshold, retraining the generation model and the discrimination model, and if the test result meets the prediction probability threshold, obtaining the MRI image synthesis model.

[0023] Furthermore, the step of inputting the CT image into the MRI image synthesis model to obtain a synthesized MRI image includes:

[0024] Using a computer vision model to detect the input CT image, and determine whether the CT image is an organ image supported by the MRI image synthesis model;

[0025] If yes, the CT image is synthesized into the corresponding MRI image.

[0026] Based on the MCT feature and the LGP feature, it is determined whether the input CT image is an organ image supported by the MRI image synthesis model.

[0027] A system for synthesizing MRI images from CT images of brain metastases based on a residual density network comprises the following steps:

[0028] Dataset acquisition module: used to obtain multimodal CT-MRI paired datasets of patients with brain metastases;

[0029] Dataset preprocessing module: used to align and resample the data in the paired dataset;

[0030] Model training module: used to train the residual density-based MRI image generation model (Dense-GAN);

[0031] Image acquisition module: used to use Dense-GAN to obtain synthetic images from brain metastasis CT to MRI.

[0032] It can be seen from the above technical solutions that, compared with the prior art, the present invention provides a multimodal image conversion method and target area determination system for brain metastases based on residual density, which has the following beneficial effects:

[0033] (1) Improve the accuracy of brain metastasis target delineation:

[0034] This invention, through synthetic MRI, will provide doctors with a more comprehensive diagnostic basis. Synthetic MRI can overcome the limitations of CT and MRI image acquisition, providing more information for target delineation of brain metastases, improving target delineation accuracy and local tumor control rates, and providing patients with more timely and effective medical services.

[0035] (2) Reduce medical costs:

[0036] Synthesizing MRI images from CT images can reduce the need for MRI scans and lower healthcare costs. MRI scans are typically expensive, equipment resources are limited, and some patients are contraindicated for MRI scans. By reducing the number of MRI scans, medical resources and costs can be saved, making healthcare more sustainable. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0038] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0039] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0040] Multimodal imaging plays a crucial role in the precise delineation of brain metastasis target volumes. However, differences in information acquisition principles between different imaging modalities, such as CT and MRI, lead to significant errors in MRI-CT image fusion registration. Furthermore, some patients have contraindications to MRI scanning, thus limiting the application of multimodal imaging in the precise delineation of brain metastasis target volumes. To address this issue, the present invention synthesizes MRI images to provide more comprehensive and realistic multimodal imaging anatomical information of tumor lesions, thereby improving the accuracy of brain metastasis target delineation.

[0041] The present invention discloses a method for converting brain metastasis radiotherapy images based on deep learning, such as Figure 1 As shown, the following steps are included:

[0042] Obtain CT-MRI paired datasets of patients with brain metastases;

[0043] Perform multimodal registration and resampling of data in paired datasets;

[0044] The paired dataset was used to complete the training of a model (Dense-GAN) for synthesizing MRI images from CT images of brain metastases based on residual density.

[0045] The CT image is input into Dense-GAN to obtain a synthetic MRI image.

[0046] Specifically, the present embodiment includes the following steps:

[0047] (1) Multimodal imaging data collection: obtaining MRI and CT image data of the head of patients with brain metastases in the same position within a preset time range;

[0048] (2) Multimodal image data registration: A multimodal registration algorithm is used to register and resample the MRI image data and CT image data to obtain a pixel-level corresponding paired registration dataset;

[0049] (3) Model training and evaluation: Design and implement a residual density-based algorithm for synthesizing MRI images from CT images of brain metastases, including the design of the model architecture, preparation of training data, model training, and optimization. The trained model is trained by inputting the CT-MRI paired data of existing cases into a pre-built Dense-GAN model, and then the trained model is tested on a validation dataset to evaluate the effectiveness of the model.

[0050] Dataset collection: Acquire a large-scale multimodal CT-MRI paired dataset of 500-1000 cases to ensure the diversity and representativeness of the dataset, covering patient data of different diseases, ages, genders and ethnicities. At the same time, comply with ethical standards and protect patient privacy. These datasets should contain CT and MRI images acquired simultaneously, with relevant clinical labels and annotations. The inclusion criteria for collecting patient data are: age over 18 years old, regardless of gender; patients with radiotherapy for brain metastases with a clear clinical diagnosis in the past 5 years; positioning enhanced CT (with target area delineation and organ at risk delineation); the same patient has a cranial enhanced MRI within 2 weeks of the positioning CT, with no restrictions on the size, number, and location of brain metastases in T1+C, T2, FLASH, and DWI sequences. The hospital needs to be able to pass ethical standards and obtain imaging data, including Dicom original image data of positioning CT and MRI, tumor delineation data, and organ at risk delineation data.

[0051] Multimodal Image Data Registration and Resampling: Paired CT and MRI images are registered and resampled using traditional image registration algorithms and interpolation techniques to ensure spatial alignment and similar spatial features. MRI and CT image data are registered using a multimodal registration algorithm. The registered data is then verified for accuracy, with data with poor registration excluded. The resampled data is standardized to 512 x 512 pixel, 5mm slice thickness MRI-CT paired data, which represents a pixel-level one-to-one correspondence between the MRI and CT image data.

[0052] Dense-GAN model design: A model for synthesizing MRI images from brain metastases using a residual density network. Select an appropriate loss function and optimization algorithm, and perform appropriate hyperparameter adjustments. The Dense-GAN model includes a generative model and a discriminative model. The generative model includes convolutional layers, residual density blocks, and upsampling, while the adversarial network includes convolutional layers, activation functions, normalization layers, and a density network. The loss function of the Dense-GAN is:

[0053]

[0054] Among them, x f is the output of the generative model, i.e. x f=G(x i ), where x i is the CT image input; D = σ1(C), where σ1 is the sigmoid activation function and C is the discrimination model output; x r This is a real MRI image.

[0055] The residual density network includes a density network, a cascade network and a local residual network. The density network is composed of a convolutional layer and an activation function. The input and output relationship of each density network is:

[0056] F d,c =σ(W d,c [F d-1 F d，1 ,…,F d，c-1 ]);

[0057] Among them, σ is the ReLU activation function, W d，c is the weight of each convolutional layer; construct features before the activation function and introduce the perceptual loss function Lpercep, then the loss function of the generated network is

[0058]

[0059] Where L1 is the absolute distance between the synthetic MRI image and the real MRI image, that is, the absolute distance between the synthetic MRI image obtained by the CT image and the real MRI image obtained by pairing with the registration algorithm; λ and η are parameters used to balance different loss factors; the PSNR model is used to calculate the L1 loss function, and its calculation formula is as follows:

[0060] PSNR=20*log 10 (MAX I )-10*log 10 (MSE)

[0061]

[0062] Among them, MAXI is the maximum possible value of the pixel. Since the data will be normalized during the training process, the maximum value is 1. I and K represent the synthetic MRI image and the real MRI image, respectively.

[0063] Model training and evaluation: The Dense-GAN model is trained, cross-validated, and evaluated using a paired CT-MRI dataset. The quality and accuracy of the MRI images synthesized by the model are evaluated and compared with real MRI images. Based on the output model, the cross-entropy loss function is optimized using the validation set to obtain a validation model. The validation model is then tested using the test set. If the test result does not meet the prediction probability threshold, the generative and discriminative models are retrained. If the test result meets the prediction probability threshold, the MRI image synthesis model is obtained.

[0064] Model deployment and application: The CT image is input into the MRI image synthesis model to obtain a synthetic MRI image. The MRI and CT data are preprocessed and verified to implement the synthetic MRI function and guide the target volume delineation of brain metastases. Previous brain metastasis target volume delineation cases are collected to evaluate the differences between synthetic MRI-guided target volume delineation and MRI-CT registration and fusion-guided target volume delineation to further verify the accuracy and feasibility of synthetic MRI-guided target volume delineation.

[0065] In this embodiment, synthesizing MRI images from CT images involves using deep learning algorithms to create synthetic images that resemble real MRI images. This synthesis technology has widespread applications in medical research and clinical practice. However, CT and MRI are two different medical imaging technologies, and their correlation is limited. Therefore, synthesizing MRI images is a relatively complex problem, and currently no mature solution has been found in academia.

[0066] Finally, this embodiment discloses a system for synthesizing MRI images from CT images of brain metastases based on a residual density network, which includes the following steps:

[0067] Dataset acquisition module: used to obtain multimodal CT-MRI paired datasets of patients with brain metastases;

[0068] Dataset preprocessing module: used to align and resample the data in the paired dataset;

[0069] Model training module: used to train the residual density-based MRI image generation model (Dense-GAN);

[0070] Image acquisition module: used to use Dense-GAN to obtain synthetic images from brain metastasis CT to MRI.

[0071] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0072] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for synthesizing MRI images from CT images of brain metastases based on residual density networks, characterized in that: The following steps are involved: Acquire MRI image data and CT image data of the head of a patient with brain metastasis in the same position within a preset time range; The MRI image data and CT image data are registered and resampled to obtain a pixel-level corresponding paired registration data set; Using the CT-MRI registration dataset, a Dense-GAN model based on residual density network is constructed to synthesize MRI images based on CT images; Using the Dense-GAN model, synthetic images from brain metastases CT to MRI are obtained.

2. The method for synthesizing MRI images from CT images of brain metastases based on residual density network according to claim 1, characterized in that: The registered dataset has associated clinical labels and annotations.

3. The method for synthesizing MRI images from CT images of brain metastases based on residual density network according to claim 1, characterized in that: The MRI image data and CT image data are registered and resampled specifically as follows: the images of the two different modalities are spatially aligned to ensure that the corresponding structures and anatomical features are in the same position; the sampling rate and pixel spacing of the image are adjusted to match the input requirements of the model.

4. The method for synthesizing MRI images from CT images of brain metastases based on residual density network according to claim 1, characterized in that: After registration, the pixel levels of the CT image data and the MRI image data of the CT-MRI paired data are in a one-to-one correspondence.

5. The method for synthesizing MRI images from CT images of brain metastases based on residual density network according to claim 1, characterized in that: The method for synthesizing MRI images from CT images of brain metastases is based on a Dense-GAN model with a residual density network. The Dense-GAN model includes a generative model and a discriminative model. The generative model includes a convolutional layer, a residual density block, and upsampling. The discriminative model includes a convolutional layer, an activation function, a normalization layer, and a density network. The residual density block includes a density network, a cascade network, and a local residual network. The density network is composed of a convolutional layer and an activation function.

6. The method for synthesizing MRI images from CT images of brain metastases based on residual density network according to claim 5, characterized in that: The loss function of the Dense-GAN is: Among them, x f is the output of the generative model, x f =G(x i ), where x i is the CT image input; D = σ1(C), where σ1 is the sigmoid activation function and C is the discrimination model output; x r This is a real MRI image.

7. A system for synthesizing brain metastasis CT images into MRI images based on residual density network, characterized in that: The following steps are involved: Dataset acquisition module: used to acquire MRI image data and CT image data of the head of patients with brain metastases in the same position within a preset time range; Dataset preprocessing module: used to align and resample MRI image data and CT image data to obtain pixel-level corresponding paired registration dataset; Model training module: used to train the MRI image generation model Dense-GAN based on the residual density network; Image acquisition module: used to use Dense-GAN to obtain synthetic images from brain metastasis CT to MRI.

Images