Pseudo-CT synthesis method based on multi-modal MRI simultaneous generation

Abstract

The present invention relates to a pseudo-CT synthesis method based on the synchronous generation of multi-modal MRI, and belongs to the field of medical image processing. The method takes a single modality MRI as input and outputs multiple other modal MRIs and pseudo-CTs at the same time. The specific steps include: obtaining a single modality MRI, inputting a preset generator, and the generator consisting of a shared encoder and J decoders; first obtaining MRIs of multiple modalities of the same period generated by the first J-1 decoders, and fusing the shared encoder and the middle layer features of the first J-1 decoders to obtain fusion features; further fusing the J-th decoder and the fusion features to obtain a pseudo-CT image, where J is a natural number. The present invention can realize the pseudo-CT synthesis of one-to-many multi-modal MRI synchronous generation, which can solve the problems of difficult and high cost in clinical multi-modal image acquisition.

Description

Technical Field

[0001] The present disclosure relates to the field of medical image processing, and in particular to a pseudo-CT synthesis method based on synchronous generation of multi-modal MRI. Background Art

[0002] MRI-guided online adaptive radiotherapy (MRIgOART) using MRL as a carrier can significantly improve the clinical benefits of cancer patients, increasing the two-year survival rate of pancreatic cancer patients by 19% and reducing the recurrence rate of breast cancer patients by 20%. It is hailed as another milestone technological advancement in the history of radiotherapy.

[0003] MRIgOART uses MR images acquired before or during fractionated treatment as its primary input source. Based on this input, it conducts workflows including region of interest delineation, plan design, verification, and execution. The plan design process simulates dose deposition to patients under different treatment conditions, specifies plan optimization objectives, and inversely solves for the treatment conditions that should be set, directly determining the quality of the patient's treatment plan. Unlike traditional CT-based planning, MRIgOART, provided only with MR images, can only reflect the proton density and magnetic relaxation properties of tissue (i.e., the inertia of the dipole moment), but not the electron density of tissue, making it unsuitable for direct dose calculation during radiotherapy planning. Obtaining electron density information (i.e., synthetic CT, sCT, or pseudo-CT) from the available MR images is a prerequisite for successful dose calculation. Current commercial methods use a strategy based on tissue segmentation and assignment, resulting in low accuracy. Existing research methods often employ simplified one-to-one (single-modality MRI directly to sCT) or many-to-one (multi-modality MRI to sCT) approaches. For example, CN202110117854.8 discloses an image generation system that generates pseudo-CT from multi-sequence MR based on deep learning, and uses the complementary features between different modalities in multimodal MRI to improve the quality of pseudo-CT generation. However, due to limited scanning time and expensive MRI costs, it is not economical to obtain multi-sequence MRI images of the same patient in clinical scenarios. Therefore, designing a new method for generating pseudo-CT from a certain easily accessible MRI modality, but with the same effect as multi-sequence modalities, has important clinical and research value. Summary of the Invention

[0004] The purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and propose a pseudo-CT synthesis method for radiotherapy plans based on the simultaneous generation of multi-modal MRI, so that the synthesis quality of pseudo-CT can be further enhanced while generating multiple modal MRIs.

[0005] To achieve the above objectives, the technical solutions of the present invention are as follows.

[0006] In a first aspect, a method for synthesizing pseudo-CT of a radiotherapy plan based on simultaneous generation of multimodal MRI is provided, the method comprising the following steps:

[0007] Obtain a single modality MRI and input it into a preset generator, which consists of a shared encoder and J identical decoders. The output features of the shared encoder based on the single modality MRI are the shared input features of the J decoders;

[0008] Fuse the intermediate layer features corresponding to the shared encoder and the first J-1 decoders to obtain multi-scale fusion features;

[0009] The first J-1 decoders are used to generate MRIs of multiple modalities at the same time based on shared input features, and then the J-th decoder is used to generate pseudo CT images based on multi-scale fusion features and shared input features, where J is a natural number.

[0010] The above technical solution realizes deep feature expression mapping between different MRI modalities through a shared encoder and a non-shared decoder to achieve high-quality generation effects, realize one-to-many multi-modality MRI and pseudo-CT generation, and solve the problems of difficulty and high cost in clinical multimodal image acquisition.

[0011] In the above technical solution, the shared encoder includes M convolutional layers and N downsampling layers; the decoder includes N upsampling layers and M convolutional layers, where M and N are set values ​​and are natural numbers.

[0012] In one implementation of the above technical solution, the convolution layer performs convolution, normalization and activation operations in sequence; the downsampling layer performs convolution, normalization, activation and pooling operations in sequence; and the upsampling layer performs deconvolution, normalization and activation operations in sequence.

[0013] In the above technical solution, while generating multiple modal MRIs, the deep features of multiple modal MRIs are fused to fully utilize the information of different modalities. That is, the intermediate layer features corresponding to the shared encoder and the first J-1 decoders are fused to obtain multi-scale fused features. The specific implementation steps include:

[0014] Obtain N downsampled features based on the shared encoder and record them as downsampled features Down in order i , i=N,N-1,…,2;

[0015] Use the jth decoder to obtain N up-sampled features, which are recorded as up-sampled features Up in order jk ,j=1,2,…,J-1,k=2,3,…,N;

[0016] From k=2 onwards, the downsampled features are Down k , Upsampling feature Up jk Fusion, obtaining fusion features

[0017] In the above technical solution, the Jth decoder generates a pseudo CT image based on multi-scale fusion features and shared input features, including:

[0018] Get the upsampled feature Up of the Jth decoder Jk , and fusion features Fusion, obtain the k-th upsampled features of the J-th decoder, k = 2, 3, ..., N;

[0019] The Nth upsampled feature of the Jth decoder is processed by M convolutional layers to output a pseudo CT image, thereby achieving the accuracy and stability of MR-CT cross-modal mapping and establishing an effective and practical MR-CT cross-modal mapping model and technology.

[0020] In the above technical solution, a generator is preset, and generative adversarial learning is used as the model framework, which is obtained through training. The training steps include:

[0021] Build a corresponding discriminator for the generator, and combine the generator and discriminator to form a training model;

[0022] Acquire multimodal datasets of patients, including multiple CT images of the same site and multimodal MRI acquired at the same time, and preprocess the data;

[0023] Pre-align CT images and multi-modality MRI, and then perform affine and deformable registration of the images using a symmetric normalized registration method with mutual information as the optimization metric;

[0024] After registration, the CT images and multi-modal MRI were normalized to the maximum and minimum values;

[0025] Input a single modality MRI to train the one-to-many architecture generator model to obtain multiple modalities of MRI and pseudo-CT, with image value ranges ranging from [0, 1];

[0026] The obtained MRI and pseudo-CT of multiple modalities are simultaneously input into the corresponding discriminator, and the sum of the probability that the discriminator judges the real image as real and the probability that the discriminator judges the fake image generated by the generator as fake is calculated as the adversarial loss;

[0027] When the training reaches the specified number of times or the total loss reaches the specified threshold, the training is stopped, the weight parameters of the model at this time are saved, and the generator at this time is used as the preset generator.

[0028] In the above technical solution, through iteration, the generator's generated images are made as close to real images as possible, while the discriminator's ability to distinguish between real images and fake ones becomes increasingly stronger, thus achieving a game of chess. Once the model reaches equilibrium, the discriminator is no longer able to distinguish between real and generated images. During the prediction phase, the discriminator is no longer needed; only the generator is used.

[0029] As a further improvement to the above technical solution, the preset generator further sets generation loss, structural similarity loss, and perceptual loss during training to further improve the accuracy of the model-generated images, and uses the sum of adversarial loss, generation loss, structural similarity loss, and perceptual loss as the total loss of training. When the training reaches a specified number of times or the loss reaches a specified threshold, the training is stopped, the weight parameters of the model at this time are saved, and the generator at this time is used as the preset generator.

[0030] in:

[0031] Adversarial loss, which is the sum of the probability that the discriminator judges a real image as real and the probability that the discriminator judges a fake image generated by the generator as fake;

[0032] Generative loss, used to penalize the difference between the generated image and the true value loss;

[0033] Structural similarity loss, used to prevent similarity loss of generator structure variation;

[0034] Perceptual loss is used to calculate the distance between the generated image and the real image in the feature space of human perception.

[0035] As a further improvement of the above technical solution, the method maps the multiple modal MRIs and pseudo-CTs generated by the generator back to their original value ranges, and restores the images to their original size for comparison with real images to evaluate the generation results.

[0036] In a second aspect, the present invention provides a computer-readable storage medium storing a computer program that can be loaded by a processor and execute any one of the above methods.

[0037] In a third aspect, the present invention proposes a pseudo-CT synthesis system for radiotherapy plans based on simultaneous generation of multimodal MRI, the system comprising the following modules:

[0038] An input module, configured to acquire a single modality MRI and input a preset generator;

[0039] The generator consists of a shared encoder and J identical decoders. The output features of the shared encoder based on a single modality MRI are the shared input features of the J decoders.

[0040] A fusion module is configured to fuse the intermediate layer features corresponding to the shared encoder and the first J-1 decoders to obtain multi-scale fusion features;

[0041] The acquisition module is configured to use the first J-1 decoders to generate MRIs of multiple modalities at the same time based on shared input features, and then use the J-th decoder to generate a pseudo CT image based on multi-scale fusion features and shared input features, where J is a natural number. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 , is a schematic diagram of a model framework flow chart of a method for synthesizing pseudo-CT of radiotherapy plans generated synchronously based on multi-modal MRI in one embodiment;

[0043] Figure 2 , is a schematic diagram of a data preprocessing process in one embodiment;

[0044] Figure 3 , is a schematic diagram of a network structure of a pseudo-CT synthesis model for radiotherapy planning generated synchronously based on multi-modal MRI in one embodiment;

[0045] Figure 4 , Comparison diagram of the generated results of the multimodal generation model according to an embodiment of the present invention and the real image DETAILED DESCRIPTION

[0046] The method of the present invention is further described in detail below with reference to the following examples. However, the present invention is not limited to the types and number of applicable MRI modalities, nor is it limited to applicable deep learning methods. Furthermore, applicable medical imaging modalities are not limited to MRI and CT. Various substitutions and modifications based on technical knowledge and customary practices in the art, without departing from the above-described technical principles of the present invention, are intended to be within the scope of the present invention.

[0047] Implementation Cases:

[0048] (1) Acquire a multimodal image dataset, including multiple CT images of the same part and multimodal MRI images obtained at the same time, and preprocess the data.

[0049] (1.1) Acquire the patient's multimodal dataset. First, collect the patient's CT images and multimodal MRI images acquired at the same location and during the same period.

[0050] This study included 50 patients with nasopharyngeal carcinoma (NPC). Four MRI modalities (T1, T1c, T1Dixon C_water, and T2) and corresponding CT images were obtained. MRI was acquired on a 3.0T MR simulator (Ingenia 3.0, Philips Medical Systems, The Netherlands). CT images were acquired on a Siemens scanner (Somatom Definition Edge, Erlangen, Germany). MRI and CT images were acquired on the same day for each patient using the same setup (including patient-specific immobilization).

[0051] (1.2) Preprocess the data.

[0052] Because the acquisition ranges of MRI and CT images do not completely coincide, registration is required. CT images are segmented using thresholding and morphological methods to remove bed position information. Pixels representing internal body structures are set to 1, while pixels not belonging to internal body structures are set to 0, resulting in a body mask image for the CT image. The CT body mask image serves as the moving image for registration, and the MRI image serves as the fixed image for registration. The registration algorithm uses the SyN algorithm from the ANTs registration tool. The SyN algorithm, also known as symmetric normalization, includes affine and deformable transformations and uses mutual information as an optimization metric. To maintain the in-plane resolution of the original images and reduce GPU training costs, the aligned images are cropped to a uniform size of 256×256. This minimizes background areas while retaining the region of interest (ROI) in the center of the brain and neck images. For the shoulder region, after removing the background at the edges, the ROI is cropped into two 256×256 tiles. During the prediction phase, the slice_size of all predicted results will be restored to 500×500, and the small blocks in the shoulder area will be combined and restored to the original slice_size of 500x500, and the overlapping areas will be represented by the average value. (See Appendix Figure 2 )

[0053] (1.2) In the experiment, the data was divided into three parts: training, validation, and test data in a ratio of 7:1:2, with the training, validation, and test sets consisting of 35, 5, and 10 patients, respectively. The model was trained on 2D data, with each image size being 256x256. Therefore, the training set contained 3974 2D images, the validation set contained 503 2D images, and the test set contained 1073 2D images. The T1 modality of MRI is widely used in clinical analysis due to its strong tissue contrast and suitability for observing anatomical structures. Therefore, this work uses the T1 modality as input. All MRI and CT images were normalized to [0, 1] using maximum-minimum normalization.

[0054] (2) Constructing a multimodal generative model based on deep learning, the multimodal generative model includes a one-to-many structured generator model and a corresponding multi-discriminator model such as Figure 3 As shown in , the generator consists of a shared encoder and J decoders with the same structure, where J is a natural number.

[0055] (2.1) Constructing a one-to-many structured generator model in the multimodal generative model. First, construct the shared encoder portion of the generator model, which includes M convolutional layers and N downsampling layers; the decoder includes N upsampling layers and M convolutional layers, where M and N are set values ​​and are natural numbers. The convolutional layers include a sequential sequence of convolution, normalization, and activation layers; the downsampling layers include a sequential sequence of convolution, normalization, activation, and pooling layers.

[0056] The shared encoder inputs a single modality MRI, and its output features are the shared input features of J decoders. In order to better preserve the common image features of single modality MRI and other modalities and CT images, the output features of the middle layer of the shared encoder are connected to the corresponding fusion module as fusion features. Figure 3 In the shared encoder, N down-sampled features are obtained. They are recorded as down-sampled features Down in order. i , i=N, N-1,…,2, these downsampled features are used for subsequent feature fusion.

[0057] (2.2) Construct the multi-branch non-shared decoder in the one-to-many structure generator model. The basic structure of all non-shared decoders is the same, including upsampling and convolution layers in sequence. Upsampling includes deconvolution, normalization and activation layers. Among them, the last decoder is added with a fusion module as a decoder for synthesizing pseudo-CT, and the other decoders are used to generate MRI of other modalities. The features obtained in the middle of the decoder for generating the MRI modality are connected to the fusion module in the decoder for synthesizing pseudo-CT as fusion features. In the schematic diagram, Figure 3 In the example, the jth decoder obtains N upsampled features, which are recorded as upsampled features Up in order. jk , j=1, 2,…, J-1, k=2, 3,…, N; J is a natural number.

[0058] (2.3) In the branch of synthetic pseudo-CT, a fusion module in a multi-branch non-shared decoder is constructed. The intermediate features of the shared encoder and the multi-branch decoder that generates other modalities of MRI are output through the fusion module and connected to the corresponding layer of the synthetic pseudo-CT decoder to achieve multi-modal and multi-scale feature fusion. The fusion module is composed of two branches. One branch uses global average pooling to extract the attention of global features, and the other branch directly uses point convolution to extract the channel attention of local features. The features obtained from the two branches are added and then passed through the Sigmoid activation layer. Finally, the activated feature map is multiplied with the input. In the schematic diagram Figure 3 In the example, starting from k=2, the downsampled features are Down k , Upsampling feature Up jk Fusion, obtaining fusion features j=1,2,…,J-1,k=2,3,…,N。 The Jth decoder and the fusion feature are further fused to obtain a pseudo CT image, including: obtaining the up-sampled feature Up of the Jth decoder Jk , and fusion features Fusion is performed to obtain the kth up-sampled feature of the Jth decoder, where k = 2, 3, ..., N. The Nth up-sampled feature of the Jth decoder is processed through M convolutional layers to output a pseudo CT image.

[0059] (2.4) As an improvement to the model, several residual modules are connected after the shared encoder, and attention modules and several residual modules are used at the input of each decoder branch to improve the accuracy of the generator in learning one modality to generate other modalities. In this embodiment, the number of residual modules connected after the shared encoder is three, and the attention module is also connected after three residual modules.

[0060] (2.5) Constructing the discriminator model in the multimodal generative model, first constructing the convolution layer and downsampling layer of the discriminator, the convolution layer includes convolution, normalization and activation layers in sequence; the downsampling includes convolution, normalization, activation and pooling layers in sequence.

[0061] (2.6) Construct a training framework. First, input a single-modality MRI image to train the one-to-many architecture generator model. This generates multi-modality MRI and pseudo-CT images, all with image values ​​in the range [0, 1]. Next, train the discriminator model. When a single-modality MRI image and a real image are simultaneously input to the corresponding discriminator, the discriminator outputs a True result. When the generated image and the corresponding real image are simultaneously input to the corresponding discriminator, the discriminator outputs a False result.

[0062] (2.7) Construct a training strategy with a single shared encoder and multiple decoder branches. First, the input single modality MRI is represented as The target modal data is expressed as It contains multiple modal MRI and CT images, and the generated results of the model are represented as γ.

[0063] In each iteration, the adversarial loss is calculated: the generator and discriminator in the multimodal generative model are trained adversarially during the training process, and the adversarial loss function is As shown below:

[0064]

[0065] in, represents the discriminator corresponding to the i-th target modality, is the generated result of the i-th decoder. The patch-based discriminator used here maps the input to a real number, which is the probability that the input sample is a real sample. However, the patch-based discriminator maps the input to an N×N feature map (N is 8 in this example). Each patch in the feature map represents the probability of being a real sample. From this feature map, we can trace back to a specific position in the original image, and see how this position affects the final output. is the square of the binormal of the feature map that the discriminator predicts as fake for the generated image, It is the square of the two-norm of the feature map predicted by the discriminator for the real image.

[0066] During the training process, the generator and the discriminator are trained alternately. For the discriminator, the ability to predict the generated image as false and the real image as true is stronger, so To constrain the discriminator; for the generator, the generated image should be able to deceive the discriminator, that is, the discriminator’s ability to predict the generated image as false becomes weaker, so it is necessary to use to constrain the generator training phase; the final goal of the adversarial loss function is

[0067] The training strategy is further improved. The generation loss, structural similarity loss, and perceptual loss are further set during training. The sum of the adversarial loss, generation loss, structural similarity loss, and perceptual loss is used as the total loss of training. After training for a specified number of times or several times, when the verification loss no longer decreases in the verification set, the training is stopped and the model weight parameters at the lowest loss are saved. The generator at this time is used as the preset generator.

[0068] in:

[0069] (2.7.1) Generation loss: In order to train the generator, we introduce a The loss is used to penalize the difference between the generated image and the real image:

[0070]

[0071] (2.7.2) Structural Similarity Loss: In order to improve the structural similarity and contrast of the model, similarity loss is used to constrain the output structure and prevent the generator from generating structural variations. Structural Similarity Loss SSIM loss is as follows:

[0072]

[0073] in are the means of the real image and the model prediction results, are the standard deviations of the real image and the model prediction results, respectively. C1 and C2 are constants, which are set to 10 in one embodiment. -5 and 9x10 -5

[0074] (2.7.3) Perception loss: To prevent distortion in the actual appearance of the generated image, we use perception loss. Perception loss calculates the distance between the generated image and the real image in the feature space of human perception, rather than in the pixel space. The formula is as follows:

[0075]

[0076] Where: φ is the pre-trained VGG-19 network extractor, w, h, d represent the width, height and depth of the selected feature layer respectively. Indicates that a certain layer of features from VGG-19 is selected to calculate the loss. Since VGG-19 uses color images as input, and medical images are often grayscale, the input single-modality MRI is copied and spliced ​​as the input of VGG-19.

[0077] (2.8) Using test data independent of the training data, a single modality MRI is input into the trained generator model to directly generate MRIs of other modalities and synthesize pseudo CT.

[0078] (2.9) In evaluation mode, test data is input into the trained model to generate multiple modalities of MRI and pseudo-CT images and the images are mapped back to the original range. At the same time, the images are restored to their original size and compared with the real images to evaluate the generated results. (See Appendix Figure 4 )

[0079] (3) Obtain application data, input a single modality MRI into the trained generator model, directly generate MRIs of other multiple modalities and synthesize pseudo CT.

[0080] After the above-mentioned deep learning-based multimodal generation model is trained, the generator model can be directly applied to generate other modal MRIs based on a single modality MRI and synthesize pseudo-CT, solving the problem of difficulty and high cost in acquiring clinical multimodal images.

[0081] From the above examples, it can be seen that the method of the present invention achieves one-to-many multi-modality MRI and pseudo-CT generation, solving the problems of difficulty and high cost in clinical multi-modality image acquisition. Using generative adversarial learning as the model framework, a shared encoder and a non-shared decoder are constructed to realize deep feature expression mapping between different MRI modalities to achieve high-quality generation effects. While generating multiple modal MRIs, the deep features of multiple modal MRIs are fused to fully utilize the information of different modalities and reduce the uncertainty of MRI to CT mapping. Ultimately, the accuracy and stability of MR-CT cross-modality mapping are achieved, and an effective and practical MR-CT cross-modality mapping model and technology are established.

[0082] Through the above description of the embodiments, those skilled in the art will clearly understand that the present disclosure can be implemented using software plus necessary general-purpose hardware. Of course, it can also be implemented using dedicated hardware, including application-specific integrated circuits, dedicated CPUs, dedicated memories, and dedicated components. Generally speaking, any function performed by a computer program can be easily implemented using corresponding hardware. Moreover, the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits, or dedicated circuits. However, for the present disclosure, software implementation is often the preferred embodiment.

[0083] In some embodiments, the above method is implemented as a pseudo-CT synthesis system for radiotherapy plans generated based on multimodal MRI synchronization, and the system includes the following modules:

[0084] An input module, configured to acquire a single modality MRI and input a preset generator;

[0085] The generator consists of a shared encoder and J identical decoders. The output features of the shared encoder based on a single modality MRI are the shared input features of the J decoders.

[0086] A fusion module is configured to fuse the intermediate layer features corresponding to the shared encoder and the first J-1 decoders to obtain multi-scale fusion features;

[0087] The acquisition module is configured to use the first J-1 decoders to generate MRI of multiple modalities of the same period based on the shared input features, and then use the J-th decoder to generate a pseudo CT image based on the multi-scale fusion features and the shared input features, where J is a natural number.

[0088] Although the embodiments of the present invention have been described above with reference to the accompanying drawings, the present invention is not limited to the above-mentioned specific embodiments and application fields. The above-mentioned specific embodiments are merely illustrative and instructive, and are not restrictive. A person skilled in the art, guided by this specification and without departing from the scope of protection of the claims of the present invention, may also devise various forms, all of which fall within the scope of protection of the present invention.

Claims

1. A pseudo-CT synthesis method based on multi-modal MRI synchronous generation, characterized in that: The method comprises the following steps: Obtain a single modality MRI and input it into a preset generator; wherein: the generator consists of a shared encoder and J decoders of the same structure. The output features of the shared encoder based on the single modality MRI are shared input features of the J decoders. The shared encoder includes M convolutional layers and N downsampling layers. A residual module is connected after the shared encoder. The decoder includes N upsampling layers and M convolutional layers. An attention module and a residual module are used at the input of each decoder branch. M and N are set values ​​and are natural numbers. First, the first J-1 decoders are used to generate MRI of multiple modalities at the same time based on the shared input features, and then the intermediate layer features corresponding to the shared encoder and the first J-1 decoders are fused to obtain multi-scale fusion features, including: obtaining N down-sampled features based on the shared encoder, and recording them as down-sampled features in order , i=N, N-1,…,2; use the j-th decoder to obtain N upsampled features, which are recorded as upsampled features in order , j=1,2,…,J-1, k=2,3,…,N; starting from k=2, the downsampled features , upsampled features Fusion, obtaining fusion features , j=1,2,…,J-1 ,k=2,3,…,N; Generate a pseudo CT image using the Jth decoder based on multi-scale fusion features and shared input features, where J is a natural number, including: Obtaining the up-sampled features of the Jth decoder , and fusion features Fusion is performed to obtain the kth up-sampled feature of the Jth decoder, where k = 2, 3, ..., N. The Nth up-sampled feature of the Jth decoder is processed through M convolutional layers to output a pseudo CT image.

2. The method according to claim 1, characterized in that The preset generator is obtained through training. The training steps include: Build a corresponding discriminator for the generator, and combine the generator and discriminator to form a training model; Acquire multimodal datasets of patients, including multiple CT images of the same site and multimodal MRI acquired at the same time, and preprocess the data; Pre-align CT images and multi-modality MRI, and then perform affine and deformable registration of the images using a symmetric normalized registration method with mutual information as the optimization metric; After registration, the CT images and multi-modal MRI were normalized to the maximum and minimum values; Input a single modality MRI to train a one-to-many architecture generator model, and obtain multiple modal MRI and pseudo-CT images, with the image value range in [0,1]; The obtained MRI and pseudo-CT of multiple modalities are simultaneously input into the corresponding discriminator, and the sum of the probability that the discriminator judges the real image as real and the probability that the discriminator judges the fake image generated by the generator as fake is calculated as the adversarial loss; When the training reaches the specified number of times or the total loss reaches the specified threshold, the training is stopped, the weight parameters of the model at this time are saved, and the generator at this time is used as the preset generator.

3. The method according to claim 2, characterized in that The preset generator further sets generation loss, structural similarity loss, and perceptual loss during training, and uses the sum of adversarial loss, generation loss, structural similarity loss, and perceptual loss as the total loss of training. When the training reaches the specified number of times or the loss reaches the specified threshold, the training is stopped, the weight parameters of the model at this time are saved, and the generator at this time is used as the preset generator; Among them: Generate loss, which is used to punish the difference between the generated image and the true value loss; Structural similarity loss, used to prevent similarity loss of generator structure variation; Perceptual loss is used to calculate the distance between the generated image and the real image in the feature space of human perception.

4. The method according to claim 2, wherein: The method maps multiple modality MRI and pseudo CT images generated by the generator from the value range [0,1] back to the original value range, and restores the images to their original size for comparison with real images to evaluate the generated results.

5. A computer-readable storage medium, characterized in that: A computer program is stored which can be loaded by a processor and execute the method according to any one of claims 1 to 4.

6. A pseudo-CT synthesis system for radiotherapy plans based on simultaneous generation of multimodal MRI, characterized by: The system includes the following modules: The input module is configured to obtain a single modality MRI and input a preset generator, wherein: the generator consists of a shared encoder and J identical decoders, the output features of the shared encoder based on the single modality MRI are shared input features of the J decoders, the shared encoder includes M convolutional layers and N downsampling layers, and a residual module is connected after the shared encoder. The decoder includes N upsampling layers and M convolutional layers, and uses an attention module and a residual module at the input end of each decoder branch. M and N are set values ​​and are natural numbers; The fusion module is configured to fuse the intermediate layer features corresponding to the shared encoder and the first J-1 decoders to obtain multi-scale fusion features, including: obtaining N downsampled features based on the shared encoder, and recording them in order as downsampled features , i=N, N-1,…,2; use the j-th decoder to obtain N upsampled features, which are recorded as upsampled features in order , j=1,2,…,J-1, k=2,3,…,N; starting from k=2, the downsampled features , upsampled features Fusion, obtaining fusion features , j=1,2,…,J-1 ,k=2,3,…,N; The acquisition module is configured to use the first J-1 decoders to generate MRI of multiple modalities at the same time based on the shared input features, and then use the J-th decoder to generate a pseudo CT image based on the multi-scale fusion features and the shared input features, where J is a natural number, including: obtaining the upsampling features of the J-th decoder , and fusion features Fusion is performed to obtain the kth up-sampled feature of the Jth decoder, where k = 2, 3, ..., N. The Nth up-sampled feature of the Jth decoder is processed through M convolutional layers to output a pseudo CT image.

Images