Nasopharynx cancer pseudo-CT generation method based on multi-modal deformation registration and Transform network

Abstract

The invention relates to the technical field of medical image processing, in particular to a nasopharyngeal carcinoma pseudo CT generation method based on multi-modal deformation registration and a Transform network, and the method comprises the steps: firstly collecting CT and MRI images of a nasopharyngeal carcinoma patient confirmed by pathology, and carrying out the preprocessing of the CT and MRI images; secondly, establishing a deformation registration network based on a CycleFCNs model, inputting the CT image as a fixed image and the MRI image as a floating image, outputting a space deformation transformation field, and performing deformation transformation on the MRI image; then, a decoder is added on the basis of the original Transform model, a loss function is set, and the weight is adjusted; the image subjected to deformation registration is used as training data to be input into the improved Transform model for training optimization; and finally, generating a pseudo-CT image from the to-be-processed MRI image by using the trained model, and performing imagery evaluation by using various evaluation indexes. The accuracy and quality of pseudo CT generation are improved, the radiation dose of a patient is reduced, and the method has important clinical application value in the field of nasopharyngeal carcinoma radiotherapy.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a method for generating pseudo CT of nasopharyngeal carcinoma based on multimodal deformation registration and Transformer network. Background Art

[0002] In the field of tumor treatment, the incidence of nasopharyngeal carcinoma is high, and radiotherapy is its key treatment method. During radiotherapy, accurate radiotherapy plan design is crucial to improving treatment efficacy and reducing damage to surrounding normal tissues. CT images have always been the basis for radiotherapy plan design because they can clearly display the anatomical structure and density information of human tissues.

[0003] However, there are some issues with CT scanning that cannot be ignored. First, CT examinations expose patients to a certain dose of radiation, which poses a potential risk to the patient's physical health, especially for patients who need multiple scans, the cumulative radiation dose may be higher. Secondly, in actual clinical practice, some situations may make it difficult to obtain high-quality CT images. For example, the patient's physical condition may not be able to tolerate multiple CT scans, or due to equipment failure, patient movement, etc., the CT image may have artifacts or poor quality.

[0004] At the same time, MRI images have many unique advantages. They can provide very high soft tissue contrast, which is of great value in showing the boundaries, size, and relationship of tumors to surrounding tissues. Moreover, MRI examinations do not cause radiation hazards and are relatively safe for patients. However, the physical properties of MRI images determine that they cannot be directly used for dose calculation in radiotherapy planning.

[0005] In recent years, with the rapid development of deep learning technology, it has shown great potential in the field of medical image processing. Deep learning methods can automatically learn the feature representation of images from a large amount of data, thereby achieving tasks such as image classification, segmentation, and generation. In the field of medical imaging, many studies have tried to use deep learning technology to solve various practical problems, such as disease diagnosis, image registration, etc.

[0006] In this context, using MRI images to generate high-quality pseudo-CT images has become a research hotspot. By converting MRI images into pseudo-CT images, the advantages of MRI images can be fully utilized while meeting the needs for CT images in radiotherapy plan design. Based on the above background, the present invention proposes an innovative pseudo-CT generation method for nasopharyngeal carcinoma based on multimodal deformation registration and Transformer network, aiming to overcome the shortcomings of the existing technology, improve the generation quality and accuracy of pseudo-CT images, provide more reliable support for the radiotherapy plan design of nasopharyngeal carcinoma, and thus improve the treatment effect and quality of life of patients. Summary of the invention

[0007] The purpose of the present invention is to provide a nasopharyngeal carcinoma pseudo CT generation method based on multimodal deformation registration and Transformer network to solve the problems raised in the above background technology.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] A method for generating pseudo-CT images of nasopharyngeal carcinoma based on multimodal deformation registration and Transformer network comprises the following steps:

[0010] Step S1, data collection and preprocessing;

[0011] Step S1-1, collecting multimodal imaging data of patients with nasopharyngeal carcinoma confirmed by pathology, the multimodal imaging data including CT images and MRI images;

[0012] Step S1-2, preprocessing the CT image and the MRI image, the preprocessing including image standardization, image resampling, image cropping and image normalization;

[0013] Step S2, multi-modal deformation registration;

[0014] Step S2-1, establish a deformable registration network based on the CycleFCNs model, and the deformable registration network is built based on Python3 and Pytorch deep learning framework;

[0015] Step S2-2, setting a loss function, the loss function includes a modality-independent neighborhood descriptor target loss term, an L2 norm regularization loss term, and a cycle loss term, taking the CT image as a fixed image and the MRI image as a floating image, inputting them into a deformable registration network, and outputting a spatial deformation transformation field;

[0016] Step S2-3, using the spatial deformation transformation field to perform deformation transformation on the MRI image, to obtain a deformed MRI image having the same anatomical structure as the CT image;

[0017] Step S3, improving the Transformer model;

[0018] Step S3-1: Based on the original Transformer model, a decoder is added to restore the encoded features into a pseudo CT image;

[0019] Step S3-2, setting a loss function, where the loss function includes a MIND loss function and a mean square error MSE loss function;

[0020] Step S3-3, extracting the human body outer contour mask of the CT image, setting the loss function weight value of the area inside the mask to a high value, and setting the loss function weight value of the area outside the mask to a low value;

[0021] Step S4: model training and optimization;

[0022] Step S4-1, the deformable MRI image and the corresponding CT image after deformation registration are used as training data and input into the improved Transformer model for training;

[0023] Step S4-2, calculating the loss function value between the pseudo CT image output by the Transformer model and the real CT image, and updating the model parameters through the back propagation algorithm to optimize the Transformer model;

[0024] Step S5, pseudo CT generation and evaluation;

[0025] Step S5-1, input the MRI image to be processed into the trained Transformer model to generate a pseudo CT image;

[0026] Step S5-2, using a variety of evaluation indicators to perform image evaluation on the generated pseudo CT image, the evaluation indicators include mean absolute error MAE, root mean square error RMSE, mean error ME, structural similarity SSIM and peak signal-to-noise ratio PSNR;

[0027] Step S5-3: Use the pseudo CT image for radiotherapy plan design, calculate the dose distribution, and compare it with the dose distribution of the radiotherapy plan designed based on the real CT image to perform dosimetric evaluation.

[0028] As a preferred solution, the formula for image standardization is: ;

[0029] in, represents the normalized image; represents the original image; Represents the mean of the original image; Represents the standard deviation of the image data; Indicates the number of pixels in the image.

[0030] As a preferred solution, the formula for image normalization is: ;

[0031] in, represents the normalized image; represents the original image; Representing images The maximum value of Representing images The minimum value of .

[0032] As a preferred solution, the calculation method of the MIND target loss term is: ;

[0033] in, is a constant used to normalize the function; Indicates the search area in space The offset within represents the spatial search area; Represents an image; represents a voxel in the image; ,in, Belongs to the set Elements of Voxel The image block centered on Voxel and voxels The L2 distance between two image patches centered at ; ;in, Voxel The neighborhood of The variance estimate of It is a voxel field The elements in .

[0034] As a preferred solution, the formula of the MSE loss function is: ;

[0035] in, is the number of pixels; For Real value; The generated pseudo value.

[0036] Pseudo-CT generation and evaluation:

[0037] The calculation formula is ,in, is the number of pixels, For Real value, is the generated pseudo CT value; The calculation formula is ,in, The number of pixels For Real value, The generated pseudo value;

[0038] The calculation formula is ,in, is the number of pixels, For Real value, is the generated pseudo CT value;

[0039] The calculation formula is: ;

[0040] in, and Respectively represent the two images to be compared; and Respectively represent images and The average brightness of all pixels; and Respectively represent images and Grayscale standard deviation; and is a constant used to avoid zero denominator and adjust the scale of calculation results;

[0041] The calculation formula of PSNR is: ,in, is the number of bits of the image; is the mean square error, that is is the number of pixels, For Real value, is the generated pseudo CT value.

[0042] It can be seen from the technical solution provided by the present invention that the method for generating pseudo CT of nasopharyngeal carcinoma based on multimodal deformation registration and Transformer network provided by the present invention has the following beneficial effects:

[0043] Improve the accuracy of pseudo-CT generation: Through multi-modal deformation registration technology, the registration problem between images of different modalities is effectively solved, making the generated pseudo-CT images more consistent with the real CT images in anatomical structure, thereby improving the accuracy of pseudo-CT generation;

[0044] Generating high-quality pseudo CT images: The improved Transformer network can better capture image features. Combined with a carefully designed loss function, the quality of pseudo CT images is further improved, making them perform better in imaging evaluation indicators.

[0045] Comprehensive evaluation ensures reliability: The pseudo-CT images are evaluated for imaging and dosimetry using a variety of evaluation indicators, which can comprehensively and objectively measure the performance of pseudo-CT images. In terms of dosimetry evaluation, the dose distribution of the radiotherapy plan designed based on real CT images is compared to ensure the accuracy and reliability of pseudo-CT images in radiotherapy plan design, providing strong support for radiotherapy of nasopharyngeal carcinoma.

[0046] Reducing patient radiation dose: Since pseudo CT images can be generated from MRI images, the radiation dose received by patients due to repeated CT scans can be reduced to a certain extent, which is beneficial to the health protection of patients;

[0047] High clinical application value: The method of the present invention has important clinical application value in the field of nasopharyngeal carcinoma radiotherapy, and is expected to improve the efficiency and quality of radiotherapy planning and improve the treatment effect of patients. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 The present invention is a flowchart of a method for generating pseudo-CT of nasopharyngeal carcinoma based on multimodal deformation registration and Transformer network. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0050] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0051] like Figure 1 As shown, an embodiment of the present invention provides a method for generating pseudo CT of nasopharyngeal carcinoma based on multimodal deformation registration and Transformer network, comprising the following steps:

[0052] Step S1, data collection and preprocessing;

[0053] Step S1-1, collecting multimodal imaging data of patients with nasopharyngeal carcinoma confirmed by pathology, the multimodal imaging data including CT images and MRI images;

[0054] Step S1-2, preprocessing the CT image and the MRI image, the preprocessing including image standardization, image resampling, image cropping and image normalization;

[0055] Step S2, multi-modal deformation registration;

[0056] Step S2-1, establish a deformable registration network based on the CycleFCNs model, and the deformable registration network is built based on Python3 and Pytorch deep learning framework;

[0057] Step S2-2, setting a loss function, the loss function includes a modality-independent neighborhood descriptor target loss term, an L2 norm regularization loss term, and a cycle loss term, taking the CT image as a fixed image and the MRI image as a floating image, inputting them into a deformable registration network, and outputting a spatial deformation transformation field;

[0058] Step S2-3, using the spatial deformation transformation field to perform deformation transformation on the MRI image, to obtain a deformed MRI image having the same anatomical structure as the CT image;

[0059] Step S3, improving the Transformer model;

[0060] Step S3-1: Based on the original Transformer model, a decoder is added to restore the encoded features into a pseudo CT image;

[0061] Step S3-2, setting a loss function, where the loss function includes a MIND loss function and a mean square error MSE loss function;

[0062] Step S3-3, extracting the human body outer contour mask of the CT image, setting the loss function weight value of the area inside the mask to a high value, and setting the loss function weight value of the area outside the mask to a low value;

[0063] Step S4: model training and optimization;

[0064] Step S4-1, the deformable MRI image and the corresponding CT image after deformation registration are used as training data and input into the improved Transformer model for training;

[0065] Step S4-2, calculating the loss function value between the pseudo CT image output by the Transformer model and the real CT image, and updating the model parameters through the back propagation algorithm to optimize the Transformer model;

[0066] Step S5, pseudo CT generation and evaluation;

[0067] Step S5-1, input the MRI image to be processed into the trained Transformer model to generate a pseudo CT image;

[0068] Step S5-2, using a variety of evaluation indicators to perform image evaluation on the generated pseudo CT image, the evaluation indicators include mean absolute error MAE, root mean square error RMSE, mean error ME, structural similarity SSIM and peak signal-to-noise ratio PSNR;

[0069] Step S5-3: Use the pseudo CT image for radiotherapy plan design, calculate the dose distribution, and compare it with the dose distribution of the radiotherapy plan designed based on the real CT image to perform dosimetric evaluation.

[0070] In this embodiment, the specific steps of data collection and preprocessing are:

[0071] Step S1-1: Data collection

[0072] Step S1-1-1, determine the data source: obtain the imaging data of nasopharyngeal carcinoma patients confirmed by pathology from the hospital's case database;

[0073] Step S1-1-2, screening qualified data: ensure that the patient's medical records and radiotherapy plan data are complete and without interference from serious underlying diseases; imaging data must include high-quality CT and MRI images;

[0074] Step S1-1-3, set inclusion criteria: only patients diagnosed with NPC and with complete CT and MRI imaging data can enter the research dataset;

[0075] Step S1-1-4, clarify the exclusion criteria: exclude cases with poor imaging data quality, such as cases with severe image artifacts, low resolution or missing data;

[0076] Step S1-2: Data preprocessing

[0077] Step S1-2-1, image standardization:

[0078] To de-center CT and MRI images, use the formula: ;

[0079] in, represents the normalized image; represents the original image; Represents the mean of the original image; Represents the standard deviation of the image data; Represents the number of pixels in the image to improve the generalization of network training results;

[0080] Step S1-2-2, image resampling: unify the resolution of all image data to 1mm×1mm×1mm;

[0081] Step S1-2-3, image cropping: According to the research needs and model input requirements, the resampled image is cropped to a suitable size, such as (256, 256, 256), to reduce the input image size without losing effective information;

[0082] Step S1-2-4, image normalization: Use the following formula to normalize the image data to the (-1,1) interval to help the neural network converge faster: ;

[0083] in, represents the normalized image; represents the original image; Representing images The maximum value of Representing images The minimum value of .

[0084] In this embodiment, the specific steps of step S2-1 are as follows:

[0085] Step S2-1-1, select programming language and deep learning framework: determine to use Python3 as the programming language and Pytorch as the deep learning framework;

[0086] Step S2-1-2, import related libraries and modules: In the Python environment, import Pytorch and other libraries and modules that may be needed, such as numpy, for data processing and model building;

[0087] Step S2-1-3, define the network structure:

[0088] Based on the architecture of the CycleFCNs model, determine the various layers of the network and their parameters; for example, set the convolutional layer's convolution kernel size, step size, padding method, and the type of activation function (such as ReLU);

[0089] Determine the input and output dimensions of the network to adapt to the data format of CT and MRI images and the task requirements of deformable registration;

[0090] Step S2-1-4, initializing network parameters: using a suitable initialization method (such as random initialization) to assign initial values ​​to the network parameters;

[0091] Step S2-1-5, configure the training environment: set training-related parameters, such as learning rate, optimizer (such as Adam optimizer), loss function, etc.; at the same time, determine the training device (such as GPU or CPU) to accelerate the network training process;

[0092] Through the above steps, the deformation registration network based on the CycleFCNs model is built, and it is configured based on Python3 and Pytorch deep learning framework to prepare for the subsequent multimodal deformation registration tasks.

[0093] In this embodiment, the specific steps of step S2-2 are as follows:

[0094] Step S2-2-1: Setting the loss function

[0095] Modality Independent Neighborhood Descriptor (MIND) objective loss term:

[0096] For images Each voxel and spatial search area in The offset within , calculate the loss term ; First calculate the L2 distance between two image patches: ;

[0097] in, Belongs to the set Elements of Voxel The image patch centered at the center; then the voxel is calculated Variance estimate of : ;in, Voxel The neighborhood of The variance estimate of It is a voxel field Elements in

[0098] at last, The loss term is calculated as: ;

[0099] in, is a constant used for normalization;

[0100] L2 norm regularization loss term: Add L2 norm regularization term to the network parameters to prevent overfitting; the specific calculation method is to sum the square sum of the network parameters and multiply it by a regularization coefficient;

[0101] Cycle loss term: By designing cycle consistency constraints, we ensure that the deformation transformation from CT image to MRI image and the deformation transformation from MRI image to CT image can be restored to each other, so as to calculate the cycle loss term;

[0102] Step S2-2-2, input image: take the CT image as a fixed image and the MRI image as a floating image, and input them into the already built deformable registration network;

[0103] Step S2-2-3, outputting the spatial deformation transformation field: after calculation by the deformation registration network, outputting the spatial deformation transformation field; this transformation field describes the spatial deformation of the MRI image relative to the CT image, and is used for subsequent deformation transformation of the MRI image to align it with the CT image in terms of anatomical structure;

[0104] Through the above steps, the setting of the loss function is completed, and the process of inputting CT and MRI images into the deformable registration network to obtain the spatial deformation transformation field is realized, laying the foundation for subsequent multimodal deformable registration.

[0105] In this embodiment, the specific steps of step S2-3 are:

[0106] Step S2-3-1, obtaining a spatial deformation transformation field: the output result obtained from step S2-2, i.e., the spatial deformation transformation field;

[0107] Step S2-3-2, applying deformation transformation: transforming the coordinates of each voxel in the MRI image according to the spatial deformation transformation field; specifically, for each voxel coordinate in the MRI image , find its corresponding coordinates in the CT image space through the spatial deformation transformation field ;

[0108] Step S2-3-3, generating a deformed MRI image: resampling the voxel values ​​of the MRI image according to the transformed coordinates to obtain a deformed MRI image having an anatomical structure consistent with the CT image;

[0109] Through the above steps, the MRI image is deformed using the spatial deformation transformation field, so that the deformed MRI image is consistent with the CT image in terms of anatomical structure, providing more accurate multimodal information for subsequent pseudo CT generation.

[0110] In this embodiment, the specific steps of step S3-1 are:

[0111] Step S3-1-1, analyze the original Transformer model structure: deeply understand the architecture of the original Transformer model, including the structure and working principle of the encoder part, and clarify the encoding feature form of its output;

[0112] Step S3-1-2, designing a decoder structure: designing a suitable decoder structure according to the generation requirements of the pseudo CT image; the decoder should be able to receive the encoding features output by the encoder and gradually restore them to a pseudo CT image with the same size and resolution as the CT image;

[0113] Step S3-1-3, integrate the decoder with the original model: add the designed decoder to the original Transformer model to ensure that the connection between the decoder and the encoder is correct and can smoothly transfer data and information;

[0114] Step S3-1-4, adjust model parameters: initialize or adjust the parameters of the entire model after adding the decoder to adapt to the new structure and task requirements; some hyperparameters may need to be adjusted according to the specific situation, such as learning rate, number of layers, number of neurons, etc.;

[0115] Through the above steps, a decoder is successfully added to the original Transformer model, enabling it to restore the encoded features into pseudo CT images, preparing for the subsequent pseudo CT generation.

[0116] In this embodiment, the specific steps of step S3-2 are:

[0117] Step S3-2-1, define the MIND loss function:

[0118] For the input image (such as deformable MRI image and corresponding CT image), calculate the gradient change around each pixel;

[0119] Calculate according to the calculation formula of MIND loss function: ;

[0120] Among them, the L2 distance between image blocks is involved and the voxel variance estimate etc. calculation steps (the specific calculation method has been described in detail in the previous steps);

[0121] The role of the MIND loss function is to extract the soft tissue boundary information in the image, which helps to ensure the consistency of the generated pseudo-CT image with the real CT image in terms of anatomical structure;

[0122] Step S3-2-2, define the mean square error MSE loss function:

[0123] For the generated pseudo CT image and the corresponding real CT image, the difference in CT values ​​of the corresponding pixels between them is calculated;

[0124] Calculate according to the calculation formula of MSE loss function: ;

[0125] in, is the number of pixels, is the true CT value, is the generated pseudo CT value;

[0126] The role of the MSE loss function is to ensure the accuracy of the CT value of the generated pseudo-CT image;

[0127] Through the above steps, the loss function settings including the MIND loss function and the mean square error MSE loss function are completed. These two loss functions will be used together in the subsequent model training and optimization process to improve the quality of generated pseudo CT images.

[0128] In this embodiment, the specific steps of step S3-3 are:

[0129] Step S3-3-1, image segmentation: Use a suitable image segmentation algorithm (such as Otsu's method) to segment the foreground and background of the CT image to obtain a preliminary binary image;

[0130] Step S3-3-2, obtaining a background mask: obtaining a background mask from a preliminary binary image by using a flood fill algorithm or other methods;

[0131] Step S3-3-3, generating an outer contour mask: inverting the CT value of the background mask, and using a morphological closing operation to repair the cavity portion of the mask edge, to obtain the desired human body outer contour mask;

[0132] Step S3-3-4, setting weight values: setting the weight value of the loss function in the area inside the mask to a higher value, and setting the weight value of the loss function in the area outside the mask to a lower value;

[0133] Through the above steps, the human body outer contour mask of the CT image is successfully extracted, and appropriate loss function weight values ​​are set for different regions, so that the model can pay more attention to the image feature learning of the internal area of ​​the human body during the training process, while reducing the influence of the external area on the model training, thereby improving the accuracy of generating pseudo CT images.

[0134] In this embodiment, the specific steps of step S4-1 are:

[0135] Step S4-1-1, prepare training data:

[0136] First, confirm that the multimodal deformation registration step has been completed and accurate deformation MRI images and corresponding CT images have been obtained; these images are the basic data for model training;

[0137] Next, data cleaning and preprocessing operations are performed on the deformable MRI and CT images; this may include checking the integrity and consistency of the data to ensure that there are no corrupted or missing parts of the image data;

[0138] Then, resize the image according to the model's input requirements; for example, if the model requires the input image to be of a fixed size , you need to crop or fill the image to make it meet the required size;

[0139] In addition, it may be necessary to normalize the image data to map the pixel values ​​of the image to a specific range (such as or ) to improve the training effect and stability of the model;

[0140] Step S4-1-2: Input training data:

[0141] The sorted and preprocessed deformable MRI images and corresponding CT images are input into the improved Transformer model according to the model’s input format requirements;

[0142] During the input process, it is necessary to ensure the correct correspondence of the data, that is, each deformable MRI image has its corresponding CT image as a reference;

[0143] Step S4-1-3, model training:

[0144] The model starts running and calculates the loss value based on the previously set loss functions (including MIND loss function and MSE loss function);

[0145] For the MIND loss function, according to its specific calculation formula, the difference in gradient changes around each pixel between the generated pseudo-CT image and the real CT image is calculated to measure the similarity of the images in anatomical structure;

[0146] For the MSE loss function, the average of the sum of squares of the differences in CT values ​​of corresponding pixels between the generated pseudo-CT image and the real CT image is calculated to evaluate the accuracy of the image CT value;

[0147] The back propagation algorithm automatically adjusts the model parameters according to the calculated loss value. Specifically, the back propagation algorithm calculates the gradient of the model parameters corresponding to the loss value, and then updates the model parameters according to the direction and size of the gradient, so that the model can gradually reduce the loss value in subsequent training.

[0148] Repeat the above process of forward propagation to calculate the loss value and back propagation to update the parameters for multiple training rounds. In each training round, the model will continue to learn and optimize until the model converges (that is, the loss value no longer decreases significantly) or reaches the preset training goal (such as reaching a certain accuracy or the upper limit of the training round).

[0149] Through the above detailed steps, the improved Transformer model can be effectively trained using the deformable registered image data, so that it can continuously learn and improve its ability to generate high-quality pseudo CT images from deformable MRI images.

[0150] In this embodiment, the specific steps of step S4-2 are:

[0151] Step S4-2-1, calculate the loss function value:

[0152] Compare the pseudo CT images output by the Transformer model with the corresponding real CT images;

[0153] Calculate the values ​​of the two loss functions according to the calculation formulas of the MIND loss function and the MSE loss function respectively;

[0154] For the MIND loss function, it is calculated according to the formula: ;

[0155] Among them, the L2 distance between image blocks is involved and the voxel variance estimate etc. calculation steps (the specific calculation method has been described in detail in the previous steps);

[0156] For the MSE loss function, it is calculated according to the following formula: ;

[0157] in, is the number of pixels; For Real value; The generated pseudo value;

[0158] Step S4-2-2, back propagation calculation gradient:

[0159] Use automatic differentiation tools (such as the autograd module in Pytorch) to automatically calculate the gradient of the loss function with respect to the model parameters based on the calculated loss function value;

[0160] This process starts from the output layer, propagates the error backward step by step, and calculates the gradient value of each layer parameter;

[0161] Step S4-2-3, update model parameters:

[0162] Based on the calculated gradient value, use an optimization algorithm (such as stochastic gradient descent, Adam optimizer, etc.) to update the model parameters;

[0163] Specifically, according to the update rule of the optimization algorithm, the model parameters are adjusted in the opposite direction of the gradient to reduce the value of the loss function;

[0164] Repeat the above steps and continuously iterate and optimize the model until the model converges or reaches the preset training conditions;

[0165] Through the above steps, the loss function value can be effectively calculated, and the model parameters can be updated through the back-propagation algorithm, so as to continuously optimize the Transformer model and improve the accuracy and quality of pseudo CT images generated.

[0166] In this embodiment, the specific steps of step S5-1 are:

[0167] Step S5-1-1, inputting the MRI image to be processed: obtaining the MRI image data for generating the pseudo CT image, and preprocessing it according to the input requirements of the model, such as resizing, normalizing, etc.; then, inputting the preprocessed MRI image into the trained Transformer model;

[0168] Step S5-1-2, model reasoning: After the Transformer model receives the input MRI image, it extracts and transforms the image features according to the mapping relationship learned during the training process; the encoder part inside the model encodes the features of the MRI image, and then the decoder part restores the encoded features into a pseudo CT image;

[0169] Step S5-1-3, output pseudo CT image: After the inference calculation of the model, a pseudo CT image corresponding to the input MRI image is finally output.

[0170] In this embodiment, the specific steps of step S5-2 are:

[0171] Step S5-2-1, calculate the mean absolute error MAE:

[0172] For the generated pseudo CT image and the corresponding real CT image, subtract the CT values ​​of their corresponding pixels and take the absolute value;

[0173] Then the absolute values ​​of all pixels are averaged to get the mean absolute error MAE, which is calculated as follows: ;

[0174] in, is the number of pixels, For Real value, is the generated pseudo CT value;

[0175] Step S5-2-2, calculate the root mean square error RMSE:

[0176] Similarly, for the generated pseudo-CT image and the real CT image, the CT values ​​of the corresponding pixels are first subtracted, and then the square sum is calculated;

[0177] Then average the sum of squares and finally take the square root to get the root mean square error RMSE, which is calculated as: ;

[0178] in, The number of pixels For Real value, The generated pseudo value;

[0179] Step S5-2-3, calculate the average error ME:

[0180] Subtract the CT values ​​of the corresponding pixels of the generated pseudo CT image and the real CT image, and then calculate the average to get the average error ME. The calculation formula is: ;

[0181] in, is the number of pixels, For Real value, The pseudo CT value generated

[0182] Step S5-2-4, calculate the structural similarity SSIM:

[0183] Calculate the average brightness, grayscale standard deviation and covariance of the generated pseudo CT image and the real CT image respectively;

[0184] Then calculate according to the SSIM calculation formula: ;

[0185] in, and Respectively represent the two images to be compared; and Respectively represent images and The average brightness of all pixels; and Respectively represent images and Grayscale standard deviation; and is a constant used to avoid zero denominator and adjust the scale of calculation results;

[0186] Step S5-2-5, calculate the peak signal-to-noise ratio PSNR:

[0187] First, calculate the mean square error (MSE) between the generated pseudo CT image and the real CT image (the calculation method is the same as the RMSE step);

[0188] Then calculate according to the PSNR calculation formula: ;

[0189] in, is the number of bits of the image; is the mean square error, that is is the number of pixels, For Real value, It is the pseudo CT value generated to avoid the denominator being zero and to adjust the scale of the calculation results;

[0190] Through the above steps, a comprehensive imaging evaluation of the generated pseudo CT images was performed using a variety of evaluation indicators, which can measure the similarity and quality between the pseudo CT images and the real CT images from different angles.

[0191] In this embodiment, the specific steps of step S5-3 are:

[0192] Step S5-3-1, radiotherapy plan design:

[0193] First, ensure that the radiotherapy planning software used can support the import and processing of pseudo-CT images;

[0194] The generated pseudo-CT images are accurately imported into the radiotherapy planning software;

[0195] Then, based on the patient's specific condition, tumor location, size, and surrounding normal tissue conditions, combined with clinical treatment requirements and goals, the software carefully sets the parameters of the radiotherapy plan; these parameters include but are not limited to the number, shape, size, angle of the irradiation field, and the dose weight of each irradiation field;

[0196] Step S5-3-2, calculate dose distribution:

[0197] After completing the setting of radiotherapy plan parameters, start the dose calculation module of the software;

[0198] This module will use specific dose calculation algorithms (such as convolution algorithm, Monte Carlo algorithm, etc.) to perform accurate dose calculation on pseudo CT images based on the set parameters and the tissue structure and density information provided by the pseudo CT images.

[0199] After calculation, the dose distribution data corresponding to the pseudo-CT image is obtained. These data are presented in the form of a three-dimensional dose matrix, which can reflect the radiation dose size at different positions.

[0200] Step S5-3-3, obtaining the dose distribution of the real CT image:

[0201] Similarly, the real CT images corresponding to the pseudo-CT images were imported into the radiotherapy planning software;

[0202] Keep the same radiotherapy plan parameter settings as those of the pseudo-CT image, run the dose calculation module again, and calculate the dose distribution data of the real CT image;

[0203] Step S5-3-4, comparing dose distribution:

[0204] Using the data analysis and visualization functions of the software, the dose distribution data of the pseudo-CT image is compared with the dose distribution data of the real CT image;

[0205] The dose volume histogram (DVH) can be drawn to intuitively show the dose volume received by each region of interest (such as tumor target area, organs at risk, etc.) at different dose levels.

[0206] At the same time, the dose distribution data can also be quantitatively analyzed to calculate the differences in some key indicators, such as the mean dose, maximum dose, minimum dose, etc.

[0207] Step S5-3-5: Evaluate dosimetric indicators:

[0208] In addition to directly comparing the dose distribution data, some commonly used dosimetric indices need to be calculated to further evaluate the dosimetric performance of pseudo-CT images;

[0209] For example, the target dose uniformity index is calculated, which is used to measure the uniformity of dose distribution within the tumor target area. The closer the value is to 1, the more uniform the dose distribution is;

[0210] Calculate the conformity index to evaluate the conformity of the radiotherapy plan to the tumor target, that is, the degree of conformity between the irradiation field shape and the target shape. The higher the value, the better the conformity.

[0211] By analyzing and comparing these dosimetric indicators, we can comprehensively evaluate the accuracy and feasibility of pseudo-CT images in radiotherapy planning design, determine whether they can replace real CT images for dose calculation, and whether they can meet the requirements of clinical treatment;

[0212] Through the above detailed steps, a comprehensive and in-depth dosimetric evaluation of pseudo-CT images can be performed, providing a reliable basis for its application in the design of NPC radiotherapy plans.

[0213] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for generating pseudo-CT images of nasopharyngeal carcinoma based on multimodal deformation registration and Transformer network, characterized by: The following steps are involved: Step S1, data collection and preprocessing; Step S1-1, collecting multimodal imaging data of patients with nasopharyngeal carcinoma confirmed by pathology, the multimodal imaging data including CT images and MRI images; Step S1-2, preprocessing the CT image and the MRI image, wherein the preprocessing includes image standardization, image resampling, image cropping and image normalization; Step S2, multi-modal deformation registration; Step S2-1, establishing a deformable registration network based on the CycleFCNs model, wherein the deformable registration network is built based on Python3 and Pytorch deep learning framework; Step S2-2, setting a loss function, wherein the loss function includes a modality-independent neighborhood descriptor target loss term, an L2 norm regularization loss term, and a cycle loss term, taking the CT image as a fixed image and the MRI image as a floating image, inputting them into a deformable registration network, and outputting a spatial deformation transformation field; Step S2-3, using the spatial deformation transformation field to perform deformation transformation on the MRI image, to obtain a deformed MRI image having the same anatomical structure as the CT image; Step S3, improving the Transformer model; Step S3-1: Based on the original Transformer model, a decoder is added to restore the encoded features into a pseudo CT image; Step S3-2, setting a loss function, where the loss function includes a MIND loss function and a mean square error MSE loss function; Step S3-3, extracting the human body outer contour mask of the CT image, setting the loss function weight value of the area inside the mask to a high value, and setting the loss function weight value of the area outside the mask to a low value; Step S4: model training and optimization; Step S4-1, the deformable MRI image and the corresponding CT image after deformation registration are used as training data and input into the improved Transformer model for training; Step S4-2, calculating the loss function value between the pseudo CT image output by the Transformer model and the real CT image, and updating the model parameters through the back propagation algorithm to optimize the Transformer model; Step S5, pseudo CT generation and evaluation; Step S5-1, input the MRI image to be processed into the trained Transformer model to generate a pseudo CT image; Step S5-2, using a variety of evaluation indicators to perform image evaluation on the generated pseudo CT image, the evaluation indicators include mean absolute error MAE, root mean square error RMSE, mean error ME, structural similarity SSIM and peak signal-to-noise ratio PSNR; Step S5-3: Use the pseudo CT image for radiotherapy plan design, calculate the dose distribution, and compare it with the dose distribution of the radiotherapy plan designed based on the real CT image to perform dosimetric evaluation.

2. According to claim 1, a method for generating pseudo-CT of nasopharyngeal carcinoma based on multimodal deformation registration and Transformer network, characterized in that: The formula for image normalization is: ； in, represents the normalized image; represents the original image; Represents the mean of the original image; Represents the standard deviation of the image data; Indicates the number of pixels in the image.

3. According to claim 1, a method for generating pseudo-CT of nasopharyngeal carcinoma based on multimodal deformation registration and Transformer network, characterized in that: The formula for image normalization is: ； in, represents the normalized image; represents the original image; Representing images The maximum value of Representing images The minimum value of .

4. According to claim 1, a method for generating pseudo-CT of nasopharyngeal carcinoma based on multimodal deformation registration and Transformer network, characterized in that: The calculation method of the MIND target loss term is: ； in, is a constant used to normalize the function; Indicates the search area in space The offset within represents the spatial search area; Represents an image; represents a voxel in the image; ,in, Belongs to the set Elements of Voxel The image block centered on Voxel and voxels The L2 distance between two image patches centered at ; ;in, Voxel The neighborhood of The variance estimate of It is a voxel field The elements in .

5. According to claim 1, a method for generating pseudo-CT of nasopharyngeal carcinoma based on multimodal deformation registration and Transformer network, characterized in that: The formula of the MSE loss function is: ； in, is the number of pixels; For Real value; The generated pseudo value.

6. According to claim 1, a method for generating pseudo-CT of nasopharyngeal carcinoma based on multimodal deformation registration and Transformer network, characterized in that: In the pseudo-CT generation and evaluation: Said The calculation formula is ,in, is the number of pixels, For Real value, is the pseudo CT value generated; The calculation formula is ,in, The number of pixels For Real value, The generated pseudo value; Said The calculation formula is ,in, is the number of pixels, For Real value, is the generated pseudo CT value; Said The calculation formula is: ； in, and Respectively represent the two images to be compared; and Respectively represent images and The average brightness of all pixels; and Respectively represent images and Grayscale standard deviation; and is a constant used to avoid zero denominator and adjust the scale of calculation results; The calculation formula of the PSNR is: ,in, is the number of bits of the image; is the mean square error, that is is the number of pixels, For Real value, is the generated pseudo CT value.

Images