Image processing method for correcting metal artifacts in radiotherapy positioning CT

Abstract

The invention provides an image processing method for correcting metal artifacts in radiotherapy positioning CT (Computed Tomography). According to the method, an attention mechanism module is utilized to enhance the generation capability of a model in a sinogram missing region and improve the correction range of the model in an image domain; in order to protect the relation between the upper layer and the lower layer of the image, the upper layer and the lower layer of the image are additionally added in input, the relevance between adjacent images is enhanced, and the feeling range of the model is widened. Wherein in the sinogram domain, the invention provides a sinogram repairing network, the image area of the metal artifact is accurately learned into the model through an attention mechanism, and the intact sinogram is used for repairing the sinogram influenced by the metal artifact. Meanwhile, in an image domain, 2.5 D image input and an attention mechanism are utilized, information of an upper layer and information of a lower layer are connected in series, and image restoration is completed globally. In addition, an additional discriminator is introduced to form multiple tasks to judge whether the output image meets the requirement or not, so that a better correction result is obtained.

Description

Technical Field

[0001] The invention relates to an image processing technology for radiotherapy positioning CT in the medical field, and in particular to a method for correcting head metal artifacts in radiotherapy positioning CT images. Background Art

[0002] Radiotherapy positioning CT is a key step in radiotherapy. It uses CT scanning technology to obtain imaging data in the patient's treatment position. These data are used to accurately delineate the target area and design the treatment plan. The purpose of radiotherapy positioning CT is to ensure that the radiation is accurately irradiated to the tumor tissue during treatment while minimizing damage to surrounding normal tissues.

[0003] Metal artifacts are a common problem in radiotherapy positioning CT, which are caused by metal implants in the patient's body (such as artificial joints, tooth fillings, steel nails, particle implants, etc.). These metal objects have a high absorption rate of X-rays, resulting in distortion and streak-like artifacts during CT image reconstruction. These artifacts not only reduce the clarity of the image, but may also affect the doctor's accuracy in outlining the tumor boundary and dose calculation, thereby affecting the accuracy and safety of radiotherapy.

[0004] In previous studies, there are three main methods for correcting CT metal artifacts: the first is a removal method based on iterative reconstruction, the second is a method based on interpolation and filling, and the third is a method based on deep learning. With the enhancement of computer performance and the optimization of algorithms, deep learning methods have higher computing power and better results, and have also made great progress in the field of medical image processing, providing more feasibility for faster and better correction of metal artifacts in CT.

[0005] Although there are many correction methods, there is no standard solution. The iterative reconstruction method and the interpolation method are both slow and introduce new artifacts, so the quality of the corrected image is not high. Most deep learning methods use generative models to fill in directly, which makes the generated CT values ​​differ greatly from the true values ​​and will significantly change the CT values ​​of other areas. Summary of the invention

[0006] In view of the technical defects of the existing methods, for example, in the sinusoidal projection, it fails to make full use of more effective pixels connected, and can only use the limited number of surrounding pixels for restoration. In the CT image, it fails to make full use of the connection between the upper and lower layers of the image, ignores the contrast problem in the image, resulting in excessive smoothing and unusable. The present invention provides an image processing method for correcting metal artifacts in radiotherapy positioning CT, by building a model using real collected data, using the attention mechanism and 2.5D image input technology to better correct the artifacts caused by metal, and optimize the data generation effect.

[0007] In order to achieve the above object, the present invention provides the following technical solutions:

[0008] The present invention provides an image processing method for correcting metal artifacts in radiotherapy positioning CT, which uses an attention mechanism module to strengthen the generation ability of a model in a sinusoidal missing area and improve the correction range of the model in an image domain; and in order to protect the connection between the upper and lower layers of the image, two layers of upper and lower images are additionally added to the input, thereby strengthening the correlation between adjacent images and improving the perception range of the model; wherein, in the sinusoidal domain, a sinusoidal repair network is proposed, and the image area of ​​the metal artifact is accurately learned into the model through the attention mechanism, and the sinusoidal image affected by the metal artifact is repaired using the intact sinusoidal image; at the same time, in the image domain, 2.5D image input and attention mechanism are used to connect the information of the upper and lower layers in series, and the image repair is completed from a global perspective; in addition, an additional discriminator is introduced to form multiple tasks to determine whether the output image meets the requirements, thereby obtaining a better correction result.

[0009] Furthermore, the specific steps of the method are as follows:

[0010] 1) According to the data characteristics, the threshold segmentation algorithm is used to segment the metal in the head area to obtain the mask of the metal area, which is recorded as M metal ; According to the segmentation results of the metal area, filter the CT slice data that needs to correct the artifacts; select the nth CT slice that needs to be corrected, recorded as At the same time, find two adjacent CT slices and record them as and Then select the metal segmentation results corresponding to the three CT images and record them as At this time, M is a binary (0 or 1) segmentation result image. At this time, the metal image can be obtained as:

[0011]

[0012] Where ⊙ represents element-by-element multiplication; the three CT images and three metal images obtained in (1) are respectively subjected to the Radon method to obtain sinusoidal images, The specific area of ​​metal contamination can be determined through the sinusoidal diagram of the metal image. Based on this, S CT Clear the contaminated area and obtain Binary map of metal contaminated areas

[0013]

[0014] 2) The six images obtained, and its upper and lower layers, The upper and lower layers are input into the attention-based sinusoidal graph filling network ASNet; ASNet is a five-layer structure similar to Unet, with two three-channel input branches and one single-channel output. Input branch 1 is responsible for extracting The output of each layer is used as the Query in the attention mechanism and is input into the corresponding layer of branch 2. The input branch 2 is responsible for extracting The results of each layer are converted into Key and Value. The attention weight is obtained by calculating the similarity between Q and K, and applied to V to obtain the calculated result, which is then input into the next layer. The mathematical expression of the attention mechanism is as follows, where d is the dimension of the input and output vectors:

[0015]

[0016] The output branch is connected to the input branch 2 through a skip connection to obtain more global information. Each output branch uses two dilated convolutions with a dilation number of 3 and an upsampling calculation with linear interpolation; the output of ASNet is written as:

[0017]

[0018] in yes The restored sinusoidal image result; finally, the image is reconstructed by inverse Radon transform to obtain the reconstructed image. The weighted mean absolute error (L1Loss) L1 and mean square error (L2 Loss) mixed loss is used to measure the difference of the sinusoidal graph. The L1 loss is used to measure the entire area, and the L2 loss is used to strengthen the attention of the generated area (metal interference area) to avoid overfitting. Since the magnitude of the L2 loss is higher, the weight is given to 0.1. In summary, the loss definition of ASNet is as follows:

[0019]

[0020] in, is a true sine diagram without metal;

[0021] 3) In the process of converting the sinusoidal image to the CT image, some additional linear artifacts will be introduced, so another correction is performed in the image domain; the original image Sinogram reconstruction and metal segmentation results Input into the multi-task image correction network (MANet), where the input of the model is and After two convolution calculations, the three The feature map of is used as the Q of the attention mechanism, acting on and In the generated K and V, the attention mechanism is used to increase the attention of the metal area in the feature map. and The results are concatenated and fused, and the artifacts are further eliminated through a Unet++ model with ResNet50 as the backbone network to obtain the output result Then Input into the pre-trained discriminator model Discriminator, and use the discriminator model to judge the correction artifacts Whether it can pass and calculate the loss; the output of MANet can be written as:

[0022]

[0023] For MANet, global L1 loss and L2 loss are used. The L2 loss result is reduced to the same level as MAE by weight 0.001, which is conducive to model training and gradient descent. In order to verify the effectiveness of MANet's output results, the discriminator is used to calculate the loss. The discriminator uses BCEWithLogitsLoss, which combines the characteristics of Sigmoid activation function and binary cross entropy (BCE) loss. Therefore, the loss function of MANet can be defined as:

[0024]

[0025] in The CT image is normal and artifact-free. The 0 in BCEWithLogitsLoss indicates that there is no metal artifact. The discriminator model is a binary classification model based on VGG16. During the training process, the data label of the image without artifact is 0 and the data label of the image with artifact is 1. The binary classification training is performed. The input image contains the image with metal artifact. and without metal artifacts

[0026] 4) CT images generated by neural networks The pixel values ​​will be changed uncontrollably, so they need to be adjusted; calculate The mean and variance of the nearest metal-free CT slice in the CT sequence are normalized and calculated at the same time. The mean and variance of are normalized and then adjusted using the histogram matching method. The pixel values ​​of each point are calculated, and then the pixels are restored according to the mean and variance to obtain the final artifact corrected image.

[0027] Furthermore, the output size of the network is 512×512×C, where C is the number of input channels. For example, the number of input channels is 3 in ASNet and MANet, and 1 in Discriminator.

[0028] Furthermore, during the model training process, all experiments were based on the Pytorch framework and accelerated training using the NVIDIA RTX A600048G GPU.

[0029] Furthermore, the learning rates of ASNet and MANet were set to 0.0001, and the Adam optimizer and cosine annealing learning rate adjustment algorithm were used. The models were trained for a total of 150 epochs.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] 1. Aiming at the metal artifacts caused by metal in the head in radiotherapy positioning CT, the present invention provides a dual-domain 2.5D metal artifact correction method based on deep learning, which can effectively correct metal artifacts, improve image quality, and reduce the impact on radiotherapy planning.

[0032] 2. The present invention integrates 2.5D image input and attention mechanism into the metal artifact correction model. It uses 2.5D image input to establish the connection between the upper and lower layers of the CT image, and uses the attention mechanism to allow the model to fully pay attention to and perceive the impact of metal artifacts in the sinusoidal graph domain and the image domain, thereby optimizing the problem of over-average filling caused by the lack of focus in the traditional network. In the image domain, the metal area is introduced as the focus of attention to the result after the sinusoidal graph is filled and the original image, which not only strengthens the effect of the model in correcting artifacts in the image domain but also improves the problem of introducing new artifacts. Finally, the adjacent artifact-free data is used for histogram matching to adjust the actual effect of the corrected image, which can be applied to positioning CT in radiotherapy, effectively improving the impact of metal artifacts on radiotherapy plans. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 Model diagram for the attention-based sinusoidal graph padding network ASNet.

[0034] Figure 2 Model diagram of the multi-task image rectification network MANet.

[0035] Figure 3 This is a graph showing the experimental results of the model data.

[0036] Figure 4 This figure shows the artifact correction effect of clinical data. DETAILED DESCRIPTION

[0037] The technical solution of the present invention is further described in detail below in conjunction with specific embodiments, and the specific effect of the present invention on metal artifact correction is demonstrated.

[0038] The data used in this paper comes from the head data in the AAPM CT-MAR challenge data and benchmark tools [1-5] and the phantom data made by the hospital. The size of all CT image slices is 512×512.

[0039] 1.AAPM CT MetalArtifact Reduction(CT-MAR)Grand Challenge:

[0040] https: / / www.aapm.org / GrandChallenge / CT-MAR

[0041] 2.AAPM CT MetalArtifact Reduction (CT-MAR)Grand Challenge BenchmarkTool:

[0042] https: / / github.com / xcist / example / tree / main / AAPM_datachallenge /

[0043] 3.M.Wu,P.FitzGerald,J.Zhang,WPSegars,H.Yu,Y.Xu,B.DeMan,"XCIST-anopen access x-ray / CT simulation toolkit,"Phys MedBiol.2022Sep 28;67(19)

[0044] 4.Yan K, Wang

[0045] 5.Goren N,Dowrick T,Avery J&Holder D.(2017).UCLH Stroke EIT Dataset-Radiology Data(CT).Zenodo.

[0046] https: / / doi.org / 10.5281 / zenodo.838704 .

[0047] An image processing method for correcting metal artifacts in radiotherapy positioning CT of the present invention comprises the following steps:

[0048] 1) After the patient completes the radiotherapy positioning CT scan, determine whether there are metal artifacts in the head. If it is confirmed that there are metal artifacts, extract the data.

[0049] 2) Use threshold segmentation to determine a series of CT slices where the metal artifacts of the head are located, and determine the one-to-one corresponding CT slices And metal segmentation results And use the following formula to obtain the complete metal image

[0050]

[0051] If the image to be removed is the first or last image containing metal, an additional normal image without artifacts is taken from the upper or lower layer. The three CT images and three metal images are obtained using the Radon method to obtain sinusoidal images. The following formula is used to remove the metal-contaminated areas in the three CT sinusoidal images:

[0052]

[0053]

[0054] 3) The six images obtained are and its upper and lower layers, And its upper and lower layers are input into the attention-based sinusoidal graph filling network ASNet, where And its upper and lower layers are input into the second branch, And its upper and lower layers are input to the first branch to obtain the corrected sinusoidal graph Using inverse Radon transform Perform image reconstruction to obtain a reconstructed CT image like Figure 1 The model diagram of the attention-based sinusoidal graph filling network ASNet is shown.

[0055] 4) Slice the original CT Reconstruction of CT images And the corresponding metal segmentation results Input into the multi-task image correction network MANet, such as Figure 2 The model diagram of the multi-task image correction network MANet shown in the figure obtains the output result calculate The mean and variance of are normalized.

[0056] 5) Find the CT sequence The nearest CT section without metal artifacts is used to calculate its mean and variance and normalize them. The normalized result is adjusted, and then the mean and variance are used to restore the pixel values ​​to obtain the final artifact corrected image.

[0057] Finally, the phantom artifact data and real clinical data were used for testing, respectively, such as Figure 3 The correction results of the model data are shown as follows: Figure 4 The figure shows the correction results of clinical data. It can be seen that the method proposed in the present invention can achieve good results not only on the phantom data, but also on the real clinical data, and almost all artifacts are corrected. Therefore, this model can establish excellent CT images, achieve good artifact correction effects and serve in clinical radiotherapy positioning CT.

[0058] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. An image processing method for correcting metal artifacts in radiotherapy positioning CT, characterized in that: The attention mechanism module is used to enhance the model's generation ability in the missing sinusoidal image area and improve the model's correction range in the image domain. In order to protect the connection between the upper and lower layers of the image, two additional layers of upper and lower images are added to the input, which strengthens the correlation between adjacent images and improves the model's perception range. In the sinusoidal image domain, a sinusoidal image restoration network is proposed, which uses the attention mechanism to accurately learn the image area of ​​the metal artifact into the model, and uses the intact sinusoidal image to repair the sinusoidal image affected by the metal artifact. At the same time, in the image domain, 2.5D image input and attention mechanism are used to connect the information of the upper and lower layers in series to complete the image restoration from a global perspective. In addition, an additional discriminator is introduced to form multiple tasks to determine whether the output image meets the requirements, thereby achieving better correction results.

2. The image processing method for correcting metal artifacts in radiotherapy positioning CT according to claim 1, characterized in that: The specific steps of this method are as follows: 1) According to the data characteristics, the threshold segmentation algorithm is used to segment the metal in the head area to obtain the mask of the metal area, which is recorded as M metal ; According to the segmentation results of the metal area, filter the CT slice data that needs to correct the artifacts; select the nth CT slice that needs to be corrected, recorded as At the same time, find two adjacent CT slices and record them as and Then select the metal segmentation results corresponding to the three CT images and record them as At this time, M is a binary (0 or 1) segmentation result image. At this time, the metal image can be obtained as: Where ⊙ represents element-by-element multiplication; The three CT images and three metal images obtained in (1) are used to obtain sinusoidal images using the Radon method. The specific area of ​​metal contamination can be determined through the sinusoidal diagram of the metal image. Based on this, S CT Clear the contaminated area and obtain Binary map of metal contaminated areas 2) The six images obtained, and its upper and lower layers, The upper and lower layers are input into the attention-based sinusoidal graph filling network ASNet; ASNet is a five-layer structure similar to Unet, with two three-channel input branches and one single-channel output. Input branch 1 is responsible for extracting The output of each layer is used as the Query in the attention mechanism and is input into the corresponding layer of branch 2. The input branch 2 is responsible for extracting The results of each layer are converted into Key and Value. The attention weight is obtained by calculating the similarity between Q and K, and applied to V to obtain the calculated result, which is then input into the next layer. The mathematical expression of the attention mechanism is as follows, where d is the dimension of the input and output vectors: The output branch is connected to the input branch 2 through a skip connection to obtain more global information. Each output branch uses two dilated convolutions with a dilation number of 3 and an upsampling calculation with linear interpolation; the output of ASNet is written as: in yes The repaired sinusoidal image result; finally, the image is reconstructed by inverse Radon transform to obtain the reconstructed CT The weighted mean absolute error (L1Loss) L1 and mean square error (L2 Loss) mixed loss is used to measure the difference of the sinusoidal graph. The L1 loss is used to measure the entire area, and the L2 loss is used to strengthen the attention of the generated area (metal interference area) to avoid overfitting. Since the magnitude of the L2 loss is higher, the weight is given to 0.

1. In summary, the loss definition of ASNet is as follows: in, is a true sine diagram without metal; 3) In the process of converting the sinusoidal image to the CT image, some additional linear artifacts will be introduced, so another correction is performed in the image domain; the original image Sinogram reconstruction and metal segmentation results Input into the multi-task image correction network (MANet), where the input of the model is and After two convolution calculations, the three The feature map of is used as the Q of the attention mechanism, acting on and In the generated K and V, the attention mechanism is used to increase the attention of the metal area in the feature map. and The results are concatenated and fused, and the artifacts are further eliminated through a Unet++ model with ResNet50 as the backbone network to obtain the output result Then Input into the pre-trained discriminator model Discriminator, and use the discriminator model to judge the correction artifacts Whether it can pass and calculate the loss; the output of MANet can be written as: For MANet, global L1 loss and L2 loss are used. The L2 loss result is reduced to the same level as MAE by weight 0.001, which is conducive to model training and gradient descent. In order to verify the effectiveness of MANet's output results, the discriminator is used to calculate the loss. The discriminator uses BCEWithLogitsLoss, which combines the characteristics of Sigmoid activation function and binary cross entropy (BCE) loss. Therefore, the loss function of MANet can be defined as: in The CT image is normal and artifact-free. The 0 in BCEWithLogitsLoss indicates that there is no metal artifact. The discriminator model is a binary classification model based on VGG16. During the training process, the data label of the image without artifact is 0 and the data label of the image with artifact is 1. The binary classification training is performed. The input image contains the image with metal artifact. and without metal artifacts 4) CT images generated by neural networks The pixel values ​​will be changed uncontrollably, so they need to be adjusted; calculate The mean and variance of the nearest metal-free CT slice in the CT sequence are normalized and calculated at the same time. The mean and variance of are normalized and then adjusted using the histogram matching method. The pixel values ​​of each point are calculated, and then the pixels are restored according to the mean and variance to obtain the final artifact corrected image.

3. The image processing method for correcting metal artifacts in radiotherapy positioning CT according to claim 2, characterized in that: The output size of the network is 512×512×C, where C is the number of input channels.

4. The image processing method for correcting metal artifacts in radiotherapy positioning CT according to claim 3, characterized in that: During the model training process, all experiments were based on the Pytorch framework and accelerated by using the NVIDIA RTX A6000 48G GPU.

5. The image processing method for correcting metal artifacts in radiotherapy positioning CT according to claim 4, characterized in that: The learning rates of ASNet and MANet are both set to 0.0001, and the Adam optimizer and cosine annealing learning rate adjustment algorithm are used. The models are trained for a total of 150 epochs.

Images