CT (Computed Tomography) metal artifact correction method based on driving image domain transformation through diffusion model

Abstract

A CT (Computed Tomography) metal artifact correction method based on image domain transformation driven by a diffusion model comprises the following steps: S1, inputting a real metal artifact image xr into a trained first diffusion model Dsx for conditional sampling operation, an output final sampling result is a domain-adapted metal artifact image # imgabs0 # S2, inputting the domain-adapted metal artifact image # imgabs1 # into a trained metal artifact correction model # imgabs2 # for artifact correction to obtain a metal artifact-corrected image # imgabs3 # S3, inputting the metal artifact-corrected image # imgabs4 # into a trained second diffusion model Dry for conditional sampling operation to obtain a metal artifact-corrected image # imgabs4 #; according to the method, the domain difference problem during reasoning of the supervised metal artifact correction model is fully considered, input domain adaptation is realized through the diffusion model, the correction performance of the supervised metal artifact correction model on simulation data is fully utilized, and the generalization performance of the model on real data is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning and computer tomography, and in particular to a CT metal artifact correction method based on image domain transformation driven by a diffusion model. Background Art

[0002] X-ray computed tomography (CT) is widely used in clinical practice due to its fast imaging speed and extremely high resolution. However, once the subject contains metal implants (such as fracture screws, neurovascular coils, pacemakers, etc.), physical mechanisms such as radiohardening, scattering, photon starvation, and nonlinear partial volume effects will occur, and introduce severe metal artifacts such as streaks, bands, or shadows to the reconstructed images.

[0003] Traditional metal artifact correction algorithms are mainly interpolation methods, which use projection data adjacent to the metal to directly replace or correct the projection data destroyed by the metal. Although this type of method can suppress some metal artifacts, it still causes the image to be over-smoothed. Some improved interpolation methods improve the accuracy of interpolated data by estimating effective prior images or embedding physical prior knowledge. However, when there are multiple metal implants in the test subject, the performance improvement of this type of method is very limited.

[0004] Deep learning has a powerful feature learning ability, so it is gradually applied to the field of metal artifact correction. Most metal artifact correction algorithms based on deep learning use supervised learning, or learn the mapping from images with metal artifacts to images without metal artifacts; or are based on dual domains, using the strategy of jointly training projection domain and image domain recovery subnetworks to improve the performance of artifact correction. Because these artifact correction algorithms based on supervised learning lack paired real data (with metal artifacts / without metal artifacts) to train the model, they can only use the commonly used simulation algorithm to segment the attenuation coefficient image to obtain the base material image, and use the multi-energy forward projection operator, noise addition, and reconstruction to obtain paired simulation data. However, due to the extremely complex physical effects of X-rays and metal implants, and the extremely difficult alignment of simulated and real spectral information, scanning protocols, scanning equipment, and reconstruction parameters, the simulated paired data is often different from the real paired data. Therefore, supervised metal artifact correction algorithms often face the problem of domain differences, which are particularly prominent in the high-frequency parts of the image (such as noise and streak artifacts), and this domain difference will reduce the generalization performance of the model, that is, the effect of reasoning on real data is often not as good as that of reasoning on simulated data.

[0005] In semi-supervised or unsupervised learning methods, the metal artifact correction model is directly trained with real data, such as adversarial generative networks (GAN, ADN). Although these methods do not consider the domain difference between simulated data and real data, the network training of these methods is unstable and cannot correct strong metal artifacts. One of them is domain adaptation during training, which aligns the real data features extracted by the encoder with the simulated data features (such as UDAMAR) by training a classifier. Although this method is more effective than supervised learning, the potential inconsistency between pre-trained real data and test data will reduce the effect of artifact correction.

[0006] Therefore, in view of the shortcomings of the prior art, it is necessary to provide a CT metal artifact correction method based on image domain transformation driven by a diffusion model to solve the shortcomings of the prior art. Summary of the invention

[0007] The purpose of the present invention is to avoid the shortcomings of the prior art and provide a CT metal artifact correction method based on driving image domain transformation through a diffusion model. The CT metal artifact correction method based on driving image domain transformation through a diffusion model fully considers the domain difference problem during the reasoning of the supervised metal artifact correction model, and realizes input domain adaptation through the diffusion model, fully utilizes the correction performance of the supervised metal artifact correction model on simulated data, and improves the generalization performance of the model on real data.

[0008] The above-mentioned purpose of the present invention is achieved by the following technical measures:

[0009] A method for CT metal artifact correction based on image domain transformation driven by a diffusion model is provided, comprising the following steps:

[0010] S1, the real metal artifact image x r Input the first diffusion model D after training sx Perform conditional sampling operation, and the final sampling result output is an image with metal artifacts after domain adaptation

[0011] S2, image with metal artifacts after domain adaptation obtained from S1 Input the trained metal artifact correction model Perform artifact correction to obtain the metal artifact corrected image

[0012] S3, the metal artifact corrected image obtained in S2 Input the trained second diffusion model D ry Perform conditional sampling operation, and the final sampling result output is a metal artifact-free image after domain adaptation.

[0013] Preferably, the first diffusion model D sx The first diffusion model D is constructed by forward process according to the training data set. sx Perform training to obtain the first diffusion model D after training sx ; and the training data set is provided with a pair of paired data and a plurality of real metal artifact-free images y r ; The paired data is a simulated metal artifact image x s and simulated metal artifact-free image y s .

[0014] Preferably, the second diffusion model D ry The second diffusion model D is constructed by a forward process according to the training data set. ry Perform training to obtain the trained second diffusion model D ry .

[0015] In the first diffusion model D sx In the forward process, the simulated metal artifact image x of the counter sampling s Add T times Gaussian noise until the metal artifact image x s The data distribution p sampled in the simulation domain s (x) converges to the prior Gaussian noise image and obtains the first diffusion model D after training sx .

[0016] In the second diffusion model D ry In the forward process, the real metal artifact-free image y r Add T times Gaussian noise until the real metal artifact-free image y r In the real domain, the sample converges to the prior Gaussian noise image and obtains the trained second diffusion model D ry .

[0017] In the first diffusion model D sx In the forward process, the simulated metal artifact image x s The data distribution p sampled in the simulation domain s (x) is expressed by equation (1), the first diffusion model D sx The objective function of the forward process is expressed by equation (2);

[0018] x s ~p s (x) …… Formula (1);

[0019]

[0020] in, is the gradient operator, θ sx The first diffusion model D sxThe learnable parameters of , ε is a priori Gaussian noise image with a mean of 0, The first diffusion model D sx The noise predictor of The first diffusion model D sx is a hyperparameter, and t is the uniform distribution time coefficient.

[0021] In the second diffusion model D ry In the forward process, the real metal artifact-free image y r The data distribution p sampled in the real domain r (y) is expressed by equation (3), the second diffusion model D ry The objective function of the forward process is expressed by equation (4);

[0022] y r ~p r (y)...Formula (3);

[0023]

[0024] Among them, θ ry The second diffusion model D ry The learnable parameters of The second diffusion model The noise predictor of The second diffusion model D ry Hyperparameters of .

[0025] Preferably, the Gaussian noise image ε is expressed by formula (5):

[0026]

[0027] Where I is the variance, is a normal distribution;

[0028] Preferably, the uniform distribution time coefficient t is expressed by formula (6):

[0029] t~Uniform({1,…,T})……Formula (6);

[0030] Among them, T is the total number of sampling times, and Uniform is uniform distribution.

[0031] In the S1, the conditional sampling operation is performed by using the first diffusion model D sx Add the real metal artifact image x to the reasoning process r The low-frequency characteristics φ(x r ) and perform T sampling operations, and then use the sampling result x of the last sampling i-1 As the final sampling result, the final sampling result is the metal artifact image after the domain adaptation.

[0032] In the conditional sampling operation in S3, the second diffusion model D ry The metal artifact corrected image obtained by adding S2 during the inference process The low frequency characteristics And perform T sampling operations, and then use the sampling result x of the last sampling i-1 As the final sampling result, the final sampling result is the metal artifact-free image after domain adaptation.

[0033] In S1, the conditional constraint of the conditional sampling operation is a conditional input constraint, and the sampling operation is represented by equations (7) to (11);

[0034]

[0035] ε i =D sx (x i ,i,φ(x r ))……Formula (8);

[0036]

[0037]

[0038] Where i is a time variable, the value range of i is [T, 1], and T = 1000, is the noise variance, and β i are all hyperparameters, and α i =1-β i , ε i is the noise of the ith prediction, x i is the sampling result of the ith time, is the result x of the i-th sampling i The initial result after denoising weighting, z is the additional Gaussian noise image, x i-1 It is the sampling result of the current output.

[0039] In S1, the conditional constraint of the conditional sampling operation is a sample enhancement constraint, and the sampling operation is represented by equations (12) to (17);

[0040]

[0041] ε i =D sx (x i ,i)...Formula (13);

[0042]

[0043] x i-1 = x′ i-1 +λ(φ(x r )-φ(x′ i-1 ))……Formula (17);

[0044] Where i is a time variable, the value range of i is [T, 1], and T = 1000, is the noise variance, and β i are all hyperparameters, and α i =1-β i , ε i is the noise in the i-th prediction, x i is the sampling result of the i-th time, is the result x of the i-th sampling i The initial result after denoising weighting, z is the additional Gaussian noise image, x′ i-1 is the initial sampling result of the current sampling, x i-1 is the sampling result of the current sampling output, and λ is the hyperparameter of the modulation.

[0045] In S3, the conditional constraint of the conditional sampling operation is a conditional input constraint, and the sampling operation is represented by equations (18) to (23);

[0046]

[0047] Where i is a time variable, the value range of i is [T, 1], and T = 1000, is the noise variance, and β i are all hyperparameters, and α i =1-β i , ε i is the noise of the ith prediction, x i is the sampling result of the i-th time, is the result x of the i-th sampling i The initial result after denoising weighting, z is the additional Gaussian noise image, x i-1 It is the sampling result of the current output.

[0048] In S3, the conditional constraint of the conditional sampling operation is a sample enhancement constraint, and the sampling operation is represented by equations (23) to (28);

[0049]

[0050] ε i =D ry (x i ,i)...Formula (24);

[0051]

[0052] Where i is a time variable, the value range of i is [T, 1], and T = 1000, is the noise variance, and β i are all hyperparameters, and α i =1-β i , ε i is the noise in the i-th prediction, x i is the sampling result of the ith time, is the result x of the i-th sampling i The initial result after denoising weighting, z is the additional Gaussian noise image, x′ i-1 is the initial sampling result of the current sampling, x i-1 is the sampling result of the current sampling output, and λ is the hyperparameter of the modulation.

[0053] Preferably, the method for constructing the above training data set is:

[0054] A1. Obtain multiple pairs of simulated metal projection data and simulated metal-free projection data, as well as multiple real metal-free artifact images y r ,Each pair of simulated metal projection data and simulated metal-free projection data is for the same object and the same anatomical position;

[0055] A2. Reconstruct each pair of simulated metal projection data and simulated metal-free projection data respectively, and obtain multiple pairs of simulated metal artifact images x s and simulated metal artifact-free image y s .

[0056] Preferably, the metal artifact correction model The training method is: the simulated metal artifact image x in the training data set is s Dot product complement of metal mask A metal artifact-only image is obtained, wherein the metal artifact-only image does not contain metal implants, wherein is the simulated metal artifact image x s Then the image containing only metal artifacts is input into the metal artifact correction model Get the artifact-corrected image, and calculate the artifact-corrected image and the simulated metal-free artifact image y according to the loss function MSE s The loss value between , and then update the metal artifact correction model Finally, the trained metal artifact correction model is obtained.

[0057] Preferably, the loss value Loss of the above loss function MSE is expressed by formula (29):

[0058]

[0059] The present invention provides a CT metal artifact correction method based on driving image domain transformation through a diffusion model, comprising the following steps: S1, converting a real metal artifact image x into a r Input the first diffusion model D after training sx Perform conditional sampling operation, and the final sampling result output is an image with metal artifacts after domain adaptation S2, image with metal artifacts after domain adaptation obtained from S1 Input the trained metal artifact correction model Perform artifact correction to obtain the metal artifact corrected image S3, the metal artifact corrected image obtained in S2 Input the trained second diffusion model D ry Perform conditional sampling operation, and the final sampling result output is a metal artifact-free image after domain adaptation. This CT metal artifact correction method based on driving image domain transformation through a diffusion model fully considers the domain difference problem during the reasoning of the supervised metal artifact correction model, realizes input domain adaptation through the diffusion model, and fully utilizes the correction performance of the supervised metal artifact correction model on simulated data, thereby improving the generalization performance of the model on real data. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The present invention is further described with reference to the accompanying drawings, but the contents in the accompanying drawings do not constitute any limitation to the present invention.

[0061] Figure 1 Flowchart of a CT metal artifact correction method based on image domain transformation driven by a diffusion model.

[0062] Figure 2 CT images of the head before and after correction.

[0063] Figure 3 These are CT images of the chest and abdomen before and after correction. DETAILED DESCRIPTION

[0064] The technical solution of the present invention is further described in conjunction with the following embodiments.

[0065] Example 1

[0066] A CT metal artifact correction method based on image domain transformation driven by a diffusion model, such as Figure 1 As shown, the following steps are included:

[0067] S1, the real metal artifact image x r Input the first diffusion model D after training sx Perform conditional sampling operation, and the final sampling result output is an image with metal artifacts after domain adaptation

[0068] S2, image with metal artifacts after domain adaptation obtained from S1 Input the trained metal artifact correction model Perform artifact correction to obtain the metal artifact corrected image

[0069] S3, the metal artifact corrected image obtained in S2 Input the trained second diffusion model D ry Perform conditional sampling operation, and the final sampling result output is a metal artifact-free image after domain adaptation.

[0070] It should be noted that the metal referred to in the present invention refers to all metal implants that may cause serious stripes, bands or shadows in the reconstructed image, including but not limited to heart stents, peripheral vascular stents, dental implants, orthopedic implants, pacemakers, etc.

[0071] The diffusion model of the present invention is a common denoising diffusion probability model DDPM. The diffusion model can transform a Gaussian distribution into any complex data distribution. Therefore, the diffusion model can describe any complex data distribution, such as the data distribution p of the metal artifact image in the simulation domain. s (x), data distribution p of the metal artifact-free image in the simulation domain s (y), data distribution p of real domain images with metal artifacts r (x), and the data distribution p of the real domain metal artifact-free image r (y). In S2, since the real metal artifact data has completed the domain transformation in S1, that is, the transition from the real domain to the simulation domain has been completed, the artifact correction model Better at handling metal artifacts in the simulation domain.

[0072] The first diffusion model D sx The first diffusion model D is constructed by forward process according to the training data set. sx Perform training to obtain the first diffusion model D after training sx ; and the training data set is provided with a pair of paired data and a plurality of real metal artifact-free images y r; The paired data is a simulated metal artifact image x s and simulated metal artifact-free image y s In the first diffusion model D sx In the forward process, the simulated metal artifact image x of the counter sampling s Add T times Gaussian noise until the metal artifact image x s The data distribution p sampled in the simulation domain s (x) converges to the prior Gaussian noise image and obtains the first diffusion model D after training sx .

[0073] The second diffusion model D ry The second diffusion model D is constructed by a forward process according to the training data set. ry Perform training to obtain the trained second diffusion model D ry In the second diffusion model D ry In the forward process, the real metal artifact-free image y r Add T times Gaussian noise until the real metal artifact-free image y r In the real domain, the sample converges to the prior Gaussian noise image and obtains the trained second diffusion model D ry .

[0074] In the first diffusion model D sx In the forward process, the simulated metal artifact image x s The data distribution p sampled in the simulation domain s (x) is expressed by equation (1), the first diffusion model D sx The objective function of the forward process is expressed by formula (2);

[0075] x s ~p s (x) …… Formula (1);

[0076]

[0077] in, is the gradient operator, θ sx The first diffusion model D sx The learnable parameters of , ε is a priori Gaussian noise image with a mean of 0, The first diffusion model D sx The noise predictor of The first diffusion model D sx is a hyperparameter, and t is the uniform distribution time coefficient.

[0078] In the second diffusion model D ry In the forward process, the real metal artifact-free image y rThe data distribution p sampled in the real domain r (y) is expressed by equation (3), the second diffusion model D ry The objective function of the forward process is expressed by equation (4);

[0079] y r ~p r (y)...Formula (3);

[0080]

[0081] Among them, θ ry The second diffusion model D ry The learnable parameters of The second diffusion model The noise predictor of The second diffusion model D ry Hyperparameters of

[0082] The Gaussian noise image ε is expressed by formula (5);

[0083]

[0084] Where I is the variance, is a normal distribution;

[0085] The uniform distribution time coefficient t is expressed by formula (6);

[0086] t~Uniform({1,…,T})……Formula (6); where T is the total number of sampling times and Uniform is uniform distribution.

[0087] It should be noted that the first diffusion model D of the present invention sx and the second diffusion model D ry The forward process is to learn a noise predictor ε θ process.

[0088] The first diffusion model D sx As an example, a metal artifact image x is sampled from the simulation domain. s , while from Gaussian noise A pure noise ε is sampled as the noise predictor ; then through This weighted sum operation results in a noise-contaminated image. To control the amount of original image information, to control the intensity of the added noise. is a hyperparameter. The larger the subscript t is, The smaller it is, the less information of the original image is retained, and the greater the noise is added. In the forward process, t gradually increases from 1 to T; finally, the image contaminated by noise is and time factor t input noise predictor In the above example, we get the predicted noise and compare it with the label ε to perform MSE loss. We continuously optimize this loss training to obtain the optimal noise predictor.

[0089] The method for constructing the training data set of the present invention is:

[0090] A1. Obtain multiple pairs of simulated metal projection data and simulated metal-free projection data, as well as multiple real metal-free artifact images y r ,Each pair of simulated metal projection data and simulated metal-free projection data is for the same object and the same anatomical position;

[0091] A2. Reconstruct each pair of simulated metal projection data and simulated metal-free projection data respectively, and obtain multiple pairs of simulated metal artifact images x s and simulated metal artifact-free image y s .

[0092] It should be noted that in A1, the simulated metal projection data and the simulated metal-free projection data are obtained by simulation using a simulation toolkit, and the real physical mechanisms are considered in the simulation process, including but not limited to radiographic hardening, scattering, photon starvation, nonlinear partial volume effect and other physical mechanisms to obtain paired image data. The simulation tools of the present invention include but are not limited to GE's CatSim, Duke University's DukeSim, etc. The pairing of the present invention refers to the simulated metal artifact image x s and simulated metal artifact-free image y s Contains the same anatomy, except that the metal artifact images contain metal implants and metal artifacts. s Obey a certain data distribution p s (x), the data distribution of metal artifact images simulated by different tools and metal implants with different materials is different. Simulated image without metal artifacts y s Obey a certain data distribution p r (y), The data distribution corresponding to the metal artifact-free images obtained by metal implants of different materials, different scanners, and different reconstruction parameters is different.

[0093] The metal artifact correction model The training method is: the simulated metal artifact image x in the training data set is s Dot product complement of metal mask A metal artifact-only image is obtained, wherein the metal artifact-only image does not contain metal implants, wherein is the simulated metal artifact image x s Then the image containing only metal artifacts is input into the metal artifact correction model Get the artifact-corrected image, and calculate the artifact-corrected image and the simulated metal-free artifact image y according to the loss function MSE s The loss value between , and then update the metal artifact correction model Finally, the trained metal artifact correction model is obtained.

[0094] The loss value Loss of the loss function MSE is expressed by formula (29):

[0095]

[0096] In the S1, the conditional sampling operation is performed by using the first diffusion model D sx Add the real metal artifact image x to the reasoning process r The low-frequency characteristics φ(x r ) and perform T sampling operations, and then use the sampling result x of the last sampling i-1 As the final sampling result, the final sampling result is the metal artifact image after the domain adaptation. The conditional sampling operation in S1 of the present invention is applied to the real metal artifact image x r Conditional sampling means using the trained first diffusion model D sx Continuously sample from the Gaussian noise image to obtain a clean image, and the sampling result x of the previous stage i , time coefficient i, and conditional constraints to ensure x r The anatomical structure, artifacts, and other information remain unchanged before and after domain adaptation.

[0097] In the conditional sampling operation in S3, the second diffusion model D ry The metal artifact corrected image obtained by adding S2 during the inference process The low frequency characteristics And perform T sampling operations, and then use the sampling result x of the last sampling i-1 As the final sampling result, the final sampling result is the metal artifact-free image after domain adaptation. The conditional sampling operation in S3 of the present invention refers to the conditional input constraint in the conditional sampling using the trained second diffusion model D sx Continuously sample from the Gaussian noise image to obtain a clean image, and the sampling result x of the previous stage i, time coefficient i, and conditional constraints to ensure x r The anatomical structure, artifacts, and other information remain unchanged before and after domain adaptation.

[0098] It should be noted that the conditional constraints of the conditional sampling operation of the present invention include but are not limited to conditional input constraints and sampling enhancement constraints.

[0099] In S1, when the conditional constraint of the conditional sampling operation is a conditional input constraint, it means that in the first diffusion model D sx During the inference process, an additional real metal artifact image x is added r The low-frequency information φ(x r ) to better maintain the consistency of image anatomical structure, metal artifacts and other information before and after domain adaptation.

[0100] The sampling operation is represented by equations (7) to (11);

[0101]

[0102] ε i =D sx (x i ,i,φ(x r ))……Formula (8);

[0103]

[0104] Where i is a time variable, the value range of i is [T, 1], and T = 1000, is the noise variance, and β i are all hyperparameters, and α i =1-β i , ε i is the noise of the ith prediction, x i is the sampling result of the ith time, is the result x of the i-th sampling i The initial result after denoising weighting, z is the additional Gaussian noise image, x i-1 It is the sampling result of the current output.

[0105] In S1, when the conditional constraint of the conditional sampling operation is a conditional input constraint, the specific agreement is that the result of this sampling is x i-1 , it is agreed that the input required for this sampling is: the result of the previous sampling x i , the time node i of this sampling, and the low-frequency information φ(x r ); When i=T, the first sampling starts, and the result of this sampling is x T-1, taking pure Gaussian noise as the result of the previous sampling x T , the time node of this sampling is T; when i=1, the last sampling is performed, and the result of this sampling, that is, the final sampling result, is x0, the time node of this sampling is 1, and the result of the previous sampling is x1.

[0106] The specific sampling process is as follows: In the i-th sampling, the first diffusion model D sx Input the result x of the previous sampling i , i and the low-frequency information φ(x r ), the first diffusion model D sx The noise ε at this stage will be predicted i ; Then, the result x of the last sampling i Subtract the weighted prediction noise Then, weight Get the image after preliminary denoising Finally, add a The weighted Gaussian noise produces the final result x of this sampling i-1 .

[0107] In S1, the conditional constraint of the conditional sampling operation is a sample enhancement constraint, which refers to using the last sampling result and the real metal artifact-free image x r , the low-frequency information corresponding to the two jointly regulates the sampling result of this time. The sampling operation is represented by equations (12) to (17);

[0108]

[0109] ε i =D sx (x i ,i)...Formula (13);

[0110]

[0111] x i-1 = x′ i-1 +λ(φ(x r )-φ(x′ i-1 ))……Formula (17);

[0112] Where i is a time variable, the value range of i is [T, 1], and T = 1000, is the noise variance, and β i are all hyperparameters, and α i =1-β i , ε iis the noise in the i-th prediction, x i is the sampling result of the i-th time, is the result x of the i-th sampling i The initial result after denoising weighting, z is the additional Gaussian noise image, x′ i-1 is the initial sampling result of the current sampling, x i-1 is the sampling result of the current sampling output, and λ is the hyperparameter of the modulation.

[0113] It should be noted that in the sample enhancement constraint in S1, the noise prediction does not take into account the low-frequency information φ(x r ), and before the final output of the sampling result in equation (17), the low-frequency information φ(x i-1 ) and the low-frequency information φ(x r ) jointly modulate the sampling result.

[0114] In S1, when the conditional constraint of the conditional sampling operation is the sample enhancement constraint, specifically, the value range of i is [T, 1], where T = 1000. It is agreed that the result of this sampling is x i-1 , the input required for this sampling is the result x of the previous sampling i , the time node i of this sampling, and the low-frequency information φ(x r ). When i=T, the first sampling starts, and the result of this sampling is x T-1 , taking pure Gaussian noise as the result of the previous sampling x T , the time node of this sampling is T; when i=1, the last sampling is performed, and the result of this sampling, that is, the final sampling result, is x0, the time node of this sampling is 1, and the result of the previous sampling is x1.

[0115] The detailed sampling process is as follows: At the i-th sampling, the diffusion model D sx Input the result x of the previous sampling i , i, the diffusion model will predict the noise ε i ; Then, the result x of the last sampling i Subtract the weighted prediction noise Then, weight Get the image after preliminary denoising Then, add a Weighted Gaussian noise produces the initial result x′ of this sampling i-1 ; Finally, based on this preliminary result, formula (17) is used to make the low-frequency information φ(x r) and the low-frequency information φ(x′) of the preliminary results of this sampling i-1 ) jointly modulate the final result x of this sampling i-1 .

[0116] In S3, the conditional constraint of the conditional sampling operation is a conditional input constraint, and the sampling operation is represented by equations (18) to (23);

[0117]

[0118] Where i is a time variable, the value range of i is [T, 1], and T = 1000, is the noise variance, and β i are all hyperparameters, and α i =1-β i , ε i is the noise of the ith prediction, x i is the sampling result of the ith time, is the result x of the i-th sampling i The initial result after denoising weighting, z is the additional Gaussian noise image, x i-1 is the sampling result of the current output. The conditional input constraints of S3 are mainly different from those of S1 in equation (19), and the others are the same. The detailed sampling process will not be described one by one.

[0119] In S3, the conditional constraint of the conditional sampling operation is a sample enhancement constraint, and the sampling operation is represented by equations (23) to (28);

[0120]

[0121] ε i =D ry (x i ,i)...Formula (24);

[0122]

[0123]

[0124] Where i is a time variable, the value range of i is [T, 1], and T = 1000, is the noise variance, and β i are all hyperparameters, and α i =1-β i , ε i is the noise in the i-th prediction, x i is the sampling result of the ith time, is the result x of the i-th sampling i The initial result after denoising weighting, z is the additional Gaussian noise image, x′ i-1 is the initial sampling result of the current sampling, x i-1 is the sampling result of the current sampling output, and λ is the hyperparameter of the modulation. For the sample enhancement constraint of S3, there is only a major difference between the conditional input constraint of S1 and the conditional input constraint of S1 in formula (28), and the other parts are the same. The detailed sampling process will not be repeated one by one.

[0125] It should also be noted that the present invention is only for metal artifact correction of 2D fan beam CT, but the present invention is still applicable to metal artifact correction of 3D cone beam CT. The difference is that the FBP algorithm is changed to a 3D spiral reconstruction algorithm or a Feldkamp-Davis-Kress (FDK) algorithm during the reconstruction process, and the 2D front projection is replaced by a 3D front projection.

[0126] The present invention can well characterize the data distribution of the simulation domain and the real domain through two diffusion models, and effectively realize the domain adaptation of "real domain-simulation domain" and "simulation domain-real domain". Moreover, under the paired training data of the supervised metal artifact correction model, the present invention only needs to train a fixed diffusion model to characterize the data distribution of the simulation domain, and a fixed artifact correction model to realize artifact correction. For the real data of any manufacturer, it is only necessary to train an additional diffusion model to characterize the distribution of the real data of the manufacturer, so as to fully utilize the advantages of the supervised artifact correction model and realize cross-model artifact correction.

[0127] This CT metal artifact correction method based on driving image domain transformation through a diffusion model fully considers the domain difference problem during the reasoning of the supervised metal artifact correction model, realizes input domain adaptation through the diffusion model, and fully utilizes the correction performance of the supervised metal artifact correction model on simulated data, thereby improving the generalization performance of the model on real data.

[0128] Example 2

[0129] A CT metal artifact correction method based on image domain transformation driven by a diffusion model, the other features are the same as those of Example 1. In this example, the chest, abdomen, and head with or without metal implants are projected and reconstructed using the CatSim toolkit to obtain a simulated metal artifact image x s and simulated metal artifact-free image y s .

[0130] Specifically, this specific embodiment uses clinical chest and abdomen data and clinical head data for model training and testing. The data comes from Philips' Brilliance iCT Elite and Brilliance iCT Elite FHD respectively. The simulation toolkit uses GeCatsim to simulate projections with / without metal implants and corresponding images with / without metal artifacts.

[0131] The main material of metal implants is titanium, and the main types are heart stents, peripheral vascular stents, etc. Their shapes are obtained by segmenting clinical real metal image data. The simulated tube voltage is set to 120kV, the distance from the source to the detector (DSD) is set to 1085.6mm, the distance from the source to the center of rotation (DSO) is set to 595mm, the size of the detector unit is 1.2858mm×1.2858mm, the number of detector units is 736, the reconstructed image pixel size is 0.6934mm×0.6934mm, and the number of projections is 1152. Poisson noise and Gaussian noise are added during the simulation process.

[0132] In this embodiment, the training data of the metal artifact correction embodiment includes: a first diffusion model D sx The training data contains 18,976 simulated images with metal artifacts, of which 8,000 are from clinical head data and 10,976 are from Mayo chest and abdomen data; the metal artifact correction model The training data of the second diffusion model D ry The training data contains 30,000 real metal artifact-free images, of which 15,000 are from clinical head data and 15,000 are from clinical chest and abdomen data.

[0133] The first diffusion model D sx Use simulated metal artifact images x in training s Train the first diffusion model D sx , to characterize the data distribution p of the simulation domain with metal artifact images s (x). In this embodiment, the first diffusion model D sx is the denoising diffusion probability model DDPM. The first diffusion model D sx The number of epochs for training is 100, and the learning rate is increased from 1×10 -4 Linearly decreases to 1×10 -6 , the optimizer is Adam optimizer, where parameters (β1, β2) = (0.9, 0.999).

[0134] Metal Artifact Correction Model During training, simulated paired training data x is used s and s , train the metal artifact correction model in a supervised learning manner , to achieve chest and abdomen metal artifact correction. The metal artifact correction model It is NAF-Net. Specifically, the metal artifact correction model The number of epochs for training is 100, and the learning rate is increased from 1×10 -4 Linearly decreases to 1×10 -6 , the optimizer is Adam optimizer, where parameters (β1, β2) = (0.9, 0.999).

[0135] The second diffusion model D ry Use real metal artifact-free images y in training r Train the second diffusion model D ry , to characterize the data distribution p of real domain images without metal artifacts r (y) The second diffusion model D in this embodiment ry is the denoising diffusion probability model DDPM. Specifically, D ry The number of epochs for training is 100, and the learning rate is increased from 1×10 -4 Linearly decreases to 1×10 -6 , the optimizer is Adam optimizer, where parameters (β1, β2) = (0.9, 0.999).

[0136] In this embodiment, the test data of the specific embodiment of metal artifact correction includes: 2317 real metal artifact images x r , of which 865 cases were from clinical head data and 1452 cases were from clinical chest and abdomen data.

[0137] The first diffusion model D obtained by training sx For a real image with metal artifacts x r Perform conditional sampling to obtain an image with metal artifacts after domain adaptation The conditional sampling in this embodiment adopts conditional input constraint, and the feature extraction method adopts Fourier transform.

[0138] The metal artifact-free image finally obtained by the present invention after domain adaptation like Figure 2 and Figure 3 As shown. Among them, Figure 2 The left side shows the metal artifact effect of the head before and after correction. Figure 3The left side shows the metal artifact effects of the chest and abdomen before and after correction. The dark gray in the enlarged image in the ROI area is the real shape of the metal. It can be observed through the ROI area that the present invention can well remove strong bright and dark bands and stripe artifacts while maintaining important anatomical structure information.

[0139] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit the scope of protection of the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the essence and scope of the technical solution of the present invention.

Claims

1. A CT metal artifact correction method based on image domain transformation driven by a diffusion model, characterized in that: The steps include: S1, the real metal artifact image x r Input the first diffusion model D after training sx Perform conditional sampling operation, and the final sampling result output is an image with metal artifacts after domain adaptation S2, image with metal artifacts after domain adaptation obtained from S1 Input the trained metal artifact correction model Perform artifact correction to obtain the metal artifact corrected image S3, the metal artifact corrected image obtained in S2 Input the trained second diffusion model D ry Perform conditional sampling operation, and the final sampling result output is a metal artifact-free image after domain adaptation.

2. The CT metal artifact correction method based on image domain transformation driven by diffusion model according to claim 1, characterized in that: The first diffusion model D sx The first diffusion model D is constructed by forward process according to the training data set. sx Perform training to obtain the first diffusion model D after training sx ; and the training data set is provided with a pair of paired data and a plurality of real metal artifact-free images y r ; The paired data is a simulated metal artifact image x s and simulated metal artifact-free image y s ; The second diffusion model D ry The second diffusion model D is constructed by a forward process according to the training data set. ry Perform training to obtain the trained second diffusion model D ry .

3. The CT metal artifact correction method based on image domain transformation driven by diffusion model according to claim 2, characterized in that: In the first diffusion model D sx In the forward process, the simulated metal artifact image x of the counter sampling s Add T times Gaussian noise until the metal artifact image x s The data distribution p sampled in the simulation domain s (x) converges to the prior Gaussian noise image and obtains the first diffusion model D after training sx ; In the second diffusion model D ry In the forward process, the real metal artifact-free image y r Add T times Gaussian noise until the real metal artifact-free image y r In the real domain, the sample converges to the prior Gaussian noise image and obtains the trained second diffusion model D ry .

4. The CT metal artifact correction method based on image domain transformation driven by diffusion model according to claim 3, characterized in that: In the first diffusion model D sx In the forward process, the simulated metal artifact image x s The data distribution p sampled in the simulation domain s (x) is expressed by equation (1), the first diffusion model Ds x The objective function of the forward process is expressed by formula (2); x s ~p s (x) ……Formula (1); in, is the gradient operator, θ sx The first diffusion model D sx The learnable parameters of , ε is a priori Gaussian noise image with a mean of 0, The first diffusion model D sx The noise predictor of is a hyperparameter, t is a uniformly distributed time coefficient; In the second diffusion model D ry In the forward process, the real metal artifact-free image y r The data distribution p sampled in the real domain r (y) is expressed by equation (3), the second diffusion model D ry The objective function of the forward process is expressed by equation (4); y r ~p r (y) …… Formula (3); Among them, θ ry The second diffusion model D ry The learnable parameters of The second diffusion model D ry Noise predictor for ; The Gaussian noise image ε is expressed by formula (5); Where I is the variance, is a normal distribution; The uniform distribution time coefficient t is expressed by formula (6); t~Uniform({1,…,T}) ……Formula (6); Among them, T is the total number of sampling times, and Uniform is uniform distribution.

5. The CT metal artifact correction method based on image domain transformation driven by diffusion model according to claim 4, characterized in that: In the S1, the conditional sampling operation is performed by using the first diffusion model D sx Add the real metal artifact image x to the reasoning process r The low-frequency characteristics φ(x r ) and perform T sampling operations, and then use the sampling result x of the last sampling i-1 As the final sampling result, the final sampling result is the metal artifact image after the domain adaptation. In the conditional sampling operation in S3, the second diffusion model D ry The metal artifact corrected image obtained by adding S2 during the inference process The low frequency characteristics And perform T sampling operations, and then use the sampling result x of the last sampling i-1 As the final sampling result, the final sampling result is the metal artifact-free image after domain adaptation.

6. The CT metal artifact correction method based on image domain transformation driven by diffusion model according to claim 5, characterized in that: In S1, the conditional constraint of the conditional sampling operation is a conditional input constraint, and the sampling operation is represented by equations (7) to (11); ε i = D sx (x i , i, φ(x r )) …… Equation (8); Where i is a time variable, the value range of i is [T, 1], and T = 1000, is the noise variance, and β i are all hyperparameters, and α i =1-β i , ε i is the noise of the ith prediction, x i is the sampling result of the i-th time, is the result x of the i-th sampling i The initial result after denoising and weighting, z is the additional Gaussian noise image, x i-1 It is the sampling result of the current output.

7. The CT metal artifact correction method based on image domain transformation driven by diffusion model according to claim 5, characterized in that: In S1, the conditional constraint of the conditional sampling operation is a sample enhancement constraint, and the sampling operation is represented by equations (12) to (17); ε i = D sx (x i , i) …… Equation (13); x i-1 = x' i-1 + λ(φ(x r )) - φ(x' i-1 ))... Equation (17); Where i is a time variable, the value range of i is [T, 1], and T = 1000, is the noise variance, and β i are all hyperparameters, and ε i is the noise in the i-th prediction, x i is the sampling result of the i-th time, is the result x of the i-th sampling i The initial result after denoising and weighting, z is the additional Gaussian noise image, x′ i-1 is the initial sampling result of the current sampling, x i-1 is the sampling result of the current sampling output, and λ is the hyperparameter of the modulation.

8. The CT metal artifact correction method based on image domain transformation driven by diffusion model according to claim 5, characterized in that: In S3, the conditional constraint of the conditional sampling operation is a conditional input constraint, and the sampling operation is represented by equations (18) to (23); Where i is a time variable, the value range of i is [T, 1], and T = 1000, is the noise variance, and βi are hyperparameters, and ε i is the noise of the ith prediction, x i is the sampling result of the i-th time, is the result x of the i-th sampling i The initial result after denoising weighting, z is the additional Gaussian noise image, x i-1 It is the sampling result of the current output.

9. The CT metal artifact correction method based on image domain transformation driven by diffusion model according to claim 5, characterized in that: In S3, the conditional constraint of the conditional sampling operation is a sample enhancement constraint, and the sampling operation is represented by equations (23) to (28); ε i = D ry (x i , i) …… Equation (24); Where i is a time variable, the value range of i is [T, 1], and T = 1000, is the noise variance, and βi are hyperparameters, and ε i is the noise in the i-th prediction, x i is the sampling result of the i-th time, is the result x of the i-th sampling i The initial result after denoising and weighting, z is the additional Gaussian noise image, x′ i-1 is the initial sampling result of the current sampling, x i-1 is the sampling result of the current sampling output, and λ is the hyperparameter of the modulation.

10. The CT metal artifact correction method based on image domain transformation driven by diffusion model according to claim 2, characterized in that: The training data set is constructed as follows: A1. Obtain multiple pairs of simulated metal projection data and simulated metal-free projection data, as well as multiple real metal-free artifact images y r ,Each pair of simulated metal projection data and simulated metal-free projection data is for the same object and the same anatomical position; A2. Reconstruct each pair of simulated metal projection data and simulated metal-free projection data respectively, and obtain multiple pairs of simulated metal artifact images x s and simulated metal artifact-free image y s ; The metal artifact correction model The training method is: the simulated metal artifact image x in the training data set is s Dot product complement of metal mask A metal artifact-only image is obtained, wherein the metal artifact-only image does not contain metal implants, wherein is the simulated metal artifact image x s Then the image containing only metal artifacts is input into the metal artifact correction model Get the artifact-corrected image, and calculate the artifact-corrected image and the simulated metal-free artifact image y according to the loss function MSE s The loss value between , and then update the metal artifact correction model Finally, the trained metal artifact correction model is obtained. The loss value Loss of the loss function MSE is expressed by formula (29):

Images