CT image metal artifact removal method based on dynamic weighted depth expansion network

Abstract

The invention provides a CT (Computed Tomography) image metal artifact removal method based on a dynamic weighted depth expansion network, which comprises the following steps of: constructing an MAR model with an implicit regular term, and manually setting a weighting function; converting the MAR model with an implicit regular term into a saddle point problem with a penalty term, and solving the saddle point problem with the penalty term by using a prime-dual hybrid gradient algorithm to obtain a double-domain iteration format containing a near-end operator; the double-domain iteration format containing the near-end operator is expanded into a convolutional neural network, and a deep expansion network is obtained; determining a weight optimization network W-Net according to the weight function, combining the weight optimization network W-Net and the deep expansion network to obtain a dynamic weighted deep expansion network, and training the dynamic weighted deep expansion network by using an existing CT data set to obtain a metal artifact removal model; and inputting a to-be-processed CT metal artifact image into the trained metal artifact removal model to obtain a reconstructed metal artifact-removed CT image. Good MAR results can be obtained for artifacts of different levels, and the effectiveness and robustness of the method are shown.

Description

Technical Field

[0001] The present invention relates to the technical field of CT metal artifact removal, and in particular to a CT image metal artifact removal method based on a dynamic weighted deep expansion network. Background Art

[0002] CT imaging technology plays an increasingly important role in the auxiliary diagnosis of clinical diseases due to its fast imaging speed and ability to present clear anatomical relationships. However, in clinical CT examinations, if metal objects (such as metal teeth, metal vertebral fixtures, metal heart stents, etc.) are implanted in the patient's body due to surgery or other reasons, under the influence of factors such as X-ray beam hardening, photon noise and partial volume effect, severe light and dark alternating ray-like or block-like metal artifacts will be produced in the reconstructed CT image. These metal artifacts will cover a large area of ​​the tissue structure information around the metal, not only showing Figure 1 The metal artifact removal (MAR) of CT images plays a vital role in clinical diagnosis and disease treatment. How to automatically remove metal artifacts from CT images has become a hot research topic.

[0003] As a key mathematical technique for MAR, variational regularization methods provide interpretability, stability, and computational efficiency with moderate local resource requirements. Since the artifact removal process involves solving mathematically ill-posed problems, regularization-based methods play a vital role. Most variational models are based on the piecewise smoothness of CT images, and introduce total variation (TV) regularization to directly repair metal trajectories, effectively suppressing metal artifacts and noise. However, the reconstruction results are prone to strong diffusion around high-density materials, and even introduce new artifacts. Another part of the method greatly improves the performance of the model by normalizing the projection domain information of the pre-segmentation, but this requires relatively accurate pre-segmentation. However, segmenting CT images contaminated by metal artifacts is also challenging. In recent years, with the remarkable success of deep learning in medical image processing, it has been realized to use deep neural networks (DNNs) to solve the problem of metal artifact removal. The deep learning-based MAR method directly transforms the metal artifact correction task into a general image restoration problem through an end-to-end learning method. It uses a multi-level neural network to automatically identify the semantic features of the data and learn the missing information from a large amount of data, which greatly improves the reconstruction accuracy of traditional methods and enhances the generalization ability. However, these methods lack sufficient embedding of inherent physical geometric constraints throughout the learning process, resulting in poor model interpretability. In addition, data-driven deep learning MAR methods also face challenges such as large data requirements and long training time.

[0004] Based on the MAR method combining data-driven and model-driven, the learning of the model is guided by the fusion imaging mechanism, which not only retains the interpretability of the model-driven method and the reliability of data prediction, but also integrates the adaptive ability of the data-driven method, which can effectively improve the effect of the model in removing metal artifacts. As a classic MAR method combining data and model-driven, an interpretable dual-domain network (IndudoNet) for CT metal artifact removal uses the proximal gradient algorithm to design an unfolded network based on the dual-domain regularization model. It not only inherits the excellent properties of the variational model, but also can use the powerful representation and learning capabilities of the neural network to improve the reconstruction accuracy. However, the results of this network are affected by the preprocessing prior on the one hand; on the other hand, directly losing the information of the metal trajectory may cause excessive effective information to be lost, thereby losing the texture structure of the reconstructed image. Summary of the invention

[0005] In view of the technical problems existing in the existing CT image metal artifact removal methods, such as being easily affected by preprocessing, insufficient reconstruction accuracy and texture loss, the present invention proposes a CT image metal artifact removal method based on a dynamic weighted deep unfolding network. The method does not rely on any reconstruction priors, cleverly sets a dynamic weight function penalty mechanism, effectively retains the texture information of the reconstructed image, and improves the diffusion problem in the reconstructed image.

[0006] In order to achieve the above object, the technical solution of the present invention is implemented as follows: a method for removing metal artifacts from CT images based on a dynamic weighted deep expansion network, comprising the following steps:

[0007] Step 1: Build a MAR model with an implicit regularization term and manually set the weight function in the MAR model;

[0008] Step 2: According to the pre-dual form of the two-norm square, the MAR model with implicit regularization term is transformed into a saddle point problem with a penalty term. The saddle point problem with the penalty term is solved using the primal-dual mixed gradient algorithm, and a dual-domain iterative format with proximal operators is obtained.

[0009] Step 3: Expand the dual-domain iterative format containing the proximal operator into a convolutional neural network to obtain a deep expanded network;

[0010] Step 4: Determine the weight optimization network W-Net according to the weight function of step 1, combine the weight optimization network W-Net with the deep expansion network to obtain a dynamic weighted deep expansion network, and use the existing CT data set to train the dynamic weighted deep expansion network to obtain a metal artifact removal model;

[0011] Step 5: Input the CT metal artifact image to be processed into the trained metal artifact removal model to obtain a reconstructed CT image with metal artifacts removed.

[0012] Preferably, the MAR model with implicit regularization term in step 1 is:

[0013] Sine diagram for multi-energy measurement with m1 and m2 as the number of projection views and detectors The MAR model with implicit regularization is:

[0014]

[0015] Among them, the original variable is the image to be reconstructed, X represents the image space, n is the number of pixels of width and height of the image to be reconstructed; regularization weight parameter λ>0, is the Radon transform, W is the weight function, ⊙ is the matrix product operation according to the corresponding elements, ∥∥ 2 represents the square operation of the second norm, and g(·) is the regularization term that introduces prior knowledge.

[0016] Preferably, the weight function is:

[0017]

[0018] Among them, max{·,·} means taking the maximum value of each element of the two matrices, and the parameter ε is a constant; represents a matrix of all 1s, Indicates that the highly attenuated projection area O t Set to 0, and the rest of the area to 1;

[0019] The high attenuation projection area O t for

[0020]

[0021] Where Ω is the metal trajectory, t is the threshold parameter, and Y i,j Represents the value of the element in the i-th row and j-th column of the sinusoidal graph Y, ∥Y∥ max Represents the maximum element value in the sine graph Y.

[0022] Preferably, the method of converting the MAR model with implicit regularization term into a saddle point problem with penalty term is:

[0023] Set the intermediate variable The implicit regularization term G(·) = λg(·) can be rewritten as

[0024]

[0025] Among them, data fidelity

[0026] According to the pre-dual form of the two-norm square, the above model is transformed into a saddle point problem:

[0027]

[0028] Among them, the dual function y represents the dual variable;

[0029] The method of solving the saddle point problem with penalty term using the primal-dual hybrid gradient algorithm is:

[0030] The update method of the dual variable y and the original variable u in the k+1th iteration is

[0031]

[0032] The dual variable y has a closed form solution

[0033]

[0034] in, Prox τG () respectively represent σF * and the neighboring operator of σG, y k ,y k+1 denote the dual variables of the kth iteration and the k+1th iteration, u k 、u k+1 denote the original variables of the kth iteration and the k+1th iteration respectively, represents the extrapolated variable used to accelerate the algorithm, σ, τ, and γ are all step size parameters, is the inverse Radon transform.

[0035] Preferably, the deep expansion network includes an original variable network u-Net and a dual variable network y-Net, the original variable network u-Net and the dual variable network y-Net both include N layers, the original variable network u-Net and the dual variable network y-Net of the corresponding layers form N network blocks, and the N network blocks are connected in this way; each parameterized network block of the dual variable network y-Net and the original variable network u-Net has the same structure: {convolution + PReLU + convolution + PReLU + convolution}; each network block adopts a residual learning method that fuses input information and network block output information;

[0036] According to the updating method of the original variable u and the closed-form solution of the dual variable y, they are respectively used as the input of each network block of the original variable network u-Net and the dual variable network y-Net. The network blocks of the dual network y-Net and the original network u-Net respectively update the original variable u and the dual variable y continuously and alternately.

[0037] Preferably, the original variable u is initialized0 = 0 and the dual variable y 0 = 0, and the dual variable y 0 , intermediate variables and W⊙Y as the input of the first layer of the dual variable network y-Net, and the output of the first layer of the dual variable network y-Net is the dual variable y of the first iteration 1 ; The original variable u 0 = 0 and intermediate variables Input the input of the first layer of the original variable network u-Net, and the output of the first layer of the original variable network u-Net is the original variable u of the first iteration 1 ; The output of the first layer of the original variable network u-Net and the output of the first layer of the dual variable network y-Net are used as the input of the second layer of the original variable network u-Net and the dual variable network y-Net; and so on, the dual variable y output of the kth layer of the dual variable network y-Net is used as the input of the second layer of the original variable network u-Net and the dual variable network y-Net. k And intermediate variables and W⊙Y as the input of the k+1th layer of the dual variable network y-Net, and the output of the k+1th layer of the dual variable network y-Net is the dual variable y k+1 , the original variable u output by the kth layer of the original variable network u-Net k and intermediate variables As the input of the k+1th layer of the original variable network u-Net, the k+1th layer of the original variable network u-Net outputs the original variable u k+1 ; The original variable output by N network blocks is the image to be reconstructed.

[0038] Preferably, the multi-energy measured sinusoidal graph Y, the weight function W and the metal track mask Tr are simultaneously input into the pre-training network Prior-Net to obtain the weight function W of the 0th iteration. 0 ;

[0039] The weight function W 0 , sinusoidal graph Y and metal trajectory mask Tr are used as inputs of the weight optimization network W-Net to dynamically update the weight function.

[0040] Preferably, the network structure of the pre-trained network Prior-Net includes three Conv+PRelu modules and a BN module connected in sequence, and also includes a residual learning unit, which adds the input data to the data output by the BN module to obtain the weight function W of the 0th iteration 0 ;

[0041] By initializing the original variable u 0 =[u 0 ,u 0 ,u0 ,u 0 ,u 0 ] and the dual variable y 0 =[y 0 ,y 0 ,y 0 ,y 0 ,y 0 ] Expand the variable space and deeply expand the network to record the original variables updated before during the update process; utilize the powerful representation ability of the network to adaptively learn the acceleration rules, and after the optimization of N network blocks, reconstruct the CT image after removing artifacts.

[0042] Preferably, the k-th layer of the weight optimization network W-Net is added to the k-th network block of the depth expansion network, and the data processing method of the k-th network block is:

[0043] The output of the kth layer of the dual variable network y-Net is:

[0044]

[0045] The output of the kth layer of the original variable network u-Net is:

[0046]

[0047] The output of the kth layer of the weight optimization network W-Net is:

[0048]

[0049] in, represents the operation function of the dual variable network y-Net, represents the operation function of the original variable network u-Net, represents the operation function of the weight optimization network W-Net, Q l represents the weight function of the k-1th layer output of the weight optimization network W-Net, is the second slice of the dual variable network y-Net output in the kth network block, u k is the original variable output by the k-1th layer of the original variable network u-Net, Represents the first slice of the dual variable network y-Net output in the k+1th network block.

[0050] Preferably, the loss function during training is The mean square error is used as the objective function for training:

[0051]

[0052] Where M represents the number of samples, Y i and Tr i They represent the ith CT training data without metal artifacts, the corresponding sinusoidal data, and the mask corresponding to the metal trajectory, respectively. For CT training data The corresponding final model outputs a CT image with metal artifacts removed.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] 1. The present invention attempts to construct a MAR model with an implicit regularization term, so as to improve the diffusion phenomenon in the reconstructed image. At the same time, in order to avoid the loss of a large amount of effective information during the reconstruction process, the present invention manually designs weights during modeling to achieve regional differentiation artifact removal;

[0055] 2. The present invention uses the primal-dual hybrid gradient algorithm (PDHG) to write the constructed MAR model into a dual-domain iterative framework with proximal operators, and learns a more appropriate representation through the regularity of the network to improve the reconstruction accuracy;

[0056] 3. The present invention considers using a shallow neural network to dynamically learn weight functions to enhance the adaptability and expression ability of the MAR model to artifacts in different regions;

[0057] 4. The present invention has conducted a large number of experiments, including CT images of different numbers of tissues, and can obtain good MAR results for artifacts of different levels, demonstrating the effectiveness and robustness of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0059] Figure 1 The following are examples of metal artifacts in CT images, where (a) is a CT image of metal teeth and (b) is a CT image of metal vertebral fixtures.

[0060] Figure 2 It is a flow chart of the present invention.

[0061] Figure 3 Implementation pipeline for metal artifact removal for depth unwrapping.

[0062] Figure 4 Implementation pipeline for metal artifact removal with dynamic weighted depth expansion.

[0063] Figure 5 This is the structural diagram of the pre-training network Prior-Net.

[0064] Figure 6 The reconstruction results and comparative experiments of sciatic CT images, where (a) is the original image, (b) is the image processed by the CG method, (c) is the image processed by LI correction, (d) is the image processed by NMAR correction, (e) is the image processed by LearnedPD, (f) is the image processed by IndudoNet, and (g) is the image processed by the present invention.

[0065] Figure 7 The reconstruction results and comparative experiments of hip joint CT images, where (a) is the image processed by LearnedPD, (b) is the image processed by IndudoNet, and (c) is the image processed by the present invention.

[0066] Figure 8 The reconstruction results and comparative experiments of abdominal CT images, where (a) is the image processed by LearnedPD, (b) is the image processed by IndudoNet, and (c) is the image processed by the present invention.

[0067] Fig. 9 for Figure 8 Magnified view of the area near the metal, where (a) is the image processed by LearnedPD, (b) is the image processed by IndudoNet, and (c) is the image processed by the present invention. DETAILED DESCRIPTION

[0068] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0069] As Figure 2 As shown, the present invention proposes a method for removing metal artifacts from CT images based on a dynamic weighted deep expansion network. First, a MAR model with a traditional implicit regularization is constructed. Then, the PDHG algorithm is used to solve the dual-domain expansion framework with a proximal operator, and the expansion network is used to iteratively update the original dual variables. Finally, the deep expansion network is used to dynamically update the weight function to achieve metal artifact removal of the dynamic weighted deep expansion network. The specific steps of the present invention are as follows:

[0070] Step 1: Build a MAR model with an implicit regularization term and manually set the weight function in the MAR model.

[0071] Sine diagram for multi-energy measurement with m1 and m2 as the number of projection views and detectors The traditional MAR model based on iterative optimization is expressed as:

[0072]

[0073] The image to be reconstructed X represents the image space, n is the number of pixels in the image width and height. The regularization weight parameter λ>0, is Radon transform, weight function is the mask of the binary metal track, which takes 1 in the metal area and 0 in the non-metal area; ⊙ is the matrix product operation according to the corresponding elements, ∥∥ 2 represents the square operation of the second norm, and g(·) is the regularization term that introduces prior knowledge.

[0074] Directly discarding the information of the metal track may cause excessive effective information to be lost, thereby losing the texture result of the image u to be reconstructed. For this reason, the present invention adopts a manually designed weight function:

[0075]

[0076] Among them, max{·,·} represents the maximum value of each element of the two matrices, and the parameter ε is set to avoid the score being 0, which is generally 10 -16 . represents a matrix of all 1s, Indicates that the projection area O t Set to 0, and the rest of the area to 1. t Represents the projection area with high attenuation, and the specific form is

[0077]

[0078] Where Ω is the metal track, indicating the area where the metal track mask Tr takes the value of 1; t is the threshold parameter, Y i,j Represents the value of the element in the i-th row and j-th column of the sinusoidal graph Y, ∥Y∥ max Represents the maximum element value in the sine graph Y.

[0079] Setting the weight function W according to the above method can realize setting the projection of the high attenuation area as a mask, and taking the inverse proportion of the square of the measured projection in other areas, which can enhance the contrast information of the edge of the reconstructed image and effectively alleviate the diffusion phenomenon around the metal in the reconstructed image and the problems of secondary artifacts.

[0080] Variational regularization is a key mathematical method for solving ill-posed inverse problems. It is widely used in MAR tasks because of its good interpretability, stability and computational efficiency. However, although the variational method can improve the effect of metal artifact removal, the reconstructed image still has a certain degree of blur, especially for the area near the metal containing CT texture details. The present invention adopts an implicit regularization term g(·) to learn a more generalized representation through the regularity of the neural network, thereby alleviating the diffusion phenomenon in the reconstructed image and improving the accuracy of MAR.

[0081] Step 2: Primal-dual hybrid gradient (PDHG) framework: According to the pre-dual form of the two-norm square, the MAR model with implicit regularization term is transformed into a saddle point problem with a penalty term. The primal-dual hybrid gradient algorithm is used to solve the saddle point problem with the penalty term, and a dual-domain iterative format with proximal terminals is obtained.

[0082] Set the intermediate variable The implicit regularization term G(·) = λg(·) rewrites the MAR model of formula (1) into the following form:

[0083]

[0084] Among them, data fidelity According to the pre-dual form of the two-norm square, the model of formula (2) is transformed into the following saddle point problem:

[0085]

[0086] Among them, the dual function y represents the dual variable. The method of solving the saddle point problem (3) using the primal-dual hybrid gradient (PDHG) algorithm is:

[0087] The update method of the dual variable y and the original variable u in the k+1th iteration is

[0088]

[0089] The dual variable y has a closed form solution

[0090]

[0091] in, Prox τG () respectively represent σF * and the neighboring operator of σG, y k ,y k+1 Denote the dual variables y and u of the kth iteration and the k+1th iteration respectively k 、u k+1 denote the original variable u of the kth iteration and the k+1th iteration respectively, represents the extrapolated variable used to accelerate the algorithm, σ, τ, and γ are all step size parameters, is the inverse Radon transform. It can be summarized as the following algorithm:

[0092]

[0093] Among them, Algorithm 1 considers the relative error between the reconstructed CT image and the target CT image and the number of iterations as the termination conditions.

[0094] The present invention converts the MAR model into a saddle point problem with a penalty term and uses the PDHG algorithm to reconstruct CT images in an iterative manner, thereby effectively removing metal artifacts. Under appropriate parameter conditions, the method exhibits good convergence. In this process, all alternating update steps only contain simple proximal operators, making the expansion process easy to execute, providing a theoretical basis for the implementation of subsequent deep network expansion algorithms.

[0095] Step 3: Based on the characteristic that neural networks can calculate any complex function, the dual-domain iterative format containing proximal terminals is expanded into convolutional neural networks to obtain a deep expanded network.

[0096] The gradient descent step for the subproblem of reconstructing the image u allows the network to learn the original variable u at the kth iteration k and The combination between them expands the solution space. Specifically, corresponding to the sth iteration, the original variable u s and As the input of the original variable network u-Net, get the updated reconstructed image u s+1 Similarly, y s , and W⊙Y as the input of the dual variable network y-Net to obtain the updated dual variable y k+1 After a given input, the dual network y-Net and the original network u-Net are continuously and alternately updated, as shown in Figure 3 The entire model depicted includes N stage updates, corresponding to N iterations of Algorithm 1, where N can be set based on observed experimental results and termination conditions. Each parameterized network block of the dual variable network y-Net and the original variable network u-Net has the same structure: {convolution + PReLU + convolution + PReLU + convolution}. In order to ensure the stability of multiple iterative updates, each network block adopts a residual learning method that fuses input information and network block output information.

[0097] For extrapolated variables The present invention no longer uses the traditional iterative rule, but instead uses u 0 =[u 0 ,u0 ,u 0 ,u 0 ,u 0 ] and y 0 =[y 0 ,y 0 ,y 0 ,y 0 ,y 0 ] Expand the variable space, allowing the network to record the reconstruction result u of the previous update step during the update process s Secondly, in order to effectively accelerate, specific compensation parameters (regularization parameter λ, step size parameters σ, τ, γ) are no longer used to control the external compensation, but the acceleration rules are adaptively learned by the network based on the powerful representation ability of the neural network. After N stages of optimization, the framework can well reconstruct the CT image after removing artifacts.

[0098] It is summarized in the following algorithm:

[0099]

[0100] in, represents the operation function of the dual variable network y-Net, represents the operation function of the original variable network u-Net, y s ,y s+1 Respectively represent the input and output of the sth network block, u s 、u s+1 Represent the input and output of the sth network block respectively. It means taking the second slice of the output of the sth network block. Indicates taking the first slice of the output of the sth network block. represents the first slice of the output of the s+1th dual variable network block, is the output of the Nth original variable network u-Net, representing the target reconstructed CT image.

[0101] This step expands the dual-domain iterative format of the proximal operator into a convolutional neural network, and through end-to-end training and learning, controls the optimization direction of the network according to the loss function to obtain the optimal network parameters and improve the accuracy of reconstructed CT images. However, with the update of the original variable u and the dual variable y, different weight functions will have different effects on the solution of the saddle point problem. In order to further improve the performance of the model, the present invention will choose to use a shallow convolutional network to dynamically learn weights to further optimize the mechanism of data fidelity.

[0102] Step 4: Determine the weight optimization network W-Net according to the manually designed weight function, combine the weight optimization network W-Net with the deep unfolding network to obtain a dynamic weighted deep unfolding network, use the existing public CT data set to train the dynamic weighted deep unfolding network to obtain a metal artifact removal model; input the CT metal artifact image to be processed into the trained metal artifact removal model to obtain a reconstructed CT image with metal artifacts removed.

[0103] The dual variable y, the original variable u and the weight function are iterated respectively using the dynamic weighted deep expansion network to obtain the reconstructed image after the metal artifacts are removed.

[0104] The proposed dynamic weighted depth expansion metal artifact removal model (DuWeNet+) process is as follows Figure 4 First, using the powerful representation ability of the neural network, the multi-energy measurement sinusoidal graph Y, the manually designed weight function W and the metal track mask Tr are simultaneously input into the pre-trained network Prior-Net to obtain the weight function W of the 0th iteration. 0 , improving the robustness of the weight function. Figure 5 The network structure of Prior-Net is shown, including three Conv+PRelu modules and a BN module connected in sequence, as well as a residual learning unit. The residual learning unit adds the input data to the data output by the BN module to obtain the weight function W of the 0th iteration. 0 . Secondly, the weight function W 0 , sinusoidal graph Y and metal track mask Tr are used as inputs of weight optimization network W-Net to dynamically update the weight function. Regarding the learning of dual variable update network y-Net and primal variable update network u-Net, except that the weight function changes, the specific update method is the same as step 3. The algorithm is summarized as follows:

[0105]

[0106] Among them, N p 、N q denote the number of slices of the primal variable and the dual variable, respectively. Represents the operation function of the pre-trained network Prior-Net, Represents the operation function of the weight optimization network W-Net, represents the operation function of the original variable network u-Net, It is the output of the Nth original variable update network u-Net, representing the target reconstructed CT image.

[0107] The loss function of the present invention is:

[0108] Paired data The mean square error is used as the objective function for training:

[0109]

[0110] Where M represents the number of samples, Y i and Tr i They represent the ith CT training data without metal artifacts, the corresponding sinusoidal data, and the mask corresponding to the metal trajectory, respectively. For CT training data The corresponding final model outputs a CT image with metal artifacts removed.

[0111] The experimental platform of the present invention is: on the NVIDIA RTX A6000 GPU, the Radon transform encapsulated by the ODL library in PyTorch is used and inverse Radon transform Implement the algorithm of the present invention.

[0112] Specific Experiment 1

[0113] Figure 6 The reconstruction and comparison results of the sciatic CT images by the present invention and some MAR methods are shown. Figure 6 As shown in (b), the sciatic CT image reconstructed by the conjugate gradient method (CG) is seriously affected by artifacts and noise, which reduces the image quality and masks key clinical information, affecting the accuracy of diagnosis. Figure 6 As shown in (c), the image corrected by linear interpolation (LI) alleviates the original artifacts to a certain extent, but there are still artifacts remaining and secondary artifacts are introduced; Figure 6 As shown in (d), the result of the normalized metal artifact removal method (NMAR) correction greatly improves the above situation. The artifacts between metals are almost completely removed, but new artifacts are still introduced at the metal boundaries and contain noise, which loses useful detail information to a certain extent. Figure 6 As shown in (e) and (f), the corrected images of the learnable primal-dual reconstruction method (LearnedPD) and the interpretable dual-domain metal artifact removal method (IndudoNet) show that the deep learning method not only performs well in removing metal artifacts, but also shows good denoising effects. However, since the information of the metal area is overly suppressed or completely ignored, some important texture information is lost in the reconstructed image, which is crucial to maintaining the naturalness and clinical value of the image. Figure 6 As shown in (g), the image obtained by the method proposed in the present invention comprehensively considers the complexity and diversity of metal artifacts, and adopts an innovative algorithm framework to effectively maintain the original texture and details of the image while significantly removing noise and correcting artifacts.

[0114] Specific Experiment 2

[0115] Figure 7 The test results of the present invention and other correction algorithms based on deep learning on hip CT images are shown. Due to the interference of metal, such as Figure 7 As shown in (a), some important information near the bones is lost in the CT image corrected by LearnedPD. Figure 7 As shown in (b), because the interpolation algorithm is not accurate enough, the learned prior knowledge cannot be reasonably applied to the image reconstruction process, which introduces some new noise structures near the bones in the IndudoNet reconstructed image. Figure 7 As shown in (c), the DuWeNet+ proposed in this invention greatly reduces the information loss caused by metal artifacts. In order to comprehensively evaluate the performance of the proposed method, Figure 8 The test results of the present invention and related algorithms when a large metal is introduced into the abdominal CT image are shown. In order to show the restoration of the texture structure near the metal in more detail, Fig. 9 A magnified view of the area near the metal is provided in . The results show that LearnedPD and IndudoNet almost lost the information of the entire vertebra, while the DuWeNet+ proposed in the present invention can still correct the artifacts well while maintaining the details.

[0116] Specific experiment three

[0117] In order to quantitatively analyze the performance difference between the present invention and other models, Table 1 shows the PSNR and SSIM results of different methods. It can be clearly seen that the method based on deep learning has obvious advantages in performance compared with the traditional algorithm. This is not only reflected in small metal samples, but also in large metal samples. The present invention also shows more competitive PSNR and SSIM index results, thus verifying the efficiency and versatility of the present invention.

[0118] Table 1 PSNR / SSIM indicators of MAR reconstructed images using different methods

[0119] Model Sciatica Hip abdomen Test set average CG 27.9771 / 0.7539 29.0561 / 0.9102 24.6970 / 0.6083 27.0251 / 0.7228 LI 28.1139 / 0.8712 29.2288 / 0.9222 26.8808 / 0.7433 -- NMAR 29.2877 / 0.9131 29.3561 / 0.9502 28.5202 / 0.8563 -- LearnedPD 48.6051 / 0.9903 48.9199 / 0.9904 39.8922 / 0.9888 47.0251 / 0.9928 IndudoNet 45.4843 / 0.9911 44.8775 / 0.9909 37.2019 / 0.9851 41.6660 / 0.9907 DuWeNet+ 52.0239 / 0.9964 52.5334 / 0.9970 50.0102 / 0.9934 52.0479 / 0.9965

[0120] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for removing metal artifacts from CT images based on a dynamic weighted deep unfolding network. It is characterized in that The steps include: Step 1: Build a MAR model with an implicit regularization term and manually set the weight function in the MAR model; Step 2: According to the pre-dual form of the two-norm square, the MAR model with implicit regularization term is transformed into a saddle point problem with a penalty term. The saddle point problem with the penalty term is solved using the primal-dual mixed gradient algorithm, and a dual-domain iterative format with proximal operators is obtained. Step 3: Expand the dual-domain iterative format containing the proximal operator into a convolutional neural network to obtain a deep expanded network; Step 4: Determine the weight optimization network W-Net according to the weight function of step 1, combine the weight optimization network W-Net with the deep expansion network to obtain a dynamic weighted deep expansion network, and use the existing CT data set to train the dynamic weighted deep expansion network to obtain a metal artifact removal model; Step 5: Input the CT metal artifact image to be processed into the trained metal artifact removal model to obtain a reconstructed CT image with metal artifacts removed.

2. The method for removing metal artifacts from CT images based on a dynamic weighted deep expansion network according to claim 1, It is characterized in that The MAR model with implicit regularization term in step 1 is: For m 1 and m 2 Sine diagram of multi-energy measurement for the number of projection views and the number of detectors The MAR model with implicit regularization is: Among them, the original variable is the image to be reconstructed, X represents the image space, n is the number of pixels of the width and height of the image to be reconstructed; the regularization weight parameter λ>0, is the Radon transform, W is the weight function, ⊙ is the matrix product operation according to the corresponding elements, || || 2 represents the square operation of the second norm, and g(·) is the regularization term that introduces prior knowledge.

3. The method for removing metal artifacts from CT images based on a dynamic weighted deep expansion network according to claim 2, It is characterized in that The weight function is: Among them, max{·,·} means taking the maximum value of each element of the two matrices, and the parameter ε is a constant; represents a matrix of all 1s, Indicates that the highly attenuated projection area O t Set to 0, and the rest of the area to 1; The high attenuation projection area O t for Where Ω is the metal trajectory, t is the threshold parameter, and Y i，j represents the value of the element in the i-th row and j-th column of the sinusoidal graph Y, ||Y|| max Represents the maximum element value in the sine graph Y.

4. The method for removing metal artifacts from CT images based on a dynamic weighted deep expansion network according to claim 2 or 3, It is characterized in that The method of converting the MAR model with implicit regularization term into a saddle point problem with penalty term is: Set the intermediate variable The implicit regularization term G(·) = λg(·) can be rewritten as Among them, data fidelity According to the pre-dual form of the two-norm square, the above model is transformed into a saddle point problem: Among them, the dual function y represents the dual variable; The method of solving the saddle point problem with penalty term using the primal-dual hybrid gradient algorithm is: The update method of the dual variable y and the original variable u in the k+1th iteration is The dual variable y has a closed form solution in, Prox τG () respectively represent σF * and the neighboring operator of σG, y k ,y k+1 denote the dual variables of the kth iteration and the k+1th iteration, u k 、u k+1 denote the original variables of the kth iteration and the k+1th iteration respectively, represents the extrapolated variable used to accelerate the algorithm, σ, τ, and γ are all step size parameters, is the inverse Radon transform.

5. The method for removing metal artifacts from CT images based on a dynamic weighted deep expansion network according to claim 4, It is characterized in that The deep expansion network includes an original variable network u-Net and a dual variable network y-Net, each of which includes N layers, and the original variable network u-Net and the dual variable network y-Net of the corresponding layers form N network blocks, which are connected in this way; each parameterized network block of the dual variable network y-Net and the original variable network u-Net has the same structure: {convolution+PReLU+convolution+PReLU+convolution}; each network block adopts a residual learning method that fuses input information and network block output information; According to the updating method of the original variable u and the closed-form solution of the dual variable y, they are respectively used as the input of each network block of the original variable network u-Net and the dual variable network y-Net. The network blocks of the dual network y-Net and the original network u-Net respectively update the original variable u and the dual variable y continuously and alternately.

6. The method for removing metal artifacts from CT images based on a dynamic weighted deep expansion network according to claim 5, It is characterized in that Initialize the original variable u 0 = 0 and the dual variable y 0 = 0, and the dual variable y 0 , intermediate variables and W⊙Y as the input of the first layer of the dual variable network y-Net, and the output of the first layer of the dual variable network y-Net is the dual variable y of the first iteration 1 ; The original variable u 0 = 0 and intermediate variables Input the input of the first layer of the original variable network u-Net, and the output of the first layer of the original variable network u-Net is the original variable u of the first iteration 1 ; The output of the first layer of the original variable network u-Net and the output of the first layer of the dual variable network y-Net are used as the input of the second layer of the original variable network u-Net and the dual variable network y-Net; and so on, the dual variable yk output of the kth layer of the dual variable network y-Net and the intermediate variable and W⊙ Y as the input of the k+1th layer of the dual variable network y-Net, and the output of the k+1th layer of the dual variable network y-Net is the dual variable y k+1 , the original variable u output by the kth layer of the original variable network u-Net k and intermediate variables As the input of the k+1th layer of the original variable network u-Net, the k+1th layer of the original variable network u-Net outputs the original variable u k+1 ; The original variable output by N network blocks is the image to be reconstructed.

7. The method for removing metal artifacts from CT images based on a dynamic weighted deep expansion network according to claim 5 or 6, It is characterized in that The multi-energy measured sinusoidal graph Y, weight function W and metal track mask Tr are simultaneously input into the pre-trained network Prior-Net to obtain the weight function W of the 0th iteration. 0 ; The weight function W 0 , sinusoidal graph Y and metal trajectory mask Tr are used as inputs of the weight optimization network W-Net to dynamically update the weight function.

8. The method for removing metal artifacts from CT images based on a dynamic weighted deep expansion network according to claim 7, It is characterized in that The network structure of the pre-trained network Prior-Net includes three Conv+PRelu modules and a BN module connected in sequence, and also includes a residual learning unit. The residual learning unit adds the input data and the data output by the BN module to obtain the weight function W of the 0th iteration 0 ; By initializing the original variable u 0 =[u 0 ,u 0 ,u 0 ,u 0 ,u 0 ] and the dual variable y 0 =[y 0 ,y 0 ,y 0 ,y 0 ,y 0 ] Expand the variable space and deeply expand the network to record the original variables updated before during the update process; utilize the powerful representation ability of the network to adaptively learn the acceleration rules, and after the optimization of N network blocks, reconstruct the CT image after removing artifacts.

9. The method for removing metal artifacts from CT images based on a dynamic weighted deep expansion network according to claim 7, It is characterized in that The k-th layer of the weight optimization network W-Net is added to the k-th network block of the depth expansion network. The data processing method of the k-th network block is: The output of the kth layer of the dual variable network y-Net is: The output of the kth layer of the original variable network u-Net is: The output of the kth layer of the weight optimization network W-Net is: in, represents the operation function of the dual variable network y-Net, represents the operation function of the original variable network u-Net, Represents the operation function of the weight optimization network W-Net, W k represents the weight function of the k-1th layer output of the weight optimization network W-Net, is the second slice of the dual variable network y-Net output in the kth network block, u k is the original variable output by the k-1th layer of the original variable network u-Net, Represents the first slice of the dual variable network y-Net output in the k+1th network block.

10. The method for removing metal artifacts from CT images based on a dynamic weighted deep expansion network according to claim 1, It is characterized in that The loss function during training is the pairing data The mean square error is used as the objective function for training: Where M represents the number of samples, Y i and Tr i They represent the ith CT training data without metal artifacts, the corresponding sinusoidal data, and the mask corresponding to the metal trajectory, respectively. For CT training data The corresponding final model outputs a CT image with metal artifacts removed.

Images