CT image reconstruction method and system based on optimized iterative network

Abstract

The present invention discloses a CT image reconstruction method and system based on an optimized iterative network. The method combines iterative optimization with deep learning, explores deep features in residual measurement by using a deep neural network prior as a regularizer, and designs an iterative network model for limited angle tomographic reconstruction. The method does not directly deploy regularization terms in the image space, but deploys regularization terms in the residual image domain, which can significantly improve the detail retention and convergence of the reconstructed image; compared with the existing scheme, the method adopts a training strategy for image post-processing and performs iterative optimization during inference. Therefore, the quality of the reconstructed image is improved in terms of detail preservation, and it can be easily extended to cone beam CT. The convergence of the iterative network model designed by the method has been verified experimentally, and it has high stability.

Description

Technical Field

[0001] The present invention relates to the field of CT imaging technology, and in particular to a CT image reconstruction method and system based on an optimized iterative network. Background Art

[0002] At present, the key to obtaining high-quality CT images is to overcome the instability of limited-angle reconstruction. In order to solve this problem, many methods have been developed, mainly including regularization-based methods and deep learning-based methods.

[0003] Regularization-based methods have been used to solve typical ill-posed inverse problems, including reconstruction from incomplete projection data. In fact, regularization methods mainly use prior knowledge, such as sparsity and low-rank properties, to constrain the feasible domain. The core of regularization methods is to integrate appropriate regularization terms into the optimization model. Sparsity, low rank, dictionary learning, and wavelet transform are commonly used priors for inverse problems in medical imaging. Although regularization-based methods can improve CT reconstruction, they still have some disadvantages, including high computational cost and sensitive hyperparameter selection.

[0004] Deep learning methods based on convolutional neural networks (CNNs) have been introduced into CT image reconstruction with good performance, such as FBP Conv-Net, HD-Net, RED-CNN, and CT-Net, which have achieved good performance in feature recovery and reconstruction accuracy in tomographic reconstruction. Although CNNs have achieved good performance, the receptive field of convolution operations is limited, resulting in them only being able to perceive image features in local areas. Although visual transformers (ViTs) have been introduced, these algorithms are only based on post-processing methods, ignoring data consistency and network stability.

[0005] In summary, on the one hand, data inconsistency is an important issue that limits the quality of reconstructed CT images; on the other hand, due to the instability of deep reconstruction networks, it is difficult to extend deep learning-based algorithms to clinical applications. In the real world, the X-ray scanning process contains a lot of random noise and interference, which will lead to the degradation of deep reconstruction networks. For example, some tiny, almost undetectable perturbations in the image and sampling domains may cause serious artifacts in the reconstruction (possibly false positives). This instability may lead to incorrect medical diagnoses in medical imaging. Therefore, it is not easy to directly extend deep learning-based methods to clinical applications. Summary of the invention

[0006] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes a CT image reconstruction method based on an optimized iterative network, which can improve the quality of the reconstructed CT image.

[0007] The present invention also provides a CT image reconstruction system, an electronic device and a computer-readable storage medium for executing the above-mentioned CT image reconstruction method based on the optimized iterative network.

[0008] According to a first aspect of the present invention, a CT image reconstruction method based on an optimized iterative network comprises:

[0009] An iterative network model composed of a plurality of iterative sub-models is constructed, wherein the iterative sub-model includes a neural network, and the iterative network model integrates the prior of the neural network as a regularizer to construct an objective function, wherein the objective function includes: Where A represents the system matrix, y (0) represents the original projection data, y represents the residual data in the projection domain, represents the neural network function, λ>0 represents the hyperparameter, represents the Frobenius norm, f represents the iterative output image; the iterative expression of the iterative network model includes: Where k represents the number of iterations, f (k) Represents the image output of the kth iteration;

[0010] The original projection data is input into the iterative network model to obtain a CT image reconstructed by the iterative network model.

[0011] The control method according to the embodiment of the present invention has at least the following beneficial effects:

[0012] In order to overcome the false negatives and false positives caused by network instability, this method combines iterative optimization and deep learning. By using deep neural network priors as regularizers to explore deep features in residual measurements, an efficient iterative network model is designed for limited angle tomographic reconstruction. This iterative network model does not directly deploy regularization terms in the image space, but deploys regularization terms in the residual image domain, which can significantly improve the detail preservation and convergence of the reconstructed image. Compared with the existing scheme based on unfolding iteration, which occupies a large amount of computing resources with projection and back-projection operations and cannot be applied to cone beam CT (CBCT), this iterative network model adopts training strategies such as image post-processing and deploys regularization terms in the residual image domain. Therefore, this iterative network model improves the quality of the reconstructed image in terms of detail preservation, and this iterative network model can be easily extended to CBCT. The convergence of the iterative network model designed by this method has been verified by experiments, proving that this iterative network model has high stability.

[0013] According to some embodiments of the present invention, an iterative network model including a plurality of said iterative sub-models is constructed by an alternating direction multiplier method.

[0014] According to some embodiments of the present invention, the neural network includes a local information extractor for extracting local features and a global feature extractor for extracting global features.

[0015] According to some embodiments of the present invention, the local information extractor is a Dense ED, which includes a densely connected encoder and a decoder. The encoder of the Dense ED is used to gradually reduce the low-dimensional features of the extracted feature map using a maximum pooling operator, and the decoder of the Dense ED is used to gradually enlarge the restored feature map.

[0016] According to some embodiments of the present invention, the global feature extractor is a ReconstructionTransformer, which includes a symmetric encoder-decoder composed of multiple transformer blocks, the number of transformer blocks in the symmetric encoder-decoder gradually increases from top to bottom, shallow features are connected to deep features through skip connections, and the feature map output by the Dense ED is combined with the feature map output by the symmetric encoder-decoder through residual connections.

[0017] According to some embodiments of the present invention, the transformer block includes an MCTA module that uses self-attention across channels, the MCTA module uses a convolutional layer and the process expression of the MCTA module includes:

[0018] Y=F p Attention(Q′,K′,V′)

[0019]

[0020] Where Y represents the feature map output by the MCTA module; Q′, K′ and V′ are matrices obtained after reshaping the tensor X. and is a 1×1 convolutional layer, is a 3×3 convolutional layer, X represents the tensor received by the MCTA module; β represents a trainable scale parameter used to control the size of the dot product of K′ and Q′ before using softmax.

[0021] According to some embodiments of the present invention, the Reconstruction Transformer further comprises a convolutional layer, which is used to extract shallow feature embedding of the feature map output by the Dense ED and input it into the symmetric encoder-decoder.

[0022] According to a second aspect of the present invention, a CT image reconstruction system based on an optimized iterative network comprises:

[0023] The iterative network construction unit is used to construct an iterative network model composed of multiple iterative sub-models, wherein the iterative sub-model includes a neural network, and the iterative network model integrates the prior of the neural network as a regularizer to construct an objective function, wherein the objective function includes: Where A represents the system matrix, y (0) represents the original projection data, y represents the residual data in the projection domain, represents the neural network function, λ>0 represents the hyperparameter, represents the Frobenius norm, f represents the iterative output image; the iterative expression of the iterative network model includes: Among them, k represents the number of iterations, f (k) Represents the image output of the kth iteration;

[0024] The CT image reconstruction unit is used to input the original projection data into the iterative network model to obtain a CT image reconstructed by the iterative network model.

[0025] Since the CT image reconstruction system adopts all the technical solutions of the CT image reconstruction method of the above embodiment, it at least has all the beneficial effects brought by the technical solutions of the above embodiment.

[0026] According to an embodiment of the third aspect of the present invention, the electronic device comprises: at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute a CT image reconstruction method based on an optimized iterative network as described above.

[0027] Since the electronic device can implement all the technical solutions of the CT image reconstruction method of the above embodiment, it at least has all the beneficial effects brought by the technical solutions of the above embodiment.

[0028] According to the computer-readable storage medium of the fourth aspect of the present invention, the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute a CT image reconstruction method based on an optimized iterative network as described above.

[0029] Since the computer-readable storage medium adopts all the technical solutions of the CT image reconstruction method of the above embodiment, it at least has all the beneficial effects brought by the technical solutions of the above embodiment.

[0030] Other features and advantages of the present invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0032] Figure 1 It is a logic block diagram of a CT image reconstruction method based on an optimized iterative network provided by an embodiment of the present invention;

[0033] Figure 2 is the reconstruction result of Case 1 provided by an embodiment of the present invention;

[0034] Figure 3 is the reconstruction result of Case 2 provided by an embodiment of the present invention

[0035] Figure 4 is an intensity distribution diagram of a simulation data result of 130° reconstruction provided by an embodiment of the present invention;

[0036] Figure 5 is a noise power spectrum diagram of chest reconstruction at 130° in different schemes of an embodiment of the present invention;

[0037] Figure 6 is the reconstruction result of different schemes of an embodiment of the present invention within a scanning angle range of 90°;

[0038] Figure 7 is the reconstruction result of Case 3 provided by an embodiment of the present invention;

[0039] Figure 8 is the reconstruction result of Case 4 provided by an embodiment of the present invention;

[0040] Fig. 9 It is a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0042] In the description of the present invention, if there is a description of first, second, etc., it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0043] In the description of the present invention, it should be understood that descriptions involving orientation, such as orientation or positional relationship indicated as up, down, etc., are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0044] In the description of the present invention, it should be noted that, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention in combination with the specific content of the technical solution.

[0045] Reference Figure 1 One embodiment of the present invention provides a CT image reconstruction method based on an optimized iterative network, the CT image reconstruction method comprising:

[0046] Step S101: construct an iterative network model composed of multiple iterative sub-models, the iterative sub-model includes a neural network, and the iterative network model integrates the prior of the neural network as a regularizer to construct an objective function, and the objective function includes: Where A represents the system matrix, y (0) represents the original projection data, y represents the residual data in the projection domain, represents the neural network function, λ>0 represents the hyperparameter, represents the Frobenius norm, f represents the iterative output image; the iterative expressions of the iterative network model include: Among them, k represents the number of iterations, f (k) Represents the image output at the kth iteration.

[0047] Step S102: input the original projection data into the iterative network model to obtain a CT image reconstructed by the iterative network model.

[0048] In order to overcome the false negative and positive artifacts caused by data inconsistency, this method uses a deep neural network prior as a regularizer to explore the deep features in the residual measurement and establishes an iterative network model composed of multiple iterative sub-models. Compared with the existing schemes that occupy a lot of computing resources with projection and back-projection operations and cannot be applied to cone beam CT (CBCT), this iterative network model adopts training strategies such as image post-processing and deploys regularization terms on the residual image domain, which improves the quality of the reconstructed image and can be easily extended to CBCT.

[0049] Based on the above embodiments, this embodiment constructs an iterative network model (this embodiment is called Iterative Residual Optimization Network, IRON) in the following manner:

[0050] The ideal CT imaging model can be expressed as a discrete linear system:

[0051] y (0) =Ax+b (1)

[0052] Where x represents the reconstructed CT image, which can be expressed as x = [x 1 , x 2 , x 3 , …x P ] T ;y (0) Represents the projection data (i.e., the measured projection data), which can be expressed as y (0) =[y 1 ,y 2 ,y 3 …y Q ] T ; b represents the projection noise, A represents the CT system matrix, which contains Q×P elements. Due to the noise in y, the solution of equation (1) can be obtained by minimizing the following objective function:

[0053]

[0054] in, Represents the Frobenius norm. Solving x directly from a very sparse projection data y is an underdetermined inverse problem, which may lead to poor quality reconstructed images with streaking artifacts. Here, a regularization term representing prior knowledge is introduced to obtain better reconstruction results, and then:

[0055]

[0056] Among them, () represents the regularization term, λ>0 is a hyperparameter, and the hyperparameter λ is used to balance the data fidelity term and the regularization term.

[0057] In this embodiment, the objective function of the iterative network model is set as:

[0058]

[0059] Among them, y (0) is a raw projection data, Represents the neural network function, and f represents the image output by the iterative network model. Definition:

[0060]

[0061] Then, the block coordinate descent method can be used to optimize equation (4) as follows:

[0062]

[0063] To update y, the following problems need to be solved:

[0064]

[0065] Calculate the partial derivative on the right side of (6), we have:

[0066]

[0067] Because the neural network is well trained to solve the problem Af = y, It can be regarded as a back propagation network, with Therefore, formula (7) can be simplified as:

[0068]

[0069] To update f, the following problems are solved:

[0070]

[0071] Calculate the partial derivative on the right side of formula (9):

[0072] λA T (y (k+1) +Af-y (0) )=0 (10)

[0073] λA T Af=λA T (y (0) -y (k+1) ) (11)

[0074] From formula (7), we have:

[0075]

[0076] By replacing formula (12) with formula (11), we have:

[0077]

[0078] Formula (13) can be simplified as:

[0079] λf=φ(y (k+1) )+λf (k) (14)

[0080] There is (k+1) :

[0081]

[0082] Combining equation (8) and equation (15), we get the formula:

[0083]

[0084] In some embodiments, Figure 1 , Figure 1 (a) is the overall framework of this iterative network model (IRON) (wherein, FBP operator refers to the filtered back projection operator, which realizes the function of reconstructing projection data into CT images. The projection operator is a projection operator, which is opposite to the FBP operator and can convert CT images into projection data), as shown in Figure 1 As shown in (b), the neural network in the iterative network model includes two modules: Dense ED (Dense Encoder-Decoder) and Reconstruction Transformer. Dense ED is used as the initial recovery subnetwork to learn texture features and local information. Dense ED outputs the initial restored image as the input of Reconstruction Transformer to extract long-range pixel interactions. In summary, the neural network of this embodiment achieves the final high-quality image by combining a local information extractor (CNN-based Dense ED) and a global feature extractor (Transformer-based Reconstruction Transformer).

[0085] Dense ED can extract multiple structural extension functions and reduce the use of computational memory. The goal of Dense ED's encoder is to extract low-dimensional features by gradually reducing the feature map using the maximum pooling operator; the decoder of Dense ED can effectively restore high-quality images by gradually enlarging the feature map. Dense connections help improve parameter utilization. In addition, skip connections connect shallow features with deep features, which can avoid detail loss and accelerate feature mapping.

[0086] The Reconstruction Transformer is used to extract long-range pixel interactions. Given a preliminary restored image from Dense ED, the Reconstruction Transformer first applies convolution to obtain shallow feature embeddings. Next, similar to Dense ED, these shallow features pass through a symmetric encoder-decoder containing multiple transformer blocks, where the number of transformer blocks gradually increases from top to bottom, applying maximum pooling and pixel cleaning operations for downsampling and upsampling respectively, and also connecting low-level features with high-level features through skip connections. The transformer block is then used to aggregate the shallow features of the encoder's image with the deep features of the decoder. Finally, the initial restored image output by Dense ED is added to the output image of the Reconstruction Transformer through a residual connection to obtain the final output image.

[0087] In some embodiments, a core component of the Reconstruction Transformer is provided, which is referred to as the MCTA module (Multi-Convolutional Transposed Attention) in this embodiment. The key idea of ​​MCTA is to use self-attention across channels instead of spatial dimensions, which helps to calculate the cross-covariance across channels, thereby generating an attention map that implicitly encodes the global context. In addition, before calculating the feature covariance, convolution is introduced to emphasize local information, thereby producing a global attention map. MCTA first generates query (Q), key (K), and value (V) projections from an input tensor X. Figure 1 As shown in (c), this process is achieved by applying 1×1 convolution to aggregate pixel cross-channel context, and then applying 3×3 convolution to encode the context in channel space, producing and in is a 1×1 convolutional layer, is a 3×3 convolutional layer. Next, the query and key projections are reshaped to obtain a transposed-attention map A from their dot product interaction, which can save computational memory. Therefore, the MCTA process is defined as:

[0088] Y=F p Attention(Q′,K′,V′) (17)

[0089]

[0090] Where Y is the output feature map, and the matrices Q′, K′, and V′ are obtained after reshaping the tensor. Here, β is a trainable scale parameter that controls the size of the dot product of K′ and Q′ before using the softmax function.

[0091] The following proves the relative error convergence characteristics of the neural network provided by this embodiment:

[0092] If the reconstruction error of the reconstruction network is L 2 Norm and corresponding label L 2 If the norm ratio is less than 1-μ (0<μ<1), then the reconstruction network has relative error convergence (RECP). If a reconstruction network satisfies RECP, it is called a well-designed and well-trained reconstruction network.

[0093] Assume that the neural network function is a well-trained neural network. It represents the output of the neural network Among them, the second and third terms are the observable component and the null space component of the error image. A relative error (REN) is designed to judge whether the network meets the requirements of RECP. Then, RECP is defined as:

[0094]

[0095] Equation (19) means that ||f ob +f nl ||≤(1-μ)‖f * ‖.

[0096] Here, the experimental results of this neural network and U-net based on simulated and real data sets are verified. As shown in Table 1 (Relative error standards of simulated and real test data sets):

[0097]

[0098] Table 1

[0099] The following proves the convergence analysis of the iterative network model provided in this embodiment:

[0100] In formula (16), Indicated as N 1 , Indicated as N 2 ,have Then formula (16) can be simplified as:

[0101]

[0102] In formula (16), Replace with "=" in formula (20). The convergence of the iterative network model is analyzed as follows:

[0103] Assume an initial image f * is the ground-truth image, and the other two components f (ob,0) and f (nl,0) is observable and unobservable, an unobservable image f (nl,0) Located in the null space of A, f (nl,0) Meet Af nl = 0 and if ||f ob ||≠0 then ||Af ob ||≠0. According to the RECP attribute, we have:

[0104] ||f (ob,0) +f (nl,0) ||<(1-μ)‖f * ‖ (twenty one)

[0105] It is worth noting that Af (nl,0) =0 and Af (ob,0) ≠0, so we can get:

[0106] (Af (nl,0) )(Af (ob,0) ) T =0 (22)

[0107] A (nl,0) (f (ob,0) ) T A T =0 (23)

[0108] f (nl,0) (Af (ob,0) ) T =0 (24) Therefore, f (ob,0) Orthogonal to f (nl,0) , from which we have:

[0109] ||f (ob,0) +f (nl,0) ||=||f (ob,0) ||+||f (nl,0) || (25)

[0110] This means:

[0111] ||f (ob,0) ||<(1-μ)‖f * ‖ (26)

[0112] Because f (0) is the output of the network, which can be expressed as:

[0113]

[0114] For k=0, according to equations (20) to (27), we obtain:

[0115]

[0116] Because y (0) -Af * =0 and Af (nl,0) =0, we can get:

[0117] y (1) =-N 1 A (ob,0) (29)

[0118] Use the forward model A to synthesize the unexplained residual data y (1) Then y (1) is fed to the reconstruction network to reconstruct Because y (1) Follow f (ob,0) Related, -f (ob,0) A new artifact image f can be reconstructed (ob,1) +f (nl,1) .

[0119]

[0120] ||f (ob,1) ||<(1-μ)||N 1 f (ob,0) || (31)

[0121] Starting from equation (20), when k = 0, we get:

[0122]

[0123] By replacing (30) with (32), we obtain:

[0124] f (1) =f * +(1-N)f (ob,0) +N 2 f (ob,1) +f (nl,0) +N 2 f (nl,1) (33)

[0125] For k=1:

[0126] y (2) =N 1 (y (0) -Af (1) )=-(1-N)N1 A (ob,0) -NAf (ob,1) (34)

[0127]

[0128] ||f (ob,2) ||<(1-μ)||(1-N)N 1 f (ob,0) +Nf (ob,1) || (36)

[0129]

[0130] By replacing (35) with (37), we obtain:

[0131] f (2) =f * +(1-N) 2 f (ob,0) +(1-N)N 2 f (ob,1) +N 2 f (ob,2) +f (nl,0) +N 2 f (nl,1) +N 2 f (nl,2) (38)

[0132] If we continue the above process, for k>1, we can easily get:

[0133]

[0134] y (k) The real image is represented as f (*,k) , that is, y (k) =Af (*,k) ,get:

[0135] f (*,k+1) =(1-N)f (*,k) -Nf (ob,k) (40)

[0136] ||f (ob,k+1) ||<(1-μ)||f (*,k+1) || (41)

[0137] Equations (40) and (41) show that:

[0138] ||f (*,1) ||=||N 1 f (ob,0) ||≤N 1 ||f (ob,0) || (42)

[0139] ||f (*,2) ||=||(1-N)f (*,1) -Nf (ob,1) ||

[0140] ≤(1-N)||f (*,1) ||+N||f (ob,1) ||

[0141] ≤(1-N)||f (*,1) ||+N(1-μ)||f (*,1) ||

[0142] ≤(1-Nμ)N 1 ||f (ob,0) || (43)

[0143] If we continue this process, we get:

[0144] ||f (*,k+1) ||≤(1-Nμ) k N 1 ||f (ob,0) || (44)

[0145] ||f (ob,k+1) ||<(1-μ)(1-Nμ) k N 1 ||f (ob,0) || (45)

[0146] When k→∞, there exists a ξ∈(0,1) that satisfies:

[0147] 0<||f (ob,∞) ||<ξ<1 (46)

[0148] Formula (46) shows that ||f (ob,∞) ||→0, formula (44) will be monotonically decreasing and can reach the connection, that is, the iterative network model will converge to a solution at the intersection of the space constrained by the measured data and the neural network space spanned by CNN and Transformer.

[0149] The following is a set of experimental results:

[0150] The simulation experiments were trained in Python using the PyTorch framework. All experiments were run on a computer equipped with a 24G NVIDIA TITAN RTX GPU, an Intel Xeon Silver 4210 CPU (2.20GHz), and 64GB of memory. The configuration of the training network is as follows: The network is trained using the Adam optimizer, and the learning rate is set to 2.5×10 -4The training times are 30 and the input sample size is 1 each time.

[0151] This scheme (IRON) is compared with other schemes: FBP Conv-Net, RED-CNN, LEARN, FISTA-net and MIST-net. LEARN and FISTA-net are high-performance iterative methods for sparse view reconstruction; MIST-net is currently the most advanced restoration method in sparse tree reconstruction; Root mean square error (RMSE), peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) are introduced to quantitatively evaluate the reconstruction results.

[0152] A. Simulation data results;

[0153] In order to verify the feasibility of this neural network, it was first trained and tested on a simulated dataset. First, 4665 512*512 pixel images were obtained from 10 patients, of which 4274 images from 8 patients were used for training and 391 images from the other 2 patients were used for testing. This experiment was mainly conducted on 114 views and 130° fan beam priority angle datasets. Experiments were also conducted on 90° fan beam datasets. All schemes except this one could not recover the features marked by the arrows in the figure.

[0154] like Figure 2 and Figure 3 , Figure 2 The reconstruction results of Case 1 are shown in row 1 as input, row 2 as difference image relative to the ground truth, row 3 as magnification ROI, and rows 4 and 5 as sagittal and coronal results. The display windows of the reconstructed image and difference image are [-160 240] HU and [-90 90] HU; Figure 3 This is the reconstruction result of Case 2.

[0155] The image reconstructed by RED-CNN is severely damaged and the image quality is seriously degraded. Compared with the clean image, FBP Conv-Net can remove artifacts to a certain extent, but it loses the edge marked by arrow "1". For the result shown by arrow "2", LEARN and FISTA-net will more or less cause the edge to be too smooth. In addition, the details marked by arrow "3" show that MIST-net will produce redundant shadows, while this scheme can avoid this error. The results show that compared with other schemes, this scheme achieves the best results in dealing with edge divergence caused by limited angle scanning.

[0156] This embodiment also quantitatively evaluates all the schemes, and the results are shown in Table 2. The results show that all relevant schemes improve image quality in three indicators. Specifically, this scheme produces the best results. This scheme has the smallest root mean square error, the largest PSNR and SSIM. These quantitative results verify that this scheme has the advantage of the best performance. The training time of all networks is also compared, and the results are shown in Table 2.

[0157]

[0158]

[0159] Table 2

[0160] In order to explore the performance of different schemes in terms of edge preservation, Figure 4 The intensity distribution diagram of the simulated data results for the 130° reconstruction is shown. It can be seen that the present IRON framework produces the closest results to the reference.

[0161] Figure 5 The noise power spectrum (NPS) of chest reconstruction at 130° for different schemes is shown, and the display window is [0, 1200] HU 2 mm 2 , Figure 5 (a) is RED-CNN, Figure 5 (b) is FBP Conv-Net, Figure 5 (c) is LEARN, Figure 5 (d) FISTA, (e) in Figure 5 is MIST-net, Figure 5 (f) is IRON. It can be seen that this scheme has a smaller error in each performance, especially in the low-frequency area, indicating that the structural error of this scheme is small.

[0162] Figure 6 The reconstruction results of different schemes with a scanning angle range of 90 are shown. Figure 6 (a) is the input, Figure 6 (b) is GT, Figure 6 (c) is RED-CNN, Figure 6 (d) FBP Conv-Net, Figure 6 (e) is LEARN, Figure 6 (f)FISTA, Figure 6 (g) is MIST-net, Figure 6 (h) is IRON, and the display window is [-160, 240]. The framework of this scheme achieves the best performance in detail preservation.

[0163] B. Clinical cardiac validation;

[0164] To further verify the performance of this scheme, a real dataset was used. The curved cylindrical detector contains 880 units. A full scan has 2200 views. The diameter of the field of view (FOV) is 49.8×49.8cm^2, and the image matrix is ​​512×512 pixels. The distances from the X-ray source to the system isocenter and the detector array are 53.85cm and 103.68cm, respectively. Since the reconstruction network is trained using the AAPM dataset, here, the trained network is transferred to the real dataset to evaluate the reconstruction performance, which is conducive to evaluating the generalization ability of the model.

[0165] Figure 7 This is the reconstruction result of Case 3. The scanning angle range is 130°. The 1st to 8th columns represent the input, Label, RED-CNN, FBP Conv-Net, LEARN, FISTA, MIST-net and IRON, respectively. The 2nd row shows the difference image relative to GT, and the 3rd row shows the enlarged ROI. The display windows of the reconstructed image and the difference image are [-800 1000] HU and [-90 90] HU. Figure 8 The reconstruction results of Case 4. The 1st to 8th columns represent input, Label, RED-CNN, FBP Conv-Net, LEARN, FISTA, MIST-net and IRON respectively. The 2nd row shows the difference image relative to GT, and the 3rd row shows the enlarged ROI. The display windows of the reconstructed image and the difference image are [-450 240] HU and [-90 90] HU. Figure 7 In the figure, the image structure is severely destroyed by RED-CNN. FBP Conv-Net loses image edges and features, as shown by arrow “1”. In addition, LEARN generates some additional light tissue in arrow “2”. Compared with RED-CNN and FBP Conv-Net, MIST-net restores the missed image boundaries but produces grayscale intensity shift. It is worth noting that this scheme produces higher quality images than other schemes.

[0166] To further demonstrate the advantages of this solution, the display window Figure 1 Compared with RED-CNN, FBP Conv-Net provides better images. However, it is still difficult to accurately define the image edge pointed by the arrow. In addition, LEARN, FISTA-net and MIST-net also give fuzzy image edges similar to the arrow-marked area with poor accuracy.

[0167] Figure 4 (b) shows the intensity distribution of the real data results, which is consistent with the results of the simulated data. The results produced by this scheme (IRON) are closest to the reference images of all other schemes.

[0168] C. Noise and interference experiments;

[0169] In the noise experiment, Gaussian noise was added to the image, and the mean and variance were set to 0 and 0.001, respectively. Then, voice projection was used during the test to verify the ability of the reconstruction model to resist voice attacks. This scheme can obtain better image quality than other schemes. In the perturbation experiment, a small but visible perturbation was added to the test image (here a letter "H" was added). D. Ablation analysis;

[0170] Based on the reconstructed images with a scanning angle range of 130°, an ablation study was conducted to explore the effectiveness of the iterations and different modules of this scheme.

[0171] (1) Validity of iterations: To evaluate the impact of the number of iterations, the test values ​​were adjusted. Figure 8 The results of different iterative reconstructions using 114 views and 130 angle data are shown. As the number of iterations increases, the performance gradually improves.

[0172] (2) Effectiveness of the proposed module: In order to study the effectiveness of the hybrid structure and the proposed module, three additional experiments were implemented. The scheme consists of two sub-networks: dense encoder-decoder (Dense ED) and ReconstructionTransformer. First, a pure convolutional neural network consisting of two Dense EDs is designed. Then, a network consisting of two Reconstruction Transformers is constructed. A traditional MSHA is also used to replace the proposed MCTA. The number of epochs for all ablation studies is set to 40. E. Convergence analysis and runtime cost;

[0173] The convergence of the scheme in terms of quantitative measurement and number of iterations is studied experimentally. The RMSE value shows that the processing effect of the scheme on the reconstructed image can be gradually improved.

[0174] This embodiment has at least the following beneficial effects:

[0175] (1) In order to overcome the false negatives and false positives caused by network instability, this method combines iterative optimization and deep learning. By using deep neural network priors as regularizers to explore deep features in residual measurements, an efficient iterative network model (IRON) is designed for limited-angle tomographic reconstruction. Instead of directly deploying regularization terms in image space, IRON deploys regularization terms in the residual image domain, which can significantly improve the detail preservation and convergence of the reconstructed image. In addition, the alternating direction method of multipliers (ADMM) is adopted to implement the novel iterative framework. Although existing advanced methods based on unfolding iterations have been successfully applied to limited-angle CT imaging, they occupy a lot of computational resources with projection and back-projection operations and cannot be applied to cone beam CT (CBCT). When testing this IRON, training strategies such as image post-processing are adopted and regularization terms are deployed in the residual image domain. Therefore, this IRON improves the quality of reconstructed images in terms of detail preservation, and the model can be easily extended to CBCT.

[0176] (2) In this IRON, a combination of two modules, Dense ED and Reconstruction Transformer, is proposed as a deep neural network. The neural network can restore the damaged features with remote valuable information (reconstructing the complete structure of the image), and can effectively fuse local features and global features. In the Transformer, a new self-attention module MCTA is also set up, which uses convolutional layers instead of linear layers, which is conducive to reducing computational costs.

[0177] (3) After verification by simulated and real cardiac clinical data sets, the IRON has a relative error convergence property (RECP). Based on the RECP property, the convergence of the IRON is also verified, indicating that the IRON can converge to a stable point under appropriate conditions, has strong stability, and can cope with the perturbations of the input data.

[0178] One embodiment of the present invention provides a CT image reconstruction system based on an optimized iterative network, the system comprising an iterative network construction unit 1001 and a CT image reconstruction unit 1002, wherein:

[0179] The iterative network construction unit 1001 is used to construct an iterative network model composed of multiple iterative sub-models, the iterative sub-model includes a neural network, and the iterative network model integrates the prior of the neural network as a regularizer to construct an objective function, and the objective function includes: Where A represents the system matrix, y (0) represents the original projection data, y represents the residual data in the projection domain, represents the neural network function, λ>0 represents the hyperparameter, represents the Frobenius norm, f represents the iterative output image; the iterative expressions of the iterative network model include: Among them, k represents the number of iterations, f (k) Represents the image output at the kth iteration.

[0180] The CT image reconstruction unit 1002 is used to input the original projection data into the iterative network model to obtain a CT image reconstructed by the iterative network model.

[0181] It should be noted that the present system embodiment and the above method embodiment are based on the same inventive concept, and therefore the relevant contents of the above method embodiment are also applicable to the present system embodiment.

[0182] Reference Fig. 9 The embodiment of the present invention further provides an electronic device, and the electronic device 6000 can be any type of smart terminal, such as a mobile phone, a tablet computer, a personal computer, etc.

[0183] Specifically, the electronic device 6000 includes: one or more control processors 6001 and a memory 6002, Fig. 9 A control processor 6001 is taken as an example.

[0184] The control processor 6001 and the memory 6002 may be connected via a bus or other means. Fig. 9 The example of connecting through bus is taken in the following.

[0185] The memory 6002 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules. The control processor 6001 executes various functional applications and data processing in the CT image system based on the optimized iterative network by running the non-transient software programs, instructions and modules stored in the memory 6002, that is, the CT image method based on the optimized iterative network in the above method embodiment is implemented.

[0186] The memory 6002 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created based on the use of a CT image system based on an optimized iterative network, etc. In addition, the memory 6002 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 6002 may optionally include a memory remotely arranged relative to the control processor 6001, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0187] The one or more modules are stored in the memory 6002, and when executed by the one or more control processors 6001, the CT image method based on the optimized iterative network in the above method embodiment is executed.

[0188] The embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by one or more control processors, for example, Fig. 9 One of the control processors 6001 executes, which can enable the one or more control processors 6001 to execute the CT image method based on the optimized iterative network in the above method embodiment.

[0189] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, i.e., may be located in one place, or may be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0190] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform. Those skilled in the art can understand that all or part of the processes in the above embodiment method can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and the program can include the process of the embodiment of the above method when executed. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0191] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge scope of ordinary technicians in the relevant technical field without departing from the purpose of the present invention.

Claims

1. A CT image reconstruction method based on an optimized iterative network, characterized in that: The method comprises: An iterative network model composed of a plurality of iterative sub-models is constructed, wherein the iterative sub-model includes a neural network, and the iterative network model integrates the prior of the neural network as a regularizer to construct an objective function, wherein the objective function includes: Where A represents the system matrix, y (0) represents the original projection data, y represents the residual data in the projection domain, represents the neural network function, λ>0 represents the hyperparameter, represents the Frobenius norm, f represents the iterative output image; the iterative expression of the iterative network model includes: Among them, k represents the number of iterations, f (k) Represents the image output of the kth iteration; The original projection data is input into the iterative network model to obtain a CT image reconstructed by the iterative network model.

2. The CT image reconstruction method based on optimized iterative network according to claim 1, characterized in that: An iterative network model including a plurality of said iterative sub-models is constructed by an alternating direction multiplier method.

3. The CT image reconstruction method based on optimized iterative network according to claim 1 or 2, characterized in that: The neural network includes a local information extractor for extracting local features and a global feature extractor for extracting global features.

4. The CT image reconstruction method based on optimized iterative network according to claim 3, characterized in that: The local information extractor is a Dense ED, which includes a densely connected encoder and a decoder. The encoder of the Dense ED is used to gradually reduce the low-dimensional features of the extracted feature map using a maximum pooling operator, and the decoder of the Dense ED is used to gradually enlarge the restored feature map.

5. The CT image reconstruction method based on optimized iterative network according to claim 4, characterized in that: The global feature extractor is a Reconstruction Transformer, which includes a symmetric encoder-decoder composed of multiple transformer blocks. The number of transformer blocks in the symmetric encoder-decoder gradually increases from top to bottom. The shallow features are connected to the deep features through skip connections, and the feature map output by the Dense ED is combined with the feature map output by the symmetric encoder-decoder through residual connections.

6. The CT image reconstruction method based on optimized iterative network according to claim 5, characterized in that: The transformer block includes an MCTA module that uses self-attention across channels, the MCTA module uses a convolutional layer and the process expression of the MCTA module includes: Y=F p Attention(Q ′ ,K ′ ,V ′ ) Wherein, Y represents the feature map output by the MCTA module; Q ′ ,K ′ and V ′ is the matrix obtained after reshaping the tensor X. Q, K, and V represent the query, key, and value generated by the tensor X. and is a 1×1 convolutional layer, is a 3×3 convolutional layer, X represents the tensor received by the MCTA module; β represents a trainable scale parameter used to control K before using softmax ′ and Q ′ The dot product size of .

7. The CT image reconstruction method based on optimized iterative network according to claim 5 or 6, characterized in that: The Reconstruction Transformer also includes a convolutional layer, which is used to extract shallow feature embedding of the feature map output by the Dense ED and input it into the symmetric encoder-decoder.

8. A CT image reconstruction system based on an optimized iterative network, characterized in that: The system comprises: The iterative network construction unit is used to construct an iterative network model composed of multiple iterative sub-models, wherein the iterative sub-model includes a neural network, and the iterative network model integrates the prior of the neural network as a regularizer to construct an objective function, wherein the objective function includes: Where A represents the system matrix, y (0) represents the original projection data, y represents the residual data in the projection domain, represents the neural network function, λ>0 represents the hyperparameter, represents the Frobenius norm, f represents the iterative output image; the iterative expression of the iterative network model includes: Among them, k represents the number of iterations, f (k) Represents the image output of the kth iteration; The CT image reconstruction unit is used to input the original projection data into the iterative network model to obtain a CT image reconstructed by the iterative network model.

9. An electronic device, characterized in that: include: at least one control processor and a memory for communicatively coupling with the at least one control processor; The memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute a CT image reconstruction method based on an optimized iterative network as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the CT image reconstruction method based on an optimized iterative network as described in any one of claims 1 to 7.

Images