Energy spectrum CT imaging method and device based on multi-domain integrated Transform iterative network

Abstract

The invention discloses an energy spectrum CT (Computed Tomography) imaging method and device based on a multi-domain integrated Transform iterative network, and the method comprises the steps: firstly building a coupling multi-task optimization framework combining energy spectrum CT image reconstruction and material decomposition based on the dependence of data in a plurality of energy channels; then, converting the optimization model into a plurality of sub-problems based on an alternating direction multiplier method; then, fusing a data driving method and a model driving method to solve each sub-problem; and finally, aiming at the defects of the convolutional neural network in global and remote image information interaction, introducing a Transform network. According to the method, a model driving method and a data driving method are creatively fused, the learning characterization capability and generalization performance of energy spectrum CT imaging are improved, errors caused by sparse angle data are processed in multiple domains based on dependency among multiple energy channel data, and adverse factors in images are effectively restrained.

Description

Technical Field

[0001] This invention relates to the field of spectral CT imaging technology, and in particular to a spectral CT imaging method and apparatus based on a multi-domain integrated Transformer iterative network. Background Technology

[0002] Spectral computed tomography (spectral CT) has shown great potential in disease detection and material analysis, and has received increasing attention in recent years. In the diagnosis of brain tumors, spectral CT can separate tumor tissue from surrounding brain tissue, revealing the tumor's location and boundaries more clearly, thus improving surgical outcomes. However, reconstructing high-quality spectral CT images typically requires acquiring full-scan projection datasets across multiple energy channels, which not only increases acquisition time but also raises radiation dose. Currently, the sparse angle problem caused by reduced projection data has become a hot research topic.

[0003] Over the past few decades, model-based and data-based methods have made significant progress, each with its own advantages and disadvantages. Model-based methods offer theoretical foundations and robustness, but the drawbacks of manually tuning parameters and designing regularizations can lead to suboptimal results. Data-based methods promise to overcome these shortcomings, but they are susceptible to various types of uncertainty and lack interpretability. In recent years, inspired by traditional iterative reconstruction algorithms, deep reconstruction networks can be designed as unfolded deep iterative reconstructions. Chen et al. introduced expert domains as regularization terms, proposing the Learned Experts' Evaluation-based Reconstruction Network (LEARN) for direct reconstruction from sparse projected data. Furthermore, Zhang et al. extended the LEARN model to a dual-domain version (LEARN++). While deep learning reconstruction methods based on convolutional neural networks (CNNs) offer good performance, convolution operators exhibit a degree of locality. To better learn the interaction of global and remote image information, Pan et al. developed a multi-domain integrated rotating Transformer network (MIST-Net), which combines rich domain features with a high-quality reconstruction Transformer.

[0004] However, the reconstruction of each energy channel data in the above-mentioned spectral CT imaging work is mostly carried out independently or in simple coupling, and the dependencies in spectral CT imaging are often ignored, which leads to a decrease in the quality of reconstructed images under sparse angle data. Summary of the Invention

[0005] To address the issue of degraded image quality under sparse angular data, this invention proposes a spectral CT imaging method and apparatus based on a multi-domain integrated Transformer iterative network, which is used to simultaneously optimize spectral CT image reconstruction and material decomposition from sparse projection.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] This invention proposes a spectral CT imaging method based on a multi-domain integrated Transformer iterative network, comprising:

[0008] Step 1: Based on the dependencies between multiple energy channels, establish a coupled multi-task optimization framework model that combines energy spectrum CT image reconstruction and material decomposition;

[0009] Step 2: Transform the above optimization framework model into multiple sub-problems using the alternating direction multiplier method;

[0010] Step 3: Integrate model-driven and data-driven methods to solve each sub-problem, and embed the Transformer network into the sub-problem solving process to improve the network's learning representation ability;

[0011] Step 4: Iterate through step 3 to complete the energy spectrum CT imaging.

[0012] Furthermore, the optimization framework model is as follows:

[0013]

[0014] Where R represents the system matrix that maps the image to the projection. This represents the tensor of the energy spectrum CT reconstructed image. This represents the energy spectrum CT projection tensor. Let A denote the material image tensor, and let A denote the system matrix of the image domain decomposition; φ(·) denotes the transformation on the reconstructed image. This represents sparse regularization based on the L1 norm of the transformation. Let F represent the material regularization term, α, β, and η be the parameters of the equilibrium terms, and F represent the norm. express The matrix formed by unfolding along the third dimension express The matrix formed by unfolding along the third dimension;

[0015] Through auxiliary variables Replace the tensor of the spectral CT reconstructed image Auxiliary variables Replacement material image tensor The equation is updated to an unconstrained augmented Lagrangian form:

[0016]

[0017] in To represent the tensor of error feedback, W and H represent the width and height of the reconstructed image, respectively, K represents the number of materials, S represents the number of energies, and λ1 and λ2 represent penalty factors.

[0018] Furthermore, in step 2, each sub-problem is represented as follows:

[0019]

[0020]

[0021]

[0022]

[0023]

[0024] in This represents the material image tensor at the (n+1)th iteration. This represents the auxiliary variable corresponding to the nth iteration. This represents the corresponding iteration in the nth iteration. This represents the corresponding iteration in the nth iteration. This represents the energy spectrum CT reconstructed image tensor at the (n+1)th iteration. This represents the auxiliary variable corresponding to the nth iteration.

[0025] Furthermore, in step 3, equation (3) is solved in the following manner:

[0026] Transformation of equation (3) based on pixel level:

[0027]

[0028] in express The value at pixel (l,k) express The value at pixel (l,s); and express and The value at pixel (i, j, k); by solving equation (8) using the gradient descent method, we obtain:

[0029]

[0030] in This represents the corresponding iteration at the (n+1)th iteration. and The value at pixel (i, j, k) Indicates the nth iteration The value at pixel (i, j, s).

[0031] Furthermore, in step 3, equation (4) is solved as follows:

[0032] With other variables fixed, we perform a first-order Taylor expansion of equation (4) and take its derivative, resulting in the following expression.

[0033]

[0034] in This indicates that the gradient is calculated with respect to the variable u.

[0035] Using neural networks Alternative The following representation is obtained.

[0036]

[0037] The network parameters and hyperparameters are updated in different iteration rounds, as expressed in equation (12).

[0038]

[0039] Furthermore, the neural network comprises three convolutional layers and a Swin Transformer module.

[0040] Furthermore, in step 3, equation (5) is solved in the following manner:

[0041] The solution for image reconstruction based on pixel level is expressed as follows:

[0042]

[0043] in, and express and The value at pixel (i, j, s);

[0044] Solving equation (14) using the gradient descent method, the iterative formula for reconstructing the image is expressed as follows:

[0045]

[0046] Where g represents the step size, and n and n+1 both represent the number of iterations.

[0047] Furthermore, in step 3, equation (6) is solved as follows:

[0048] With other variables fixed and irrelevant variables removed, the optimization formula for image regularization is expressed as follows:

[0049]

[0050] make Equation (16) is transformed into

[0051]

[0052] Equation (17) is approximately expressed as

[0053]

[0054] Where δ is a scalar quantity that balances the terms. The closed form is represented as

[0055]

[0056] Wherein, parameter γ represents a contraction threshold and the combination of δ and α;

[0057] Introducing the left inverse of φ(·) Equation (19) is expressed as

[0058]

[0059] in, Represented as As the identity matrix, φ(·) is designed as φ(·)=C2(ReLU(C1(·))), where C1(·) and C2(·) denote the convolution operators separated by the linear rectified function ReLU. The structure is designed as a symmetrical structure of φ(·).

[0060] Furthermore, in step 3, based on the obtained parameters, substituting them into formula (7) yields the result. and

[0061] Another aspect of the present invention proposes a spectral CT imaging device based on a multi-domain integrated Transformer iterative network, comprising:

[0062] An optimized framework building module is used to establish a coupled multi-task optimization framework model that combines energy spectrum CT image reconstruction and material decomposition based on the dependencies between multiple energy channels.

[0063] An optimized framework transformation module is used to transform the above optimized framework model into multiple sub-problems using the alternating direction multiplier method.

[0064] The sub-problem solving module is used to integrate model-driven and data-driven methods to solve each sub-problem, and embeds the Transformer network into the sub-problem solving process to improve the network's learning representation ability;

[0065] The iterative execution module is used to iteratively execute the sub-problem solving module to complete energy spectral CT imaging.

[0066] Compared with the prior art, the present invention has the following beneficial effects:

[0067] This invention innovatively integrates model-driven and data-driven methods, enhancing the learning representation ability and generalization performance of spectral CT imaging. Based on the dependencies between data from multiple energy channels, it processes errors introduced by sparse angle data in multiple domains, effectively suppressing undesirable factors in the image. Experimental results verify the effectiveness of this invention in spectral CT imaging tasks. Attached Figure Description

[0068] Figure 1 This is one of the flowcharts of a spectral CT imaging method based on a multi-domain integrated Transformer iterative network according to an embodiment of the present invention;

[0069] Figure 2 This is the second flowchart of a spectral CT imaging method based on a multi-domain integrated Transformer iterative network according to an embodiment of the present invention;

[0070] Figure 3 This is a schematic diagram of the network structure built based on Transformer in an embodiment of the present invention;

[0071] Figure 4 This is a schematic diagram of the structure of a spectral CT imaging device based on a multi-domain integrated Transformer iterative network according to an embodiment of the present invention;

[0072] Figure 5 An example diagram of test results in a thoracic dataset provided in an embodiment of the present invention. Detailed Implementation

[0073] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments:

[0074] like Figure 1 As shown, a spectral CT imaging method based on a multi-domain integrated Transformer iterative network includes:

[0075] S1: Based on the dependencies between multiple energy channels, a coupled multi-task optimization framework model combining energy spectrum CT image reconstruction and material decomposition is established.

[0076] The coupled multi-task optimization framework combining energy spectral CT image reconstruction and material decomposition proposed in this invention is as follows:

[0077]

[0078] Where R represents the system matrix that maps the image to the projection. This represents the tensor of the energy spectrum CT reconstructed image. This represents the energy spectrum CT projection tensor. Let A denote the material image tensor, and let A denote the system matrix of the image domain decomposition; φ(·) denotes the transformation on the reconstructed image. This represents sparse regularization based on the L1 norm of the transformation. Let F represent the material regularization term, α, β, and η be the parameters of the equilibrium terms, and F represent the norm. express The matrix formed by unfolding along the third dimension express The matrix is ​​formed by unfolding along the third dimension.

[0079] To optimize the above process, this invention introduces auxiliary variables. Replace the tensor of the spectral CT reconstructed image Auxiliary variables Replacement material image tensor The equation is updated to an unconstrained augmented Lagrangian form as follows:

[0080]

[0081] Among them, multipliers To represent the tensor of error feedback, W and H represent the width and height of the reconstructed image, respectively, K represents the number of materials, S represents the number of energies, and λ1 and λ2 represent penalty factors.

[0082] S2: Equation (2) is transformed into 5 sub-problems (material decomposition, material regularization, image reconstruction, image regularization, and Lagrange operator solution) by the alternating direction multiplier method. Each sub-problem is represented as follows:

[0083]

[0084]

[0085]

[0086]

[0087]

[0088] To improve imaging quality, this invention employs a nonlinear operator transformation function to replace φ(·) and As a sparse regularization term.

[0089] S3: Integrate model-driven and data-driven methods to solve each sub-problem, and embed the Transformer network into the sub-problem solving process to improve the network's learning representation ability.

[0090] Based on the above mathematical model, this invention expands its iterative process into a neural network, the overall architecture and composition of which are as follows: Figure 2 As shown. This invention mainly solves the above sub-problems using gradient descent algorithm and neural network, where equations (4) and (6) are optimized using neural network.

[0091] Solving equation (3): Equation (3) can be transformed based on pixel level as follows:

[0092]

[0093] in express The value at pixel (l,k) express The value at pixel (l,s). and express and The value at pixel (i, j, k). Equation (8) is solved using the gradient descent method, and the solution formula is as follows.

[0094]

[0095] in Indicates the nth iteration The value at pixel (i, j, s).

[0096] Solving equation (4): With other variables fixed, equation (4) can be expressed as

[0097]

[0098] Expanding the above equation using a first-order Taylor series and taking the derivative, we obtain the following expression:

[0099]

[0100] in This indicates that the gradient of the variable u is calculated.

[0101] Using neural networks Alternative The following representation can be obtained.

[0102]

[0103] The network parameters and hyperparameters are updated in different iteration rounds, and equation (12) can be expressed as follows:

[0104]

[0105] To address the problem that convolutional neural networks cannot effectively learn the interaction between global and long-range image information, this invention introduces a Transformer structure in the solution of equation (4). This network includes convolutional blocks and a Swing Transformer structure. To simultaneously acquire local and global information, a residual structure is employed. The network contains three convolutional layers and one Swing Transformer (STB) module. Each STB module contains one convolutional layer and four Swing Transformer Layers (STLs). The network is described above as follows: Figure 3 As shown, from left to right, they are the overall network structure, the STB module structure, and the STL structure.

[0106] Solving Equation (5): The solution for image reconstruction based on pixel level can be expressed as follows:

[0107]

[0108] in, and express and The value at pixel (i, j, s) is obtained by solving equation (14) using the gradient descent method. The iterative formula for reconstructing the image can be expressed as follows:

[0109]

[0110] To improve the flexibility of the network, the step size g in this invention is obtained through iterative learning.

[0111] Solving equation (6): After fixing other variables and removing irrelevant variables, the optimization formula for image regularization can be expressed as follows:

[0112]

[0113] make Equation (16) can be transformed into

[0114]

[0115] Solving equation (17): Equation (17) can be approximately expressed as

[0116]

[0117] Where δ is a scalar that balances the terms, and further... The closed form can be represented as

[0118]

[0119] Here, the parameter γ represents a contraction threshold and the combination of δ and α. Introducing the left inverse of φ(·) The above expression can be expressed as

[0120]

[0121] Among them, left reverse It can be represented as For the identity matrix, this invention employs a convolutional neural network to perform sparsification processing on the image. φ(·) is designed as φ(·)C2(ReLU(C1(·))), where C1(·) and C2(·) represent convolution operators separated by a linear rectified function (ReLU). C1(·) corresponds to a convolutional layer with a kernel size of 3 and 1 channel, and C2(·) corresponds to a convolutional layer with a kernel size of 3 and 256 channels. The structure is designed as a symmetrical structure of φ(·).

[0122] Solving equation (7): and The solution can be expressed as

[0123]

[0124] S4: Iterate through S3 to complete the energy spectrum CT imaging.

[0125] Based on the above embodiments, such as Figure 4 As shown, this invention also proposes a spectral CT imaging device based on a multi-domain integrated Transformer iterative network, comprising:

[0126] An optimized framework building module is used to establish a coupled multi-task optimization framework model that combines energy spectrum CT image reconstruction and material decomposition based on the dependencies between multiple energy channels.

[0127] An optimized framework transformation module is used to transform the above optimized framework model into multiple sub-problems using the alternating direction multiplier method.

[0128] The sub-problem solving module is used to integrate model-driven and data-driven methods to solve each sub-problem, and embeds the Transformer network into the sub-problem solving process to improve the network's learning representation ability;

[0129] The iterative execution module is used to iteratively execute the sub-problem solving module to complete energy spectral CT imaging.

[0130] In summary, this invention proposes a spectral CT imaging method and device based on a multi-domain ensemble Transformer iterative network, combining the advantages of traditional models and deep learning. This invention possesses the advantages of both model-driven and data-driven approaches, exhibiting strong stability and learning representation capabilities. Various experiments have demonstrated the superior performance and generalization ability of this method. Figure 5 This is an example of the test results of the method of the present invention in thoracic cavity data.

[0131] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A spectral CT imaging method based on a multi-domain integrated Transformer iterative network, characterized in that, include: Step 1: Based on the dependencies between multiple energy channels, establish a coupled multi-task optimization framework model that combines energy spectrum CT image reconstruction and material decomposition; Step 2: Transform the above optimization framework model into multiple sub-problems using the alternating direction multiplier method; Step 3: Integrate model-driven and data-driven methods to solve each sub-problem, and embed the Transformer network into the sub-problem solving process to improve the network's learning representation ability; Step 4: Iterate through step 3 to complete the energy spectrum CT imaging.

2. The spectral CT imaging method based on a multi-domain integrated Transformer iterative network according to claim 1, characterized in that, The optimization framework model is as follows: Where R represents the system matrix that maps the image to the projection. This represents the tensor of the energy spectrum CT reconstructed image. This represents the energy spectrum CT projection tensor. Let A denote the material image tensor, and let A denote the system matrix of the image domain decomposition; φ(·) denotes the transformation on the reconstructed image. This represents sparse regularization based on the L1 norm of the transformation. Let F represent the material regularization term, α, β, and η be the parameters of the equilibrium terms, and F represent the norm. express The matrix formed by unfolding along the third dimension express The matrix formed by unfolding along the third dimension; Through auxiliary variables Replace the tensor of the spectral CT reconstructed image Auxiliary variable u replaces the material image tensor The equation is updated to an unconstrained augmented Lagrangian form: in To represent the tensor of error feedback, W and H represent the width and height of the reconstructed image, respectively, K represents the number of materials, S represents the number of energies, and λ1 and λ2 represent penalty factors.

3. The energy spectrum CT imaging method based on a multi-domain integrated Transformer iterative network according to claim 2, characterized in that, In step 2, each sub-problem is represented as follows: in Let u represent the material image tensor at the (n+1)th iteration. (n) Let u represent the auxiliary variable u at the nth iteration. This represents the corresponding iteration in the nth iteration. This represents the corresponding iteration in the nth iteration. This represents the energy spectrum CT reconstructed image tensor at the (n+1)th iteration. This represents the auxiliary variable corresponding to the nth iteration.

4. The spectral CT imaging method based on a multi-domain integrated Transformer iterative network according to claim 3, characterized in that, In step 3, equation (3) is solved in the following manner: Transformation of equation (3) based on pixel level: in express The value at pixel (l,k) express The value at pixel (l,s); u i,j,k , and Represents u, and The value at pixel (i, j, k); by solving equation (8) using the gradient descent method, we obtain: in This represents the corresponding iteration at the (n+1)th iteration. and The value at pixel (i, j, k) Indicates the nth iteration The value at pixel (i, j, s).

5. The energy spectrum CT imaging method based on a multi-domain integrated Transformer iterative network according to claim 3, characterized in that, In step 3, equation (4) is solved in the following manner: With other variables fixed, we perform a first-order Taylor expansion of equation (4) and take its derivative, resulting in the following expression. in This indicates that the gradient is calculated with respect to the variable u. Using neural network Ψ(u (n) ) alternative The following representation is obtained. The network parameters and hyperparameters are updated in different iteration rounds, as expressed in equation (12).

6. The spectral CT imaging method based on a multi-domain integrated Transformer iterative network according to claim 5, characterized in that, The neural network contains three convolutional layers and a Swing Transformer module.

7. The spectral CT imaging method based on a multi-domain integrated Transformer iterative network according to claim 3, characterized in that, In step 3, equation (5) is solved in the following manner: The solution for image reconstruction based on pixel level is expressed as follows: in, and express and The value at pixel (i, j, s); Solving equation (14) using the gradient descent method, the iterative formula for reconstructing the image is expressed as follows: Where g represents the step size, and n and n+1 both represent the number of iterations.

8. The energy spectrum CT imaging method based on a multi-domain integrated Transformer iterative network according to claim 3, characterized in that, In step 3, equation (6) is solved as follows: With other variables fixed and irrelevant variables removed, the optimization formula for image regularization is expressed as follows: make Equation (16) is transformed into Equation (17) is approximately expressed as Where δ is a scalar quantity that balances the terms. The closed form is represented as Wherein, parameter γ represents a contraction threshold and the combination of δ and α; Introducing the left inverse of φ(·) Equation (19) is expressed as in, Represented as As the identity matrix, φ(·) is designed as φ(·)=C2(ReLU(C1(·))), where C1(·) and C2(·) denote the convolution operators separated by the linear rectified function ReLU. The structure is designed as a symmetrical structure of φ(·).

9. The spectral CT imaging method based on a multi-domain integrated Transformer iterative network according to claim 3, characterized in that, In step 3, based on the obtained parameters, substituting them into formula (7) yields the result. and 10. A spectral CT imaging device based on a multi-domain integrated Transformer iterative network, characterized in that, include: An optimized framework building module is used to establish a coupled multi-task optimization framework model that combines energy spectrum CT image reconstruction and material decomposition based on the dependencies between multiple energy channels. An optimized framework transformation module is used to transform the above optimized framework model into multiple sub-problems using the alternating direction multiplier method. The sub-problem solving module is used to integrate model-driven and data-driven methods to solve each sub-problem, and embeds the Transformer network into the sub-problem solving process to improve the network's learning representation ability; The iterative execution module is used to iteratively execute the sub-problem solving module to complete energy spectral CT imaging.

Images