Sparse angle CT reconstruction method and model training method based on deep learning

Abstract

The invention discloses a sparse angle CT reconstruction method and model training method based on depth expansion and residual optimization, a depth expansion and residual optimization network comprises a local residual optimization module and a non-local residual optimization module, and the local residual optimization module comprises a wavelet U-Net module and a dense space attention module. The non-local residual optimization module comprises a shift window transformer module and a cavity dense channel attention module, and inputs the sparsely sampled projection data into the depth expansion and residual optimization network to obtain an initial image; residual projection data is acquired and reconstructed in combination with the projection data, and a residual image is obtained; inputting the residual image into a local residual optimization module for iteration to obtain an iterated residual image and a channel feature image, processing the residual image and a residual image obtained by last iteration, inputting the processed image into a non-local residual optimization module for iteration to obtain a channel feature image; and the local residual optimization module and the non-local residual optimization module are alternately used subsequently for iteration until a target image is obtained.

Description

Technical Field

[0001] The present invention belongs to the technical field of image reconstruction, and relates to, but is not limited to, a sparse-angle CT reconstruction method and a model training method based on deep learning. Background Art

[0002] Computed Tomography (CT) is an inspection technique widely used in the field of medical imaging. By combining X-ray and computer processing technology, cross-sectional images of the human body can be obtained. However, the ionizing radiation in CT examinations may pose potential hazards to the human body, especially for patients who need multiple scans. Research shows that the carcinogenic risk of a single CT examination can reach 0.05%-0.1%, and the risk for pediatric patients is even 3-4 times higher than that of adults. Therefore, in CT examinations, the ALARA (As Low As Reasonably Achievable) principle must be strictly followed, and the radiation dose should be reduced as much as possible while ensuring image quality.

[0003] As an important approach to achieving low-dose scanning, sparse-angle CT reconstruction technology significantly reduces the radiation dose by reducing the number of projection angles (usually reduced to 1 / 3 to 1 / 5 of conventional scanning), while still maintaining sufficient image information. For patients who need long-term follow-up, sparse-angle scanning has important clinical application value. However, how to reconstruct image quality that meets the diagnostic requirements from limited projection data has become the main challenge faced by this technology.

[0004] In related technologies, there are two categories: analytical methods and iterative methods. Analytical methods are represented by Filtered BackProjection (FBP) and Feldkamp-Davis-Kress (FDK) algorithms. They are widely used in clinics due to their advantages such as high computational efficiency and fast reconstruction speed. However, these methods have high requirements for projection data and will produce serious streak artifacts and noise under sparse-angle scanning conditions, resulting in a significant decline in image quality and seriously affecting diagnostic accuracy. The iterative reconstruction algorithm transforms the reconstruction problem into an objective function optimization problem by establishing a mathematical model. However, when the projection data is overly undersampled and lacks additional prior information, there are still defects such as streak artifacts, large image noise, and unclear structure in its reconstruction results.

[0005] Therefore, how to improve the quality of sparse-angle CT reconstructed images has become an urgent problem to be solved. Summary of the Invention

[0006] In view of this, an embodiment of the present invention provides a sparse-angle CT reconstruction method based on deep learning, which at least solves the problem of low quality of sparse-angle CT reconstructed images in the related art.

[0007] According to the first aspect of the embodiments of the present invention, there is provided a sparse-angle CT reconstruction method based on deep learning, including:

[0008] Input the sparsely sampled projection data into a deep unfolding and residual optimization network to obtain an initial image; and perform image reconstruction based on the initial image, the projection data, and the filtered back-projection algorithm to obtain a first residual image;

[0009] Input the first residual image into a local residual optimization module to obtain a second residual image and a first channel feature image with the number of channels halved; and add the initial image and the second residual image to obtain a third residual image after the first iteration; the deep unfolding and residual optimization network includes a local residual optimization module and a non-local residual optimization module, and the local residual optimization module is composed of a wavelet U-Net module and a dense spatial attention module;

[0010] Perform image reconstruction based on the third residual image, the projection data, and the filtered back-projection algorithm to obtain a fourth residual image;

[0011] Input the fourth residual image and the first channel feature image into the non-local residual optimization module to obtain a fifth residual image and a second channel feature image with the number of channels halved; and add the second residual image and the fifth residual image to obtain a sixth residual image after the second iteration; the non-local residual optimization module is composed of a shifted window transformer module and a dilated dense channel attention module;

[0012] Perform image reconstruction again based on the sixth residual image, the projection data, and the filtered back-projection algorithm to obtain a seventh residual image; and input the seventh residual image and the second channel feature image into the local residual optimization module for the third iteration until the number of iterations is reached to obtain the target image.

[0013] According to the second aspect of the embodiments of the present invention, there is provided a training method for a sparse-angle CT reconstruction model based on deep learning, including:

[0014] Input the sparsely sampled projection data samples into a deep unfolding and residual optimization network to obtain initial image samples;

[0015] Perform image reconstruction based on the initial image samples, the projection data samples, and the filtered back-projection algorithm to obtain first residual image samples;

[0016] Input the first residual image sample into the local residual optimization module to obtain a second residual image sample and a first channel feature image sample with the number of channels halved; and add the initial image sample and the second residual image sample to obtain a third residual image sample after the first iteration; the deep unfolding and residual optimization network includes a local residual optimization module and a non-local residual optimization module, and the local residual optimization module is composed of a wavelet U-Net module and a dense spatial attention module;

[0017] Based on the third residual image sample, the projection data sample, and the filtered back-projection algorithm, perform image reconstruction to obtain a fourth residual image sample;

[0018] Input the fourth residual image sample and the first channel feature image sample into the non-local residual optimization module to obtain a fifth residual image and a second channel feature image sample with the number of channels halved; and add the second residual image sample and the fifth residual image sample to obtain a sixth residual image sample after the second iteration; the non-local residual optimization module is composed of a shifted window transformer module and a dilated dense channel attention module;

[0019] Perform image reconstruction again based on the sixth residual image sample, the projection data sample, and the filtered back-projection algorithm to obtain a seventh residual image sample; and input the seventh residual image sample and the second channel feature image sample into the local residual optimization module for the third iteration until the number of iterations is reached to obtain a reconstructed image sample;

[0020] Based on the reconstructed image sample, the label image, and the reference image, train the deep unfolding and residual optimization network to be trained until the trained deep unfolding and residual optimization network is obtained.

[0021] According to the solution provided by the embodiments of the present invention, the deep unfolding and residual optimization network consists of a local residual optimization module and a non-local residual optimization module. The local residual optimization module (LROM module) is used to learn local information of edges and texture features, and retains the main structure and multi-scale feature information of the image. In addition, the non-local residual optimization module (NLROM module) serves as a non-local residual optimization module, which can extract long-range pixel interactions. First, a residual optimization objective function is used to minimize the reconstruction error to the greatest extent. The prior of the integrated residual optimization module (Residual Optimization Module, ROM) is used as regularization to explore deep features in the residual data. In addition, block coordinate descent is adopted to implement a novel iterative framework. During the iteration process, the Wavelet U-Net and the shifted window transformer module are alternately used to capture local and long-range pixel interactions simultaneously. For the above two different networks, corresponding iterative interaction modules are designed for them. Through the iterative interaction module, the intermediate feature maps of each iterative block will be transmitted and connected to the next iteration. The iterative interaction module will establish an additional path to interact the deep features extracted by the network between different iterative blocks. Therefore, the deep unfolding and residual optimization network can improve the network performance by extracting deep features and obtain better visual effects. Description of the Drawings

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained according to these drawings, where:

[0023] Figure 1 It is a schematic flowchart of a sparse angle CT reconstruction method based on deep learning provided by the embodiments of the present invention;

[0024] Figure 2 It is an overall flowchart of a deep unfolding and residual optimization network provided by the embodiments of the present invention;

[0025] Figure 3 It is a schematic diagram of the overall structure of a local residual optimization module provided by the embodiments of the present invention;

[0026] Figure 4 It is a schematic diagram of the overall structure of a non-local residual optimization module provided by the embodiments of the present invention;

[0027] Figure 5 It is the reconstructed images of different reconstruction algorithms at 128 sparse angles provided by the embodiments of the present invention;

[0028] Figure 6 The enlarged view of the region of interest provided by the embodiment of the present invention Figure 5 in;

[0029] Figure 7 The cross-sectional view (128 degrees) along the red line (located at Figure 5 ) of the experimental results of a different method provided by the embodiment of the present invention;

[0030] Figure 8 The schematic diagram of the reconstruction image effect of a different reconstruction algorithm provided by the embodiment of the present invention at 64 sparse angles;

[0031] Figure 9 The enlarged view of the region of interest provided by the embodiment of the present invention Figure 7 in;

[0032] Figure 10 The cross-sectional view (64 degrees) along the red cross-section line (located at Figure 8 ) of the experimental results of different methods;

[0033] Figure 11 The schematic diagram of the reconstruction image effect of different reconstruction methods provided by the embodiment of the present invention at 32 sparse angles;

[0034] Figure 12 The enlarged view of the region of interest provided by the embodiment of the present invention Figure 10 in;

[0035] Figure 13 The cross-sectional view (32 degrees) along the red cross-section line (located at Figure 11 ) of the experimental results of different methods provided by the embodiment of the present invention;

[0036] Figure 14 The box plot of the reconstruction results of different methods at 128 angles (a), 64 angles (b), and 32 angles (c) provided by the embodiment of the present invention. Detailed implementation manners

[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. The following embodiments are used to illustrate the present invention, but are not used to limit the scope of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0038] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.

[0039] It should be noted that the terms "first / second / third" involved in the embodiments of the present invention are only used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first / second / third" can be interchanged with a specific order or sequence when allowed, so that the embodiments of the present invention described herein can be implemented in an order other than that illustrated or described herein.

[0040] Those skilled in the art of the present technology can understand that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as the general understanding of those of ordinary skill in the art to which the embodiments of the present invention belong. It should also be understood that terms such as those defined in a general dictionary should be understood to have a meaning consistent with the meaning in the context of the prior art, and will not be interpreted with an idealized or overly formal meaning unless specifically defined as herein.

[0041] Figure 1 The flowchart of a sparse-angle CT reconstruction method based on deep learning provided for the embodiments of the present invention can be executed by an electronic device, such as a computer, a server, etc.

[0042] As Figure 1 shown, the sparse-angle CT reconstruction method based on deep learning includes:

[0043] S101. Input the sparsely sampled projection data into a deep unfolding and residual optimization network to obtain an initial image; and perform image reconstruction based on the initial image, the projection data, and the filtered back-projection algorithm to obtain a first residual image.

[0044] In the embodiments of the present invention, the projection data is obtained by penetrating an object (usually a part of the human body) with X-rays from multiple angles and recording the X-ray absorption amounts at different positions. These absorption amounts reflect the information of the internal structure of the object because different tissue types have different absorption rates for X-rays. The projection data is essentially the "shadow" or contour information of the object from different perspectives. First, the projection data of the object (such as the human body) is obtained by a sparse-angle CT device. The projection data is input into a deep unfolding and residual optimization network to perform image reconstruction using the FBP algorithm to obtain an initial image, and image reconstruction is performed using the initial image, the projection data, and the filtered back-projection algorithm to obtain a first residual image.

[0045] S102. Input the first residual image into the local residual optimization module to obtain the second residual image and the first channel feature image with the number of channels halved; and add the initial image and the second residual image to obtain the third residual image after the first iteration. The local residual optimization module consists of a wavelet U-Net module and a DSAR.

[0046] In an embodiment of the present invention, the local residual optimization module (LROM module) consists of a wavelet U-Net module and a dense spatial attention module. The local residual optimization module is used to learn local information of edges and texture features, and retains the main structure and multi-scale feature information of the image. The design of the wavelet U-Net module is based on the U-Net structure and combines discrete wavelet transform (DWT) and inverse wavelet transform (IWT). The encoder aims to extract low-dimensional features by gradually shrinking the feature map using max pooling operations. The decoder can effectively restore high-quality images by gradually expanding the feature map. Dense connections help improve parameter utilization. At the same time, DWT is applied to the input image in the encoder to separate the local detail information and low-frequency information of the image, enhancing the network's sensitivity to edges, textures, and local structures. IWT is used in the decoder to restore the high-frequency details of the image, enhancing the edge sharpness and structural details of the image, and alleviating the problem that detail information may be lost when the image is processed by pooling operations or convolutional layers. In addition, skip connections connect shallow features with deep features to avoid detail loss and accelerate feature mapping. The dense spatial attention module is mainly composed of a dense residual layer and a spatial attention layer. Input the first residual image into the local residual optimization module, and process it through the wavelet U-Net module and the dense spatial attention module in the local residual optimization module to obtain the second residual image and the first channel feature image with the number of channels halved. Add the initial image and the second residual image to obtain the third residual image after the first iteration.

[0047] S103. Based on the third residual image, projection data, and the filtered back-projection algorithm, perform image reconstruction to obtain the fourth residual image.

[0048] In an embodiment of the present invention, obtain the second residual projection data based on the third residual image and the projection data, and use the FBP algorithm to perform image reconstruction on the second residual projection data to obtain the fourth residual image.

[0049] S104. Input the fourth residual image and the first channel feature image into the non-local residual optimization module to obtain the fifth residual image and the second channel feature image with the number of channels halved; and add the second residual image and the fifth residual image to obtain the sixth residual image after the second iteration. The non-local residual optimization module consists of a shifted window transformer module and a dilated dense channel attention module.

[0050] In an embodiment of the present invention, the non-local residual optimization module (NLROM module) is composed of a shifted window transformer module and a dilated dense channel attention module. The shifted window transformer module applies convolution to the feature image from the local residual optimization module to obtain shallow features. The dilated dense channel attention module consists of dense residual connections and channel attention. The fourth residual image and the first channel feature image are input into the non-local residual optimization module, and after being processed by the shifted window transformer module and the dilated dense channel attention module, a fifth residual image and a second channel feature image with the number of channels halved are obtained. Then, the second residual image and the fifth residual image are added together to obtain the sixth residual image after the second iteration. The non-local residual optimization module serves as a non-local residual optimization module to learn the non-local regularization features of the residual image, which can extract long-range pixel interactions and capture global information at different levels.

[0051] S105. Again, based on the sixth residual image, the projection data, and the filtered back-projection algorithm, a seventh residual image is obtained; and the seventh residual image is input into the local residual optimization module for the third iteration process until the preset number of iterations is reached to obtain the target image.

[0052] In an embodiment of the present invention, after the second iteration ends, the third iteration is carried out again. The third residual projection data is obtained from the sixth residual image and the projection data obtained after the second iteration ends, and image reconstruction is performed on the third residual projection data through the FBP algorithm to obtain the seventh residual image. Then, the seventh residual image and the second channel feature image are input into the LROM for the third iteration process. The multiple iteration processes are continuously executed until the preset number of iterations is reached to obtain the target image reconstructed from the projection data. The local residual optimization module and the non-local residual optimization module alternate in subsequent iterations until the number of iterations is reached.

[0053] Among them, after each iteration, the finally obtained is the residual image obtained by adding the residual image output by the non-local residual optimization module or the local residual optimization module and the residual image after the previous iteration. The non-local residual optimization module or the local residual optimization module not only outputs the residual image after each iteration process, but also outputs a channel feature image with the number of channels halved.

[0054] Among them, if the preset number of iterations is 10 times, the local residual optimization module is used in odd-numbered iterations, and the non-local residual optimization module is used in even-numbered iterations. That is, in the 1st, 3rd, 5th, 7th, and 9th iterations, the residual image is processed by the local residual optimization module, and in the 2nd, 4th, 6th, 8th, and 10th iterations, the residual image is processed by the non-local residual optimization module.

[0055] As Figure 2 shown, Figure 2This is the overall flowchart of a depth unfolding and residual optimization network provided by an embodiment of the present invention. In Figure 2 , the sparsely sampled projection data y (0) is reconstructed by the FBP algorithm to obtain the initial image x (0) . Then, y (0) and x (0) are input into the depth unfolding and residual optimization network to start the first iteration. In odd iterations, it has the same structure as iteration3, and in even iterations, it has the same structure as iteration2. In the first iteration, x (0) passes through A (forward projection) and is subtracted from the sparsely sampled projection data y (0) to obtain the residual projection data. The residual projection data passes through FBP (filtered back projection) to obtain the residual image. The residual image is input into the local residual optimization module (composed of a wavelet U-Net module and a dense spatial attention module) to obtain the processed residual image and the channel feature image H 1 with the number of channels halved. The processed residual image and x (0) are added together to obtain x 1 . Thus, the first iteration is completed and the second iteration begins. After x 1 passes through A (forward projection) and is subtracted from the sparsely sampled projection data y (0) to obtain the residual projection data. The residual projection data passes through FBP (filtered back projection) to obtain the residual image. The residual image and H 1 are input into the non-local residual optimization module (composed of a shifted window transformer module and a dilated dense channel attention module) to obtain the processed residual image and H 2 with the number of channels halved. The processed residual image and x 1 are added together to obtain x 2 . Thus, the second iteration is completed and the third iteration begins. After x 2 passes through A (forward projection) and is subtracted from the sparsely sampled projection data y (0) to obtain the residual projection data. The residual projection data passes through FBP (filtered back projection) to obtain the residual image. The residual image and H 2 are input into the local residual optimization module (composed of a wavelet U-Net module and a dense spatial attention module) to obtain the processed residual image and the channel feature image H 3 with the number of channels halved. The processed residual image and x 2 are added together to obtain x 3 . Thus, the third iteration is completed and the fourth iteration begins. The local residual optimization module and the non-local residual optimization module are continuously alternately used for iteration until the preset number of iterations 10 is reached to obtain the target image x 10. The Deep Unfolding and Residual Optimization Network can combine local and non-local information of the residual image and improve the overall performance of the model through the interaction of deep features during the iterative process.

[0056] In summary, the present invention proposes two iterative interaction modules for establishing the interaction relationship between different iterations. As Figure 2 shown, this module is embedded inside the LROM and NLROM, and outputs the intermediate feature H k+1 which is transmitted between the iterative modules and connected to the next iteration. The iterative interaction module proposed by the present invention can effectively expand the network depth, thus significantly improving the model performance; since the conversion from deep features to pixel-level features is avoided in the entire iterative interaction module path, this module not only realizes the connection between iterations, but also can extract deeper features, further enhancing the model's ability to maintain structural details. According to the two different modes of ROM, namely LROM and NLROM, the iterative interaction module is correspondingly designed in two forms to achieve matching. As Figure 2 shown, in the LROM, the iterative interaction module is a dense spatial attention module, which consists of a dense residual and a spatial attention mechanism. This design aims to promote feature reuse, optimize the information flow between layers, and enhance the learning ability of the spatial relationship in the feature map. While in the NLROM, the iterative interaction module is a dilated dense channel attention module, which consists of a dense residual, multi-scale dilated convolutions, and a channel attention mechanism. By adopting convolutional layers with different dilation rates, this module can effectively capture multi-scale features, significantly expand the receptive field of the network, and at the same time enhance the model's attention ability to the information in the channel dimension. This targeted design enables the iterative interaction module to better adapt to the task requirements of different residual optimizations, thus comprehensively improving the network performance.

[0057] In the embodiment of the present invention, based on residual optimization, a Deep Unfolding and Residual Optimization Network is established and then the target image is obtained. The principle is as follows:

[0058] To solve for x from the sparse-view data y (0) , usually, consistent constraints are imposed in the image space. Due to the redundant information in the image domain, this method may not be able to fully utilize the inherent consistency features of the data. Inspired by the iterative framework, the present invention uses an objective function focusing on residual optimization for the Deep Unfolding and Residual Optimization Network, as shown in the following (1):

[0059]

[0060] In the above formula (1), the argmin function is used to find the parameter value that makes the objective function reach the minimum value. The first term represents the regularization function of the residual image in the image domain, and the second term It represents data consistency in the projection domain, where λ is a hyperparameter that balances the two terms. A is a matrix, and y is an intermediate variable representing the residual in the projection domain, that is, the difference between the predicted projection data and the projection of the real image. It is a residual-optimized reconstruction network used to combine the prior information of the unobservable components of the residual data. In this study, It consists of two parts. One part is composed of the operator of the Filtered Backprojection (FBP) algorithm, and the other part is the residual-optimized network in the image domain. So It is a residual image. Therefore, the first term is to minimize the image domain noise caused by network instability, and the second term is to minimize the measured noise in the projection domain. The following formula (2) can be defined:

[0061]

[0062] In the above formula, L is the loss function.

[0063] Then, the block coordinate descent method can be used to optimize formula (2), as shown in the following (3):

[0064]

[0065] In the above formula (3), K is the total number of iterations, and the argminL function refers to the process of finding the parameter values that minimize the loss function L.

[0066] To update y, the following formula (4) problem needs to be solved:

[0067]

[0068] By calculating the partial derivative of the right side of formula (4), the following formula (5) can be obtained:

[0069]

[0070] In the embodiment of the present invention, since the neural network is designed specifically to solve the problem Ax = y, the of formula (5) can be regarded as the backpropagation of the network. Therefore, it can be obtained that It should be noted that this is an approximation. This approximation has been confirmed to be feasible through the convergence analysis of existing articles, and the present invention will not elaborate on this too much. In the iterative reconstruction framework, this approximation has a good effect and brings convenience to the next simplification of the formula. Therefore, updating y can obtain the following formula (6):

[0071]

[0072] Similarly, to update x, solve the following problem of formula (7):

[0073]

[0074] Calculate the partial derivative on the right side of formula (7), and the following formula (8) can be obtained:

[0075] λA T Ax = λA T (y (0) -y (k+1) ) (8)

[0076] In the above formula, A T is the transpose matrix of A.

[0077] From formula (5), the following formula (9) can be obtained:

[0078]

[0079] Because And substituting (9) into (8), the following formula (10) can be obtained:

[0080] λA T Ax = A T y (k+1) +λA T Ax (k) (10)

[0081] Also because The following formula (11) can be obtained:

[0082]

[0083] Combining formula (6) and (11), the update equation can be obtained as shown in the following (12):

[0084]

[0085] In the embodiments of the present invention, it is desired to minimize equation (2) to reconstruct a high-quality image. Specifically, in order to reduce the residuals and artifacts in the image domain, the first term can be minimized. In order to reduce the residuals in the projection domain and suppress the noise in the projection domain, it is achieved by minimizing the second term. After convergence through the iterative process, the residuals in the projection domain should theoretically approach zero.

[0086] It is understandable that in the embodiments of the present invention, the deep unfolding and residual optimization network consists of a local residual optimization module and a non-local residual optimization module. The local residual optimization module is used to learn local information of edges and texture features, and retains the main structure and multi-scale feature information of the image. In addition, the non-local residual optimization module serves as a non-local residual optimization module, which can extract long-range pixel interactions. First, a residual optimization objective function is used to minimize the reconstruction error, and the prior of the residual optimization module is integrated as regularization to explore the deep features in the residual data. In addition, block coordinate descent is adopted to implement a novel iterative framework. During the iteration, the wavelet U-Net and the shifted window transformer module are alternately used to capture local and long-range pixel interactions simultaneously. For the above two different networks, corresponding iterative interaction modules are designed. Through the iterative interaction module, the intermediate feature maps of each iterative block will be transmitted and connected to the next iteration. The iterative interaction module will establish an additional path to interact the deep features extracted by the network between different iterative blocks. Therefore, the deep unfolding and residual optimization network can improve the network performance by extracting deep features and obtain better visual effects.

[0087] In some embodiments of the present invention, the image reconstruction based on the initial image, the projection data, and the filtered backprojection algorithm in S102 to obtain the first residual image can be implemented through S1021 to S1022, and the description is as follows.

[0088] S1021. The initial image is orthogonally projected to obtain target projection data, and the target projection data is subtracted from the projection data to obtain the first residual projection data.

[0089] S1022. Based on the filtered backprojection algorithm, the first residual projection data is reconstructed to obtain the first residual image.

[0090] In some embodiments of the present invention, the initial image is processed using the orthogonal projection technique to obtain target projection data, and the target projection data is subtracted from the projection data to obtain the first residual projection data. The filtered backprojection algorithm is used to reconstruct the first residual projection data to obtain the first residual image.

[0091] Further, the image reconstruction based on the third residual image, the projection data, and the filtered backprojection algorithm to obtain the fourth residual image is specifically to subtract the orthogonally projected third residual image from the projection data, and use the filtered backprojection algorithm to reconstruct the subtracted residual data to obtain the fourth residual image. The specific steps of the image reconstruction based on the sixth residual image, the projection data, and the filtered backprojection algorithm are the same.

[0092] In some embodiments of the present invention, inputting the first residual image into the LROM to obtain the second residual image and the first channel feature image with the number of channels halved in S103 can be implemented through S1031 to S1033, and the details are described as follows.

[0093] S1031. Perform dimensional processing on the first residual image by using the first dense residual layer, the first spatial attention layer, the wavelet transform layer, the inverse wavelet transform layer, the convolutional layer, the batch normalization layer, and the activation layer to obtain the second residual image.

[0094] In some embodiments of the present invention, the wavelet U-Net module includes the first dense residual layer, the first spatial attention layer, the wavelet transform layer, the inverse wavelet transform layer, the convolutional layer, the batch normalization layer, and the activation layer; the dense spatial attention module includes the second dense residual layer and the second spatial attention layer. Perform dimensional processing on the first residual image by using the first dense residual layer, the first spatial attention layer, the wavelet transform layer, the inverse wavelet transform layer, the convolutional layer, the batch normalization layer, and the activation layer to obtain the second residual image.

[0095] S1032. Input the second residual image into the second dense residual layer and the second spatial attention layer in sequence to obtain the first channel feature image, and the output of the second dense residual layer is the input of the second spatial attention layer.

[0096] In some embodiments of the present invention, the dense spatial attention module includes the second dense residual layer and the second spatial attention layer. Input the second residual image into the second dense residual layer to obtain the processed residual image. Ensure that each layer can obtain the feature maps of all previous layers through dense connection, and then strengthen the flow of these features through residual connection to reduce information loss. Input the processed residual image into the second spatial attention layer to obtain the first channel feature image.

[0097] As Figure 3 shown, Figure 3 is a schematic diagram of the overall structure of a local residual optimization module provided by an embodiment of the present invention. x k represents the feature image of H×W×C jointly composed of the red block and the blue block, and the blue block represents extracting the C / 2 channel feature image of x k and then combining it with H kFusion is performed to obtain a new H×W×C feature image, which is input into the Wavelet U-Net module. In the Wavelet U-Net module, the new H×W×C feature image first undergoes processing through a convolutional layer, a normalization layer, and an activation function layer, and then undergoes feature dimensionality reduction through multiple wavelet transform layers, convolutional layers, normalization layers, and activation function layers. When the features are reduced to a certain dimension, an inverse wavelet transform layer is performed after passing through a convolutional layer and a dense residual layer, and upsampling is performed through multiple convolutions, normalizations, activation functions, and inverse wavelet transforms to restore to the original dimension. After being processed by the Wavelet U-Net module, x k with the same dimension as x k +1 is obtained. The green arrow indicates that the "skip connection" is a direct connection from the downsampling path (left descending arm) to the upsampling path (right ascending arm) to the corresponding feature map. When in the right ascending arm, the feature maps of the green arrow and the blue arrow are concatenated and then undergo a series of processes. x k+1 is input into the dense spatial attention module (including a dense residual layer and a spatial attention layer) to obtain a C / 2-channel feature image H k+1 .

[0098] In some embodiments of the present invention, inputting the fourth residual image and the first channel feature image into the NLROM in S105 to obtain the fifth residual image and the second channel feature image with the number of channels halved can be implemented through S1051 to S1052, which is described in the following steps.

[0099] S1051: Halve the number of channels of the fourth residual image to obtain a halved-channel feature image, and fuse the halved-channel feature image and the first channel feature image to obtain a second fused image.

[0100] S1052: Based on the second fused image, a layer normalization layer, a window-based multi-head self-attention layer, a multi-layer perceptron layer, a shifted window multi-head self-attention layer, and a window merging layer, obtain the fifth residual image; and input the fifth residual image into the dilated dense channel attention module to obtain the second channel feature image.

[0101] In some embodiments of the present invention, the shifted window transformer module consists of a window-based multi-head attention layer and two-layer multi-layer perceptron, or a shifted window multi-head attention layer and two-layer multi-layer perceptron, which halves the number of channels of the fourth residual image to obtain a channel feature image after halving, and fuses the channel feature image after halving with the first channel feature image to obtain a second fused image. Specifically, the layer normalization layer, window-based multi-head self-attention layer, multi-layer perceptron layer, shifted window multi-head self-attention layer, and window merging layer in the shifted window transformer module are used to process the second fused image to obtain a fifth residual image, and the fifth residual image is input into the dilated dense channel attention module layer, and after being processed by the dilated convolutional layer and the dense connection layer, a second channel feature image is obtained. Using the shifted window transformer module not only focuses on locally adjacent pixels, but also can explore the relationship between pixels in the window, thereby obtaining non-local regularization features and improving the performance and expressive ability of feature extraction.

[0102] In some embodiments of the present invention, obtaining the fifth residual image based on the second fused image, layer normalization layer, window-based multi-head self-attention layer, multi-layer perceptron layer, shifted window multi-head self-attention layer, and window merging layer in S1052 can be composed of S201 to S205 and is described through the following steps.

[0103] S201: Input the second fused image into the layer normalization layer and the window-based multi-head self-attention layer in sequence to obtain a first feature image processed by the attention mechanism; the output of the layer normalization layer is the input of the window-based multi-head self-attention layer.

[0104] In some embodiments of the present invention, the layer normalization layer (LayerNormalization layer) normalizes the input second fused image to obtain a normalized feature image, and inputs the normalized feature image into the window-based multi-head self-attention layer, and the output is usually a first feature image processed by the attention mechanism. The weighted multi-head self-attention layer is a variant of the self-attention mechanism (Self-Attention), which flexibly captures features in the input image by introducing weights (Weighted) to perform weighted fusion on the outputs of different attention heads (Head).

[0105] S202: Add the first feature image and the second fused image to obtain a second feature image; and input the second feature map into the layer normalization layer and the multi-layer perceptron layer in sequence to obtain a third feature map; the output of the layer normalization layer is the input of the multi-layer perceptron layer.

[0106] In some embodiments of the present invention, the first feature image and the second fused image are added to obtain a second feature image. The second feature image is input into a layer normalization layer for normalization processing to obtain a normalized feature image. Finally, the normalized feature image is input into a multi-layer perceptron layer (MLP) to obtain a third feature map.

[0107] S203. Add the third feature map and the first feature image to obtain a fourth feature image, and sequentially input the fourth feature image into a layer normalization layer and a multi-head self-attention layer with shifted windows to obtain a fifth feature image processed by the attention mechanism; the output of the layer normalization layer is the input of the multi-head self-attention layer with shifted windows.

[0108] S204. Add the fifth feature image and the fourth feature map to obtain a sixth feature map, and sequentially input the sixth feature map into a layer normalization layer and a multi-layer perceptron layer to obtain a seventh feature map; the output of the layer normalization layer is the input of the multi-layer perceptron layer.

[0109] In some embodiments of the present invention, the third feature map and the first feature image are added to obtain a fourth feature map. The fourth feature image is sequentially input into a layer normalization layer to obtain a normalized image, and the normalized feature image is input into a multi-head self-attention layer with shifted windows. The output is a fifth feature image processed by the attention mechanism, and its dimension is consistent with the spatial resolution and the number of channels of the input feature map. Add the fifth feature image and the fourth feature map to obtain a sixth feature map. Finally, the sixth feature image is sequentially processed by a layer normalization layer and a multi-layer perceptron layer. This process is similar to S202 and will not be described here. The weighted multi-head self-attention layer can learn the attention relationship between pixel-level features within the window and aggregate the features based on the learned attention weights, thereby extracting richer pixel-level feature information.

[0110] S204. Add the seventh feature image and the sixth feature image to obtain an eighth feature image, and sequentially input it into a layer normalization layer and a window merging layer to obtain a fifth residual image; the output of the layer normalization layer is the input of the window merging layer.

[0111] In some embodiments of the present invention, the seventh feature image and the sixth feature image are added to obtain an eighth feature image. The eighth feature image is input into a layer normalization layer for normalization processing to obtain a normalized feature image. Finally, the normalized feature image is input into a window merging layer to reorganize the window between different stages, realizing resolution reduction and information integration to obtain a fifth residual image.

[0112] As Figure 4 shown, Figure 4Schematic diagram of the overall structure of a non-local residual optimization module provided by an embodiment of the present invention. In Figure 4 where, x k represents a feature image of H×W×C composed of a red block and a blue block. The blue block represents the feature image of C / 2 channels extracted from x k , and then it is fused with H k to obtain a new feature image of H×W×C. The new feature image of H×W×C first passes through a layer normalization layer for feature scaling and stabilization, and then is input into a weighted multi-head self-attention layer to calculate attention scores to obtain a first feature image. The first feature image and the new feature image of H×W×C are added together to obtain a second feature image. Then, the second feature image is processed through a layer normalization layer and a multi-layer perceptron layer to obtain a third feature image, and the third feature map and the first feature image are added together to obtain a fourth feature map. The fourth feature map passes through a layer normalization layer and a weighted multi-head self-attention layer for shifted window attention calculation to obtain a fifth feature image. The fifth feature image and the fourth feature map are added together to obtain a sixth feature map, and the sixth feature map is input into a layer normalization layer and a multi-layer perceptron layer to obtain a seventh feature image. The seventh feature image and the sixth feature image are added together to obtain an eighth feature image. Finally, the eighth feature image passes through a layer normalization layer and a window merging layer to reorganize the window between different stages, realizing the reduction of resolution and the integration of information, and obtaining x k with the same dimension as x k+1 . x k+1 is input into a dilated dense channel attention module, which is composed of a convolutional layer, a dense residual connection, and a channel attention layer (channel attention mechanism). In the dense residual connection, each line of a different color represents directly connecting each layer to all other layers, enhancing feature propagation and encouraging feature reuse. Each layer receives the feature maps of all previous layers as input and passes its own feature map to all subsequent layers. After passing through the convolutional layer and the dense residual connection, it is input into the channel attention layer to dynamically adjust the weight relationship between channels, enhancing the network's learning ability for key features, and obtaining a feature image of C / 2 channels H k+1 .

[0113] The present invention provides a training method for a sparse angle CT reconstruction model based on deep learning, which is implemented through the following S301 to S307, and the following is a specific description.

[0114] S301. Input sparse sampled projection data samples into a deep unfolding and residual optimization network to obtain initial image samples; and perform image reconstruction based on the initial image samples, projection data samples, and filtered back-projection algorithm to obtain first residual image samples.

[0115] S302. Input the first residual image sample into the local residual optimization module in the deep expansion and residual optimization network to be trained to obtain a second residual image sample and a first channel feature image sample with half the number of channels; and add the initial image sample and the second residual image sample to obtain a third residual image sample after the first iteration; the local residual optimization module consists of a wavelet U-Net module and a dense spatial attention module.

[0116] S303 : Perform image reconstruction based on the third residual image sample, the projection data sample, and a filtered back-projection algorithm to obtain a fourth residual image sample.

[0117] S304. Input the fourth residual image sample and the first channel feature image sample into the non-local residual optimization module to obtain a fifth residual image and a second channel feature image sample with half the number of channels; and add the second residual image sample and the fifth residual image sample to obtain a sixth residual image sample after the second iteration; the non-local residual optimization module is composed of a shifted window transformer module and a void dense channel attention module.

[0118] S305. Perform image reconstruction again based on the sixth residual image sample, the projection data sample, and the filtered back projection algorithm to obtain a seventh residual image sample; and input the seventh residual image sample and the second channel feature image sample into the local residual optimization module for a third iteration until the number of iterations is reached to obtain a reconstructed image sample.

[0119] In the embodiment of the present invention, steps S301 to S305 are similar to steps S101 to S105 and are not described in detail here.

[0120] S305 , training the depth expansion and residual optimization network to be trained based on the reconstructed image samples, the label image and the reference image until a trained depth expansion and residual optimization network is obtained.

[0121] In the embodiments of the present invention, the reference image refers to a real image used as a benchmark when evaluating model performance. It can be an original high-quality image or a standard image that has been manually annotated or processed.

[0122] The labeled image is an image generated by annotating or segmenting the original image. Each pixel value in the labeled image represents the category or region (such as foreground, background, specific object, etc.) to which it belongs. The deep expansion and residual optimization network to be trained is trained based on the reconstructed image, the labeled image, and the reference image until the trained deep expansion and residual optimization network is obtained.

[0123] In some embodiments of the present invention, S306 may be implemented through S3061 to S3062, which is described in the following steps.

[0124] S3061. Obtain the mean square error based on the reconstructed image sample and the labeled image, and obtain the structural similarity based on the reconstructed image and the reference image.

[0125] In some embodiments of the present invention, the MSE (mean square error) between the reconstructed image sample and the labeled image is used to measure the error between two image pixel points. The structural similarity (Structural Similarity Index Measure, SSIM) between the reconstructed image sample and the reference image measures the difference between images from the overall structural features of the images, comprehensively comparing the brightness, contrast, and structural similarity of the two images. The mean square error is calculated by the following formula (12):

[0126]

[0127] In the above formula (12), x iout represents the reconstructed image sample, x ilabel represents the labeled image, N represents the number of images, L MSE represents the mean square error, and i represents the pixel points of the image.

[0128] The structural similarity is calculated by the following formula (13):

[0129]

[0130] In the above formula (13), and respectively represent the average gray levels of the output result x iout and the real image x ilabel , and respectively represent the standard deviations of the gray levels of the output result x iout and the real image x ilabel , c1 and c2 are respectively set to 0.0004 and 0.0036, and L SSIM is the structural similarity.

[0131] S3062. Sum the mean square error and the structural similarity to obtain the target loss, and use the target loss to adjust the parameters of the depth unfolding and residual optimization network to be trained until the training conditions are met, obtaining the trained depth unfolding and residual optimization network.

[0132] In some embodiments of the present invention, the target loss is obtained by the following formula (14):

[0133]

[0134] In the above formula (14), γ is the balance parameter, γ is set to 0.5, and L Com is the target loss.

[0135] To prove the effectiveness of the method proposed by the present invention, a series of experiments were carried out:

[0136] 1. Experimental setup

[0137] To evaluate the effectiveness of a sparse-angle CT reconstruction method based on deep learning proposed by the present invention, the present invention used the public dataset of "2016 NIH-AAPM-Mayo Clinic Low-Dose CT Grand Challenge" of Mayo Clinic as the simulation dataset. The image dataset of the present invention contains 5936 full-dose 512×512 CT images with a thickness of 1 mm from 10 patients. The forward projection was performed on it using a simulated fan beam to obtain the corresponding projection data. The original size of the images in the dataset was 512×512, but it was adjusted to 256×256. The forward projection algorithm uses the distance-driven method, and the simulated rotation range of the detector is 360° in total. The projection simulation parameters are the same as those in Chapter 3, as shown in Table 3-1 specifically. To study the reconstruction ability of the method proposed by the present invention under different sparse degrees, the present invention constructed three different sparse-angle cases, namely 128 angles, 64 angles, and 32 angles, which were uniformly sampled within 0° to 360°. 5322 CT slice data of 8 randomly selected patients were used as the training set, and 460 CT slice data of the remaining 2 patients were used as the test set.

[0138] The experiment of the present invention relies on the hardware platform of NVIDIA RTX 3090 24GB GPU and Intel i7-9700k 3.6GHz CPU. The software is executed in Python using the PyTorch framework. The Adam optimizer is used to optimize the network, and the initial learning rate is 1×e -4 , which gradually decreases to 0. The two exponential decay factors are set to 0.9 and 0.999 respectively. During network training, the number N t of iterative blocks = 10, training for 100 epochs, the batch size is 1, and the feature map has C = 96 channels. The parameter settings of the shifted window transformer module follow. In the wavelet U-Net module, the convolution kernel size is 3×3, and for both iterative interaction modules, the convolution kernel size is 3×3, and the dilation rates of the three dilated convolutions are 1, 2, and 3 respectively.

[0139] 2. Algorithm comparison

[0140] In this invention, several representative methods under three angle settings (i.e., 128, 64, 32) are selected for comparison, including the method based on Filtered back projection convolutionalneural network (FBPConvNet) (Method 2), the method of Manifold andgraph integrative convolutional network (MAGIC) (Method 3), the method of Combining integrated attention with iterative optimizationlearning (CAIR) (Method 5), and the method of Meta inversion network (MetaInv-Net) (Method 4). The method based on filtered back projection (Method 1) is one of the most widely used analytical reconstruction algorithms, which has the advantages of fast reconstruction speed and simple algorithm. Method 2 is a deep learning-based image post-processing algorithm that can use a U-Net denoiser to solve the noise and artifact problems in FBP reconstructed images. Method 3 introduces graph convolution and uses an iterative unfolding scheme to achieve parallel processing in the image space and the manifold space, more effectively improving the reconstruction effect. Method 5 realizes good reconstruction of sparse-angle CT images by introducing an integrated attention module as deep prior information and combining iterative optimization learning. Method 4 unfolds the iterative algorithm into the network and constructs a meta-learning model by introducing a hypernetwork to learn the initialization of the conjugate gradient module, more effectively improving the reconstruction effect.

[0141] To ensure the accuracy of the reconstruction effect of the comparative algorithms involving deep learning, this invention adopts the same training strategy as the original reference for model training. At the same time, to verify the good generality and effectiveness of the proposed algorithm, this invention evaluates the network model on different test data sets with sparse angles of 128, 64, and 32 respectively, and uses the Peak Signal-to-Noise Ratio (PSNR), StructuralSimilarity Index Measure (SSIM), and RootMean Square Error (RMSE) as evaluation indicators to quantify the reconstruction quality of each algorithm.

[0142] 3. Visual effect evaluation results

[0143] The following experimental comparison diagrams list the CT images generated by different reconstruction methods. The first row shows thoracic images with a display window of [-1150, 350]; the first and second rows show abdominal images with a display window of [-150, 250] HU. From left to right are the ground truth image GT, Method 1, Method 2, Method 3, Method 5, Method 4, and the reconstruction result of the present application.

[0144] As Figure 5 shown, Figure 5 are the reconstructed images of different reconstruction algorithms provided by the embodiments of the present invention at 128 sparse angles. Due to insufficient projection data, slight streak artifacts appear in the traditional Method 1. Although this method can identify most of the structural information, it performs poorly in restoring tissue features and cannot meet the requirements of precise medical evaluation. Although other comparison methods can effectively remove most of the artifacts, it is often difficult to achieve an ideal balance between artifact removal and detail retention. Although Method 2 can effectively remove most of the strip artifacts, it is still affected by the remaining artifacts, resulting in the loss of most details and affecting the visual perception of the image. Methods such as Method 3, Method 5, and Method 4 make up for some of the missing features of Method 2, but they are still limited by sparse projections, and their reconstruction results appear too smooth compared to the reference image, resulting in the loss of fine features such as tiny blood vessel structures and edge details. In contrast, the method proposed by the present invention has achieved significant improvements in both artifact suppression and feature retention, and the reconstruction result is closest to the reference image.

[0145] As Figure 6 shown, Figure 6 is the enlarged view of the region of interest provided by the embodiments of the present invention Figure 5 In the key region marked by the red arrow in the ROI region (a specific part or region in the image that the user is particularly concerned about), the method of the present invention not only successfully removes the noise, but also restores the fine details of the lung vascular structure with high fidelity, and makes the edges of the internal lesions of the liver clearer and sharper. To further intuitively demonstrate the advantages of the method of the present invention, a one-dimensional intensity profile diagram of Figure 7 is drawn. Figure 7 is the cross-sectional view (128 angles) of the experimental results of a different method provided by the embodiments of the present invention along the red line (located at Figure 5 ). By observing the region marked by the blue arrow, it can be found that the line intensity profile obtained by the method of the present invention is closest to the reference image. This high degree of coincidence not only intuitively proves that the method of the present study has significant advantages in maintaining the characteristics of the original data, but also indicates that it has obvious improvements in detail processing and accuracy compared to other methods, further verifying the effectiveness and reliability of the method. Figure 8 is a schematic diagram of the effect of the reconstructed images of a different reconstruction algorithm provided by the embodiments of the present invention at 64 sparse angles. Figure 9Provided by the embodiments of the present invention Figure 7 The enlarged view of the region of interest in Figure 8 and Figure 9 The comparison shows the effects of different reconstruction algorithms based on 64-angle sparse projection data and the enlarged display of their ROIs. As the amount of projection data decreases, various reconstruction methods show varying degrees of influence on visual quality. It is worth noting that traditional method 1 exhibits significant striping artifacts during the reconstruction process, resulting in a serious lack of diagnostic information for soft tissues except bones. Although deep learning-based method 2 significantly reduces sparse-angle artifacts, it is difficult to maintain the continuity of tissue boundary morphological features and details due to the incompleteness of the projection data. Methods 3 and 4 can significantly suppress artifacts and improve tissue edge details, but their reconstruction results have an over-smoothing phenomenon in key anatomical regions, thus affecting clinical observation and evaluation. Method 5 has a more obvious denoising effect, but due to the sparsity of the original data, it still faces the challenge of losing high-frequency feature information.

[0146] As Figure 9 shown, compared with other methods, in the regions marked by the red and blue arrows, the method proposed by the present invention exhibits optimal performance in terms of image details and clear edge structures, and can accurately identify the subtle features required in clinical evaluation. In addition, in order to analyze the pixel value changes of the image in more depth. Similarly, a horizontal cross-section was made at the red line in Figure 8 to further explore the pixel value changes in this region. The results are as Figure 10 shown, Figure 10 is the cross-section diagram (64 angles) of the experimental results of different methods along the red cross-section line (located in Figure 8 ). Similarly, the method of the present invention also shows the best performance in pixel value fitting. Figure 10 is the CT image reconstructed from the projection data sampled at 32 angles by various algorithms. In the case of extremely sparse projection angles, there is almost no valuable information in the image domain, which poses a huge challenge to the reconstruction of fine structural details. The reconstruction result of method 1 contains a large amount of artifacts and noise, making it difficult to identify the details in the image. Although method 2 has made some progress in suppressing some artifacts, the reconstructed image still appears blurred, especially in the abdominal and thoracic regions, where the details of tissue structures and blood vessels are insufficiently displayed. Although method 3 performs well in reducing artifacts, it does not achieve an ideal effect in detail reconstruction. Methods 5 and 4 algorithms do better in eliminating stripe artifacts and provide a better visual effect than the previous methods. But from Figure 11 ( Figure 11As can be seen from b(5) and b(6) in the schematic diagram of the reconstruction images of different reconstruction methods provided by the embodiments of the present invention at 32 sparse angles, there are still problems of blurred edges in some regions of the images reconstructed by these two methods, and the detailed information is also too smooth to provide accurate diagnostic information. Compared with other reconstruction methods, the method of the present invention can effectively suppress artifacts when the sampling angle is greatly reduced, and reconstruct clearer edge tissue structures between different tissues, showing the best results in both artifact suppression and detail preservation.

[0147] Observation Figure 12 ( Figure 12 provided by the embodiments of the present invention Figure 10 the ROI in the enlarged view of the region of interest in (), as shown in a(1) in the figure, at the blood vessel pointed by the red arrow, the reconstruction result of the method of the present invention is closest to the label image. Figure 12 The place pointed by the red arrow in c(1) also shows that the method proposed by the present invention has better image tissue restoration ability compared with the comparative method. Observation Figure 12 It can be seen that the method of the present invention has unique advantages in terms of the detailed blood vessel structure of the reconstructed image, the visual effect of the image tissue, etc. Especially in repairing more subtle structures and tissues of bones, the generated image is closer to the real situation of the reference image.

[0148] To further evaluate the performance of the method proposed by the present invention, a horizontal section of the results with 32 projection angles was made, as shown by the red solid line in Figure 11 The contour can reflect the pixel fitting ability of the model, and the visualization result is as shown in Figure 13 ( Figure 13 the sectional view (32 angles) along the red section line (located in Figure 11 ) of the experimental results of different methods provided by the embodiments of the present invention. Through Figure 13 the indication of the blue arrow in, it can be observed that the method of the present invention has better pixel value fitting results, which strongly proves that the method of the present invention has stronger structure preservation and stripe artifact reduction capabilities. This result is consistent with Figure 12 the visual evaluation conclusion described. Through the comparative analysis of different angles, it can be proved that the reconstruction algorithm proposed by the present invention has significant advantages in the task of reconstructing sparse projection data, especially in terms of the balance between artifact suppression and detail preservation, providing a higher quality solution for medical image reconstruction.

[0149] 3. Quantitative comparative analysis results

[0150] Table 1 shows the quantitative evaluation results of different comparison methods at 128, 64, and 32 angles, presented in the form of the average value of each index. Among them, the optimal numerical results are marked in bold. It can be seen from the table that the method proposed in the present invention is significantly better than other methods in all evaluation indexes. This result is highly consistent with the visual observation effect, further verifying the excellent performance of the method of the present invention in detail restoration and artifact removal. Figure 14 ( Figure 14 (Box plots of the reconstruction results of different methods at 128 angles (a), 64 angles (b), and 32 angles (c) provided by the embodiments of the present invention. The following application is Method 6 in the figure. The gold standard represents the highest standard or best practice and is a reference point for measuring other methods.)

[0151] Table 1 Quantitative indexes of different methods

[0152]

[0153]

[0154] 4. Conclusions and discussions

[0155] The present invention proposes to eliminate the reconstruction error by minimizing the residual image. By optimizing this difference part, the restoration accuracy in the region with a larger attenuation coefficient can be significantly improved. Based on this, the present invention proposes a sparse-angle CT reconstruction method based on deep learning.

[0156] First, a new optimization objective function is used to overcome the artifacts caused by sparse-angle measurements. At the same time, by introducing ROM as a regularization term into the optimization process, the deep features in the residual data are deeply extracted. In addition, block coordinate descent is used to implement a novel iterative framework. Second, during the iterative process, a local residual optimization module mainly composed of a wavelet U-Net module and a non-local residual optimization module mainly composed of a shifted window transformer module are alternately used to simultaneously capture local details and the interaction relationship between long-distance pixels. The shifted window transformer module can construct hierarchical feature maps and has a computational complexity linearly related to the image size to reduce the computational cost while protecting the reconstructed image features. Finally, in view of the different networks used in the local residual optimization module and the non-local residual optimization module, the present invention designs two different iterative interaction modules, namely, a dense spatial attention module and a dilated dense channel attention module, and embeds them correspondingly. They can transfer and connect the intermediate feature maps of each iterative block to the next iteration. By establishing an additional path, this module avoids the direct conversion from deep features to pixel-level features, thereby being able to more effectively extract deep features, further improving the reconstruction performance and visual effect. The experimental results show that the method of the present invention can effectively improve the quality of sparse-angle CT reconstructed images.

[0157] Those of ordinary skill in the art can realize that the units and method steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments of the present invention.

[0158] The above embodiments are only used to illustrate the embodiments of the present invention, rather than to limit the embodiments of the present invention. Those of ordinary skill in the relevant technical field can also make various changes and modifications without departing from the spirit and scope of the embodiments of the present invention. Therefore, all equivalent technical solutions also belong to the scope of the embodiments of the present invention, and the patent protection scope of the embodiments of the present invention should be defined by the claims.

Claims

1. A sparse angle CT reconstruction method based on deep learning, characterized in that, Including: Inputting the sparsely sampled projection data into a deep unfolding and residual optimization network to obtain an initial image; And performing image reconstruction based on the initial image, the projection data, and the filtered back-projection algorithm to obtain a first residual image; Inputting the first residual image into a local residual optimization module to obtain a second residual image and a first channel feature image with the number of channels halved; and adding the initial image and the second residual image to obtain a third residual image after the first iteration; the deep unfolding and residual optimization network includes a local residual optimization module and a non-local residual optimization module, and the local residual optimization module consists of a wavelet U-Net module and a dense spatial attention module; Performing image reconstruction based on the third residual image, the projection data, and the filtered back-projection algorithm to obtain a fourth residual image; Inputting the fourth residual image and the first channel feature image into a non-local residual optimization module to obtain a fifth residual image and a second channel feature image with the number of channels halved; and adding the second residual image and the fifth residual image to obtain a sixth residual image after the second iteration; the non-local residual optimization module consists of a shifted window transformer module and a dilated dense channel attention module; Performing image reconstruction again based on the sixth residual image, the projection data, and the filtered back-projection algorithm to obtain a seventh residual image; and inputting the seventh residual image and the second channel feature image into the local residual optimization module for the third iteration until the number of iterations is reached to obtain the target image.

2. The method according to claim 1, characterized in that The performing image reconstruction based on the initial image, the projection data, and the filtered back-projection algorithm to obtain a first residual image includes: Projecting the initial image orthogonally to obtain target projection data, and subtracting the target projection data from the projection data to obtain first residual projection data; Performing image reconstruction on the first residual projection data based on the filtered back-projection algorithm to obtain a first residual image.

3. The method according to claim 1, characterized in that, The wavelet U-Net module includes a first dense residual layer, a first spatial attention layer, a wavelet transform layer, an inverse wavelet transform layer, a convolutional layer, a batch normalization layer, and an activation layer; the dense spatial attention module includes a second dense residual layer and a second spatial attention layer; The inputting the first residual image into a local residual optimization module to obtain a second residual image and a first channel feature image with the number of channels halved includes: Performing dimensional processing on the first residual image using the first dense residual layer, the first spatial attention layer, the wavelet transform layer, the inverse wavelet transform layer, the convolutional layer, the batch normalization layer, and the activation layer to obtain the second residual image; Sequentially inputting the second residual image into the second dense residual layer and the second spatial attention layer to obtain the first channel feature image, and the output of the second dense residual layer is the input of the second spatial attention layer.

4. The method according to claim 1, wherein The shifted window transformer module consists of a window-based multi-head attention layer and a multi-layer perceptron layer, or consists of a multi-head attention layer with shifted windows and the multi-layer perceptron layer; the dilated dense channel attention module includes a dilated convolutional layer, a dense connection layer, and a channel attention layer; The step of inputting the fourth residual image and the first channel feature image into the non-local residual optimization module to obtain a fifth residual image and a second channel feature image with the number of channels halved includes: Halving the number of channels of the fourth residual image to obtain a channel feature image after halving, and fusing the channel feature image after halving and the first channel feature image to obtain a second fused image; Obtaining the fifth residual image based on the second fused image, a layer normalization layer, a window-based multi-head self-attention layer, a multi-layer perceptron layer, a multi-head self-attention layer with shifted windows, and a window merging layer; and obtaining the second channel feature image based on the fifth residual image, the dilated convolutional layer, the dense connection layer, and the channel attention layer.

5. The method according to claim 4, characterized in that, The step of obtaining the fifth residual image based on the second fused image, a layer normalization layer, a window-based multi-head self-attention layer, a multi-layer perceptron layer, a multi-head self-attention layer with shifted windows, and a window merging layer includes: Sequentially inputting the second fused image into the layer normalization layer and the window-based multi-head self-attention layer to obtain a first feature image processed by the attention mechanism; the output of the layer normalization layer is the input of the window-based multi-head self-attention layer; Adding the first feature image and the second fused image to obtain a second feature image; and sequentially inputting the second feature image into the layer normalization layer and the multi-layer perceptron layer to obtain a third feature image; the output of the layer normalization layer is the input of the multi-layer perceptron layer; Adding the third feature map and the first feature image to obtain a fourth feature map, and sequentially inputting the fourth feature image into the layer normalization layer and the multi-head self-attention layer with shifted windows to obtain a fifth feature image processed by the attention mechanism; the output of the layer normalization layer is the input of the multi-head self-attention layer with shifted windows; Adding the five feature image and the fourth feature map to obtain a sixth feature map, and sequentially inputting the sixth feature map into the layer normalization layer and the multi-layer perceptron layer to obtain a seventh feature map; the output of the layer normalization layer is the input of the multi-layer perceptron layer; Adding the seventh feature image and the sixth feature image to obtain an eighth feature image, and sequentially inputting the eighth feature image into the layer normalization layer and the window merging layer to obtain the fifth residual image; the output of the layer normalization layer is the input of the window merging layer.

6. A training method for a sparse angle CT reconstruction model based on deep learning, characterized in that, including: Inputting the sparsely sampled projection data samples into the deep unfolding and residual optimization network to obtain initial image samples; And performing image reconstruction based on the initial image samples, the projection data samples, and the filtered back-projection algorithm to obtain first residual image samples; Inputting the first residual image samples into the local residual optimization module to obtain second residual image samples and first channel feature image samples with the number of channels halved; Add the initial image sample and the second residual image sample to obtain a third residual image sample after the first iteration; the deep unfolding and residual optimization network includes a local residual optimization module and a non-local residual optimization module, and the local residual optimization module consists of a wavelet U-Net module and a dense spatial attention module; Based on the third residual image sample, the projection data sample, and the filtered back-projection algorithm, perform image reconstruction to obtain a fourth residual image sample; Input the fourth residual image sample and the first-channel feature image sample into the non-local residual optimization module to obtain a fifth residual image and a second-channel feature image sample with the number of channels halved; Add the second residual image sample and the fifth residual image sample to obtain a sixth residual image sample after the second iteration; the non-local residual optimization module consists of a shifted window transformer module and a dilated dense channel attention module; Perform image reconstruction again based on the sixth residual image sample, the projection data sample, and the filtered back-projection algorithm to obtain a seventh residual image sample; and input the seventh residual image sample and the second-channel feature image sample into the local residual optimization module for the third iteration until the number of iterations is reached to obtain a reconstructed image sample; Based on the reconstructed image sample, the label image, and the reference image, train the deep unfolding and residual optimization network to be trained until the trained deep unfolding and residual optimization network is obtained.

7. The method according to claim 6, characterized in that, The training of the deep unfolding and residual optimization network to be trained based on the reconstructed image sample, the label image, and the reference image until the trained deep unfolding and residual optimization network is obtained includes: Obtain the mean square error based on the reconstructed image sample and the label image, and obtain the structural similarity based on the reconstructed image sample and the reference image; Sum the mean square error and the structural similarity to obtain the target loss, and use the target loss to adjust the parameters of the deep unfolding and residual optimization network to be trained until the training conditions are met to obtain the trained deep unfolding and residual optimization network.

Images