Method and apparatus for 3D CT image reconstruction based on single projection data

Abstract

The present invention discloses a method and device for 3D CT image reconstruction based on single-frame projection data. The 3D CT image reconstruction method comprises the following steps: obtaining a CT image to be reconstructed; generating corresponding single-frame KV projection data from the acquired CT image to be reconstructed; extracting features from the acquired single-frame KV projection data using an encoder in a convolutional neural network with an attention mechanism and a multi-scale fusion strategy; performing a three-dimensional transformation on the extracted features; reconstructing the features using the encoder in the convolutional neural network to obtain a 3D reconstructed CT image; calculating the grayscale value difference between the obtained 3D reconstructed CT image and the CT image to be reconstructed; determining whether the grayscale value difference between the two images reaches a global optimal solution; if so, the reconstruction is successful and a final 3D CT image is output; otherwise, the above steps are repeated until the grayscale value difference between the two images reaches a global optimal solution. The present invention can reconstruct high-quality CT images using only single-frame KV projection data, shortens reconstruction time, and reduces radiation dose to the patient.

Description

Technical Field

[0001] This invention belongs to the field of CT image reconstruction technology, specifically relating to a method and apparatus for three-dimensional CT image reconstruction based on single projection data. Background Technology

[0002] CT imaging technology is a high-performance, non-invasive diagnostic technique that plays a crucial role in assisting clinical diagnosis. CT scans a specific area of ​​the body using X-ray beams. A flat-panel digital detector receives the X-rays passing through this area, converts them into visible light, then converts them into electrical signals via photoelectric conversion, and finally into digital signals via an analog-to-digital converter for computer processing. However, with the increasing frequency of CT imaging use, traditional CT scans are time-consuming, and the high radiation doses during scans pose a growing concern for public health. Therefore, shortening scan time and reducing radiation doses have become urgent needs for CT development.

[0003] Currently, the mainstream method is to reconstruct CT images using sparse projection. Chinese patent application CN201610987138.4 proposes a fast CT reconstruction method using multi-scale sparse projection data. This method employs convex optimization iteration and regularization constraints to reconstruct downsampled projection data. Chinese patent CN201711420601.8 proposes a method and apparatus for reconstructing CT sparse projection images under sampling angle constraints. This method obtains the pseudo-inverse matrix of the projection equation solution from the projection data, then generates a random solution set, replaces the solutions in this set accordingly, and obtains the optimal solution through continuous iteration, thus obtaining the final reconstruction result.

[0004] The above methods offer some improvements in reconstruction speed and radiation dose reduction compared to traditional reconstruction algorithms. However, ultra-sparse projection reconstruction of CT images has not yet been achieved, meaning that a high-quality CT image can be reconstructed using only a single projection data. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention proposes a method and apparatus for three-dimensional CT image reconstruction based on single-image projection data.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] On one hand, this invention discloses a method for reconstructing three-dimensional CT images based on single-image projection data, comprising the following steps:

[0008] Step 1: Obtain the CT scan to be reconstructed;

[0009] Step 2: Generate the corresponding single KV projection data from the CT scan to be reconstructed obtained in Step 1;

[0010] Step 3: Use an encoder with attention mechanism and multi-size fusion strategy in a convolutional neural network to extract features from the single KV projection data obtained in Step 2;

[0011] Step 4: Perform 3D transformation on the features extracted in Step 3;

[0012] Step 5: Use the encoder in the convolutional neural network to reconstruct features and obtain the 3D reconstructed CT.

[0013] Step Six: Calculate the difference in grayscale values ​​between the 3D reconstructed CT obtained in Step Five and the CT to be reconstructed;

[0014] Determine whether the difference in grayscale values ​​between the two reaches the global optimal solution;

[0015] If the target is reached, the reconstruction is successful, and the final 3D CT image is output.

[0016] Otherwise, repeat steps three through six until the difference in grayscale values ​​between the two reaches the global optimal solution.

[0017] Based on the above technical solution, the following improvements can be made:

[0018] As a preferred approach, the attention mechanism specifically includes the following steps:

[0019] A1: Pass the input feature map through a global max pooling layer and a global average pooling layer respectively;

[0020] A2: The output feature maps of both are transmitted to a multi-layer vector machine, which then performs further feature extraction.

[0021] A3: The generated intermediate feature map is processed to obtain the channel attention map;

[0022] A4: Multiply the channel attention map with the input feature map to obtain a secondary intermediate feature map;

[0023] A5: Pass the secondary intermediate feature maps through a global max pooling layer and a global average pooling layer, respectively;

[0024] A6: Perform feature concatenation on the output feature maps of the two;

[0025] A7: The spliced ​​feature map is processed to obtain the spatial attention map;

[0026] A8: Multiply the spatial attention map with the secondary intermediate feature map to obtain the final output feature map.

[0027] Using the preferred scheme described above, the attention mechanism includes channel attention and spatial attention. Attention maps are inferred sequentially in the channel and spatial dimensions, respectively. Then, the attention map is multiplied by the input feature map for adaptive optimization to obtain the region of interest.

[0028] As a preferred approach, the fused feature map obtained through the multi-scale fusion strategy is specifically as follows:

[0029]

[0030] Among them, G k F represents the fused feature map output from the k-th layer of the encoder. k This represents the input feature map of the k-th layer of the encoder. Represents upsampling 2 i-k C represents the concatenation operation, D sconv @2 i-k This indicates an expansion rate of 2. i-k Hollow convolution.

[0031] By adopting the above-mentioned preferred scheme, the multi-scale fusion strategy effectively combines the deep and shallow features of the encoder, captures more detailed features and passes them to the decoder, thereby reconstructing a high-quality CT.

[0032] As a preferred approach, in step five, the calculation of grayscale value difference specifically involves the calculation of the mean square error loss. The formula for calculating the mean square error loss is as follows:

[0033]

[0034] Where m is the number of samples, y i For 3D reconstructed CT, y′ i For CT scans to be reconstructed.

[0035] Using the above-mentioned preferred scheme, the gray value difference calculation involves calculating the mean squared error loss between the 3D reconstructed CT generated by the network and the CT to be reconstructed, backpropagating this loss in the network to obtain the gradient of the network parameters, and updating the network parameters through the gradient descent optimization method to minimize the mean squared error loss, i.e., the global optimal solution.

[0036] As a preferred approach, the convolutional neural network is trained before feature extraction. The training steps are as follows:

[0037] B1: Preprocess the CT and KV projection data to be reconstructed;

[0038] B2: The CT scan to be reconstructed and its corresponding KV projection data are combined into sample pairs and divided into training set, test set and validation set according to a certain ratio;

[0039] B3: Input the training set into the convolutional neural network for training and 3D conversion to obtain a 3D reconstructed CT scan;

[0040] B4: Calculate the mean squared error loss by comparing the 3D reconstructed CT with the CT to be reconstructed. Backpropagate the obtained loss in a convolutional neural network to obtain the gradient of the network parameters. Update the network parameters using the gradient descent optimization method to minimize the mean squared error loss.

[0041] B5: After each training iteration, use the validation set to verify whether the mean squared error loss on the validation set is minimized.

[0042] If so, then save the weight;

[0043] If not, continue training. If the mean squared error loss on the validation set still does not decrease after training reaches the threshold number of iterations, then adjust the learning rate.

[0044] B6: Repeat B3-B5 until the training results of the convolutional neural network reach the expected level and the mean squared error loss on the training set is minimized.

[0045] Using the above-mentioned preferred scheme, the trained convolutional neural network is obtained.

[0046] On the other hand, the present invention also discloses a three-dimensional CT image reconstruction device based on single projection data, comprising:

[0047] The projection data acquisition module is used to acquire the CT to be reconstructed and generate corresponding single KV projection data from the acquired CT to be reconstructed.

[0048] The model building module is used to construct a CT reconstruction model based on a convolutional neural network. The CT reconstruction model includes: an encoder with an attention mechanism and a multi-size fusion strategy, a 3D transformation unit, and a decoder for feature reconstruction.

[0049] The 3D reconstruction module is used to input the CT to be reconstructed into the CT reconstruction model, obtain the 3D reconstructed CT, and calculate the gray value difference between the 3D reconstructed CT and the CT to be reconstructed, and determine whether the gray value difference between the two reaches the global optimal solution.

[0050] If the target is reached, the reconstruction is successful, and the final 3D CT image is output.

[0051] Otherwise, the gradient descent method is repeatedly used to find the optimal solution until the difference in gray values ​​between the two reaches the global optimum.

[0052] As a preferred approach, the attention mechanism includes: a channel attention module and a spatial attention module;

[0053] The channel attention module is used to pass the input feature map through a global max pooling layer and a global average pooling layer, respectively, and then pass the output feature maps of the two layers to a multi-layer vector machine for further feature extraction. The resulting intermediate feature map is then processed to obtain the channel attention map.

[0054] The spatial attention module is used to multiply the channel attention map with the input feature map to obtain a secondary intermediate feature map. The secondary intermediate feature map is then passed through a global max pooling layer and a global average pooling layer, respectively. The output feature maps of the two layers are then concatenated. The concatenated feature map is then processed to obtain the spatial attention map.

[0055] Using the preferred scheme described above, the attention mechanism includes channel attention and spatial attention. Attention maps are inferred sequentially in the channel and spatial dimensions, respectively. Then, the attention map is multiplied by the input feature map for adaptive optimization to obtain the region of interest.

[0056] As a preferred approach, the fused feature map obtained through the multi-scale fusion strategy is specifically as follows:

[0057]

[0058] Among them, G k F represents the fused feature map output from the k-th layer of the encoder. k This represents the input feature map of the k-th layer of the encoder. Represents upsampling 2 i-k C represents the concatenation operation, D sconv @2 i-k This indicates an expansion rate of 2. i-k Hollow convolution.

[0059] By adopting the above-mentioned preferred scheme, the multi-scale fusion strategy effectively combines the deep and shallow features of the encoder, captures more detailed features and passes them to the decoder, thereby reconstructing a high-quality CT.

[0060] As a preferred approach, in the 3D reconstruction module, the calculation of grayscale value differences specifically involves the calculation of the mean squared error loss. The formula for calculating the mean squared error loss is as follows:

[0061]

[0062] Where m is the number of samples, y i For 3D reconstructed CT, y′ i For CT scans to be reconstructed.

[0063] Using the above-mentioned preferred scheme, the gray value difference calculation involves calculating the mean squared error loss between the 3D reconstructed CT generated by the network and the CT to be reconstructed, backpropagating this loss in the network to obtain the gradient of the network parameters, and updating the network parameters through the gradient descent optimization method to minimize the mean squared error loss, i.e., the global optimal solution.

[0064] As a preferred approach, training a convolutional neural network specifically includes the following steps:

[0065] B1: Preprocess the CT and KV projection data to be reconstructed;

[0066] B2: The CT scan to be reconstructed and its corresponding KV projection data are combined into sample pairs and divided into training set, test set and validation set according to a certain ratio;

[0067] B3: Input the training set into the convolutional neural network for training and 3D conversion to obtain a 3D reconstructed CT scan;

[0068] B4: Calculate the mean squared error loss by comparing the 3D reconstructed CT with the CT to be reconstructed. Backpropagate the obtained loss in a convolutional neural network to obtain the gradient of the network parameters. Update the network parameters using the gradient descent optimization method to minimize the mean squared error loss.

[0069] B5: After each training iteration, use the validation set to verify whether the mean squared error loss on the validation set is minimized.

[0070] If so, then save the weight;

[0071] If not, continue training. If the mean squared error loss on the validation set still does not decrease after training reaches the threshold number of iterations, then adjust the learning rate.

[0072] B6: Repeat B3-B5 until the training results of the convolutional neural network reach the expected level and the mean squared error loss on the training set is minimized.

[0073] Using the above-mentioned preferred scheme, the trained convolutional neural network is obtained.

[0074] The present invention provides a method and apparatus for reconstructing three-dimensional CT images based on single-image projection data, which has the following beneficial effects:

[0075] First, deep learning methods are used for training, which is faster and more convenient than traditional reconstruction algorithms, and requires no human intervention, making it an end-to-end training process.

[0076] Second, high-quality CT images can be reconstructed using only a single KV projection data frame. Shifting from sparse projection to single-frame projection data reconstruction of 3D CT tomography results in shorter reconstruction times and lower radiation doses to patients while maintaining high image quality.

[0077] Third, an attention mechanism is added to make the network pay more attention to the regions of interest during training, resulting in more accurate feature extraction.

[0078] Fourth, a multi-scale fusion strategy is added to better combine the shallow and deep features of the network encoder, obtain fine-grained fused features, and pass the fused features to the decoder to generate the final reconstruction result. Attached Figure Description

[0079] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0080] Figure 1 A flowchart of a three-dimensional CT image reconstruction method provided in an embodiment of the present invention.

[0081] Figure 2 A block diagram of a decoder provided in an embodiment of the present invention.

[0082] Figure 3 A flowchart illustrating the attention mechanism provided in an embodiment of the present invention.

[0083] Figure 4 A flowchart of channel attention provided for embodiments of the present invention.

[0084] Figure 5 A flowchart of spatial attention provided for embodiments of the present invention.

[0085] Figure 6 A flowchart of a multi-scale fusion strategy provided in an embodiment of the present invention.

[0086] Figure 7 This is a schematic diagram of the structure of the three-dimensional CT image reconstruction device provided in an embodiment of the present invention. Detailed Implementation

[0087] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0088] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0089] Using ordinal numbers such as “first,” “second,” “third,” etc. to describe ordinary objects merely indicates different instances of similar objects and is not intended to imply that the objects being described must have a given order in time, space, sequence, or any other way.

[0090] Furthermore, the expression "includes" is an "open-ended" expression, which only means that the corresponding component exists and should not be interpreted as excluding additional components.

[0091] To achieve the objectives of this invention, some embodiments of the three-dimensional CT image reconstruction method and apparatus based on single-image projection data, such as Figure 1 As shown, the three-dimensional CT image reconstruction method includes the following steps:

[0092] Step 1: Obtain the CT scan to be reconstructed;

[0093] Step 2: Generate the corresponding single KV projection data from the CT scan to be reconstructed obtained in Step 1;

[0094] Step 3: Use an encoder with attention mechanism and multi-size fusion strategy in a convolutional neural network to extract features from the single KV projection data obtained in Step 2;

[0095] Step 4: Perform 3D transformation on the features extracted in Step 3;

[0096] Step 5: Use the encoder in the convolutional neural network to reconstruct features and obtain the 3D reconstructed CT.

[0097] Step Six: Calculate the difference in grayscale values ​​between the 3D reconstructed CT obtained in Step Five and the CT to be reconstructed;

[0098] Determine whether the difference in grayscale values ​​between the two reaches the global optimal solution;

[0099] If the target is reached, the reconstruction is successful, and the final 3D CT image is output.

[0100] Otherwise, repeat steps three through six until the difference in grayscale values ​​between the two reaches the global optimal solution.

[0101] Convolutional neural networks (CNNs) are a deep learning method that includes an encoder and a decoder. The encoder learns the features of the projected data, and the decoder reconstructs the features learned by the encoder to generate the final result. To better extract image features, attention mechanisms and multi-scale fusion strategies are added to the intermediate layers of the encoder to obtain deeper features and pass them to the decoder.

[0102] 3D conversion maps the 2D features extracted by the encoder into corresponding 3D features, which are then passed to the decoder for feature reconstruction to obtain the reconstructed result.

[0103] Furthermore, in some embodiments, the 3D transformation step is performed synchronously after the multi-scale fusion strategy.

[0104] Furthermore, in some specific embodiments, the encoder consists of 10 layers of two-dimensional convolutions, with each pair of layers forming a two-dimensional residual convolution block. Each two-dimensional residual convolution block consists of a two-dimensional convolutional layer, a normalization layer, and a ReLU activation function. The number of channels in the convolutional kernels are 64, 128, 256, 512, and 1024, respectively. Downsampling is performed through pooling layers to extract local features of the image, while residual connections are introduced to alleviate gradient vanishing.

[0105] like Figure 2 As shown, the decoder consists of 5 layers of 3D transposed convolutions. Each 3D transposed convolution consists of a 3D transposed convolution layer, a normalization layer, and a ReLU activation function. The number of convolution kernel channels corresponds to the number of convolution kernel channels in each layer of the encoder. Simultaneously, upsampling is performed to reconstruct the features extracted in the encoding stage, generate the final result, and calculate the mean squared error loss with the CT to be reconstructed.

[0106] To further optimize the implementation effect of the present invention, in some other embodiments, the remaining technical features are the same, the difference being that, for example... Figure 3 As shown, the attention mechanism specifically includes the following steps:

[0107] A1: As Figure 4 As shown, the input feature map is passed through a global max pooling layer and a global average pooling layer, respectively.

[0108] A2: The output feature maps of both are transmitted to a multi-layer vector machine, which then performs further feature extraction.

[0109] A3: The generated intermediate feature map is passed through a 1×1 convolution, a normalization layer, and a Sigmoid activation function to obtain the channel attention map;

[0110] A4: Multiply the channel attention map with the input feature map to obtain a secondary intermediate feature map;

[0111] A5: As Figure 5As shown, the secondary intermediate feature map is used as a new input feature map and then passed through a global max pooling layer and a global average pooling layer, respectively.

[0112] A6: Perform feature concatenation on the output feature maps of the two;

[0113] A7: The spliced ​​feature map is processed through a 7×7 convolution, a normalization layer, and a Sigmoid activation function to obtain a spatial attention map;

[0114] A8: Multiply the spatial attention map with the secondary intermediate feature map to obtain the final output feature map.

[0115] Using the preferred scheme described above, the attention mechanism includes channel attention and spatial attention. Attention maps are inferred sequentially in the channel and spatial dimensions, respectively. Then, the attention map is multiplied by the input feature map for adaptive optimization to obtain the region of interest.

[0116] Channel attention and spatial attention can be represented by the following formulas:

[0117] M c (F)=σ(MLP(AvgPool(F)+MLP(MaxPool(F)))

[0118] M s (F)=σ(f 7×7 ([AvgPool(F);MaxPool(F)]))

[0119] Among them, M * (·) represents the attention map of the output, σ(·) represents the sigmoid function, MLP represents a multi-level vector machine, AvgPool and MaxPool represent average and max pooling respectively, and f 7×7 (·) represents a 7×7 convolution, and F is the input feature map.

[0120] To further optimize the implementation effect of the present invention, in some other embodiments, the remaining feature technologies are the same, the difference being that the fused feature map obtained by the multi-scale fusion strategy is specifically as follows:

[0121]

[0122] Among them, G k F represents the fused feature map output from the k-th layer of the encoder. k This represents the input feature map of the k-th layer of the encoder. Represents upsampling 2 i-k C represents the concatenation operation, D sconv @2 i-k This indicates an expansion rate of 2. i-k Hollow convolution.

[0123] By adopting the above-mentioned preferred scheme, the multi-scale fusion strategy effectively combines the deep and shallow features of the encoder, captures more detailed features and passes them to the decoder, thereby reconstructing a high-quality CT.

[0124] like Figure 6 As shown, specifically, the third layer of the encoder section is used as an example to illustrate the implementation process of the multi-scale fusion strategy. The fusion strategies of other corresponding layers are similar:

[0125] 1) Obtain the feature maps of the third, fourth, and fifth layers of the encoder respectively, and use them as input feature maps;

[0126] 2) The input feature map is convolved with 3×3 to obtain the corresponding features of each layer, which are used as intermediate feature maps;

[0127] 3) The intermediate feature maps of the fourth and fifth layers are upsampled by 2 and 4 times respectively, while maintaining the same resolution as the intermediate feature map of the third layer;

[0128] 4) Concatenate the intermediate feature maps along the channel dimension;

[0129] 5) Input the stitched feature map into dilated convolutions with dilation rates of 1, 2 and 4 to obtain feature maps of different scales;

[0130] 6) Then, the feature maps at different scales are stitched together along the channel dimension;

[0131] 7) The spliced ​​feature map is convolved with 1×1 to obtain the final output feature map of the third layer of the encoder.

[0132] To further optimize the implementation effect of the present invention, in some other embodiments, the remaining features are the same, except that in step five, the calculation of grayscale value difference is specifically the calculation of mean square error loss, and the formula for calculating mean square error loss is:

[0133]

[0134] Where m is the number of samples, y i For 3D reconstructed CT, y′ i For CT scans to be reconstructed.

[0135] Using the above-mentioned preferred scheme, the gray value difference calculation involves calculating the mean squared error loss between the 3D reconstructed CT generated by the network and the CT to be reconstructed, backpropagating this loss in the network to obtain the gradient of the network parameters, and updating the network parameters through the gradient descent optimization method to minimize the mean squared error loss, i.e., the global optimal solution.

[0136] To further optimize the implementation effect of the present invention, in some other embodiments, the remaining feature technologies are the same, except that the convolutional neural network is trained before feature extraction using the convolutional neural network. The training steps are as follows:

[0137] B1: Preprocess the CT and KV projection data to be reconstructed. The preprocessing operation is to normalize and standardize the KV projection data to the [0,1] interval and adjust it to 128×128 pixels. The CT to be reconstructed is also standardized to the [0,1] interval and the voxel size is adjusted to 128×128×128.

[0138] B2: The CT scan to be reconstructed and its corresponding KV projection data are combined into a sample pair and divided into a training set, a test set and a validation set in a 6:2:2 ratio;

[0139] B3: Input the training set into the convolutional neural network for training and 3D conversion to obtain a 3D reconstructed CT scan;

[0140] B4: Calculate the mean squared error loss by comparing the 3D reconstructed CT with the CT to be reconstructed. Backpropagate the obtained loss in a convolutional neural network to obtain the gradient of the network parameters. Update the network parameters using the gradient descent optimization method to minimize the mean squared error loss.

[0141] B5: After each training iteration, use the validation set to verify whether the mean squared error loss on the validation set is minimized.

[0142] If so, then save the weight;

[0143] If not, continue training. If the mean squared error loss on the validation set still does not decrease after training reaches the threshold number of iterations, then adjust the learning rate.

[0144] B6: Repeat B3-B5 until the training results of the convolutional neural network reach the expected level and the mean squared error loss on the training set is minimized.

[0145] Using the above-mentioned preferred scheme, the trained convolutional neural network is obtained.

[0146] Specifically, the global optimal solution is determined and the weights are saved by the network's performance on the validation set. In other words, after each training epoch, the network is validated on the validation set. If the mean squared error loss on the validation set is minimized, the weights are saved; if it is not minimized, and the model continues to train for a threshold number of epochs (e.g., 10 epochs) without decreasing, the learning rate is adjusted to half of its original value until it reaches the minimum.

[0147] In some specific embodiments, the network is trained for a total of 100 epochs with an initial learning rate of 1e-5 and an optimizer of Adam.

[0148] In some embodiments, additional evaluation metrics can be used for quantitative analysis to assess network performance. Specifically, the evaluation metrics are root mean square error (RMSE), mean absolute error (MAE), peak signal-to-noise ratio (PSNR), and structural similarity measure (SSIM), with the first two metrics being as low as possible and the latter two metrics being as high as possible.

[0149] The formula is shown below:

[0150]

[0151]

[0152]

[0153]

[0154] Where m is the number of samples, y i For the reconstructed CT generated by the network, y′ i For the CT scan to be reconstructed, MaxValue represents the maximum value of the reconstructed CT scan generated by the network, u y , Let u represent the mean and variance of y, respectively. y′ , Let σ represent the mean and variance of y′, respectively. yy′ This is the covariance between y and y′, C1=(K1L) 2 And C2=(K2L) 2 These are constants used to maintain stability; typically, K1 = 0.01, K2 = 0.03, and L is the maximum value in the data.

[0155] On the other hand, embodiments of the present invention also disclose a three-dimensional CT image reconstruction device based on single projection data, such as... Figure 7 As shown, it includes:

[0156] The projection data acquisition module is used to acquire the CT to be reconstructed and generate corresponding single KV projection data from the acquired CT to be reconstructed.

[0157] The model building module is used to construct a CT reconstruction model based on a convolutional neural network. The CT reconstruction model includes: an encoder with an attention mechanism and a multi-size fusion strategy, a 3D transformation unit, and a decoder for feature reconstruction.

[0158] The 3D reconstruction module is used to input the CT to be reconstructed into the CT reconstruction model, obtain the 3D reconstructed CT, and calculate the gray value difference between the 3D reconstructed CT and the CT to be reconstructed, and determine whether the gray value difference between the two reaches the global optimal solution.

[0159] If the target is reached, the reconstruction is successful, and the final 3D CT image is output.

[0160] Otherwise, the gradient descent method is repeatedly used to find the optimal solution until the difference in gray values ​​between the two reaches the global optimum.

[0161] Convolutional neural networks (CNNs) are a deep learning method that includes an encoder and a decoder. The encoder learns the features of the projected data, and the decoder reconstructs the features learned by the encoder to generate the final result. To better extract image features, attention mechanisms and multi-scale fusion strategies are added to the intermediate layers of the encoder to obtain deeper features and pass them to the decoder.

[0162] 3D conversion maps the 2D features extracted by the encoder into corresponding 3D features, which are then passed to the decoder for feature reconstruction to obtain the reconstructed result.

[0163] Furthermore, in some embodiments, the 3D transformation step is performed synchronously after the multi-scale fusion strategy.

[0164] Furthermore, in some specific embodiments, the encoder consists of 10 layers of two-dimensional convolutions, with each pair of layers forming a two-dimensional residual convolution block. Each two-dimensional residual convolution block consists of a two-dimensional convolutional layer, a normalization layer, and a ReLU activation function. The number of channels in the convolutional kernels are 64, 128, 256, 512, and 1024, respectively. Downsampling is performed through pooling layers to extract local features of the image, while residual connections are introduced to alleviate gradient vanishing.

[0165] The decoder consists of 5 layers of 3D transposed convolutions. Each 3D transposed convolution consists of a 3D transposed convolution layer, a normalization layer, and a ReLU activation function. The number of convolution kernel channels corresponds to the number of convolution kernel channels in each layer of the encoder. Simultaneously, upsampling is performed to reconstruct the features extracted in the encoding stage, generate the final result, and calculate the mean squared error loss with the CT to be reconstructed.

[0166] To further optimize the implementation effect of the present invention, in some other embodiments, the remaining features are the same, except that the attention mechanism includes: a channel attention module and a spatial attention module;

[0167] The channel attention module is used to pass the input feature map through a global max pooling layer and a global average pooling layer, respectively, and then pass the output feature maps of the two layers to a multi-layer vector machine for further feature extraction. The resulting intermediate feature map is then passed through a 1×1 convolution, a normalization layer, and a sigmoid activation function to obtain the channel attention map.

[0168] The spatial attention module is used to multiply the channel attention map with the input feature map to obtain a secondary intermediate feature map. The secondary intermediate feature map is then passed through a global max pooling layer and a global average pooling layer, respectively. The output feature maps of the two layers are concatenated. The concatenated feature map is then passed through a 7×7 convolution, a normalization layer, and a sigmoid activation function to obtain the spatial attention map.

[0169] Using the preferred scheme described above, the attention mechanism includes channel attention and spatial attention. Attention maps are inferred sequentially in the channel and spatial dimensions, respectively. Then, the attention maps are multiplied by the input feature map for adaptive optimization to obtain the region of interest. The spatial attention map is multiplied by the secondary intermediate feature map to obtain the final output feature map.

[0170] Channel attention and spatial attention can be represented by the following formulas:

[0171] M c (F)=σ(MLP(AvgPool(F)+MLP(MaxPool(F)))

[0172] M s (F)=σ(f 7×7 ([AvgPool(F);MaxPool(F)]))

[0173] Among them, M * (·) represents the attention map of the output, σ(·) represents the sigmoid function, MLP represents a multi-level vector machine, AvgPool and MaxPool represent average and max pooling respectively, and f 7×7 (·) represents a 7×7 convolution, and F is the input feature map.

[0174] To further optimize the implementation effect of the present invention, in some other embodiments, the remaining feature technologies are the same, the difference being that the fused feature map obtained by the multi-scale fusion strategy is specifically as follows:

[0175]

[0176] Among them, G k F represents the fused feature map output from the k-th layer of the encoder. k This represents the input feature map of the k-th layer of the encoder. Represents upsampling 2 i-k C represents the concatenation operation, D sconv @2 i-k This indicates an expansion rate of 2. i-k Hollow convolution.

[0177] By adopting the above-mentioned preferred scheme, the multi-scale fusion strategy effectively combines the deep and shallow features of the encoder, captures more detailed features and passes them to the decoder, thereby reconstructing a high-quality CT.

[0178] Specifically, the implementation process of the multi-scale fusion strategy is illustrated using the third layer of the encoder as an example; the fusion strategies for other corresponding layers are similar.

[0179] 1) Obtain the feature maps of the third, fourth, and fifth layers of the encoder respectively, and use them as input feature maps;

[0180] 2) The input feature map is convolved with 3×3 to obtain the corresponding features of each layer, which are used as intermediate feature maps;

[0181] 3) The intermediate feature maps of the fourth and fifth layers are upsampled by 2 and 4 times respectively, while maintaining the same resolution as the intermediate feature map of the third layer;

[0182] 4) Concatenate the intermediate feature maps along the channel dimension;

[0183] 5) Input the stitched feature map into dilated convolutions with dilation rates of 1, 2 and 4 to obtain feature maps of different scales;

[0184] 6) Then, the feature maps at different scales are stitched together along the channel dimension;

[0185] 7) The spliced ​​feature map is convolved with 1×1 to obtain the final output feature map of the third layer of the encoder.

[0186] To further optimize the implementation effect of the present invention, in some other embodiments, the remaining features are the same, except that in the 3D reconstruction module, the grayscale value difference calculation is specifically the calculation of the mean squared error loss, and the formula for calculating the mean squared error loss is:

[0187]

[0188] Where m is the number of samples, y i For 3D reconstructed CT, y′ i For CT scans to be reconstructed.

[0189] Using the above-mentioned preferred scheme, the gray value difference calculation involves calculating the mean squared error loss between the 3D reconstructed CT generated by the network and the CT to be reconstructed, backpropagating this loss in the network to obtain the gradient of the network parameters, and updating the network parameters through the gradient descent optimization method to minimize the mean squared error loss, i.e., the global optimal solution.

[0190] To further optimize the implementation effect of the present invention, in some other embodiments, the remaining features and techniques are the same, the difference being that the training of the convolutional neural network specifically includes the following steps:

[0191] B1: Preprocess the CT and KV projection data to be reconstructed. The preprocessing operation is to normalize and standardize the KV projection data to the [0,1] interval and adjust it to 128×128 pixels. The CT to be reconstructed is also standardized to the [0,1] interval and the voxel size is adjusted to 128×128×128.

[0192] B2: The CT scan to be reconstructed and its corresponding KV projection data are combined into a sample pair and divided into a training set, a test set and a validation set in a 6:2:2 ratio;

[0193] B3: Input the training set into the convolutional neural network for training and 3D conversion to obtain a 3D reconstructed CT scan;

[0194] B4: Calculate the mean squared error loss by comparing the 3D reconstructed CT with the CT to be reconstructed. Backpropagate the obtained loss in a convolutional neural network to obtain the gradient of the network parameters. Update the network parameters using the gradient descent optimization method to minimize the mean squared error loss.

[0195] B5: After each training iteration, use the validation set to verify whether the mean squared error loss on the validation set is minimized.

[0196] If so, then save the weight;

[0197] If not, continue training. If the mean squared error loss on the validation set still does not decrease after training reaches the threshold number of iterations, then adjust the learning rate.

[0198] B6: Repeat B3-B5 until the training results of the convolutional neural network reach the expected level and the mean squared error loss on the training set is minimized.

[0199] The present invention provides a method and apparatus for reconstructing three-dimensional CT images based on single-image projection data, which has the following beneficial effects:

[0200] First, deep learning methods are used for training, which is faster and more convenient than traditional reconstruction algorithms, and requires no human intervention, making it an end-to-end training process.

[0201] Second, high-quality CT images can be reconstructed using only a single KV projection data frame. Shifting from sparse projection to single-frame projection data reconstruction of 3D CT tomography results in shorter reconstruction times and lower radiation doses to patients while maintaining high image quality.

[0202] Third, an attention mechanism is added to make the network pay more attention to the regions of interest during training, resulting in more accurate feature extraction.

[0203] Fourth, a multi-scale fusion strategy is added to better combine the shallow and deep features of the network encoder, obtain fine-grained fused features, and pass the fused features to the decoder to generate the final reconstruction result.

[0204] The above-mentioned implementation methods can be carried out in parallel or in a cross-cutting manner.

[0205] It should be understood that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0206] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

Claims

1. A method for reconstructing three-dimensional CT images based on single-image projection data, characterized in that, Includes the following steps: Step 1: Obtain the CT scan to be reconstructed; Step 2: Generate the corresponding single KV projection data from the CT scan to be reconstructed obtained in Step 1; Step 3: Use an encoder with attention mechanism and multi-size fusion strategy in a convolutional neural network to extract features from the single KV projection data obtained in Step 2; Step 4: Perform 3D transformation on the features extracted in Step 3; Step 5: Use the encoder in the convolutional neural network to reconstruct features and obtain the 3D reconstructed CT. Step Six: Calculate the difference in grayscale values ​​between the 3D reconstructed CT obtained in Step Five and the CT to be reconstructed; Determine whether the difference in grayscale values ​​between the two reaches the global optimal solution; If the target is reached, the reconstruction is successful, and the final 3D CT image is output. Otherwise, repeat steps three through six until the difference in grayscale values ​​between the two reaches the global optimal solution; The attention mechanism specifically includes the following steps: A1: Pass the input feature map through a global max pooling layer and a global average pooling layer respectively; A2: The output feature maps of both are transmitted to a multi-layer vector machine, which then performs further feature extraction. A3: The generated intermediate feature map is processed to obtain the channel attention map; A4: Multiply the channel attention map with the input feature map to obtain a secondary intermediate feature map; A5: Pass the secondary intermediate feature maps through a global max pooling layer and a global average pooling layer, respectively; A6: Perform feature concatenation on the output feature maps of the two; A7: The spliced ​​feature map is processed to obtain the spatial attention map; A8: Multiply the spatial attention map with the secondary intermediate feature map to obtain the final output feature map; the fused feature map obtained through the multi-scale fusion strategy is as follows: Among them, G k F represents the fused feature map output from the k-th layer of the encoder. k This represents the input feature map of the k-th layer of the encoder. Represents upsampling 2i-k2 i-k C represents the concatenation operation, D sconv @2 i-k This indicates an expansion rate of 2. i-k Hollow convolution.

2. The three-dimensional CT image reconstruction method according to claim 1, characterized in that, In step five, the calculation of grayscale value difference specifically involves the calculation of mean square error loss. The formula for calculating mean square error loss is as follows: Where m is the number of samples, y i For 3D reconstruction CT, y i 'This is a CT scan to be reconstructed.' 3. The three-dimensional CT image reconstruction method according to claim 1, characterized in that, Before using a convolutional neural network for feature extraction, the convolutional neural network is trained. The training steps are as follows: B1: Preprocess the CT and KV projection data to be reconstructed; B2: The CT scan to be reconstructed and its corresponding KV projection data are combined into sample pairs and divided into training set, test set and validation set according to a certain ratio; B3: Input the training set into the convolutional neural network for training and 3D conversion to obtain a 3D reconstructed CT scan; B4: Calculate the mean squared error loss between the 3D reconstructed CT and the CT to be reconstructed, backpropagate the obtained loss in the convolutional neural network to obtain the gradient of the network parameters, and update the network parameters through the gradient descent optimization method to minimize the mean squared error loss. B5: After each training iteration, use the validation set to verify whether the mean squared error loss on the validation set is minimized. If so, then save the weight; If not, continue training. If the mean squared error loss on the validation set still does not decrease after training reaches the threshold number of iterations, then adjust the learning rate. B6: Repeat B3-B5 until the training result of the convolutional neural network reaches the expected level and the mean squared error loss on the training set is minimized.

4. A three-dimensional CT image reconstruction device based on single-image projection data, characterized in that, include: The projection data acquisition module is used to acquire the CT to be reconstructed and generate corresponding single KV projection data from the acquired CT to be reconstructed. The model building module is used to construct a CT reconstruction model based on a convolutional neural network. The CT reconstruction model includes: an encoder with an attention mechanism and a multi-size fusion strategy, a 3D transformation unit, and a decoder for feature reconstruction. The three-dimensional reconstruction module is used to input the CT to be reconstructed into the CT reconstruction model to obtain the three-dimensional reconstructed CT, and to calculate the gray value difference between the two and the CT to be reconstructed, and to determine whether the gray value difference between the two reaches the global optimal solution. If the target is reached, the reconstruction is successful, and the final 3D CT image is output. Otherwise, the gradient descent method is repeatedly used to find the optimal solution until the difference in gray values ​​between the two reaches the global optimum. The attention mechanism includes: a channel attention module and a spatial attention module; The channel attention module is used to pass the input feature map through a global max pooling layer and a global average pooling layer, respectively, and then transmit the output feature maps of the two layers to a multi-layer vector machine for further feature extraction; the generated intermediate feature map is then processed to obtain the channel attention map. The spatial attention module is used to multiply the channel attention map with the input feature map to obtain a secondary intermediate feature map. The secondary intermediate feature map is then passed through a global max pooling layer and a global average pooling layer, respectively. The output feature maps of the two layers are then concatenated. The concatenated feature map is then processed to obtain the spatial attention map. The fused feature map obtained through the multi-scale fusion strategy is as follows: Among them, G k F represents the fused feature map output from the k-th layer of the encoder. k This represents the input feature map of the k-th layer of the encoder. Represents upsampling 2 i-k C represents the concatenation operation, D sconv @2 i-k This indicates an expansion rate of 2. i-k Hollow convolution.

5. The three-dimensional CT image reconstruction apparatus according to claim 4, characterized in that, In the 3D reconstruction module, the calculation of grayscale value difference is specifically the calculation of mean squared error loss, and the formula for calculating mean squared error loss is: Where m is the number of samples, y i For 3D reconstruction CT, y' i For CT scans to be reconstructed.

6. The three-dimensional CT image reconstruction apparatus according to claim 4, characterized in that, Training a convolutional neural network specifically includes the following steps: B1: Preprocess the CT and KV projection data to be reconstructed; B2: The CT scan to be reconstructed and its corresponding KV projection data are combined into sample pairs and divided into training set, test set and validation set according to a certain ratio; B3: Input the training set into the convolutional neural network for training and 3D conversion to obtain a 3D reconstructed CT scan; B4: Calculate the mean squared error loss between the 3D reconstructed CT and the CT to be reconstructed, backpropagate the obtained loss in the convolutional neural network to obtain the gradient of the network parameters, and update the network parameters through the gradient descent optimization method to minimize the mean squared error loss. B5: After each training iteration, use the validation set to verify whether the mean squared error loss on the validation set is minimized. If so, then save the weight; If not, continue training. If the mean squared error loss on the validation set still does not decrease after training reaches the threshold number of iterations, then adjust the learning rate. B6: Repeat B3-B5 until the training result of the convolutional neural network reaches the expected level and the mean squared error loss on the training set is minimized.

Images