Low-dose CT image restoration method and device based on neural network multi-level feature learning

Abstract

The invention relates to a low-dose CT image restoration method and device based on neural network multilevel feature learning, and the method comprises the steps: carrying out the preprocessing of a CT image data set, and obtaining a preprocessed image; based on the preprocessed image, extracting image coarse features through a coarse feature extraction module to obtain a coarse feature map; based on the coarse feature map, enhancing image detail features through a detail enhancement module to obtain a detail enhanced feature map; obtaining a first reconstructed image through an image reconstruction module based on the coarse feature map; obtaining a second reconstructed image through a reconstructed image detail module based on the detail enhanced feature map; adding the first reconstructed image and the second reconstructed image to obtain a restored image; and calculating a loss function, and training a recovery model. According to the method, the low-dose CT image restoration processing effect can be effectively improved.

Description

Technical Field

[0001] The present invention relates to a low-dose CT image restoration method and device based on multi-level feature learning of neural network. Background Art

[0002] Computed tomography (CT) is a basic non-invasive diagnostic method in modern medicine. It has the characteristics of fast scanning time and clear images. It can detect smaller lesions and more subtle lesions, and plays an extremely important role in clinical diagnosis. However, the X-rays used in CT scans can penetrate the human body and interact with atoms in body tissues to form images. X-ray radiation can have a negative impact on the human body, especially when exposed to high doses for a long time, which may cause cell damage, increased cancer risk, genetic effects, and radiation dermatitis. Therefore, when receiving X-ray examinations or treatments, it is necessary to reduce the radiation dose of X-rays to reduce this hazard. However, when the X-ray dose is at a low level, the image signal-to-noise ratio will decrease, resulting in serious artifacts and noise in low-dose CT images, affecting the accuracy of medical imaging diagnosis results.

[0003] With the continuous development of deep learning, it has had an all-round impact on LDCT images (low-dose CT images), including improving image quality, reducing radiation dose, diagnostic efficiency and algorithm upgrades, so that machines can better assist doctors in diagnosis and treatment, providing new research ideas for the field of medical image processing. Compared with traditional denoising algorithms that rely on prior knowledge and are only effective for specific noise types, deep learning-based methods have better denoising effects and significant advantages in adapting to a variety of noise types. Although existing deep learning-based methods have advantages in denoising effects and image quality to a certain extent, they have certain limitations in image feature extraction and cannot effectively extract rich image information and maintain edge detail information.

[0004] Low-dose CT images are often disturbed by noise, resulting in problems such as degraded image quality, blurred details, and blurred edges. If key detail information and edge clarity are lost during the processing, it may mislead the doctor's diagnosis and affect the accurate positioning and analysis of abnormal areas. Therefore, maintaining the detail information and edge clarity of low-dose CT images is the key to ensuring accurate diagnosis and reliable analysis, and is important for improving the quality of medical images and the credibility of clinical applications. Summary of the invention

[0005] The purpose of the present invention is to provide a low-dose CT image restoration method and device based on multi-level feature learning of neural network, which can effectively improve the processing effect of low-dose CT image restoration.

[0006] Based on the same inventive concept, the present invention has two independent technical solutions:

[0007] 1. A low-dose CT image restoration method based on multi-level feature learning of neural network, comprising the following steps:

[0008] Step 1: Preprocess the CT image data set to obtain a preprocessed image;

[0009] Step 2: Based on the preprocessed image, the coarse features of the image are extracted by a coarse feature extraction module to obtain a coarse feature map;

[0010] Step 3: Based on the coarse feature map, the image detail features are enhanced by a detail enhancement module to obtain a detail enhanced feature map;

[0011] Step 4: Based on the coarse feature map, a first reconstructed image is obtained by reconstructing an image module; based on the detail enhancement feature map, a second reconstructed image is obtained by reconstructing an image detail module;

[0012] Step 5: Add the first reconstructed image and the second reconstructed image to obtain a restored image;

[0013] Step 6: Calculate the loss function and train the restoration model.

[0014] Furthermore, in step 2, the coarse feature extraction module includes a first coarse feature extraction module, a second coarse feature extraction module, and a third coarse feature extraction module, and the input end and the output end of the first coarse feature extraction module, the second coarse feature extraction module, and the third coarse feature extraction module are connected in sequence.

[0015] Furthermore, in step 2, the coarse feature extraction module is composed of a shift convolution module and a channel attention module, and the shift convolution module and the channel attention module are cascaded.

[0016] Furthermore, in step 3, the output ends of the first coarse feature extraction module, the second coarse feature extraction module, and the third coarse feature extraction module are respectively connected to the first detail enhancement module, the second detail enhancement module, and the third detail enhancement module, and the output ends of the first detail enhancement module, the second detail enhancement module, and the third detail enhancement module are cascaded.

[0017] Furthermore, in step 3, the first detail enhancement module, the second detail enhancement module, and the third detail enhancement module all adopt a residual learning module, and the residual learning module adopts a mode combining long skip connections and short skip connections.

[0018] Furthermore, in step 4, the reconstructed image detail module includes a residual learning module, a channel attention module, and a pixel attention module. The output end of the residual learning module is connected to the input end of the channel attention module, and the output end of the channel attention module is connected to the input end of the pixel attention module.

[0019] Furthermore, in step 4, the reconstructed image module includes a residual learning module, a channel attention module, and a spatial attention module. The output end of the residual learning module is connected to the input end of the channel attention module, and the output end of the channel attention module is connected to the input end of the spatial attention module.

[0020] Furthermore, in step 6, the loss function is a combination of the mean absolute error function L1 Loss and the multi-scale structural similarity index MS-SSIM error function.

[0021] 2. A low-dose CT image restoration device, used to execute the above method.

[0022] The present invention has the beneficial effects:

[0023] The present invention preprocesses a CT image data set to obtain a preprocessed image; based on the preprocessed image, a coarse feature extraction module is used to extract image coarse features to obtain a coarse feature map; based on the coarse feature map, a detail enhancement module is used to enhance image detail features to obtain a detail enhancement feature map; based on the coarse feature map, a first reconstructed image is obtained through an image reconstruction module; based on the detail enhancement feature map, a second reconstructed image is obtained through a reconstructed image detail module; the first reconstructed image and the second reconstructed image are added to obtain a restored image; a loss function is calculated to train a restoration model. The present invention adopts a coarse-to-fine feature extraction method, which integrates feature extraction, image detail enhancement and image reconstruction modules into an end-to-end model, thereby gradually extracting more detailed and rich features, which can effectively improve the low-dose CT image restoration processing effect.

[0024] The coarse feature extraction module of the present invention includes a first coarse feature extraction module, a second coarse feature extraction module, and a third coarse feature extraction module, wherein the input end and the output end of the first coarse feature extraction module, the second coarse feature extraction module, and the third coarse feature extraction module are connected in sequence; the coarse feature extraction module is composed of a shift convolution module and a channel attention module, and the shift convolution module and the channel attention module are cascaded. In the coarse feature extraction stage, the present invention introduces a shift convolution and a channel attention module to realize an optimized feature extraction network, which can effectively extract the local structural information of the image and more accurately capture the important features in the image.

[0025] The output ends of the first coarse feature extraction module, the second coarse feature extraction module, and the third coarse feature extraction module of the present invention are respectively connected to the first detail enhancement module, the second detail enhancement module, and the third detail enhancement module, and the output ends of the first detail enhancement module, the second detail enhancement module, and the third detail enhancement module are cascaded. The first detail enhancement module, the second detail enhancement module, and the third detail enhancement module all adopt residual learning modules, and the residual learning modules adopt a mode combining long jump connections and short jump connections. The present invention adopts a residual learning module in the detail enhancement stage, and adopts a mode combining short jump connections and long jump connections, which effectively increases the robustness and generalization ability of the network, helps the network better capture subtle changes and detail information in the image, and improves the image restoration effect.

[0026] The image detail reconstruction module of the present invention comprises a residual learning module, a channel attention module, and a pixel attention module. The output terminal of the residual learning module is connected to the input terminal of the channel attention module, and the output terminal of the channel attention module is connected to the input terminal of the pixel attention module. The image detail reconstruction module of the present invention introduces a feature attention module, combines the channel attention and pixel attention mechanisms, and utilizes a local residual learning method, which effectively improves the flexibility of the module in processing different types of information.

[0027] The image reconstruction module of the present invention comprises a residual learning module, a channel attention module, and a spatial attention module. The output terminal of the residual learning module is connected to the input terminal of the channel attention module, and the output terminal of the channel attention module is connected to the input terminal of the spatial attention module. The image reconstruction module of the present invention effectively integrates multi-level features at different scales by organically combining the extended convolution and attention mechanism, thereby improving the effect of image reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a schematic diagram of the structure of the low-dose CT image restoration network of the present invention;

[0029] Figure 2 is a schematic structural diagram of an image detail reconstruction module of the present invention;

[0030] Figure 3 is a schematic structural diagram of an image reconstruction module of the present invention;

[0031] Figure 4 This is the first set of denoising result comparison chart;

[0032] Figure 5 This is the comparison chart of the second set of denoising results;

[0033] Figure 6 This is the comparison chart of the third set of denoising results;

[0034] Figure 7 It is the enlarged picture of different areas;

[0035] Figure 8 It is the difference image obtained by subtracting different denoising results from the LDCT image;

[0036] Fig. 9 It is a box plot of the denoising results of different denoising methods on the AAPM test set;

[0037] Fig.10 It is a quantitative representation of ROI;

[0038] Fig.11 This is a comparison chart of denoising results for various noise types under ablation experiments. DETAILED DESCRIPTION

[0039] The present invention is described in detail below in conjunction with the various embodiments shown in the accompanying drawings, but it should be noted that these embodiments are not limitations of the present invention, and any equivalent transformations or substitutions in functions, methods, or structures made by ordinary technicians in the field based on these embodiments are all within the scope of protection of the present invention.

[0040] Embodiment 1:

[0041] Low-dose CT image restoration method based on multi-level feature learning of neural network

[0042] like Figure 1 As shown, the low-dose CT image restoration method based on multi-level feature learning of neural network of the present invention comprises the following steps:

[0043] Step 1: Preprocess the low-dose CT image dataset to obtain a preprocessed image.

[0044] Assume that the image size of the original dataset is A×A, and randomly crop several a×a-sized areas on the noisy image as training images, combined with other data augmentation operations to increase the dataset.

[0045] Step 2: Based on the preprocessed image, the coarse features of the image are extracted through the coarse feature extraction module to obtain a coarse feature map.

[0046] X and Y are represented as the input and output of the overall network architecture of the present invention.

[0047] The coarse feature extraction stage consists of three coarse feature extraction modules, which are used to extract initial features from the input LDCT images.

[0048] First, a 3×3 convolution operation is used to expand the number of channels within the model. This step helps to improve the network’s expressiveness and learning capabilities, allowing the network to better process detailed information in the image and obtain a preprocessed image.

[0049] After that, we enter the coarse feature extraction module. The whole process can be expressed as:

[0050] f0=Conv(X)

[0051] f1=D RFE_1 (f0)

[0052] f2=D RFE_2 (f1)

[0053] f3=D RFE_3 (f2)

[0054] Among them, D RFE_i (g) Perform a coarse feature extraction operation on the image. Through the above steps, the coarse feature extraction stage can extract initial features from the input LDCT image, providing effective information for the subsequent detail enhancement stage and image reconstruction stage. These coarse features can help the network better understand the overall structure and general features of the image, and provide a data basis for the processing of the subsequent stages.

[0055] like Figure 1 As shown, the coarse feature extraction module includes a first coarse feature extraction module, a second coarse feature extraction module, and a third coarse feature extraction module, and the input end and output end of the first coarse feature extraction module, the second coarse feature extraction module, and the third coarse feature extraction module are connected in sequence. The coarse feature extraction module is composed of a shift convolution module and a channel attention module, and the shift convolution module and the channel attention module are cascaded.

[0056] Shifted convolution module:

[0057] Shifted convolution is an improved convolution operation that can translate the input feature map and capture the spatial relationship in the image by learning the translation parameters. The traditional convolution operation extracts features within a fixed neighborhood, while shifted convolution allows feature extraction within a larger receptive field. By translating the features of each pixel, shifted convolution can obtain a wider range of contextual information, thereby improving the network's ability to extract features. By introducing the translation operation, shifted convolution can better capture the local structural information of the image.

[0058] The specific structure and principle of the shift convolution module are as follows:

[0059] Translation unit: The translation unit is the core of the shift convolution module, which is responsible for translating the input feature map. This operation is usually implemented through a specific algorithm or matrix operation, so that each pixel or feature point in the input feature map can be moved according to a predetermined rule.

[0060] Convolution kernel: In the shifted convolution module, the convolution kernel is used to extract features from the translated input feature map. Unlike traditional fixed-position convolution kernels, the convolution kernel in the shifted convolution module can move within a larger receptive field to capture a wider range of spatial information.

[0061] Translation parameter learning mechanism: In order to optimize the translation operation, the shift convolution module usually contains a translation parameter learning mechanism. This mechanism can automatically adjust the translation parameters according to the content of the input feature map and the goal of network training to find the optimal translation strategy.

[0062] Channel Attention Module:

[0063] The channel attention module contains the following key parts:

[0064] Squeeze part: The feature map is compressed from H×W×C to 1×1×C through global average pooling (GAP), that is, global average pooling is performed on each channel to obtain the average pooled feature of each channel, which is a value.

[0065] Excitation part: The global feature vector is transformed nonlinearly through several fully connected (FC) layers, usually including a ReLU activation function and a sigmoid activation function. The output value is compressed between 0 and 1, generating a weight vector 1×1×C with the same length as the number of input channels.

[0066] The core of the channel attention module is to assign a weight to each channel, which reflects the importance of the channel to the final task. The specific workflow is as follows:

[0067] Feature transformation: First, the input feature map is transformed, such as global average pooling, to obtain the global information of each channel.

[0068] Attention weight calculation: Next, the attention weight of each channel is calculated through the fully connected layer and activation function. These weights represent the contribution of each channel to the final feature representation.

[0069] Feature reweighting: Finally, the input feature map is multiplied by the calculated attention weight to obtain the reweighted feature representation. In this way, important channel features are amplified, while unimportant channel features are suppressed.

[0070] The present invention uses a channel attention module and a shifted convolution in cascade to construct an efficient feature extraction network. The channel attention module can automatically learn and selectively amplify or suppress the feature responses of different channels, thereby increasing the network's attention to important features. By cascading the channel attention module with the shifted convolution, high-level features of the image can be further extracted. This design of the coarse feature extraction stage enables the network to focus more on important features while retaining the local structural information of the image. The network can more accurately capture the important features in the image, which helps to improve the performance of the overall model and ensure better results in image restoration tasks.

[0071] Step 3: Based on the coarse feature map, the image detail features are enhanced by a detail enhancement module to obtain a detail enhanced feature map.

[0072] The detail enhancement stage is to further refine each step of the coarse feature extraction stage, and finally cascade the extracted detail information to obtain the detail enhancement result of the image. The whole process can be expressed as:

[0073] d1=D DEM_1 (f1)

[0074] d2=D DEM_2 (f2)

[0075] d3=D DEM_3 (f3)

[0076] d=D CONCAT (d1,d2,d3)

[0077] Among them, D DEM_i (g) To perform detail enhancement on the image, D CONCAT (g) is a cascade operation. In the detail enhancement stage, the coarse features are further refined and fused, and finally the detail enhancement effect of the image is achieved.

[0078] like Figure 1 As shown, the output ends of the first coarse feature extraction module, the second coarse feature extraction module, and the third coarse feature extraction module are respectively connected to the first detail enhancement module, the second detail enhancement module, and the third detail enhancement module, and the output ends of the first detail enhancement module, the second detail enhancement module, and the third detail enhancement module are cascaded. The first detail enhancement module, the second detail enhancement module, and the third detail enhancement module all adopt a residual learning module, and the residual learning module adopts a mode combining long jump connection and short jump connection.

[0079] In order to enable the network to more clearly capture the subtle changes and details in the image, a residual learning module is introduced to process the detail enhancement of the image. The residual learning module is an architecture based on residual learning. By skipping the direct connection of certain layers and adding the input features to the output features, it can effectively transfer and retain important features, alleviate the problem of gradient disappearance, and accelerate the convergence of network training. In order to help information directly skip some layers and retain more original information, a short skip connection mechanism is adopted. Short skip connection refers to connecting the output of certain layers directly to the input of subsequent layers to form a direct feedback path, which is helpful for the propagation of gradients and the training of the model. Long skip connections are used to further enhance the ability of the network. Long skip connections refer to connecting the output of certain layers to distant subsequent layers to form a longer feedback path. This connection can help the network span multiple levels and combine feature representations at different levels to effectively capture global and local information in the image. The design of the detail enhancement module based on residual learning enables the network to better process the detail information in the image and improve the restoration effect of image quality.

[0080] Step 4: Based on the coarse feature map, obtain a first reconstructed image through an image reconstruction module; based on the detail enhancement feature map, obtain a second reconstructed image through an image detail reconstruction module.

[0081] (I) Image detail reconstruction module

[0082] The image detail reconstruction stage is to reconstruct the extracted image detail information, aiming to restore the complete detail information of the low-dose CT image. The process is expressed as:

[0083] r0=Conv(d)

[0084] r=D IDM (r0)

[0085] Among them, D IDM (g) is to perform an operation of reconstructing details on the image, and finally obtain complete detail information in the low-dose CT image. This stage is designed to improve the visual quality of low-dose CT images, reduce noise and artifacts, and improve the diagnostic accuracy and reliability of the images.

[0086] like Figure 2 As shown, the reconstructed image detail module includes a residual learning module, a channel attention module, and a pixel attention module. The output end of the residual learning module is connected to the input end of the channel attention module, and the output end of the channel attention module is connected to the input end of the pixel attention module.

[0087] In the image detail enhancement module, in order to increase the flexibility of the module in processing different types of information, the present invention utilizes local residual learning, combines channel attention and pixel attention mechanisms, and proposes an image detail enhancement module combined with a dual attention mechanism.

[0088] First, in order to further enhance the module's ability to process image details, a channel attention mechanism is introduced that can automatically learn and selectively amplify or suppress feature responses of different channels. By applying channel attention to each channel in the feature map, the network's attention to important features can be improved and its dependence on irrelevant features can be reduced; secondly, a pixel attention mechanism is adopted that can adjust the feature map according to pixel-level weights. By learning pixel attention weights, the network's attention to specific pixels can be improved, so that subtle changes and detail information in the image can be better captured, further enhancing the module's detail enhancement capabilities; finally, in order to help the network better capture local detail information in the image and avoid averaging of global features, a local residual learning module is adopted that can define local areas on the feature map and learn the differences between local features through residual connections.

[0089] By combining channel attention and pixel attention mechanisms and utilizing local residual learning, the feature attention module can process different types of information more flexibly and effectively enhance the details of the image. This design enables the module to better adapt to image features in different scenarios and improve the detail enhancement effect.

[0090] (II) Image reconstruction module

[0091] The image reconstruction stage is to reconstruct the extracted coarse feature information, aiming to reduce the noise of low-dose CT images and restore low-dose CT images. The process is expressed as:

[0092] l0=Conv(f3)

[0093] l=D RIM (l0)

[0094] Among them, D RIM (g) is the operation of reconstructing the image. The image reconstruction stage uses the extracted coarse feature information to gradually restore the low-dose CT image structure. This stage is designed to improve the visual quality of the reconstructed image, reduce artifacts and noise, and better support the application and analysis of medical imaging.

[0095] like Figure 3 As shown, the reconstructed image module includes a residual learning module, a channel attention module, and a spatial attention module. The output end of the residual learning module is connected to the input end of the channel attention module, and the output end of the channel attention module is connected to the input end of the spatial attention module.

[0096] In the image reconstruction stage, in order to better utilize multi-scale features and significantly improve the quality of image reconstruction, residual learning is used, combined with channel attention and spatial attention mechanisms, which can more effectively fuse various features at different scales, allowing the module to adapt to different types of images and effectively enhance the detail information in the image.

[0097] First, in order to improve the accuracy of image reconstruction, a residual structure is introduced to help the network better capture small changes in the image. The residual structure uses jump connections to add input features to output features, allowing the network to directly learn the differences, propagate feature information faster in the network, help reduce gradient vanishing, and improve the network's learning ability and reconstruction effect.

[0098] Secondly, it can automatically learn and selectively adjust the feature responses of different channels to enhance the expression of important features. The spatial attention mechanism can automatically learn and adjust the feature weights of different spatial positions to better capture the local detail information in the image.

[0099] Spatial Attention Module:

[0100] The spatial attention module usually contains the following key steps:

[0101] Global pooling: Perform global maximum pooling and global average pooling in the channel dimension on an input feature map F of size H×W×C to obtain two feature maps of size H×W×1.

[0102] Feature splicing: The results of global maximum pooling and global average pooling are spliced ​​according to channels to obtain a feature map of size H×W×2.

[0103] Convolution operation: Perform a convolution operation on the concatenated result (such as a 7×7 convolution kernel) to obtain a feature map of size H×W×1.

[0104] Activation function: The spatial attention weight matrix is ​​obtained through the Sigmoid activation function.

[0105] The spatial attention module weights specific spatial locations in the feature map to highlight the areas that contribute most to the task and suppress irrelevant or redundant areas. The specific workflow is as follows:

[0106] Feature pooling: First, global maximum pooling and global average pooling are performed on the input feature map in the channel dimension to obtain the global information of each spatial position.

[0107] Feature concatenation and convolution: Then, the pooled features are concatenated and the importance of spatial positions is further learned through convolution operations.

[0108] Attention weight generation: Finally, the spatial attention weight matrix is ​​generated through the Sigmoid activation function, which encodes which positions need to be emphasized or suppressed.

[0109] By combining the channel attention and spatial attention mechanisms with dilated convolution, the reconstructed image module can make full use of various features at different scales and provide enhanced detail information during the reconstruction process. The design of the reconstructed image module for multi-level feature fusion enables the network to effectively fuse feature representations at different scales, greatly improving the effect of image reconstruction, significantly enhancing the ability to restore detail information in the image, enabling the network to better adapt to the characteristics of different types of images, and improving the quality of reconstructed images.

[0110] Step 5: Add the first reconstructed image and the second reconstructed image to obtain a restored image.

[0111] The reconstructed image detail information (second reconstructed image) and the reconstructed image (first reconstructed image) information are added together to obtain a final restored image.

[0112]

[0113] Step 6: Calculate the loss function and train the restoration model.

[0114] The loss function is a combination of the mean absolute error function L1 Loss and the multi-scale structural similarity index MS-SSIM error function.

[0115] Mean absolute error (L1 Loss) is widely used as an objective function in low-level image processing tasks. L1 Loss measures the quality of the reconstructed image by calculating the intensity difference between the output pixel and the real image, so it is often used in many low-dose CT image denoising algorithms. However, since L1 Loss is not very sensitive to uniform deviations, some image details may be lost, and optimizing L1 Loss cannot fully express all the details of the CT image. Therefore, in order to better capture the detailed features of the image and improve the quality and fidelity of image generation and processing, the network of the present invention introduces a multi-scale structural similarity index (MS-SSIM, Multi-Scale Structural Similarity). MS-SSIM is a structural similarity measurement method that not only takes into account the brightness difference of the image, but also the similarity of structural information and texture. Through multi-scale comparison, MS-SSIM can better capture and retain the detailed features of the image. In order to make full use of the complementary advantages of these two error functions, they can be combined by weighted summation or using multi-objective optimization methods, comprehensively considering the sensitivity of L1 Loss to overall intensity differences and the ability of MS-SSIM to capture detail features, to obtain a more comprehensive and accurate objective function, thereby improving the effect of image processing tasks, achieving a better balance between the importance of overall structure and detail features, and providing better visual effects and quantitative indicators. The loss function formula is as follows:

[0116]

[0117] Among them, P is the set of all pixels, p is its central pixel, is the structural similarity measure at p. In this embodiment, the dependence of all loss functions on P is ignored, and α=0.84 is set according to prior experience; α represents a priori parameters; NDCT represents normal-dose CT images, and LDCT represents low-dose CT images;

[0118] MS-SSIM stands for Multiscale Structural Similarity Index;

[0119] N represents N pairs of images;

[0120] Net() represents the network model of the present invention;

[0121] Represents the Gaussian convolution parameters.

[0122] The beneficial effects of the present invention are further illustrated below in conjunction with comparative experiments.

[0123] In the experiment, the AAPM dataset from the 2016 LDCT Challenge was used, which contains paired CT images of 10 anonymous patients. Each LDCT image has a corresponding NDCT image (normal dose CT image), each image is 512×512 pixels, 812 CT images are randomly selected as the training set, and 80 untrained images are used as the test set.

[0124] (I) Training process

[0125] The experiment is based on the Pytorch framework, and the network is trained and tested using a computer equipped with an NVIDIA GeForce RTX 2080SUPER GPU. During training, the images are randomly cropped into patches of size 64×64 for training, and the network is optimized using the default parameters of the Adam optimizer. In the experiment, the batch size is 10, the learning rate is 0.001, and the number of iterations is 100. The parameters used in the comparative experiment are set according to the parameters of the original experiment.

[0126] (II) Comparative Experiment

[0127] In order to further verify the effectiveness of the proposed network, the network of the present invention is experimentally compared with REDCNN, edge-based EDCNN, wavelet and CNN-based WavResNet, and Transformer-based CTformer in terms of visual effects and quantitative indicators. Among them, the objective evaluation indicators in the quantitative analysis include structural similarity (SSIM) based on structural differences, peak signal-to-noise ratio (PSNR) based on pixel grayscale difference, gradient amplitude similarity deviation (GMSD) based on gradient value changes, feature similarity index measure (FSIM) based on feature differences, and variance information fidelity (VIFs) based on visual perception. Among them, the higher the index values ​​of SSIM, PSNR, FSIM, and VIFs, the better the quality of the denoised CT image. GMSD measures the structural similarity between two images, that is, the similarity between the two images in gradient distribution and structural features. When the GMSD value is small, it indicates that the structural features of the two images are more similar, their gradient distributions are also closer, and the processed image is closer to the original image, retaining more detail information and structural features.

[0128] By analyzing and utilizing the texture features in CT images, doctors can more accurately detect and locate the characterization and classification of lesions, and provide a more reliable basis for the treatment and evaluation of patients. The comparative experiment of the present invention uses five different methods to compare and analyze the results of two groups of slices after processing. The experimental results are as follows: Figure 4 , Figure 5 and Figure 6As shown in the figure, the CT images of all axial views are displayed in the [-160, 240] HU window. In order to verify the effectiveness of the algorithm proposed in this invention, five objective indicators and five denoising algorithms are used for experimental comparison, and their values ​​are marked in Figure 4 , Figure 5 and Figure 6 In the upper left and right corners of the Figure 7 Some regions (ROI, Region of Interest) are enlarged in the figure. The tissue structure of the human body can be observed in the NDCT image. For the LDCT image with low radiation dose, there are obvious stripe artifacts and speckle noise. Figure 4 (a) Figure 5 (a) and Figure 6 Compared with the NDCT of (a), it can be observed that Figure 4 (b) Figure 5 (b) and Figure 6 There are a lot of mottled noise and stripe artifacts in the LDCT image in (b), but from the perspective of overall noise reduction, the five experimental methods all have a certain inhibitory effect on noise and artifacts. Although WavResNet eliminates some noise, it introduces additional artifacts. REDCNN does not have image distortion and blurring problems, but it does cause problems of over-smoothing and loss of texture details in the image. In contrast, EDCNN is better than other methods in retaining image details, but subtle stripe artifacts can still be seen. CTformer removes most of the speckle noise in the LDCT image, but still cannot effectively remove artifacts. From the perspective of perceptual performance, the method proposed in this invention (our) removes a large amount of noise level while maintaining edge information, and the edges of the ROI are clear, with minimal shadows and stripe artifacts. Comparative experiments have shown that the method proposed in this invention can effectively suppress artifacts, restore texture and details, and is closer to NDCT images.

[0129] Figure 8It is the difference image obtained by subtracting the different denoising results from the LDCT image. The detail residuals in different images can show the denoising results of different methods. The more details in different images, the more noise is suppressed during the denoising process. The order of the artifact residuals in the images after denoising by different methods is: WavResNet < RED-CNN < EDCNN < CTformer < Ourmethod. Among them, the artifact residual of WavResNet is relatively large, which means that this method fails to handle the noise well and may cause a certain degree of image information loss. There are slight differences in the performance of EDCNN and RED-CNN during the denoising process. The use of a unidirectional convolutional neural network structure in EDCNN may lead to information loss. This is because when performing a convolution operation on an image, if the convolution kernel only moves in one direction, the noise located at the image edge may not be fully processed, affecting the denoising effect. RED-CNN adopts a residual learning strategy, which reduces the interference of noise on image details to a certain extent. CTformer shows good performance in the denoising task. Although CTformer is slightly inferior to the convolutional neural network in terms of image detail modeling, it can still relatively accurately remove noise and improve the image quality. The experimental results show that the network structure (our) proposed in the present invention can suppress noise to the greatest extent, has strong denoising ability, and can restore the detail information of the image.

[0130] Table 1 Denoising performance of each denoising algorithm on the test set

[0131]

[0132] As shown in Table 1, a summary table of the average PSNR, SSIM, GMSD, FSIM, and VIFs values of the images obtained by processing 80 LDCT images in the test dataset with different methods is given. The optimal values are shown in bold. It can be seen from Table 1 that the method (our) proposed in the present invention shows relatively excellent performance in all objective indicators, especially with obvious improvements in the two indicators of PSNR and SSIM. Specifically, WavResNet performs poorly in terms of PSNR, while REDCNN and WavResNet have low scores in terms of VIFS, which is consistent with the situation of over-smoothing and low contrast in the visual judgment of the images. The use of a unidirectional convolutional neural network structure in EDCNN may lead to information loss, affecting the denoising effect, and its indicator performance is not as good as that of Ctformer. Comprehensive comparison shows that the method of the present invention has the best scores in all 5 indicators. The experiment shows that this method has good noise and artifact suppression (PSNR), feature information preservation (FSIM and VIFs), and the visual effect is closer to the NDCT image (GMSD and SSIM).

[0133] Through the box plots of PSNR and SSIM, the distribution characteristics of different methods in the test dataset are compared, such as Fig. 9 shown. The box plot summarizes a set of data through five statistics: maximum, minimum, upper and lower quartiles, and median. By observing the width of the 12 frames in the figure and the distribution range of PSNR and SSIM, the method of the present invention has better robustness. From the gray line (median) of the box plot, the order of different denoising methods is: (PSNR) WavResNet < REDCNN < EDCNN < CTformer < Our, (SSIM) WavResNet < REDCNN < EDCNN < CTformer < Our. The highest median confirms the average quantization performance of the method of the present invention (our). Since doctors tend to pay more attention to ROI in actual clinical applications, according to Fig.10 the comparison of quantifying the ROI results obtained by different methods using PSNR and SSIM, the following can be observed: Under the same ROI, the PSNR and SSIM values of WavResNet are the lowest, indicating that its performance in image quality is inferior to other methods. The comparison between EDCNN and RECNN in the ROI area is similar, without obvious differences. Although CTformer has achieved certain advantages in some ROI areas. Generally speaking, the method proposed in the present invention (our) performs excellently in all five metrics. Therefore, based on the above results, it can be concluded that the method proposed in the present invention has better performance in terms of PSNR and SSIM compared to other methods, and has great advantages in the task of low-dose CT image denoising.

[0134] (III) Ablation experiments

[0135] To study the impact of different sub-modules on the network performance, a group of ablation experiments conducted on the Mayo dataset were compared. The training dataset and the test dataset are the same as those in the comparative experiment. "w / o" indicates whether it conforms to the loss function. The model using the proposed composite loss function instead of L1 loss is denoted as ①, the model "w / o dilated convolution" (using dilated convolution instead of the traditional convolutional layer) is denoted as ②, and the network proposed in this paper is denoted as ③. Table 2 shows the average metric values of each model in the ablation experiment when processing 80 groups of test sets. The denoising effects of each model on another set of representative CT images in the test set are as Fig.11 shown, with the display window level being [-160, 240] HU. It can be seen that the method proposed in this paper has obtained the best scores in each metric. The model with only L1 loss has sub-optimal PSNR and SSIM values, while the model without the dilated convolution module performs the worst. The above experimental results show that the dilated convolution module and the composite loss function have a positive effect on the model.

[0136] Table 2 Denoising performance of each model on the test set during ablation experiment

[0137]

[0138] Embodiment 2:

[0139] Low-dose CT image restoration device

[0140] The low-dose CT image restoration device is used to execute the above-mentioned low-dose CT image restoration method based on multi-level feature learning of neural network.

[0141] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present invention. They are not intended to limit the scope of protection of the present invention. Any equivalent implementation methods or changes that do not deviate from the technical spirit of the present invention should be included in the scope of protection of the present invention.

[0142] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, the embodiments should be considered exemplary and non-restrictive in all respects, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims be included in the present invention.

Claims

1. A low-dose CT image restoration method based on multi-level feature learning of neural networks, characterized in that: The steps include: Step 1: Preprocess the CT image data set to obtain a preprocessed image; Step 2: Based on the preprocessed image, the coarse features of the image are extracted by a coarse feature extraction module to obtain a coarse feature map; Step 3: Based on the coarse feature map, the image detail features are enhanced by a detail enhancement module to obtain a detail enhanced feature map; Step 4: Based on the coarse feature map, a first reconstructed image is obtained by reconstructing an image module; based on the detail enhancement feature map, a second reconstructed image is obtained by reconstructing an image detail module; Step 5: Add the first reconstructed image and the second reconstructed image to obtain a restored image; Step 6: Calculate the loss function and train the restoration model.

2. The low-dose CT image restoration method based on multi-level feature learning of neural network according to claim 1, characterized in that: In step 2, the coarse feature extraction module includes a first coarse feature extraction module, a second coarse feature extraction module, and a third coarse feature extraction module, and the input end and the output end of the first coarse feature extraction module, the second coarse feature extraction module, and the third coarse feature extraction module are connected in sequence.

3. The low-dose CT image restoration method based on neural network multi-level feature learning according to claim 1 is characterized in that: In step 2, the coarse feature extraction module is composed of a shift convolution module and a channel attention module, and the shift convolution module and the channel attention module are cascaded.

4. The low-dose CT image restoration method based on neural network multi-level feature learning according to claim 2 is characterized in that: In step 3, the output ends of the first coarse feature extraction module, the second coarse feature extraction module, and the third coarse feature extraction module are respectively connected to the first detail enhancement module, the second detail enhancement module, and the third detail enhancement module, and the output ends of the first detail enhancement module, the second detail enhancement module, and the third detail enhancement module are cascaded.

5. The low-dose CT image restoration method based on neural network multi-level feature learning according to claim 4 is characterized in that: In step 3, the first detail enhancement module, the second detail enhancement module, and the third detail enhancement module all adopt a residual learning module, and the residual learning module adopts a mode combining long skip connections and short skip connections.

6. The low-dose CT image restoration method based on neural network multi-level feature learning according to claim 1 is characterized in that: In step 4, the reconstructed image detail module includes a residual learning module, a channel attention module, and a pixel attention module. The output end of the residual learning module is connected to the input end of the channel attention module, and the output end of the channel attention module is connected to the input end of the pixel attention module.

7. The low-dose CT image restoration method based on neural network multi-level feature learning according to claim 1 is characterized in that: In step 4, the reconstructed image module includes a residual learning module, a channel attention module, and a spatial attention module. The output end of the residual learning module is connected to the input end of the channel attention module, and the output end of the channel attention module is connected to the input end of the spatial attention module.

8. The low-dose CT image restoration method based on neural network multi-level feature learning according to claim 1 is characterized in that: In step 6, the loss function is a combination of the mean absolute error function L1 Loss and the multi-scale structural similarity index MS-SSIM error function.

9. The low-dose CT image restoration method based on neural network multi-level feature learning according to claim 8, characterized in that: The loss function formula is as follows: Among them, P is the set of all pixels, p is its central pixel, is the structural similarity measure at p, α represents the prior parameter; NDCT represents the normal-dose CT image, and LDCT represents the low-dose CT image; MS-SSIM stands for Multiscale Structural Similarity Index; N represents N pairs of images; Net() represents the network model; Represents the Gaussian convolution parameters.

10. A low-dose CT image restoration device based on multi-level feature learning of neural networks, characterized in that: Used to perform the method according to any one of claims 1 to 9.

Images