Image denoising method and device
By constructing and training deep learning neural network models and processing three-dimensional CT image data, the problem of poor denoising effect of industrial CT images is solved, and high-quality image denoising and imaging quality improvement is achieved.
Patent Information
- Application Number
- CN202411787700.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-06
- Publication Date
- 2025-03-28
AI Technical Summary
The prior art lacks mature technical solutions to directly denoise industrial CT images end-to-end using deep learning, resulting in poor noise removal effect of CT images and affecting imaging quality.
The deep learning neural network model is built and trained, and the three-dimensional CT image data is processed through downsampling, feature extraction and upsampling modules, combined with the Transformer Encoder module, the model's feature fusion capability is enhanced, and the OTSU automatic threshold segmentation and weighted loss function is used to improve the denoising performance.
It effectively removes noise in CT images, improves imaging quality, enhances image noise reduction effect, and can quickly obtain high-quality and clear images under lossless conditions, observes the fine structure shapes inside the object, and detects processing defects or abnormalities.
Smart Images
Figure CN119722507A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to an image denoising method and device. Background Art
[0002] As the manufacturing and testing requirements for metal parts and batteries become increasingly stringent, the use of CT imaging to complete non-destructive quality testing of metal industrial parts or batteries has become a new research hotspot.
[0003] The industrial CT detection method is the optimal non-destructive testing technology for battery testing. The non-contact scanning can ensure that the battery is not damaged. The reconstructed three-dimensional image can not only reproduce the surface structure of the battery, but also capture the fine structural morphology, processing defects, anomalies, etc. inside the battery.
[0004] Due to the particularity of the battery, scattering noise and beam hardening noise may be caused in CT imaging; at the same time, the photon fluctuation noise of the ray source and the thermal noise of the ray detector during CT ray scanning will affect the imaging quality of CT. Therefore, it is extremely important to remove the noise of CT images in order to obtain higher quality imaging results.
[0005] In recent years, deep learning has shown great promise in the field of CT image denoising. However, the exploration of using deep learning to directly perform end-to-end denoising on industrial CT images is still in its infancy. Summary of the invention
[0006] The purpose of the present invention is to propose an image denoising method, aiming to solve the problem that the prior art still lacks a mature technical solution for directly denoising industrial CT images end-to-end using deep learning.
[0007] The present invention provides an image denoising method, comprising the following steps:
[0008] Build a deep learning neural network model and complete training;
[0009] Collecting 3D CT image data to be denoised, splitting the collected 3D CT image data into N layers of 2D image data, inputting them into a trained deep learning neural network model, and outputting the denoised results of the N layers of 2D image data;
[0010] re-joining the denoised two-dimensional image data into three-dimensional image data;
[0011] The reassembled three-dimensional image data is further processed to obtain the final denoised three-dimensional CT image data.
[0012] Preferably, building a deep learning neural network model and completing training specifically includes the following steps:
[0013] Data acquisition step: acquiring multiple pairs of three-dimensional CT image data, wherein each pair of three-dimensional CT image data includes a low-quality three-dimensional CT image data and a high-quality three-dimensional CT image data of the same object; the low-quality three-dimensional CT image data refers to three-dimensional CT image data with pixels lower than a first preset threshold; the high-quality three-dimensional CT image data refers to three-dimensional CT image data with pixels higher than a second preset threshold;
[0014] The data preprocessing step is to perform data preprocessing on the plurality of pairs of the three-dimensional CT image data to obtain a preprocessed paired three-dimensional CT image data set (lq′ i , hq′ i ), where lq′ i is the i-th low-quality 3D CT image data, hq′ i is the i-th high-quality three-dimensional CT image data;
[0015] Network model training steps: Use the paired three-dimensional CT image data set to train the deep learning neural network model to obtain a trained deep learning neural network model.
[0016] Preferably, the data preprocessing step specifically includes the following steps:
[0017] Filling multiple pairs of the three-dimensional CT image data to obtain a height and a width of a preset fixed multiple;
[0018] The window width and window level of the filled three-dimensional CT image data are adjusted to obtain a preprocessed paired three-dimensional CT image data set.
[0019] Preferably, the inverse process of preprocessing the re-stitched three-dimensional image data comprises the following steps:
[0020] Performing a padding removal operation on the reassembled three-dimensional image data to obtain three-dimensional image data having an original height and width;
[0021] The window width and window position are restored for the three-dimensional image data with the original height and width to obtain the final denoised three-dimensional CT image data.
[0022] Preferably, in the data preprocessing step, a normalization operation is performed on the adjusted three-dimensional CT image data to obtain a preprocessed paired three-dimensional CT image data set; in the inverse processing of preprocessing the re-spliced three-dimensional image data, a denormalization operation is performed on the three-dimensional image data with the original height and width to obtain denormalized three-dimensional image data, and then the window width and window level of the denormalized three-dimensional image data are restored to obtain the final denoised three-dimensional CT image data.
[0023] Preferably, the training method of the deep learning neural network model includes:
[0024] The preprocessed paired 3D CT image dataset (lq′ i , hq′ i ) is split into N layers of two-dimensional image data, and the high-quality two-dimensional image data of each layer corresponds to the low-quality two-dimensional image data, and a two-dimensional aligned data set (lq′ ij , hq′ ij ), where lq′ ij is the low-quality 3D CT image data corresponding to the low-quality 2D CT image data at the jth layer, hq′ i The i-th high-quality three-dimensional CT image data corresponds to the high-quality two-dimensional CT image data at the j-th layer;
[0025] The low-quality two-dimensional CT image data is used as the input of the industrial CT image denoising deep neural network, and the high-quality two-dimensional CT image data is used as the corresponding standard to perform feature recognition on the low-quality two-dimensional CT image data;
[0026] The Adam optimization algorithm is used, the initial learning rate is set to 0.0001, and the cosine annealing algorithm is used to adjust the learning rate.
[0027] Preferably, the loss function of the deep learning neural network model is based on L1Loss. First, the input preprocessed paired three-dimensional CT image data set is automatically threshold segmented using the OTSU algorithm to obtain the segmentation results of the foreground and background parts. L1Loss is calculated for the foreground and background respectively, and the losses of the foreground and background parts are weighted to obtain the final loss function.
[0028] Preferably, the weighting coefficients of the foreground and background are set to 1.6 and 0.4 respectively, that is, the L1Loss of the foreground multiplied by 1.6 plus the L1Loss of the background multiplied by 0.4.
[0029] Preferably, the deep learning neural network model includes a downsampling module, a feature extraction module and an upsampling module; the output of the downsampling module is connected to the upsampling module and the feature extraction module; the output of the feature extraction module is connected to the input of the upsampling module;
[0030] A downsampling module is used to reduce the height and width of the input three-dimensional CT image data, the input is the preprocessed paired three-dimensional CT image data set, and according to the pairing relationship in the paired three-dimensional CT image data set, each high-quality three-dimensional CT image data is used as a standard to output feature data of the low-quality three-dimensional CT image data corresponding to the high-quality three-dimensional CT image data;
[0031] A feature extraction module is used to enhance the ability of the deep learning neural network model to capture the global field of view, input the feature data of the low-quality three-dimensional CT image data output by the downsampling module, and output the feature data of the low-quality three-dimensional CT image data after feature extraction;
[0032] The up-sampling module is used to gradually restore the image denoising result and output feature data of the low-quality three-dimensional CT image data corresponding to the high-quality three-dimensional CT image data, wherein the magnification factor is the same as the reduction factor of the down-sampling module.
[0033] Preferably, the upsampling module includes a TransformerEncoder module, the input feature map is feature-encoded through feature embedding, and the features are further learned through layer normalization and multi-head attention, and then the results after layer normalization and multi-head attention learning are subjected to a residual connection fusion feature; the fused features are subjected to layer normalization and a multi-layer perceptron and then subjected to a residual connection fusion feature to obtain the data processing result of the Transformer Encoder module.
[0034] Preferably, the input preprocessed paired three-dimensional CT image data set is convolved with the result of automatic threshold segmentation and superposition connection using the OTSU algorithm and then input into the downsampling module.
[0035] The present invention also provides an image denoising device, comprising:
[0036] The image denoising deep learning module receives the three-dimensional CT image data to be denoised, uses the deep learning neural network model trained in the above-mentioned image denoising method to denoise the three-dimensional CT image data, and outputs the denoised N-layer two-dimensional CT image data.
[0037] Preferably, the image denoising device further comprises:
[0038] A data acquisition module is used to acquire three-dimensional CT image data and input it into an image denoising deep learning module;
[0039] The image processing module is used to process the denoised N-layer two-dimensional image data output by the image denoising deep learning module to generate denoised three-dimensional CT image data.
[0040] Compared with the prior art, the present invention has the following beneficial effects:
[0041] (1) The present invention processes the input feature data through a downsampling module so that the height and width of the input data are twice the output result, which effectively improves the speed of the model, further improves the accuracy of the model, and enhances the image denoising effect.
[0042] (2) The present invention sets a feature extraction module and uses convolution operation to construct the feature extraction module, thereby enhancing the ability of the image denoising deep model to capture the global field of view. At the same time, a Transformer Encoder module is added to combine the Transformer model with the CNN network model. While having CNN's super feature extraction ability for images, the model's feature fusion ability for image context is enhanced.
[0043] (3) The present invention sets an upsampling module and constructs the upsampling module using a convolution operation with residual, thereby increasing the height and width of the feature data by 2 times, thereby improving the robustness of the model, enhancing the model effect, and gradually restoring the CT image denoising results.
[0044] (4) The present invention divides the image into foreground and background by setting OTSU automatic threshold segmentation, explicitly inputs the segmentation result into the model structure, and gives different weighting coefficients to the foreground and background in the loss function, so that the model can focus more on learning the characteristics of the foreground part, further improving the denoising performance of the model.
[0045] (5) The present invention pre-processes the model input data, outputs N layers of two-dimensional image denoising results after model processing, and then re-splices the two-dimensional image into three-dimensional data after data post-processing, effectively removing image noise and improving imaging quality, and has advantages in retaining the object's fine structural information and suppressing a certain degree of edge blur. At the same time, high-quality and clear images can be quickly obtained without damaging the object, and the fine internal structure can be observed. For objects with processing defects or abnormalities, the quality inspection of the object can be completed without contacting the damaged object. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.
[0047] Figure 1 A schematic diagram of the connection of a deep learning neural network model according to an embodiment of the present invention;
[0048] Figure 2 A schematic diagram of the structure of a downsampling module according to an embodiment of the present invention;
[0049] Figure 3 A schematic diagram of the structure of a feature extraction module according to an embodiment of the present invention;
[0050] Figure 4 This is a schematic diagram of the structure of an upsampling module according to an embodiment of the present invention;
[0051] Figure 5 A schematic diagram of the structure of a Transformer Encoder module according to an embodiment of the present invention;
[0052] Figure 6 The figure is a flow chart of an image denoising method according to an embodiment of the present invention. DETAILED DESCRIPTION
[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0054] like Figure 6 As shown, the present invention provides an image denoising method, comprising the following steps:
[0055] Build a deep learning neural network model and complete training;
[0056] Collecting 3D CT image data to be denoised, splitting the collected 3D CT image data into N layers of 2D image data, inputting them into a trained deep learning neural network model, and outputting the denoised results of the N layers of 2D image data;
[0057] re-joining the denoised two-dimensional image data into three-dimensional image data;
[0058] The reassembled three-dimensional image data is further processed to obtain the final denoised three-dimensional CT image data.
[0059] According to a specific embodiment of the present invention, building a deep learning neural network model and completing training specifically includes the following steps:
[0060] Data acquisition step: acquiring multiple pairs of three-dimensional CT image data, wherein each pair of three-dimensional CT image data includes a low-quality three-dimensional CT image data and a high-quality three-dimensional CT image data of the same object; the low-quality three-dimensional CT image data refers to three-dimensional CT image data with pixels lower than a first preset threshold; the high-quality three-dimensional CT image data refers to three-dimensional CT image data with pixels higher than a second preset threshold;
[0061] The data preprocessing step is to perform data preprocessing on the plurality of pairs of the three-dimensional CT image data to obtain a preprocessed paired three-dimensional CT image data set (lq′ i , hq′ i), where lq′ i is the i-th low-quality 3D CT image data, hq′ i is the i-th high-quality three-dimensional CT image data;
[0062] Network model training steps: Use the paired three-dimensional CT image data set to train the deep learning neural network model to obtain a trained deep learning neural network model.
[0063] According to a specific embodiment of the present invention, the data preprocessing step specifically includes the following steps:
[0064] Filling multiple pairs of the three-dimensional CT image data to obtain a height and a width of a preset fixed multiple;
[0065] The window width and window level of the filled three-dimensional CT image data are adjusted to obtain a preprocessed paired three-dimensional CT image data set.
[0066] According to a specific embodiment of the present invention, the reverse processing process of preprocessing the re-stitched three-dimensional image data includes the following steps:
[0067] Performing a padding removal operation on the reassembled three-dimensional image data to obtain three-dimensional image data having an original height and width;
[0068] The window width and window position are restored for the three-dimensional image data with the original height and width to obtain the final denoised three-dimensional CT image data.
[0069] According to a specific embodiment of the present invention, in the data preprocessing step, a normalization operation is performed on the adjusted three-dimensional CT image data to obtain a preprocessed paired three-dimensional CT image data set; in the inverse processing of the preprocessing of the re-spliced three-dimensional image data, a denormalization operation is performed on the three-dimensional image data with the original height and width to obtain denormalized three-dimensional image data, and then the window width and window position of the denormalized three-dimensional image data are restored to obtain the final denoised three-dimensional CT image data.
[0070] According to a specific embodiment of the present invention, the training method of the deep learning neural network model includes:
[0071] The preprocessed paired 3D CT image dataset (lq′ i , hq′ i ) is split into N layers of two-dimensional image data, and the high-quality two-dimensional image data of each layer corresponds to the low-quality two-dimensional image data, and a two-dimensional aligned data set (lq′ ij , hq′ ij ), where lq′ ijis the low-quality 3D CT image data corresponding to the low-quality 2D CT image data at the jth layer, hq′ i The i-th high-quality three-dimensional CT image data corresponds to the high-quality two-dimensional CT image data at the j-th layer;
[0072] The low-quality two-dimensional CT image data is used as the input of the industrial CT image denoising deep neural network, and the high-quality two-dimensional CT image data is used as the corresponding standard to perform feature recognition on the low-quality two-dimensional CT image data;
[0073] The Adam optimization algorithm is used, the initial learning rate is set to 0.0001, and the cosine annealing algorithm is used to adjust the learning rate.
[0074] During training, the preprocessed paired 3D CT image dataset (lq′ i , hq′ i ) is divided into training set, validation set and test set, which are used for training, validation and testing respectively.
[0075] According to a specific embodiment of the present invention, the loss function of the deep learning neural network model is based on L1Loss. First, the input preprocessed paired three-dimensional CT image data set is automatically threshold segmented using the OTSU algorithm to obtain the segmentation results of the foreground and background parts. L1Loss is calculated for the foreground and background respectively, and the losses of the foreground and background parts are weighted to obtain the final loss function.
[0076] According to a specific embodiment of the present invention, the weighting coefficients of the foreground and background are set to 1.6 and 0.4 respectively, that is, the L1Loss of the foreground is multiplied by 1.6 plus the L1Loss of the background is multiplied by 0.4.
[0077] According to a specific embodiment of the present invention, the deep learning neural network model includes a downsampling module, a feature extraction module and an upsampling module; the output of the downsampling module is connected to the upsampling module and the feature extraction module; the output of the feature extraction module is connected to the input of the upsampling module;
[0078] A downsampling module is used to reduce the height and width of the input three-dimensional CT image data, the input is the preprocessed paired three-dimensional CT image data set, and according to the pairing relationship in the paired three-dimensional CT image data set, each high-quality three-dimensional CT image data is used as a standard to output feature data of the low-quality three-dimensional CT image data corresponding to the high-quality three-dimensional CT image data;
[0079] A feature extraction module is used to enhance the ability of the deep learning neural network model to capture the global field of view, input the feature data of the low-quality three-dimensional CT image data output by the downsampling module, and output the feature data of the low-quality three-dimensional CT image data after feature extraction;
[0080] The up-sampling module is used to gradually restore the image denoising result and output feature data of the low-quality three-dimensional CT image data corresponding to the high-quality three-dimensional CT image data, wherein the magnification factor is the same as the reduction factor of the down-sampling module.
[0081] According to a specific embodiment of the present invention, the upsampling module includes an Encoder module in a Transformer model, the input feature map is feature-encoded through feature embedding, and the features are further learned through layer normalization and multi-head attention, and then the results after layer normalization and multi-head attention learning are subjected to a residual connection fusion feature; the fused features are subjected to layer normalization and a multi-layer perceptron and then subjected to a residual connection fusion feature to obtain the data processing result of the Transformer Encoder module.
[0082] According to a specific embodiment of the present invention, the input preprocessed paired three-dimensional CT image data set is convolved with the result of automatic threshold segmentation and superposition connection by the OTSU algorithm and then input into the downsampling module.
[0083] The present invention also provides an image denoising device, comprising:
[0084] The image denoising deep learning module receives the three-dimensional CT image data to be denoised, uses the deep learning neural network model trained in the image denoising method according to any one of claims 1 to 6 to perform denoising on the three-dimensional CT image data, and outputs the denoised N-layer two-dimensional CT image data.
[0085] According to a specific embodiment of the present invention, it also includes:
[0086] A data acquisition module is used to acquire three-dimensional CT image data and input it into an image denoising deep learning module;
[0087] The image processing module is used to process the denoised N-layer two-dimensional image data output by the image denoising deep learning module to generate denoised three-dimensional CT image data.
[0088] Example 1
[0089] The image denoising method of the present invention is described in detail below according to a specific embodiment of the present invention.
[0090] The present invention provides an image denoising method, comprising the following steps:
[0091] Build a deep learning neural network model and complete training;
[0092] Collecting 3D CT image data to be denoised, splitting the collected 3D CT image data into N layers of 2D image data, inputting them into a trained deep learning neural network model, and outputting the denoised results of the N layers of 2D image data;
[0093] re-joining the denoised two-dimensional image data into three-dimensional image data;
[0094] The reassembled three-dimensional image data is further processed to obtain the final denoised three-dimensional CT image data.
[0095] Example 2
[0096] The image denoising method of the present invention is described in detail below according to a specific embodiment of the present invention.
[0097] The present invention provides an image denoising method, comprising the following steps:
[0098] Build a deep learning neural network model and complete training;
[0099] Collecting 3D CT image data to be denoised, splitting the collected 3D CT image data into N layers of 2D image data, inputting them into a trained deep learning neural network model, and outputting the denoised results of the N layers of 2D image data;
[0100] re-joining the denoised two-dimensional image data into three-dimensional image data;
[0101] The reassembled three-dimensional image data is further processed to obtain the final denoised three-dimensional CT image data.
[0102] Among them, building a deep learning neural network model and completing training specifically includes the following steps:
[0103] Data acquisition step: acquiring multiple pairs of three-dimensional CT image data, wherein each pair of three-dimensional CT image data includes a low-quality three-dimensional CT image data and a high-quality three-dimensional CT image data of the same object; the low-quality three-dimensional CT image data refers to three-dimensional CT image data with pixels lower than a first preset threshold; the high-quality three-dimensional CT image data refers to three-dimensional CT image data with pixels higher than a second preset threshold;
[0104] The data preprocessing step is to perform data preprocessing on the plurality of pairs of the three-dimensional CT image data to obtain a preprocessed paired three-dimensional CT image data set (lq′ i , hq′ i ), where lq′ i is the i-th low-quality 3D CT image data, hq′ i is the i-th high-quality three-dimensional CT image data;
[0105] Network model training steps: Use the paired three-dimensional CT image data set to train the deep learning neural network model to obtain a trained deep learning neural network model.
[0106] The data preprocessing step specifically includes the following steps:
[0107] Filling multiple pairs of the three-dimensional CT image data to obtain a height and a width of a preset fixed multiple;
[0108] Adjusting the window width and window level of the filled three-dimensional CT image data;
[0109] A normalization operation is performed on the adjusted three-dimensional CT image data to obtain a preprocessed paired three-dimensional CT image data set.
[0110] The reverse processing of preprocessing the reassembled three-dimensional image data includes the following steps:
[0111] Performing a padding removal operation on the reassembled three-dimensional image data to obtain three-dimensional image data having an original height and width;
[0112] Performing a denormalization operation on the three-dimensional image data after the padding is removed to obtain denormalized three-dimensional image data;
[0113] The window width and window position are restored for the three-dimensional image data to obtain the final denoised three-dimensional CT image data.
[0114] The training method of the deep learning neural network model includes:
[0115] The preprocessed paired 3D CT image dataset (lq′ i , hq′ i ) is divided into training set, validation set and test set;
[0116] Each CT image data is split into N layers of two-dimensional image data, and the high-quality two-dimensional image data of each layer corresponds to the low-quality two-dimensional image data, and a two-dimensional aligned data set (lq′ ij , hq′ ij ), where lq′ ij is the low-quality 3D CT image data corresponding to the low-quality 2D CT image data at the jth layer, hq′ i The i-th high-quality three-dimensional CT image data corresponds to the high-quality two-dimensional CT image data at the j-th layer;
[0117] The low-quality two-dimensional CT image data is used as the input of the industrial CT image denoising deep neural network, and the high-quality two-dimensional CT image data is used as the corresponding standard to perform feature recognition on the low-quality two-dimensional CT image data;
[0118] The loss function of the deep learning neural network model is based on L1Loss. First, the input preprocessed paired three-dimensional CT image data set is automatically threshold segmented using the OTSU algorithm to obtain the segmentation results of the foreground and background parts, and L1Loss is calculated for the foreground and background respectively. The losses of the foreground and background parts are weighted to obtain the final loss function. The weighting coefficients of the foreground and background parts are set to 1.6 and 0.4 respectively, that is, the L1Loss of the foreground is multiplied by 1.6 plus the L1Loss of the background is multiplied by 0.4; the Adam optimization algorithm is used, the initial learning rate is set to 0.0001, and the cosine annealing algorithm is used to adjust the learning rate.
[0119] The deep learning neural network model includes a downsampling module, a feature extraction module and an upsampling module; the output of the downsampling module is connected to the upsampling module and the feature extraction module; the output of the feature extraction module is connected to the input of the upsampling module;
[0120] A downsampling module is used to reduce the height and width of the input three-dimensional CT image data, the input is the preprocessed paired three-dimensional CT image data set, and according to the pairing relationship in the paired three-dimensional CT image data set, each high-quality three-dimensional CT image data is used as a standard to output feature data of the low-quality three-dimensional CT image data corresponding to the high-quality three-dimensional CT image data;
[0121] A feature extraction module is used to enhance the ability of the deep learning neural network model to capture the global field of view, input the feature data of the low-quality three-dimensional CT image data output by the downsampling module, and output the feature data of the low-quality three-dimensional CT image data after feature extraction;
[0122] The up-sampling module is used to gradually restore the image denoising result and output feature data of the low-quality three-dimensional CT image data corresponding to the high-quality three-dimensional CT image data, wherein the magnification factor is the same as the reduction factor of the down-sampling module.
[0123] Among them, the upsampling module includes a TransformerEncoder module. The input feature map is feature-encoded through feature embedding, and then the features are further learned through layer normalization and multi-head attention. The results after layer normalization and multi-head attention learning are subjected to a residual connection fusion feature. The fused features are subjected to layer normalization and a multi-layer perceptron and then subjected to a residual connection fusion feature to obtain the data processing result of the Transformer Encoder module.
[0124] Among them, the input preprocessed paired three-dimensional CT image data set is convolved with the result of automatic threshold segmentation and superposition connection by the OTSU algorithm and then input into the downsampling module.
[0125] Example 3
[0126] What is different from Example 1 and Example 2 is that the specific implementation of the deep learning neural network model is different.
[0127] In this embodiment, Figure 1 As shown, the deep learning neural network model includes three down-sampling modules, namely, a first down-sampling module, a second down-sampling module and a third down-sampling module, and three up-sampling modules, namely, a first up-sampling module, a second up-sampling module and a third up-sampling module.
[0128] The input of the first downsampling module is the result of a 3×3 convolution of low-quality three-dimensional CT image data and its superposition connection with automatic threshold segmentation by the OTSU algorithm. OTSU is the Otsu method, which is an adaptive threshold segmentation method. The automatic threshold segmentation result is added before entering the main structure of the model to divide the image into two parts: foreground and background. The foreground is the area where the object is located, which is the main part that needs to be denoised. We hope that the model can have a better denoising effect on the foreground part. The segmentation result is explicitly added directly in the first step of the model structure so that the model can better distinguish the foreground and background. At the same time, the foreground and background information are also used in the loss function for weighted calculation of the loss to further improve the performance. The output of the first downsampling module is the feature data of low-quality three-dimensional CT image data identified based on high-quality three-dimensional CT image data. The feature data of the low-quality three-dimensional CT image data is input into the second downsampling module and the third upsampling module.
[0129] The input of the second down-sampling module is the feature data of the low-quality three-dimensional CT image data output by the first down-sampling module, and the output feature data is input into the third down-sampling module and the second up-sampling module. The input of the third down-sampling module is the feature data output by the second down-sampling module, and the output feature data is input into the feature extraction module.
[0130] The structure of each downsampling module is as follows Figure 2 As shown, except for the first downsampling module which inputs low-quality three-dimensional CT image data and undergoes a 3×3 convolution, the remaining downsampling modules input feature data of low-quality three-dimensional CT image data and output feature data after downsampling. The height and width of the input data are both twice the output result, and the image data is reduced. In addition, the TransformerEncoder module is used in each downsampling module, which increases the context feature extraction and learning capabilities of the deep learning neural network model.
[0131] like Figure 2As shown in the figure, the downsampling module uses a residual connection method, including 3×3 convolution, ReLU activation function, Transformer Encoder module and Stride convolution. The feature data first passes through a 3×3 convolution and ReLU activation function, and the result of the Transformer Encoder module passes through a 3×3 convolution and ReLU activation function, and then is added to the input feature data that has passed the residual connection, and finally passes through a Stride convolution and a ReLU activation function before being output; among them, Stride convolution refers to a convolution with a stride of 2, which mainly reduces the height and width of the feature data by 2 times.
[0132] The structure of the feature extraction module is as follows: Figure 3 As shown, the input is the feature data output by the third down-sampling module, and the feature data after feature extraction is output to the first up-sampling module.
[0133] like Figure 3 As shown in the figure, the feature extraction module consists of three 3×3 convolutions, three ReLU activation functions and a Transformer Encoder module. The feature data first passes through a 3×3 convolution and ReLU activation function, then passes through the Transformer Encoder module, and finally passes through two 3×3 convolutions and ReLU activation functions to obtain the final output feature data after feature extraction.
[0134] like Figure 4 As shown, the input of the first upsampling module is the result of the skip connection of the feature data output by the third downsampling module and the feature extraction module, and the output feature data is input into the second upsampling module. The input of the second upsampling module is the result of the skip connection of the feature data output by the second downsampling module and the first upsampling module, and the output feature data is input into the third upsampling module. The input of the third upsampling module is the result of the skip connection of the feature data output by the first downsampling module and the second upsampling module, and the output feature data is input into a 3×3 convolution, and the prediction result of the industrial CT image denoising model is output after convolution. After the data of the upsampling module, the height and width of the output feature are both twice that of the input, and the Transformer Encoder module is used in each upsampling module, which increases the context feature extraction and learning capabilities of the deep learning neural network model.
[0135] The upsampling module is similar to the downsampling structure, using a residual connection method, including 3×3 convolution, ReLU activation function, Transpose convolution and Transformer Encoder modules. The feature data first passes through a Transpose convolution and a ReLU activation function, and then passes through a 3×3 convolution and ReLU activation function. After that, it passes through a Transformer Encoder module to further fuse the features, and then passes through a 3×3 convolution and ReLU activation function and is added to the feature data after the first ReLU activation function of the residual connection to obtain the final output. Transpose convolution is a transposed convolution, which mainly serves to increase the height and width of the feature data by 2 times.
[0136] The Transformer Encoder module structure is as follows Figure 5 As shown in the figure, the input feature map is encoded by feature embedding, and further learned by Layer Norm and Multi-Head Attention. Here, a residual connection is performed to fuse the features; the fused features are passed through LayerNorm and MLP (Multi-layer Perceptron) and then a residual connection is performed to fuse the features to obtain the result of the Transformer Encoder module. Combining Transformer with CNN not only has the super strong feature extraction ability of CNN for images, but also strengthens the model's feature fusion ability for image context.
[0137] The above descriptions are only optional embodiments of the present invention, and are not intended to limit the patent scope of the present invention. All equivalent structural changes made using the contents of the present invention's specification and drawings, or directly / indirectly applied in other related technical fields, are included in the patent protection scope of the present invention.
Claims
1. An image denoising method, characterized in that: The steps include: Build a deep learning neural network model and complete training; Collecting 3D CT image data to be denoised, splitting the collected 3D CT image data into N layers of 2D image data, inputting them into a trained deep learning neural network model, and outputting the denoised results of the N layers of 2D image data; re-joining the denoised two-dimensional image data into three-dimensional image data; The reassembled three-dimensional image data is further processed to obtain the final denoised three-dimensional CT image data.
2. The image denoising method according to claim 1, characterized in that: Building a deep learning neural network model and completing training specifically includes the following steps: Data acquisition step: acquiring multiple pairs of three-dimensional CT image data, wherein each pair of three-dimensional CT image data includes a low-quality three-dimensional CT image data and a high-quality three-dimensional CT image data of the same object; The data preprocessing step is to perform data preprocessing on the plurality of pairs of the three-dimensional CT image data to obtain a preprocessed paired three-dimensional CT image data set (lq′ i , hq′ i ), where lq′ i is the i-th low-quality 3D CT image data, hq′ i is the i-th high-quality three-dimensional CT image data; Network model training steps: Use the paired three-dimensional CT image data set to train the deep learning neural network model to obtain a trained deep learning neural network model.
3. The image denoising method according to claim 2, characterized in that: The data preprocessing steps specifically include the following steps: Filling multiple pairs of the three-dimensional CT image data to obtain a height and a width of a preset fixed multiple; Adjusting the window width and window level of the filled three-dimensional CT image data to obtain a preprocessed paired three-dimensional CT image data set; The reverse processing of the reassembled 3D image data includes the following steps: Performing a padding removal operation on the reassembled three-dimensional image data to obtain three-dimensional image data having an original height and width; The window width and window position are restored for the three-dimensional image data with the original height and width to obtain the final denoised three-dimensional CT image data.
4. The image denoising method according to claim 3, characterized in that: In the data preprocessing step, a normalization operation is performed on the adjusted three-dimensional CT image data to obtain a preprocessed paired three-dimensional CT image data set; in the inverse processing of the preprocessing of the re-spliced three-dimensional image data, a denormalization operation is performed on the three-dimensional image data with the original height and width to obtain denormalized three-dimensional image data, and then the window width and window level of the denormalized three-dimensional image data are restored to obtain the final denoised three-dimensional CT image data.
5. The image denoising method according to claim 2, characterized in that: The training method of the deep learning neural network model includes: The preprocessed paired 3D CT image dataset (lq′ i , hq′ i ) is split into N layers of two-dimensional image data, and the high-quality two-dimensional image data of each layer corresponds to the low-quality two-dimensional image data, and a two-dimensional aligned data set (lq′ ij , hq′ ij ), where lq′ ij is the low-quality 3D CT image data corresponding to the low-quality 2D CT image data at the jth layer, hq′ i The i-th high-quality three-dimensional CT image data corresponds to the high-quality two-dimensional CT image data at the j-th layer; The low-quality two-dimensional CT image data is used as the input of the industrial CT image denoising deep neural network, and the high-quality two-dimensional CT image data is used as the corresponding standard to perform feature recognition on the low-quality two-dimensional CT image data; The Adam optimization algorithm is used, the initial learning rate is set to 0.0001, and the cosine annealing algorithm is used to adjust the learning rate.
6. The image denoising method according to claim 5, characterized in that: The loss function of the deep learning neural network model is based on L1Loss. First, the input preprocessed paired three-dimensional CT image data set is automatically threshold segmented using the OTSU algorithm to obtain the segmentation results of the foreground and background. L1Loss is calculated for the foreground and background respectively, and the losses of the foreground and background are weighted to obtain the final loss function.
7. The image denoising method according to claim 6, characterized in that: The deep learning neural network model includes a downsampling module, a feature extraction module and an upsampling module; the output of the downsampling module is connected to the upsampling module and the feature extraction module; the output of the feature extraction module is connected to the input of the upsampling module; A downsampling module is used to reduce the height and width of the input three-dimensional CT image data, the input is the preprocessed paired three-dimensional CT image data set, and according to the pairing relationship in the paired three-dimensional CT image data set, each high-quality three-dimensional CT image data is used as a standard to output feature data of the low-quality three-dimensional CT image data corresponding to the high-quality three-dimensional CT image data; A feature extraction module is used to enhance the ability of the deep learning neural network model to capture the global field of view, input the feature data of the low-quality three-dimensional CT image data output by the downsampling module, and output the feature data of the low-quality three-dimensional CT image data after feature extraction; The up-sampling module is used to gradually restore the image denoising result and output feature data of the low-quality three-dimensional CT image data corresponding to the high-quality three-dimensional CT image data, wherein the magnification factor is the same as the reduction factor of the down-sampling module.
8. The image denoising method according to claim 7, characterized in that: The upsampling module includes a TransformerEncoder module. The input feature map is feature-encoded through feature embedding, and then the features are further learned through layer normalization and multi-head attention. The results after layer normalization and multi-head attention learning are subjected to a residual connection fusion feature. The fused features are subjected to layer normalization and a multi-layer perceptron and then subjected to a residual connection fusion feature to obtain the data processing result of the Transformer Encoder module.
9. The image denoising method according to claim 6, characterized in that: The input preprocessed paired 3D CT image dataset is convolved with the result of automatic threshold segmentation and superposition connection by the OTSU algorithm and then input into the downsampling module.
10. An image denoising device, characterized in that: include: An image denoising deep learning module receives the three-dimensional CT image data to be denoised, performs denoising processing on the three-dimensional CT image data using the deep learning neural network model trained in the image denoising method according to any one of claims 1 to 8, and outputs the denoised N-layer two-dimensional CT image data; A data acquisition module is used to acquire three-dimensional CT image data and input it into an image denoising deep learning module; The image processing module is used to process the denoised N-layer two-dimensional image data output by the image denoising deep learning module to generate denoised three-dimensional CT image data.