A two-stage bone imaging denoising method integrating perceptual loss and dense residual attention convolutional blocks

Abstract

The present invention discloses a two-stage bone imaging denoising method that integrates perceptual loss and dense residual attention convolution blocks, aiming at the problem that the signal-to-noise ratio and resolution of bone imaging are low, which affect the doctor's diagnosis and the automatic detection of lesions. Taking dense residual attention convolution blocks as the basic convolution units, a two-stage bone imaging denoising network combined with dense residual attention convolution blocks is proposed. First, the noise level of bone imaging is estimated by a noise estimation network, and then the designed backbone denoising network is used to obtain the denoised bone imaging. Finally, the fused perceptual loss mixed loss function is used to optimize the bone imaging denoising effect. The bone imaging denoising method disclosed by the present invention can retain the detailed features of the lesions in the bone imaging while effectively reducing the noise of the bone imaging.

Description

Technical Field

[0001] The present invention relates to the field of deep learning, and in particular to a two-stage bone imaging denoising method that integrates perceptual loss and dense residual attention convolution blocks. Background Art

[0002] Traditional medical image denoising algorithms are primarily categorized into three types: iterative reconstruction algorithms, projection domain algorithms, and image post-processing algorithms. While these denoising methods can reduce image noise to a certain extent, they are limited by factors such as device computing power and projection data, and thus cannot achieve satisfactory results.

[0003] In recent years, with the rapid development of deep learning, many scholars have begun to apply it to the task of medical image denoising. References: "Chen H, Zhang Y, Zhang W H. Low-dose CT via convolutional neural network[J]. Biomedical Optics Express, 2017, 8(2): 679-694." The convolutional neural network is trained on low-dose CT images in a block-by-block manner. Compared with the traditional algorithm, this method has greatly improved in various evaluation indicators; References: "Kang E, Min J, Ye J C. A deep convolutional neural network using directional wavelets for low-dose X-ray CT reconstruction[J]. Medical Physics, 2017, 44(10): 360-375." The deep convolutional neural network is used to perform wavelet transform on low-dose CT images to extract the directional component of the artifact, thereby suppressing the noise unique to CT images; References: "Su Ning, Ye Hanming, Wang Xiaoyao. Research on multi-sensor docking posture fusion algorithm based on Adam optimization[J]. Modern Manufacturing Technology and Equipment, 2022, 58(2): 35-37."; References: "Keisuke U, Koichi O, Masami G, et al. Quantitative evaluation of deep convolutional neural network-based image denoising for low-dose computed tomography [J]. Visual computing for industry, biomedicine, and art, 2021, 4 (1): 21-26. "A denoising convolutional neural network was used to reduce the noise of CT images while maintaining image clarity. The image was prevented from being over-smoothed by cropping the CNN model. Currently, deep learning-based denoising algorithms mainly focus on MRI or CT images, and there is very little research on denoising methods for SPECT bone imaging. Summary of the Invention

[0004] The present invention discloses a two-stage bone imaging denoising method that integrates perceptual loss and dense residual attention convolution blocks, comprising the following steps: (1) dividing a training set and a test set; (2) combining dense connections, residual connections and a coordinated attention mechanism to propose a dense residual attention convolution block; (3) using the dense residual attention convolution block as a basic unit, constructing a two-stage SPECT denoising network based on U-Net; (4) optimizing the denoising effect of the backbone denoising network using a hybrid loss function that integrates perceptual loss; (5) training a network model and saving the parameters of the model after training; (6) using the saved final model to remove bone imaging noise and output the denoised bone image. The bone imaging denoising method disclosed by the present invention can effectively reduce the noise in the bone imaging image while retaining the detailed features of the SPECT bone imaging lesion.

[0005] The technical solution provided by the present invention is: a two-stage bone imaging denoising method integrating perceptual loss and dense residual attention convolution blocks, characterized by comprising the following steps:

[0006] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0007] A two-stage bone imaging denoising method integrating perceptual loss and dense residual attention convolutional blocks is proposed, wherein the calculation method comprises the following steps:

[0008] Step 1: Use the preprocessed bone images and the corresponding noisy bone images as the training set and test set, and divide them into the training set and test set. The specific steps are as follows:

[0009] (1) Converted into grayscale images through normalization, each case contains two images of the front and back of the human body;

[0010] (2) To improve the signal-to-noise ratio of bone imaging, the bone imaging shoulder to pelvic region is obtained by cropping, and the image is adjusted to N × N pixel size, where N is a positive integer;

[0011] (3) The original image and the noisy bone image are used as a set of data, and the preprocessed grayscale data are used as labels, and divided into a training set and a test set;

[0012] Step 2: Construct a dense residual attention convolution block. The entire dense residual attention convolution block consists of 1×1, 3×3 convolution, ReLU activation function and coordinated attention mechanism. The calculation structure of the convolution block is as follows:

[0013] x1=σ(Conv 3×3 (x in ))

[0014] x2=σ(Conv 1×1 (concat(x1,xin ))

[0015] x3=σ(Conv 3×3 (x2))

[0016] x out =CA(x in +x3+x2)

[0017] Where: x in represents the bone imaging of the input convolutional block, x out Represents the bone imaging after the dense residual attention convolution block calculation, that is, the bone imaging feature map; σ() represents the BN batch normalization and ReLU nonlinear transformation; Conv 3×3 (), Conv 1×1 () represent 3×3 convolution and 1×1 convolution operations respectively; concat represents feature concatenation, and x1 is set to c1×h×w, x in is c2×h×w, concat(x1,x in ) is calculated as (c1+c2)×h×w; CA stands for coordinated attention mechanism. Assuming that the size of the two input features is c×h×w, the calculation process is as follows:

[0018]

[0019]

[0020] Where: c represents the number of channels; h represents the height; w represents the width; x represents the width. c (h,i) indicates the i-th pixel when the channel is c and the height is h; x c (j,w) represents the jth pixel when the channel is c and the width is w; Indicates that in channel c, the feature information of the wth row of height h is summed and averaged; The calculation method is the same;

[0021]

[0022] Where: F1(·) represents the 1×1 convolution operation after feature concatenation; δ(·) represents the nonlinear transformation, which is the combination of batch normalization and PReLU activation function;

[0023]

[0024]

[0025] Where: f h 、f w They represent the vectors of f after dimensionality reduction along the h and w directions respectively; μ(·) represents the Sigmoid activation function operation;

[0026]

[0027] Where: x c (i,j) represents the bone image of the input coordinated attention mechanism; x' c (i, j) represents the bone image after calculation by the coordinated attention mechanism, that is, the bone image feature map;

[0028] Step 3: The denoising network is mainly composed of the dense residual attention convolution blocks proposed in step 2. There are 2*(n+m) convolution blocks in total, where n and m are both positive integers greater than 0. n is the number of U-Net layers in the noise estimation network, and m is the number of U-Net layers in the backbone denoising network. The overall denoising model is designed as a two-stage architecture consisting of a noise estimation network and a backbone denoising network. The noise estimation network outputs the bone image noise level, and the backbone denoising network outputs the final denoised bone image.

[0029] Step 4: Use the hybrid loss function of cross entropy loss and fusion perception loss to calculate the output losses of the noise estimation network and the backbone denoising network respectively, and use the Adam optimizer to minimize the loss function;

[0030] The cross entropy loss function calculation formula is:

[0031]

[0032] Where: c k represents the kth bone image, P(c k ) represents the actual noise level, q(c k ) represents the model prediction noise level;

[0033] The hybrid loss function of the fusion perceptual loss combines the perceptual loss function with the mean square error:

[0034] The formula for calculating the mean square error is:

[0035]

[0036] Where: w, h represent the width and height of bone imaging; y i 、x i represent the noise-reduced bone image and the original image, respectively;

[0037] The perceptual loss calculation formula is:

[0038]

[0039] Where: Represents the image features extracted by the deep learning model; Represents the original bone imaging features extracted by the deep learning model; represents the denoised bone imaging features extracted by the deep learning model; h represents height; w represents width;

[0040] L H =(1-λ)L mse +λL prec ,0≤λ≤1

[0041] Where: λ is a constant used to balance the two loss functions; L mse represents the mean square error loss function, which is used to guide the model to perform pixel-by-pixel denoising; L prec Represents the perceptual loss function, which is used to mse Constrain and preserve the detailed features of bone imaging;

[0042] Step 5: The test set data preprocessed in step 1 is fed into the network model built in step 3. The loss value is calculated using the hybrid loss function designed in step 4 that integrates the perceptual loss. The noise reduction effect of the model is evaluated using the peak signal-to-noise ratio. The final network model is saved based on the optimal parameters and recorded as the EMANet model.

[0043] The peak signal-to-noise ratio formula is:

[0044]

[0045] Where: W×H represents the size of bone imaging; MAX I Indicates the maximum pixel value in bone imaging;

[0046] Step 6: Use the saved EMANet model to output the final denoised bone image.

[0047] The present invention provides a two-stage bone imaging denoising method that integrates perceptual loss and dense residual attention convolution blocks. With dense residual attention convolution blocks as the basic convolution unit, a two-stage bone imaging denoising network combined with dense residual attention convolution blocks is proposed. First, the noise level of the bone image is estimated by a noise estimation network, and then the designed backbone denoising network is used to obtain the denoised bone image. Finally, the fused perceptual loss mixed loss function is used to optimize the bone imaging denoising effect. The bone imaging denoising method disclosed by the present invention can effectively reduce the bone imaging noise while retaining the detailed features of the bone imaging lesions.

[0048] Beneficial effects:

[0049] Compared with the current mainstream medical impact denoising methods, this invention has the following beneficial effects:

[0050] (1) A two-stage bone imaging denoising method that integrates perceptual loss and dense residual attention convolution blocks can effectively reduce bone imaging noise while preserving lesion detail features;

[0051] (2) Compared with conventional medical denoising methods, the model has a stronger ability to learn noise features and has a more obvious advantage in denoising due to the use of dense residual attention convolution blocks for feature extraction, a two-stage denoising architecture based on U-Net, and a hybrid loss function that integrates perceptual loss; BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a flow chart of the present invention; Poisson noise is added to bone imaging to obtain a noisy image, the noisy image is sent to a noise estimation network to obtain a bone imaging noise level, the noisy image and the bone imaging noise level are sent together to a backbone denoising network to obtain a denoised bone image, and a cross entropy and a hybrid loss function are used to optimize the training process;

[0053] Figure 2 It is the dense residual connection structure of the present invention; 3×3Conv represents 3×3 convolution; 1×1Conv represents 1×1 convolution; BN represents batch normalization; CA represents coordinated attention mechanism;

[0054] Figure 3 The overall structure of the noise reduction network of the present invention is as follows: Poisson noise of different intensities is added to the bone image to obtain a noisy image, which is then fed into the noise estimation network to obtain the bone image noise level. The noisy image and the bone image noise level are then fed into the backbone denoising network to obtain a denoised bone image.

[0055] Figure 3 The symbols are explained as follows:

[0056] Block diagram (a): Noise estimation network;

[0057] Block diagram (b): Backbone denoising network;

[0058] Skip connection: skip connection operation;

[0059] RDC Blcok: Dense Residual Attention Convolution Block;

[0060] Maxpooling: maximum pooling layer;

[0061] Up-sampling: up sampling;

[0062] Concat: feature concatenation operation;

[0063] Add: feature fusion operation;

[0064] Figure 4Comparison of the visual effects of noise reduction using the present invention and other medical noise reduction network models; Figure (a) shows the noise reduction effect at the scapula in bone imaging; Figure (b) shows the noise reduction effect at the sternum in bone imaging; Figure (c) shows the noise reduction effect at a small rib lesion in bone imaging; DETAILED DESCRIPTION

[0065] The present invention will be described in detail below with reference to the accompanying drawings. Figure 1 The present invention is implemented as follows:

[0066] Step 1: Preprocess the bone image. The specific processing method is as follows:

[0067] (1) Normalize the bone image and convert the original bone image file into a visible grayscale image;

[0068] (2) adding different levels of Poisson noise to the bone images and resizing the images to 256 × 256 pixels;

[0069] (3) The dataset is divided into a training set and a test set, where the training set contains 13,500 images.

[0070] The test set contains 320 images;

[0071] Step 2: Construct a dense residual attention convolution block. The entire dense residual attention convolution block consists of 1×1, 3×3 convolution, ReLU activation function and coordinated attention mechanism. The calculation structure of the convolution block is as follows:

[0072] x1=σ(Conv 3×3 (x in ))

[0073] x2=σ(Conv 1×1 (concat(x1,x in ))

[0074] x3=σ(Conv 3×3 (x2))

[0075] x out =CA(x in +x3+x2)

[0076] Where: x in represents the bone image of the input convolutional block, x out Represents the bone imaging after the dense residual attention convolution block calculation, that is, the bone imaging feature map; σ() represents the BN batch normalization and ReLU nonlinear transformation; Conv 3×3 (), Conv 1×1() represent 3×3 convolution and 1×1 convolution operations respectively; concat represents feature concatenation, and x1 is set to c1×h×w, x in is c2×h×w, then concat(x1,x in ) is calculated as (c1+c2)×h×w; CA stands for coordinated attention mechanism. Assuming that the size of the two input features is c×h×w, the calculation process is as follows:

[0077]

[0078]

[0079] Where: Indicates that in channel c, the feature information of the wth row of height h is summed and averaged; The calculation method is the same;

[0080]

[0081] Where: F1(·) represents the 1×1 convolution operation after feature concatenation; δ(·) represents the nonlinear transformation, which is the combination of batch normalization and PReLU activation function;

[0082]

[0083]

[0084] Where: f h 、f w They represent the vectors of f after dimensionality reduction along the h and w directions respectively; μ(·) represents the Sigmoid activation function operation;

[0085]

[0086] Where: x c (i,j) represents the bone image of the input coordinated attention mechanism; x' c (i, j) represents the bone image after calculation by the coordinated attention mechanism, that is, the bone image feature map;

[0087] Step 3: The overall denoising model is designed as a two-stage architecture consisting of a noise estimation network and a backbone denoising network. It consists of the dense residual attention convolutional blocks proposed in Step 2 and contains a total of 16 convolutional blocks. The number of U-Net layers in the noise estimation network is 3, and the number of U-Net layers in the backbone denoising network is 5. The noise estimation network outputs the bone image noise level, and the backbone denoising network outputs the final denoised bone image.

[0088] Step 4: Calculate the output loss of the noise estimation network through the cross entropy loss function and the output loss of the backbone denoising network through the hybrid loss function of the fused perceptual loss. Use the Adam optimizer to minimize the loss function.

[0089] The cross entropy loss function calculation formula is:

[0090]

[0091] Where: c k represents the kth bone image, P(c k ) represents the actual noise level, q(c k ) represents the model prediction noise level;

[0092] Among them, the hybrid loss function of the fusion perceptual loss combines the perceptual loss function with the mean square error:

[0093] The formula for calculating the mean square error is:

[0094]

[0095] Where: w and h represent the width and height of the bone image. In this example, w and h are both 255; y i 、x i represent the noise-reduced bone image and the original image, respectively;

[0096] The perceptual loss calculation formula is:

[0097]

[0098] Where: Represents the image features extracted by the VGG-16 model; Represents the original bone imaging features extracted by the VGG-16 model; represents the denoised bone imaging features extracted by the VGG-16 model; h represents height; w represents width;

[0099] L H =(1-λ)L mse +λL prec ,0≤λ≤1

[0100] Where: λ is a constant, and in this example, λ is 0.1; L mse represents the mean square error loss function, which is used to guide the model to perform pixel-by-pixel denoising; L prec Represents the perceptual loss function, which is used to mse Constrain and preserve the detailed features of bone imaging;

[0101] Step 5: Input the test set preprocessed in step 1 into the network built in steps 2 and 3, and calculate the loss value using the loss function set in step 4. Evaluate the noise reduction effect of the model using the peak signal-to-noise ratio. Save the final network model based on the optimal parameters and record it as the EMANet model:

[0102] The peak signal-to-noise ratio calculation formula is:

[0103]

[0104] Step 6: Use the saved EMANet model to output the final denoised bone image. Figure 4 The following are the noise reduction effect diagrams of each model.

[0105] The effects of the implementation method of the present invention are demonstrated in the following table: The noise reduction effects of various models and the method of the present invention on bone imaging are shown in Table 1:

[0106] Table 1 Peak signal-to-noise ratio values ​​of different algorithms

[0107]

[0108]

[0109] As can be seen from the above table, the two-stage bone imaging denoising method proposed in the present invention, which integrates perceptual loss and dense residual attention convolution blocks, has the best denoising effect.

Claims

1. A two-stage bone imaging denoising method that integrates perceptual loss and dense residual attention convolution blocks, characterized by The following steps are involved: Step 1: Preprocess the bone image. The specific processing method is as follows: (1) Normalize the bone image and convert the original bone image file into a visible grayscale image; (2) adding different levels of Poisson noise to the bone image and resizing the image to N × N pixel size, where N is a positive integer; (3) Divide the preprocessed bone imaging dataset into a training set and a test set; Step 2: Construct a dense residual attention convolution block. The entire dense residual attention convolution block consists of 1×1, 3×3 convolution, ReLU activation function and coordinated attention mechanism. The calculation structure of the convolution block is as follows: x1=σ(Conv 3×3 (x in )) x2=σ(Conv 1×1 (concat(x1,x in )) x3=σ(Conv 3×3 (x2)) x out =CA(x in +x3+x2) Where: x in represents the bone imaging of the input convolutional block, x out Represents the bone imaging after the dense residual attention convolution block calculation, that is, the bone imaging feature map; σ() represents the BN batch normalization and ReLU nonlinear transformation; Conv 3×3 (·), Conv 1×1 (·) represents 3×3 convolution and 1×1 convolution respectively; concat represents feature concatenation, and x1 is set to c1×h×w, x in is c2×h×w, then concat(x1,x in ) is calculated as (c1+c2)×h×w; CA stands for coordinated attention mechanism. Assuming that the size of the two input features is c×h×w, the calculation process is as follows: Where: c represents the number of channels; h represents the height; w represents the width; x represents the width. c (h,i) indicates the i-th pixel when the channel is c and the height is h; x c (j,w) represents the jth pixel when the channel is c and the width is w; Indicates that in channel c, the feature information of the wth row of height h is summed and averaged; The calculation method is the same; Where: F1(·) represents the 1×1 convolution operation after feature concatenation; δ(·) represents the nonlinear transformation, which is the combination of batch normalization and PReLU activation function; Where: f h 、f w They represent the vectors of f after dimensionality reduction along the h and w directions respectively; μ(·) represents the Sigmoid activation function operation; Where: x c (i,j) represents the bone image of the input coordinated attention mechanism; x' c (i, j) represents the bone image after calculation by the coordinated attention mechanism, that is, the bone image feature map; Step 3: The overall denoising model is designed as a two-stage architecture consisting of a noise estimation network and a backbone denoising network. It is composed of the dense residual attention convolutional blocks proposed in Step 2 and contains a total of 2*(n+m) convolutional blocks, where n and m are both positive integers greater than 0. n is the number of U-Net layers in the noise estimation network, and m is the number of U-Net layers in the backbone denoising network. The noise estimation network outputs the bone image noise level, and the backbone denoising network outputs the final denoised bone image. Step 4: Use the training set obtained in step 1 to train the network model built in step 3. Use a hybrid loss function of cross entropy loss and fusion perception loss to calculate the output loss of the noise estimation network and the backbone denoising network, and use the Adam optimizer to minimize the loss function. The cross entropy loss function calculation formula is: Where: c k represents the kth bone image, P(c k ) represents the actual noise level, q(c k ) represents the model prediction noise level; The calculation formula of the hybrid loss function of fusion perception loss is as follows: THE H =(1-λ)L mse +λL prec ,0≤λ≤1 Where: λ is a constant used to balance the two loss functions; L mse represents the mean square error loss function, which is used to guide the model to perform pixel-by-pixel denoising; L prec Represents the perceptual loss function, which is used to mse Constrain and preserve the detailed features of bone imaging; The formula for calculating the mean square error is: Where: w, h represent the width and height of the bone image, both are positive integers; y i 、x i represent noise-reduced bone imaging and native bone imaging, respectively; The perceptual loss calculation formula is: Where: Represents the image features extracted by the deep learning model; Represents the original bone imaging features extracted by the model; represents the denoised bone imaging features extracted by the model; h represents height; w represents width; Step 5: The test set data preprocessed in step 1 is fed into the network model built in step 3. The loss value is calculated using the hybrid loss function designed in step 4, and the noise reduction effect of the model is evaluated by the peak signal-to-noise ratio. The final network model is saved according to the optimal parameters and recorded as the EMANet model. The peak signal-to-noise ratio calculation formula is: Where: MAX I Indicates the maximum pixel value in bone imaging; Step 6: Use the saved EMANet model to output the final denoised bone image.

Images