A method and apparatus for enhancing a fatty liver image
By performing grayscale mapping and logarithmic transformation on ultrasound images, multiplicative speckle noise is converted into additive noise. An improved U-Net model is then used for denoising, which solves the problem of noise interference in ultrasound images and improves image quality and diagnostic accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG UNIV
- Filing Date
- 2025-07-25
- Publication Date
- 2026-05-01
AI Technical Summary
Multiplicative speckle noise in ultrasound images leads to low image contrast and blurred tissue boundaries, affecting diagnostic accuracy and automated analysis. Traditional image processing methods are difficult to adapt to individual differences.
A deep learning denoising model based on the U-Net model is adopted to transform multiplicative speckle noise into additive noise through grayscale mapping and logarithmic transformation. The model is then combined with encoder and decoder networks for feature extraction and fusion, preserving liver tissue details and boundary information.
It significantly improves image quality, effectively removes noise, preserves liver tissue details and boundary information to the maximum extent, avoids the detail blurring problem in traditional filtering methods, and improves diagnostic accuracy.
Smart Images
Figure CN120807327B_ABST
Abstract
Description
A method and apparatus for enhancing images of fatty liver. Technical Field
[0001] This invention relates to the field of image processing technology, and specifically to a method and apparatus for enhancing images of fatty liver. Background Technology
[0002] Fatty liver is a common liver disease, often with subtle early symptoms. However, without timely intervention, it can progress to steatohepatitis, liver fibrosis, and even cirrhosis. Therefore, early detection and accurate grading of fatty liver are of great clinical significance. Ultrasound imaging, due to its non-invasive, low-cost, and real-time characteristics, is one of the most widely used imaging methods for fatty liver screening. However, ultrasound images have inherent limitations, such as low image contrast, blurred tissue boundaries, and significant speckle noise interference, which severely affect the diagnostic accuracy of physicians and the automated image analysis capabilities.
[0003] Noise in ultrasound images primarily exists in the form of multiplicative speckle noise. This noise not only masks minute structures in the image but also reduces the expressive power of texture information, posing challenges to image-based feature extraction and fat grading algorithms. Traditional image processing methods, such as median filtering, Gaussian filtering, and anisotropic diffusion, while reducing noise to some extent, often result in a loss of image detail, especially when dealing with complex backgrounds or tissue boundaries, easily leading to structural blurring. Furthermore, these traditional methods generally rely on manually designed parameters, lacking specificity and robustness, and are difficult to adapt to individual differences between images. Summary of the Invention
[0004] The purpose of this invention is to solve the problem of speckle noise interference in ultrasound images mentioned in the background art, and to propose a method and apparatus for enhancing fatty liver images.
[0005] A first aspect of the present invention provides a method for enhancing images of fatty liver, the method comprising:
[0006] Acquire an ultrasound image of the liver; the ultrasound image is a grayscale image;
[0007] The grayscale values of the ultrasound image are mapped to a preset range and then logarithmically transformed to obtain the first target image;
[0008] The first target image is input into a pre-trained denoising model to obtain a second target image; the denoising model is a deep learning model obtained by improving the U-Net model.
[0009] The second target image is post-processed to obtain an enhanced liver image.
[0010] Optionally, the denoising model includes an encoder network and a decoder network, wherein the encoder network consists of a first branch and a second branch; wherein:
[0011] The first branch is used to calculate the grayscale gradient of the second target image to obtain a first edge feature map; and to perform a multi-step downsampling operation on the first edge feature map to obtain edge feature maps of multiple sizes.
[0012] The second branch is used to extract features from the second target image using multi-step convolution, pooling and attention mechanisms, and to receive corresponding edge feature maps at different stages for fusion to obtain global feature maps of multiple sizes;
[0013] The decoder network is used to fuse global feature maps of multiple sizes to obtain a second target image.
[0014] Optionally, the computation process of the decoder network includes:
[0015] The global feature maps of multiple sizes output by the encoder network are fused and restored to their original size to obtain the residual image;
[0016] The residual image is added to the first target image to obtain the second target image.
[0017] Optionally, the first branch includes a convolutional block 0, a downsampling module 1, and a downsampling module 2 connected in sequence; the encoder network includes a convolutional block 1, a base block 1, a downsampling fusion block 1, a base block 2, a downsampling fusion block 2, a base block 3, a downsampling fusion block 3, and a base block 4 connected in sequence; the decoder network includes an upsampling fusion block 1, a base block 5, an upsampling fusion block 2, a base block 6, an upsampling fusion block 3, a base block 7, a convolutional block 2, and a residual fusion block connected in sequence; wherein:
[0018] Upsampling fusion block 1 receives the output of base block 4 as input; there are skip connections between convolution block 0 and downsampling fusion block 1, between downsampling module 1 and downsampling fusion block 2, between downsampling module 2 and downsampling fusion block 3, between base block 3 and upsampling fusion block 1, between base block 2 and upsampling fusion block 2, between base block 1 and upsampling fusion block 3, and between input and residual fusion block.
[0019] Optionally, basic block 1, basic block 2, basic block 3, basic block 4, basic block 5, basic block 6, and basic block 7 have the same structure; the operation process of any basic block includes:
[0020] A convolutional layer with a kernel size of 3*3 is used to perform convolution operations on the input feature map X0 to obtain feature map X1;
[0021] The feature map X1 is non-linearly activated using the GeLU activation function to obtain the feature map X2;
[0022] Global average pooling is performed on feature map X2 to obtain feature vector X3;
[0023] A single-layer neural network is used to map the feature vector X3 to the attention weight vector T;
[0024] The feature map X2 is weighted using a weight vector T to obtain the feature map X4;
[0025] A convolutional layer with a kernel size of 3*3 is used to perform a convolution operation on feature map X4 to obtain feature map X5;
[0026] The feature map X5 and feature map X0 are summed to obtain feature map X6 as the output.
[0027] Optionally, the downsampling fusion block 1, downsampling fusion block 2, and downsampling fusion block 3 have the same structure; the operation process of any downsampling fusion block includes:
[0028] The edge feature map output by the first branch and the global feature map output by the upper-level basic block are concatenated along the channel dimension to obtain feature map Y1;
[0029] A convolutional layer with a kernel size of 1*1 is used to transform the channel dimension of feature map Y1 to obtain feature map Y2;
[0030] Max pooling with a window size of 2*2 and a stride of 2 is used to spatially compress feature map Y2, resulting in feature map Y3 as the output.
[0031] Optionally, the upsampling fusion block 1, upsampling fusion block 2, and upsampling fusion block 3 have the same structure; the operation process of any one of the upsampling fusion blocks includes:
[0032] The feature map output by the upper-level basic block is upsampled to obtain feature map Z1;
[0033] Feature map Z1 and the global feature map output by the encoder network are concatenated along the channel dimension to obtain feature map Z2;
[0034] A convolutional layer with a kernel size of 1*1 is used to transform the channel dimension of feature map Z2, resulting in feature map Z3 as the output.
[0035] Optionally, the convolutional block 1 includes a convolutional layer with a kernel size of 3*3, using GeLU as the activation function.
[0036] Optionally, the convolutional block 2 includes a convolutional layer with a kernel size of 3*3, using Tanh as the activation function.
[0037] A second aspect of the present invention provides an enhancement device for fatty liver images, used to implement the above-described method, the device comprising:
[0038] An image acquisition module is used to acquire ultrasound images of the liver; the ultrasound images are grayscale images.
[0039] The noise transformation module is used to map the grayscale values of the ultrasound image to a preset range and perform a logarithmic transformation to obtain a first target image;
[0040] The model calling module is used to input the first target image into a pre-trained denoising model to obtain the second target image;
[0041] The image restoration module is used to post-process the second target image to obtain an enhanced liver image.
[0042] The beneficial effects of this invention are:
[0043] This invention proposes a method for enhancing images of fatty liver. The method includes: acquiring an ultrasound image of the liver; mapping the grayscale values of the ultrasound image to a preset range and performing a logarithmic transformation to obtain a first target image; inputting the first target image into a pre-trained denoising model to obtain a second target image; and post-processing the second target image to obtain an enhanced liver image.
[0044] By performing grayscale mapping and logarithmic transformation on ultrasound images, multiplicative speckle noise is converted into additive noise, facilitating effective noise removal by a pre-trained denoising model and significantly improving image quality. Simultaneously, the deep learning-based pre-trained denoising model maximizes the preservation of liver tissue details and boundary information during denoising, avoiding the detail blurring problem common in traditional filtering methods. Attached Figure Description
[0045] Figure 1 is a flowchart of an image enhancement method for fatty liver provided by an embodiment of the present invention;
[0046] Figure 2 is a network architecture diagram of a denoising model provided in an embodiment of the present invention;
[0047] Figure 3 is a schematic diagram of the structure of a basic block provided in an embodiment of the present invention;
[0048] Figure 4 is a structural diagram of an enhancement device for fatty liver images provided in an embodiment of the present invention. Detailed Implementation
[0049] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.
[0050] This invention provides a method for enhancing images of fatty liver. Referring to Figure 1, Figure 1 is a flowchart of a method for enhancing images of fatty liver provided by this invention. The method includes the following steps:
[0051] S101, acquire ultrasound images of the liver.
[0052] S102, map the grayscale values of the ultrasound image to a preset range and perform a logarithmic transformation to obtain the first target image.
[0053] S103, input the first target image into the pre-trained denoising model to obtain the second target image.
[0054] S104, Post-process the second target image to obtain an enhanced liver image.
[0055] The ultrasound images are grayscale images.
[0056] The enhancement method for fatty liver images provided in this invention transforms multiplicative speckle noise into additive noise by performing grayscale mapping and logarithmic transformation on the ultrasound image. This facilitates effective noise removal by a pre-trained denoising model, significantly improving image quality. Furthermore, the deep learning-based pre-trained denoising model maximizes the preservation of liver tissue details and boundary information during denoising, avoiding the detail blurring problem common in traditional filtering methods.
[0057] In one embodiment, during the acquisition and imaging process of ultrasound images, due to the influence of the imaging mechanism, a large amount of speckle noise formed by coherent echo interference is usually superimposed on the image. This type of noise significantly reduces the spatial resolution of the image, especially affecting the detail rendering of organs and tissues. An ultrasound image can be considered as being composed of both real tissue reflection signals and noise signals. The mathematical expression model of an ultrasound image can be represented as: Where s(i,j) is the ideal ultrasonic reflection signal; v(i,j) is the multiplicative speckle noise.
[0058] Because multiplicative noise is a type of nonlinear noise, a logarithmic transformation is performed on the model: After the logarithmic transformation, the original multiplicative speckle noise is approximately converted into additive noise that follows a Gaussian distribution, which facilitates subsequent denoising processing using deep learning models.
[0059] In one implementation, the original grayscale values are first mapped to [1, 2]. Then, a logarithmic function with base 2 is used to map the grayscale values to [0, 1], resulting in the first target image. Specifically: Where (i, j) is the pixel position, and x and y are the original grayscale value and the processed grayscale value, respectively.max It is the maximum grayscale value.
[0060] In one embodiment, the denoising model is a deep learning model obtained by improving the U-Net model. In one implementation, see Figure 2, which is a network architecture diagram of a denoising model provided by an embodiment of the present invention.
[0061] The denoising model includes an encoder network and a decoder network. The encoder network consists of a first branch and a second branch; where:
[0062] The first branch is used to calculate the grayscale gradient of the second target image to obtain a first edge feature map; the first edge feature map is then subjected to multiple downsampling operations to obtain edge feature maps of multiple sizes. Specifically, the first branch includes a convolutional block 0, a downsampling module 1, and a downsampling module 2 connected in sequence.
[0063] The second branch is used to extract features from the second target image using multi-step convolution, pooling, and attention mechanisms, and receives corresponding edge feature maps at different stages for fusion to obtain global feature maps of multiple sizes. Specifically, the encoder network includes convolution block 1, base block 1, downsampling fusion block 1, base block 2, downsampling fusion block 2, base block 3, downsampling fusion block 3, and base block 4 connected in sequence.
[0064] The decoder network is used to fuse global feature maps of multiple sizes to obtain a second target image. Specifically, the decoder network's operation includes: fusing the global feature maps of multiple sizes output by the encoder network and restoring them to their original sizes to obtain a residual image; adding the residual image to the first target image to obtain the second target image. Specifically, the decoder network includes, in sequence, an upsampling fusion block 1, a base block 5, an upsampling fusion block 2, a base block 6, an upsampling fusion block 3, a base block 7, a convolutional block 2, and a residual fusion block.
[0065] Among them, the upsampling fusion block 1 receives the output of the base block 4 as input; there are skip connections between the convolution block 0 and the downsampling fusion block 1, between the downsampling module 1 and the downsampling fusion block 2, between the downsampling module 2 and the downsampling fusion block 3, between the base block 3 and the upsampling fusion block 1, between the base block 2 and the upsampling fusion block 2, between the base block 1 and the upsampling fusion block 3, and between the input and the residual fusion block.
[0066] The denoising model provided in this invention involves an edge feature extraction branch (first branch) in the encoder network to calculate the grayscale gradient of the input image, extract edge feature maps, and fuse them with the backbone network (second branch). This edge-guided mechanism enhances image structural information and organizational contours, effectively suppresses detail blurring during denoising, and improves structural fidelity. In the decoder network, the output residual image is added to the first target image to form the final output. This residual learning mechanism allows the model to focus more on learning the distribution and characteristics of noise, improving convergence speed and denoising accuracy, and reducing artifacts and over-smoothing.
[0067] In one implementation, convolutional block 0 uses the Sobel operator to calculate the gradient of the image to obtain an edge feature map. Downsampling module 1 and downsampling module 2 use bilinear interpolation to halve both the width and height of the image.
[0068] In one implementation, convolutional block 1 uses a convolutional layer with a kernel size of 3*3 to process the input image, and uses the GeLU activation function.
[0069] In one implementation, basic block 1, basic block 2, basic block 3, basic block 4, basic block 5, basic block 6, and basic block 7 have the same structure. Referring to Figure 3, Figure 3 is a schematic diagram of the structure of a basic block provided in an embodiment of the present invention. The operation process of any basic block includes:
[0070] Step 1: Perform convolution operation on the input feature map X0 using a convolutional layer with a kernel size of 3*3 to obtain feature map X1.
[0071] Step 2: Apply the GeLU activation function to the feature map X1 nonlinearly to obtain the feature map X2.
[0072] Step 3: Perform global average pooling on feature map X2 to obtain feature vector X3.
[0073] Step 4: Use a single-layer neural network to map the feature vector X3 to the attention weight vector T.
[0074] Step 5: Weight the feature map X2 using the weight vector T to obtain the feature map X4.
[0075] Step 6: Perform convolution operation on feature map X4 using a convolutional layer with a kernel size of 3*3 to obtain feature map X5.
[0076] Step 7: Sum feature map X5 and feature map X0 to obtain feature map X6 as the output.
[0077] This foundational block combines convolution, GeLU activation, attention mechanisms, and residual connections, enabling strong feature extraction and enhancement of key regions. Global average pooling introduces contextual information and adaptively adjusts channel weights, effectively highlighting key structures and suppressing noise interference. Simultaneously, the residual structure helps preserve original features, improving training stability and denoising performance.
[0078] In one implementation, downsampling fusion block 1, downsampling fusion block 2, and downsampling fusion block 3 have the same structure. The operation process of any downsampling fusion block includes:
[0079] Step 1: Concatenate the edge feature map output by the first branch and the global feature map output by the upper-level basic block in the channel dimension to obtain feature map Y1.
[0080] Step 2: Use a convolutional layer with a kernel size of 1*1 to transform the channel dimension of feature map Y1 to obtain feature map Y2.
[0081] Step 3: Spatial compression of feature map Y2 is performed using max pooling with a window size of 2*2 and a stride of 2, resulting in feature map Y3 as the output.
[0082] In one implementation, upsampling fusion block 1, upsampling fusion block 2, and upsampling fusion block 3 have the same structure. The operation process of any upsampling fusion block includes:
[0083] Step 1: Upsample the feature map output by the upper-level basic block to obtain feature map Z1.
[0084] Step 2: Concatenate feature map Z1 and the global feature map output by the encoder network along the channel dimension to obtain feature map Z2.
[0085] Step 3: Use a convolutional layer with a kernel size of 1*1 to transform the channel dimension of feature map Z2, and obtain feature map Z3 as the output.
[0086] In one implementation, convolutional block 2 uses a convolutional layer with a kernel size of 3*3 to process the input feature map, and uses the Tanh activation function.
[0087] It should be noted that, unless otherwise specified, the convolution stride of the convolutional layer is 1 by default, and the activation function uses GeLU by default.
[0088] In one implementation, to verify the effectiveness of the proposed denoising model, the performance of the proposed improved model was compared with that of the mainstream U-Net model. Evaluation metrics included Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM). At the same noise level ( The comparison results are shown in Table 1. Improved Model 3 is the complete denoising model. Improved Model 1, compared to Improved Model 3, removes the first branch. Improved Model 2, compared to Improved Model 3, removes the base block and replaces it with a general two-layer convolution.
[0089] As shown in Table 1, the improved denoising model outperforms the comparative model on all evaluation metrics, demonstrating the effectiveness of the architectural improvement. Ablation experiments show that the first branch and the base block contribute to the improvement of PSNR and SSIM, respectively.
[0090] Table 1:
[0091]
[0092] This invention provides a method for enhancing images of fatty liver. Referring to Figure 4, Figure 4 is a structural diagram of an image enhancement device for fatty liver provided in this invention. The device includes:
[0093] The image acquisition module is used to acquire ultrasound images of the liver.
[0094] The noise transformation module is used to map the grayscale values of the ultrasound image to a preset range and perform a logarithmic transformation to obtain the first target image.
[0095] The model call module is used to input the first target image into the pre-trained denoising model to obtain the second target image.
[0096] The image restoration module is used to post-process the second target image to obtain an enhanced liver image.
[0097] The ultrasound images are grayscale images.
[0098] An enhancement device for fatty liver images provided in this invention converts multiplicative speckle noise into additive noise by performing grayscale mapping and logarithmic transformation on the ultrasound image. This facilitates effective noise removal by a pre-trained denoising model, significantly improving image quality. Furthermore, the deep learning-based pre-trained denoising model maximizes the preservation of liver tissue details and boundary information during denoising, avoiding the detail blurring problem common in traditional filtering methods.
[0099] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.
Claims
1. A method for enhancing images of fatty liver, characterized in that, The method includes: acquiring an ultrasound image of the liver; the ultrasound image is a grayscale image; mapping the grayscale values of the ultrasound image to a preset range and performing a logarithmic transformation to obtain a first target image; inputting the first target image into a pre-trained denoising model to obtain a second target image; the denoising model is a deep learning model improved based on the U-Net model; post-processing the second target image to obtain an enhanced liver image; the denoising model includes an encoder network and a decoder network, the encoder network consisting of a first branch and a second branch; wherein: the first branch is used to calculate the grayscale gradient of the second target image to obtain a first edge feature map; performing multi-step downsampling operations on the first edge feature map to obtain edge feature maps of multiple sizes; the second branch is used to extract features from the second target image using multi-step convolution, pooling, and attention mechanisms, and receiving corresponding edge feature maps at different stages for fusion to obtain global feature maps of multiple sizes; the decoder network is used to fuse global feature maps of multiple sizes to obtain the second target image.
2. The method for enhancing images of fatty liver according to claim 1, characterized in that, The operation process of the decoder network includes: fusing global feature maps of multiple sizes output by the encoder network and restoring them to their original sizes to obtain a residual image; adding the residual image to the first target image to obtain a second target image.
3. The method for enhancing images of fatty liver according to claim 2, characterized in that, The first branch includes a convolutional block 0, a downsampling module 1, and a downsampling module 2 connected in sequence; the encoder network includes a convolutional block 1, a base block 1, a downsampling fusion block 1, a base block 2, a downsampling fusion block 2, a base block 3, a downsampling fusion block 3, and a base block 4 connected in sequence. The decoder network includes an upsampling fusion block 1, a base block 5, an upsampling fusion block 2, a base block 6, an upsampling fusion block 3, a base block 7, a convolutional block 2, and a residual fusion block connected in sequence; wherein: the upsampling fusion block 1 receives the output of the base block 4 as input; there are skip connections between the convolutional block 0 and the downsampling fusion block 1, between the downsampling module 1 and the downsampling fusion block 2, between the downsampling module 2 and the downsampling fusion block 3, between the base block 3 and the upsampling fusion block 1, between the base block 2 and the upsampling fusion block 2, between the base block 1 and the upsampling fusion block 3, and between the input and the residual fusion block.
4. The method for enhancing images of fatty liver according to claim 3, characterized in that, Basic blocks 1, 2, 3, 4, 5, 6, and 7 have the same structure. The operation process of any basic block includes: performing convolution operation on the input feature map X0 using a convolutional layer with a kernel size of 3*3 to obtain feature map X1; performing non-linear activation on feature map X1 using the GeLU activation function to obtain feature map X2; performing global average pooling on feature map X2 to obtain feature vector X3; mapping feature vector X3 to attention weight vector T using a single-layer neural network; weighting feature map X2 using weight vector T to obtain feature map X4; performing convolution operation on feature map X4 using a convolutional layer with a kernel size of 3*3 to obtain feature map X5; summing feature map X5 and feature map X0 to obtain feature map X6 as output.
5. The method for enhancing images of fatty liver according to claim 3, characterized in that, The downsampling fusion block 1, downsampling fusion block 2, and downsampling fusion block 3 have the same structure. The operation process of any downsampling fusion block includes: concatenating the edge feature map output by the first branch and the global feature map output by the upper-level base block in the channel dimension to obtain feature map Y1; performing channel dimension transformation on feature map Y1 using a convolutional layer with a kernel size of 1*1 to obtain feature map Y2; and performing spatial compression on feature map Y2 using max pooling with a window size of 2*2 and a stride of 2 to obtain feature map Y3 as the output.
6. The method for enhancing images of fatty liver according to claim 3, characterized in that, The upsampling fusion block 1, upsampling fusion block 2, and upsampling fusion block 3 have the same structure; The operation process of any upsampling fusion block includes: upsampling the feature map output by the upper-level base block to obtain feature map Z1; concatenating feature map Z1 and the global feature map output by the encoder network in the channel dimension to obtain feature map Z2; and performing channel dimension transformation on feature map Z2 using a convolutional layer with a kernel size of 1*1 to obtain feature map Z3 as the output.
7. The method for enhancing images of fatty liver according to claim 3, characterized in that, The convolutional block 1 contains a convolutional layer with a kernel size of 3*3, using GeLU as the activation function.
8. The method for enhancing images of fatty liver according to claim 3, characterized in that, The convolutional block 2 contains a convolutional layer with a kernel size of 3*3, using Tanh as the activation function.
9. An enhancement device for images of fatty liver, characterized in that, An apparatus for implementing a method for enhancing fatty liver images as described in any one of claims 1-8, the apparatus comprising: an image acquisition module for acquiring an ultrasound image of the liver; the ultrasound image being a grayscale image; a noise transformation module for mapping the grayscale values of the ultrasound image to a preset range and performing a logarithmic transformation to obtain a first target image; a model invocation module for inputting the first target image into a pre-trained denoising model to obtain a second target image; and an image restoration module for post-processing the second target image to obtain an enhanced liver image.