A multi-scale ultrasound image denoising method based on SRGAN model
Through the multi-scale ultrasonic image denoising method based on the SRGAN model, the problem of difficulty in removing medical image noise in the prior art is solved, and the image clarity and contrast are enhanced while retaining detailed information, which is enhanced in the diagnostic value of ultrasonic images.
Patent Information
- Application Number
- CN202310790703.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-29
- Publication Date
- 2025-09-02
- Estimated Expiration
- 2043-06-29
AI Technical Summary
Existing image denoising methods are difficult to effectively remove noise while retaining the details of the edges of medical image textures, affecting image clarity and contrast, and hindering doctors from identifying and analyzing pathological features.
Using a multi-scale ultrasonic image denoising method based on the SRGAN model, by constructing an ultrasonic image data set and training the generator and discriminator, image reconstruction and noise reduction are performed using perceived loss and content loss functions, and multi-scale denoising processing is performed in combination with Gaussian pyramid and diffusion coefficient.
While protecting structure and edges, effectively suppress noise, enhance image contrast, improve image quality, and help doctors better analyze ultrasound image content.
Smart Images
Figure CN116777789B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of image processing technology, and particularly relates to a multi-scale ultrasound image denoising method based on an SRGAN model. Background Art
[0002] Medical images carry information about the heart, brain, and nerves, playing a crucial role in capturing internal information about the human body. However, due to the limitations of their inherent imaging principles, medical images are susceptible to noise and artifacts, resulting in unclear images and poor contrast, which can hinder doctors' ability to identify and analyze patient pathological features. Therefore, medical image denoising is an essential preprocessing technique for further processing.
[0003] The primary purpose of image denoising is to enhance the quality of the target image and improve visual quality. This process can be viewed as a process of retrieving the original, clear image from a noisy image. Unlike conventional images, medical images have low contrast and most contain signal-related noise, making it difficult to remove this noise using traditional natural image denoising techniques. The inherent characteristic information contained in medical images must be preserved. Therefore, when researching medical image denoising methods, it is necessary to ensure that these methods not only perform image denoising but also retain detailed information such as image texture and edges, thereby improving the quality of medical images and enhancing visual quality. Furthermore, research on medical image denoising algorithms can promote the development of automated medical image diagnosis technology and reduce the risks of auxiliary diagnosis, thus possessing inestimable value. Summary of the Invention
[0004] In response to the above-mentioned deficiencies in the prior art, the multi-scale ultrasound image denoising method based on the SRGAN model provided by the present invention solves the problem that existing image denoising methods are difficult to accurately reduce the noise of medical images.
[0005] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is: a multi-scale ultrasound image denoising method based on the SRGAN model, comprising the following steps:
[0006] S1. Acquire ultrasound images and construct an ultrasound image dataset;
[0007] S2. Build and train the SRGAN model using ultrasound image datasets.
[0008] S3. Perform multi-scale denoising on the processed ultrasound image based on the trained SRGAN model to obtain the denoised ultrasound image.
[0009] Furthermore, in step S2, the training data of the SRGAN model is an ultrasound image with a magnification factor of 2 between low resolution and high resolution.
[0010] Furthermore, the SRGAN model includes a generator and a discriminator;
[0011] The loss function of the SRGAN model is the perceptual loss, which is expressed as:
[0012]
[0013] Where, l SR is the perceptual loss, To combat losses, is the content loss; among them, the adversarial loss is the loss of the discriminator correctly judging the reconstructed image, and the content loss is the reconstructed image With reference image I HR The Euclidean distance between the feature representations Its expression is:
[0014]
[0015] Where, φ i,j (·) is the feature map obtained by the j-th convolution before the i-th maxpooling layer in the VGG19 network, W i,j and H i,j are the width and height of the corresponding feature map in the VGG19 network, respectively. x and y are the count values of the width and height of the corresponding feature map in the VGG19 network, respectively.
[0016] Furthermore, in step S3, the method of performing multi-scale denoising on the ultrasonic image to be processed by the multi-scale denoising model is specifically as follows:
[0017] S31, performing Gaussian filtering on the ultrasonic image to be processed and then down-sampling the image to be processed to obtain first to third Gaussian pyramid images G1, G2, and G3 respectively;
[0018] S32, performing upsampling processing on the third and second Gaussian pyramid images G3 and G2 respectively, to obtain a second prediction image G'2 and a first prediction image G'1 respectively;
[0019] S33, performing difference operations on the second Gaussian pyramid image G2 and the second prediction image G'2, and on the first Gaussian pyramid image G1 and the first prediction image G'1, respectively, to obtain a second residual image D2 and a first residual image D1 respectively;
[0020] S34, performing denoising on the third Gaussian pyramid image G3 using a denoising model to obtain a third denoised image R3;
[0021] S35. Reconstruct the third denoised image R3 using the trained SRGAN model and a 2x magnification factor to obtain a second reconstructed image S2.
[0022] S36, calculating a second weight according to the homogeneous region and the boundary region of the second reconstructed image S2
[0023] S37, according to The second reconstructed image S2 and the second residual image D2 are added together and subjected to denoising processing by a denoising model to obtain a second denoised image R2;
[0024] S38. Reconstruct the second denoised image R2 using the trained SRGAN model and a 2x magnification factor to obtain a first reconstructed image S1.
[0025] S39, calculating a first weight according to the homogeneous region and the boundary region of the first reconstructed image S1
[0026] S310, according to The first reconstructed image S1 and the first residual image D1 are added together and then subjected to denoising processing by the denoising model to obtain a first denoised image R1 as the denoised ultrasound image output by the multi-scale denoising model.
[0027] Furthermore, the diffusion coefficient of the noise reduction model is:
[0028]
[0029] λ2=α
[0030] Where λ1 and λ2 represent the diffusion coefficients along the tangent direction and the contour direction, respectively; α determines the diffusion amount; β determines the edge enhancement amount; (μ1-μ2) 2 What is measured is the local coherence, s 2 is the stopping level, μ1 and μ2 are the contrast along the gradient and contour directions, respectively, and exp(·) is the exponential function.
[0031] Furthermore, the second weight and the first weight The calculation formula is:
[0032]
[0033] Where, To reconstruct the image S n The weight of , its subscript n=1,2, when n=1, is the first weight corresponding to the first reconstructed image S1. When n=2, is the second weight corresponding to the first reconstructed image S1, softPlus(·) means limiting the f(|max-min|) value to the range of [0-1]. f(|max-min|) is the difference between the maximum and minimum pixel values in the 3×3 area of the reconstructed image, and its absolute value is limited to the range of [0,255]. (p,q) is the pixel value in the reconstructed image.
[0034] Furthermore, the expression for adding the first / second reconstructed image and the first / second residual image is:
[0035]
[0036] Where R n '(p,q) is the reconstructed image S n and the residual image D n The pixel value of the added image, D n (p,q) is the residual image D n The pixel value, S n (p,q) is the reconstructed image S n Pixel value, n=1,2, when n=1, S n is the first reconstructed image, D n is the first residual image, when n=2, S n is the second reconstructed image, D n is the second residual image, and (p,q) is the pixel value.
[0037] The beneficial effects of the present invention are:
[0038] (1) Compared with traditional image denoising methods, the method of the present invention is superior in terms of structure, edge protection and noise suppression;
[0039] (2) The denoised image obtained by the method of the present invention retains more subtle features, suppresses image speckle noise while enhancing contrast, and is more stable, especially in the case of heavy noise;
[0040] (3) From the perspective of visual effects, the method of the present invention achieves better denoising effect than the traditional method, with stronger contrast and more obvious structural details, which is more conducive to doctors analyzing the content of ultrasound images. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1 Flowchart of the multi-scale ultrasound image denoising method based on the SRGAN model provided by the present invention. DETAILED DESCRIPTION
[0042] The specific embodiments of the present invention are described below to facilitate understanding of the present invention by those skilled in the art. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations utilizing the concepts of the present invention are protected.
[0043] The embodiment of the present invention provides a multi-scale ultrasound image denoising method based on the SRGAN model. Figure 1 As shown, the following steps are included:
[0044] S1. Acquire ultrasound images and construct an ultrasound image dataset;
[0045] S2. Build and train the SRGAN model using ultrasound image datasets.
[0046] S3. Perform multi-scale denoising on the processed ultrasound image based on the trained SRGAN model to obtain the denoised ultrasound image.
[0047] In step S1 of the embodiment of the present invention, a real ultrasound image acquired by an ultrasound diagnostic apparatus is used, and the ultrasound image is preprocessed into an image of 512×512 size by resizing, thereby constructing an ultrasound image dataset.
[0048] In step S2 of the embodiment of the present invention, since the downsampling and upsampling in the original Laplacian pyramid model both use a scaling factor of 2, the training data of the SRGAN model of this embodiment uses ultrasound images with a magnification factor of 2 between low resolution and high resolution, so as to ensure that the image size is restored to the input size after being processed by the denoising algorithm.
[0049] Specifically, the SRGAN model in the embodiment of the present invention includes a generator and a discriminator; the generator is responsible for synthesizing high-resolution images, and the discriminator is used to determine whether a given image comes from the generator or a real sample. Through a game-like adversarial process, the generator is able to reconstruct a given low-resolution image into a high-resolution image.
[0050] The generator in this embodiment adopts a deep ResNet architecture (deep ResNet, SRResNet), in which the deep residual module realizes efficient feature extraction and can suppress image noise to a certain extent. At the same time, the trained sub-pixel convolution layer is used to improve the resolution of the input image. In addition to the deep residual module and the sub-pixel convolution layer, a convolution module is added to the input and output parts of the entire generator to realize data adjustment and enhancement.
[0051] The discriminator in this embodiment uses LeakyReLU activation and avoids the maximum pooling problem of the entire network. The middle part uses multiple convolution modules for feature extraction, and the final output is a 1-dimensional tensor.
[0052] In this embodiment, in order to prevent the reconstructed image from being too balanced, the loss function of the SRGAN model is defined as the perceptual loss, which is expressed as follows:
[0053]
[0054] Where, l SR is the perceptual loss, To combat losses, is the content loss; among them, the adversarial loss is the loss of the discriminator correctly judging the reconstructed image, and the content loss is the reconstructed image With reference image I HR The Euclidean distance between the feature representations
[0055] In this embodiment, during the process of determining content loss:
[0056] Assumptions To reconstruct the image is the probability of a natural HR image, and the probability of the discriminator for all training samples is defined as:
[0057]
[0058] Based on this, the content loss is to reconstruct the high-resolution image SR through SRResNet, extract the feature maps of the reference image and the reconstructed image using the VGG19 network, and then construct a loss function using the Euclidean distance between the feature maps. Assume φ i,j represents the feature map obtained by the j-th convolution before the i-th maxpooling layer in the VGG19 network, W i,j and H i,j are the dimensions of the corresponding feature maps in the VGG19 network. Based on this, in this embodiment, the reconstructed image With reference image I HR The Euclidean distance between the feature representations Defined as content loss, its expression is:
[0059]
[0060] Where, φ i,j (·) is the feature map obtained by the j-th convolution before the i-th maxpooling layer in the VGG19 network, W i,j and H i,jare the width and height of the corresponding feature map in the VGG19 network, respectively. x and y are the count values of the width and height of the corresponding feature map in the VGG19 network, respectively.
[0061] In step S3 of the embodiment of the present invention, the method of performing multi-scale denoising on the ultrasonic image to be processed by the multi-scale denoising model is specifically as follows:
[0062] S31, performing Gaussian filtering on the ultrasonic image to be processed and then down-sampling the image to be processed to obtain first to third Gaussian pyramid images G1, G2, and G3 respectively;
[0063] S32, performing upsampling processing on the third and second Gaussian pyramid images G3 and G2 respectively, to obtain a second prediction image G'2 and a first prediction image G'1 respectively;
[0064] S33, performing difference operations on the second Gaussian pyramid image G2 and the second prediction image G'2, and on the first Gaussian pyramid image G1 and the first prediction image G'1, respectively, to obtain a second residual image D2 and a first residual image D1 respectively;
[0065] S34, performing denoising on the third Gaussian pyramid image G3 using a denoising model to obtain a third denoised image R3;
[0066] S35. Reconstruct the third denoised image R3 using the trained SRGAN model and a 2x magnification factor to obtain a second reconstructed image S2.
[0067] S36, calculating a second weight according to the homogeneous region and the boundary region of the second reconstructed image S2
[0068] S37, according to The second reconstructed image S2 and the second residual image D2 are added together and subjected to denoising processing by a denoising model to obtain a second denoised image R2;
[0069] S38. Reconstruct the second denoised image R2 using the trained SRGAN model and a 2x magnification factor to obtain a first reconstructed image S1.
[0070] S39, calculating a first weight according to the homogeneous region and the boundary region of the first reconstructed image S1
[0071] S310, according to The first reconstructed image S1 and the first residual image D1 are added together and then subjected to denoising processing by the denoising model to obtain a first denoised image R1 as the denoised ultrasound image output by the multi-scale denoising model.
[0072] In steps S34, S37 and S37 of this embodiment, the diffusion coefficient of the denoising model for performing denoising on the image is;
[0073]
[0074] λ2=α
[0075] Where λ1 and λ2 represent the diffusion coefficients along the tangent direction and the contour direction, respectively; α determines the diffusion amount; β determines the edge enhancement amount; (μ1-μ2) 2 What is measured is the local coherence, s 2 is the stopping level, μ1 and μ2 are the contrast along the gradient and contour directions, respectively, and exp(·) is the exponential function.
[0076] Among them, λ1 and λ2 determine the diffusion amount of each eigenvector of the structure, β is usually set to 1, which is a corrosion, and its absolute value determines the amount of edge enhancement; in (μ1-μ2) 2 A very small homogeneous region becomes isotropic due to the diffusion of λ1≈λ2=α. Corresponding to (μ1-μ2) 2 >s 2 The fully specular area of is only related to the diffusion in the contour direction. In this case, the diffusion coefficient is negative and the edge is enhanced in the tangential direction. Assuming μ1 ≥ μ2, the consistency measure in the denoising model is defined as (μ1-μ2) 2 , to provide useful information about the coherence of the structure.
[0077] In steps S36 and S39 of this embodiment, the second weight and the first weight S1 The formula for calculating (p,q) is:
[0078]
[0079] Where, To reconstruct the image S n The weight of , its subscript n=1,2, when n=1, is the first weight corresponding to the first reconstructed image S1. When n=2, is the second weight corresponding to the first reconstructed image S1, softPlus(·) means limiting the f(|max-min|) value to the range of [0-1]. f(|max-min|) is the difference between the maximum and minimum pixel values in the 3×3 area of the reconstructed image, and its absolute value is limited to the range of [0,255]. (p,q) is the pixel value.
[0080] In this embodiment, by calculating the above weights, when the reconstructed image and the residual image are added, different weights are assigned to the homogeneous region and the boundary region in the residual image, thereby effectively removing noise while enhancing boundary details.
[0081] In steps S37 and S310 of this embodiment, the expression for adding the first / second reconstructed image and the first / second residual image is:
[0082]
[0083] Where R n '(p,q) is the reconstructed image S n and the residual image D n The pixel value of the added image, D n (p,q) is the residual image D n The pixel value, S n (p,q) is the reconstructed image S n Pixel value, n=1,2, when n=1, S n is the first reconstructed image, D n is the first residual image, when n=2, S n is the second reconstructed image, D n is the second residual image, and (p,q) is the pixel value.
[0084] Specific embodiments are used in the present invention to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present invention.
[0085] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and it should be understood that the scope of protection of the present invention is not limited to such specific descriptions and embodiments. Those skilled in the art can make various other specific variations and combinations based on the technical teachings disclosed in the present invention without departing from the essence of the present invention, and such variations and combinations are still within the scope of protection of the present invention.
Claims
1. A multi-scale ultrasound image denoising method based on the SRGAN model, characterized in that: The following steps are involved: S1. Acquire ultrasound images and construct an ultrasound image dataset; S2. Build and train the SRGAN model using ultrasound image datasets. S3. Perform multi-scale denoising on the processed ultrasound image based on the trained SRGAN model to obtain the denoised ultrasound image. Specifically: S31, performing Gaussian filtering on the ultrasonic image to be processed and then down-sampling processing to obtain the first to third Gaussian pyramid images respectively ; S32, for the third and second Gaussian pyramid images Perform upsampling processing respectively, and obtain the second prediction map and the first prediction graph ; S33, for the second Gaussian pyramid image and the second prediction graph , and the first Gaussian pyramid image and the first prediction graph Perform difference operations between them respectively, and obtain the second residual image and the first residual graph ; S34, the third Gaussian pyramid image is processed by the denoising model G 3. Perform noise reduction processing to obtain the third noise reduction image R 3; S35. Use the trained SRGAN model to use a 2x magnification factor to denoise the third image. R 3. Reconstruct and obtain the second reconstructed image S 2; S36, based on the second reconstructed image S 2 homogeneous areas and boundary areas, calculate the second weight ; S37, according to , the second reconstructed image S 2 and the second residual image D After adding 2, the noise reduction model is used to perform noise reduction processing to obtain the second noise reduction image R 2; S38. Use the trained SRGAN model to use a 2x magnification factor to perform the second denoising on the image. R 2. Reconstruct and obtain the first reconstructed image S 1; S39, based on the first reconstructed image S 1 homogeneous area and boundary area, calculate the first weight ; S310, according to , the first reconstructed image S 1 and the first residual image D 1 After adding, the noise reduction model is used to perform noise reduction processing to obtain the first noise reduction image R 1. Denoised ultrasound image as the output of the multiscale denoising model.
2. The multi-scale ultrasound image denoising method based on the SRGAN model according to claim 1 is characterized in that: In step S2, the training data of the SRGAN model is an ultrasound image with a magnification factor of 2 between low resolution and high resolution.
3. The multi-scale ultrasound image denoising method based on the SRGAN model according to claim 1, characterized in that: The SRGAN model includes a generator and a discriminator; The loss function of the SRGAN model is the perceptual loss, which is expressed as: Where, is the perceptual loss, To combat losses, is the content loss; among them, the adversarial loss is the loss of the discriminator correctly judging the reconstructed image, and the content loss is the reconstructed image With reference image The Euclidean distance between the feature representations , whose expression is: Where, The first i The first layer before the maxpooling layer j The feature map obtained by convolution, and are the width and height of the corresponding feature maps in the VGG19 network, x and y are the count values of the width and height of the corresponding feature map in the VGG19 network, respectively.
4. The multi-scale ultrasound image denoising method based on the SRGAN model according to claim 1, characterized in that: The diffusion coefficient of the noise reduction model is: Where, and denote the diffusion coefficients along the tangent direction and the contour direction, respectively. To determine the amount of diffusion, The amount of edge enhancement is determined by What is measured is the local coherence, is the stop level, and are the contrast along the gradient and contour directions, is an exponential function.
5. The multi-scale ultrasound image denoising method based on the SRGAN model according to claim 1, characterized in that: The second weight and the first weight The calculation formula is: Where, To reconstruct the image The weight of n =1,2, when n =1, The first reconstructed image S 1 corresponds to the first weight, when n =2, The first reconstructed image S The second weight corresponding to 1, Indicates that The value is limited to the range [0-1]. To reconstruct the difference between the maximum pixel value and the minimum pixel value in the 3×3 area of the image, the absolute value is taken to limit the range to [0,255]. is the pixel value.
6. The multi-scale ultrasound image denoising method based on the SRGAN model according to claim 5, characterized in that: The expression for adding the first / second reconstructed image and the first / second residual image is: Where, To reconstruct the image and residual image D n The pixel value of the added image, is the residual image D n The pixel value of To reconstruct the image The pixel value of n =1,2, when n =1, is the first reconstructed image, D n is the first residual image, when n =2, is the second reconstructed image, D n is the second residual image, is the pixel value.