Ultrasonic image denoising method and system based on improved diffusion model

Abstract

The invention discloses an ultrasonic image denoising method and system based on an improved diffusion model, belongs to the technical field of image processing, and ingeniously fuses the advantages of a diffusion model (DM) and a generative adversarial network (GAN). And aiming at the bottleneck of low DM sampling speed, the inverse process is innovatively replaced by the GAN, so that the processing efficiency is remarkably improved. Meanwhile, a generator and a discriminator in the model are both based on a U-Net architecture, and a structural feature extraction network is integrated, so that the denoising capability is effectively enhanced, image blurring is reduced, and details are finely reserved. By optimizing a DM sampling mechanism and a GAN training strategy, algorithm operation is accelerated, convergence stability is enhanced, and high efficiency and reliability in practical application are ensured. Besides, the introduced structural feature extraction module ensures that key structural information of the medical image is not lost in the denoising process, the diagnostic value of the denoised image is greatly improved, and a new breakthrough is brought to the field of ultrasonic image analysis.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and more particularly to an ultrasonic image denoising method and system based on an improved diffusion model. Background Art

[0002] In the field of medical imaging, ultrasound examination has been widely used in clinical practice as a non-invasive, economical and efficient diagnostic method. Its high tissue penetration and real-time imaging capabilities provide doctors with rich diagnostic information. However, limited by the diversity of imaging environments and equipment performance, ultrasound images are often interfered by noise to varying degrees, especially the prevalence of speckle noise, which significantly reduces the contrast and clarity of the image and blurs the boundaries of tissue structures, posing a huge challenge to doctors' accurate diagnosis.

[0003] Traditional image denoising methods, such as median filtering, mean filtering and Gaussian filtering, can suppress noise to a certain extent, but these methods often sacrifice image details, resulting in blurred edge information and making it difficult to meet the requirements of medical image analysis for detail accuracy.

[0004] In recent years, with the rapid development of deep learning technology, especially the widespread application of convolutional neural networks (CNN) and generative adversarial networks (GAN), new ideas have been provided for ultrasound image denoising. For example, the DsCNN method combines traditional speckle noise filters with deep learning technology to achieve preprocessing and noise suppression of ultrasound images, and introduces structural similarity metrics as part of the loss function to retain important details of the image. At the same time, GAN-based denoising methods, especially the generator model using a fully convolutional U-Net architecture, can more effectively transmit shallow details to the deep network, thereby protecting tissue structure details while suppressing noise.

[0005] Nevertheless, existing methods are still insufficient in dealing with large-scale and complex noise, and the balance between denoising effect and retaining edge information needs to be solved urgently. As an emerging technology in the field of deep learning, the diffusion model (DM) has shown great potential in the field of image denoising with its excellent sample generation quality and training stability. However, the high computational cost and slow sampling speed of the diffusion model limit its popularity in practical applications.

[0006] Therefore, developing a new ultrasound image denoising method and system that can both efficiently denoise and accurately preserve image details is an issue that needs to be urgently addressed by those skilled in the art. Summary of the invention

[0007] In view of this, the present invention provides an ultrasonic image denoising method and system based on an improved diffusion model. By combining the ultrasonic image denoising model of DM and GAN, the sampling speed of the model is improved, and a network module for extracting tissue structure features is added, so that the model can better retain detail information.

[0008] In order to achieve the above object, the present invention adopts the following technical solution:

[0009] An ultrasonic image denoising method based on an improved diffusion model comprises the following steps:

[0010] Acquire an original ultrasound image, and input the original ultrasound image into an improved ultrasound image denoising model for denoising to obtain a denoised ultrasound image;

[0011] The improved ultrasonic image denoising model includes an ultrasonic denoising module and a structural feature extraction module that combine a diffusion model and a generative adversarial network.

[0012] Optionally, the specific process of performing denoising processing using the improved ultrasound image denoising model includes:

[0013] The ultrasonic original image is subjected to a forward denoising process of a diffusion model to generate a noisy image, and the noisy image is input into a generator to generate a denoised image;

[0014] Sampling the denoised image to obtain a sampled image;

[0015] Extracting tissue structure features of the original ultrasound image and the denoised image using a feature extraction network;

[0016] The image at time t-1 in the forward denoising process, the sampled image at the same time, and the tissue structure features are input into the discriminator for discrimination, and a denoised image that meets the output requirements is output.

[0017] Optionally, the process of establishing an ultrasonic denoising module combining a diffusion model and a generative adversarial network includes:

[0018] For the diffusion model, the adversarial loss is used to match the conditional GAN ​​generator p during training. θ (x t-1 |x t ) and the true denoised distribution q(x t-1 |x t ), by minimizing the divergence D at each denoising step adv To define:

[0019]

[0020] Among them, x t is the noise state of the ultrasound image at time t, θ is the basic parameter of the denoising model, is the mathematical expectation;

[0021] To establish adversarial training, the time-dependent discriminator is represented as Among them, φ is the basic parameter, with N-dimensional x t-1 and x t As input, and judge x t-1 Is it x t A denoised version that meets the denoising requirements;

[0022] The training method of the discriminator is:

[0023]

[0024] The total pass of the generator and the discriminator is maximized To train the generator; the loss function for:

[0025]

[0026] Optionally, the structural feature extraction module includes two VGG16 networks, which are used to extract the tissue structure features of the original ultrasound image and the tissue structure features of the denoised image respectively.

[0027] Optionally, the loss function of the structural feature extraction module is:

[0028]

[0029] Among them, f vgg (·) is the function of extracting the network, W represents the network parameters, I n represents the original ultrasound image, f d (I n ) represents the denoised image.

[0030] On the other hand, the present invention provides an ultrasonic image denoising system based on an improved diffusion model, wherein the system is used to implement the ultrasonic image denoising method based on the improved diffusion model, comprising:

[0031] Ultrasonic image acquisition module, used for acquiring original ultrasonic images;

[0032] Ultrasonic denoising module, used for performing noise addition processing, noise reduction processing and noise reduction judgment on the original ultrasonic image;

[0033] The structural feature extraction module is used to extract the tissue structural features of the original ultrasound image and the noisy image respectively.

[0034] Optionally, the ultrasonic denoising module includes:

[0035] A noise adding unit, used for performing forward noise adding processing on the original ultrasonic image by using a diffusion model to generate a noisy image;

[0036] A generator, used for generating a denoised image from the noisy image;

[0037] A sampling module, used for sampling the denoised image to obtain a sampled image;

[0038] The discriminator is used to discriminate the image at time t-1, the sampled image at the same time, and the tissue structure features in the forward denoising process to obtain the denoised ultrasound image.

[0039] Through the above technical solutions, it can be known that compared with the prior art, the present invention discloses an ultrasound image denoising method and system based on an improved diffusion model. By combining the efficient sampling of the improved DM and the stable denoising ability of the enhanced GAN, the present invention can more effectively remove noise from ultrasound images, while reducing image blur and retaining more detail information. The optimized DM sampling mechanism and GAN training strategy significantly improve the processing speed and convergence stability of the algorithm, making the method more reliable and efficient in practical applications. Furthermore, the present invention introduces a structural feature extraction module to ensure that key tissue structure information is not lost or weakened during the denoising process, thereby improving the medical diagnostic value of the denoised image. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0041] Figure 1 Schematic diagram of the overall framework of the improved ultrasonic image denoising model provided by the present invention.

[0042] Figure 2 Visualization of denoising results for different types of ultrasound images. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] The embodiment of the present invention discloses an ultrasonic image denoising method based on an improved diffusion model, comprising the following steps:

[0045] Acquire an original ultrasound image, input the original ultrasound image into an improved ultrasound image denoising model for denoising, and obtain a denoised ultrasound image;

[0046] The improved ultrasound image denoising model includes an ultrasound denoising module and a structural feature extraction module that combine a diffusion model and a generative adversarial network. The ultrasound image denoising model DG-ExNet that combines DM and GAN provided in an embodiment of the present invention uses GAN instead of the inverse process, considering that the sampling speed of DM is slow. At the same time, in order to better retain the detail information, the generator and the discriminator adopt the U-Net structure, and a structural feature extraction network is added.

[0047] like Figure 1 As shown in Figure 1, the ultrasonic denoising process is mainly divided into two parts: the structural feature extraction process in the gray part on the left and the ultrasonic image denoising process on the right. The light orange part is the forward diffusion process, and the light blue part is the reverse process combined with GAN.

[0048] Specifically, the specific process of denoising using the improved ultrasound image denoising model includes:

[0049] The ultrasonic original image is subjected to the forward denoising process of the diffusion model to generate a noisy image, and the noisy image is input into the generator to generate a denoised image;

[0050] Sampling the denoised image to obtain a sampled image;

[0051] The feature extraction network is used to extract the tissue structure features of the original ultrasound image and the denoised image, and the structural loss function is constructed through the L2 norm;

[0052] The image at time t-1 in the forward denoising process, the sampled image at the same time, and the tissue structure features are input into the discriminator for discrimination, and a denoised image that meets the output requirements is output.

[0053] In another embodiment, since conditional GANs have been shown to be able to model complex conditional distributions in the image domain, they are used to approximate the true denoised distribution q(x t-1 |x t ).

[0054] Specifically, the forward diffusion setting is similar to the common diffusion model, with the main assumption that T is assumed to be small (T ≤ 8) and each diffusion step has a larger variance schedule β t For the diffusion model, the adversarial loss is used to match the conditional GAN ​​generator p θ (x t-1 |xt ) and the true denoised distribution q(x t-1 |x t ), by minimizing the divergence D at each denoising step adv To define:

[0055]

[0056] Among them, x t is the noise state of the ultrasound image at time t, θ is the basic parameter of the denoising model, is the mathematical expectation, D adv The goal is to identify the difference between the predicted denoised image and the sampled denoised image.

[0057] To establish adversarial training, the time-dependent discriminator is represented as Among them, φ is the basic parameter, with N-dimensional x t-1 and x t As input, and judge x t-1 Is it x t A denoised version that meets the denoising requirements;

[0058] The discriminator is trained as follows:

[0059]

[0060] Among them, from the generator p θ (x t-1 |x t ) and the fake samples generated from the generator q(x t-1 |x t ), and the first expected value needs to be compared with the true sample generated by the unknown generator q(x t-1 |x t ), so the identity is used to directly calculate the value generation discriminator:

[0061] q(x t-1 |x t ) = ∫dx 0 q(x 0 )q(x t ,x t-1 |x 0 ) = ∫dx 0 q(x 0 )q(x t-1 |x 0 )q(x t |x t-1 );

[0062] By maximizing to train the generator; the total loss function of the generator and the discriminator is for:

[0063]

[0064] In the process of ultrasonic denoising, the generator and discriminator are adjusted to the U-Net structure. This structure is roughly similar to the U-Net network structure used in NCSN++. This U-Net structure has multiple residual network modules and an attention module. The design of the network hyperparameters in the specific experiment is shown in Table 1.

[0065] Table 1 Hyperparameter design of each structure in U-Net network

[0066]

[0067] Furthermore, a time-dependent network structure is designed in the ultrasonic denoising model, using a convolutional network with ResNet blocks. t and t as conditions, trying to distinguish between true and false x t-1 By changing x t and x t-1 Connect as input to implement x t Condition. For all layers, a LeakyReLU activation function with a negative slope of 0.2 is used, and a mini-batch standard deviation layer is used after all ResNet blocks. The specific architecture is shown in Table 2.

[0068] Table 2 Specific architecture of ultrasonic denoising model

[0069]

[0070] Furthermore, in ultrasound images, it is difficult to preserve the details of tissue structure while smoothing uniform areas to suppress speckle noise. Existing methods usually use different metrics or constraints to preserve details and improve the effect of speckle noise suppression. Therefore, it is very important to effectively enhance detail information. In this regard, the embodiment of the present invention proposes to use a VGG network to extract the details of tissue structure of real ultrasound images, and use these details to construct a loss function to fine-tune the speckle noise suppression network, thereby enhancing its ability to protect details.

[0071] It turns out that different convolutional layers are able to capture image features at different semantic levels in convolutional neural networks. These features can be divided into four categories: low-level features, such as object edges, intermediate features, such as edge combinations, etc. High-level features, such as object parts of regions, and complete object features. In the low-level convolutional layers of the network, general features of the image, such as boundaries or textures, can be extracted. As the depth of the network increases, these features are gradually combined to form a representation of the object in the image. At the same time, when extracting boundaries, the network is robust to noise. Therefore, a deep network can be used to extract tissue structure information from ultrasound images in the low-level convolutional layers.

[0072] In order to obtain the prior objective function of tissue structure, it is assumed that the ultrasound image should not lose details after speckle noise suppression, and its tissue structure details should be as consistent as possible with the structure before processing. Therefore, the structural feature extraction module includes two VGG16 networks, which are used to extract the tissue structure features of the original ultrasound image and the tissue structure features of the denoised image respectively. The tissue structure information of the ultrasound image in the output of the fourth convolutional layer in the network structure is extracted.

[0073] According to the fact that the tissue structure should be as similar as possible before and after speckle noise suppression, the loss function of the structural feature extraction module is:

[0074]

[0075] Among them, f vgg (·) is the function of extracting the network, W represents the network parameters, I n represents the original ultrasound image, f d (I n ) represents the denoised image.

[0076] Furthermore, the loss function of the overall model is:

[0077]

[0078] The above loss function consists of two parts, one is the generator network loss function; the other is the organizational structure loss function. ω is used to balance the two parts of the objective function. Based on multiple experiments, the value is set to 0.001. Two organizational structure feature extraction networks are used as feature extractors, and their parameters are fixed.

[0079] Furthermore, in order to further illustrate the effect of the denoising method provided by the present invention, three data sets with different tissue structures are selected in the embodiments of the present invention. The HC18 data set (fetal head circumference ultrasound), CAMUS data set (cardiac ultrasound), and brachial plexus data set are used to compare the denoising method used in the present invention with six other methods, namely three traditional methods: NLM, Bitonic, ASWMean, and four deep learning methods: RED, RDN10, RED-SENet, and GAN-RW. The comparison results are shown in Table 3. The method using deep learning shows better results than the traditional denoising method.

[0080] Table 3 Comparison of denoising effects between the denoising model of the present invention and the traditional denoising model

[0081]

[0082] Figure 2 The denoising visualization results of different types of ultrasound images. The figure on the right of each sub-image is the enlarged effect of the corresponding red box in the figure, which is mainly convenient for observing the details of the image. All methods can reduce the impact of noise to a certain extent. Although NLM was proposed in 2013, due to the similarity of non-local regions, a good regional smoothing effect can be obtained, but the boundaries of tissue structures are relatively blurred. Bitonic and ASWMean retain more edge information than NLM. The four deep learning methods basically remove noise, and there is not much difference in appearance to the naked eye. Relatively speaking, GAN-RW loses more structural information. In terms of visual effects, the present invention retains clear tissue boundaries.

[0083] On the other hand, the present invention provides an ultrasonic image denoising system based on an improved diffusion model, the system is used to implement the above-mentioned ultrasonic image denoising method based on the improved diffusion model, comprising:

[0084] Ultrasonic image acquisition module, used for acquiring original ultrasonic images;

[0085] Ultrasonic denoising module, used for performing noise addition processing, noise reduction processing and noise reduction judgment on the original ultrasonic image;

[0086] The structural feature extraction module is used to extract the tissue structure features of the original ultrasound image and the noisy image respectively.

[0087] Optionally, the ultrasonic denoising module includes:

[0088] A noise adding unit, used for performing forward noise adding processing on the original ultrasound image by using a diffusion model to generate a noisy image;

[0089] A generator, used to generate a denoised image from a noisy image;

[0090] A sampling module, used for sampling the denoised image to obtain a sampled image;

[0091] The discriminator is used to discriminate the image at time t-1, the sampled image at the same time, and the tissue structure characteristics in the forward denoising process to obtain the denoised ultrasound image.

[0092] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0093] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An ultrasonic image denoising method based on an improved diffusion model, characterized in that: The following steps are involved: Acquire an original ultrasound image, and input the original ultrasound image into an improved ultrasound image denoising model for denoising to obtain a denoised ultrasound image; The improved ultrasonic image denoising model includes an ultrasonic denoising module and a structural feature extraction module that combine a diffusion model and a generative adversarial network.

2. The ultrasonic image denoising method based on the improved diffusion model according to claim 1, characterized in that: The specific process of performing denoising processing using the improved ultrasonic image denoising model includes: The ultrasonic original image is subjected to a forward denoising process of a diffusion model to generate a noisy image, and the noisy image is input into a generator to generate a denoised image; Sampling the denoised image to obtain a sampled image; Extracting tissue structure features of the original ultrasound image and the denoised image using a feature extraction network; The image at time t-1 in the forward denoising process, the sampled image at the same time, and the tissue structure features are input into the discriminator for discrimination, and a denoised image that meets the output requirements is output.

3. The ultrasonic image denoising method based on the improved diffusion model according to claim 2, characterized in that: The process of establishing the ultrasonic denoising module combining the diffusion model and the generative adversarial network includes: For the diffusion model, the adversarial loss is used to match the conditional GAN ​​generator p during training. θ (x t-1 |x t ) and the true denoised distribution q(x t-1 |x t ), by minimizing the divergence D at each denoising step adv To define: Among them, x t is the noise state of the ultrasound image at time t, θ is the basic parameter of the denoising model, is the mathematical expectation; To establish adversarial training, the time-dependent discriminator is represented as Among them, φ is the basic parameter, with N-dimensional x t-1 and x t As input, and judge x t-1 Is it x t A denoised version that meets the denoising requirements; The training method of the discriminator is: By maximizing to train the generator; the total loss function of the generator and the discriminator is for:

4. The ultrasonic image denoising method based on the improved diffusion model according to claim 2, characterized in that: The structural feature extraction module includes two VGG16 networks, which are used to extract the tissue structural features of the original ultrasound image and the tissue structural features of the denoised image respectively.

5. The ultrasonic image denoising method based on the improved diffusion model according to claim 4, characterized in that: The loss function of the structural feature extraction module is: Among them, f vgg (·) is the function of extracting the network, W represents the network parameters, I n represents the original ultrasound image, f d (I n ) represents the denoised image.

6. An ultrasonic image denoising system based on an improved diffusion model, characterized in that: The system is used to implement an ultrasonic image denoising method based on an improved diffusion model as described in any one of claims 1 to 5, comprising: Ultrasonic image acquisition module, used for acquiring original ultrasonic images; Ultrasonic denoising module, used for performing noise addition processing, noise reduction processing and noise reduction judgment on the original ultrasonic image; The structural feature extraction module is used to extract the tissue structural features of the original ultrasound image and the noisy image respectively.

7. The ultrasonic image denoising system based on the improved diffusion model according to claim 6, characterized in that: The ultrasonic denoising module comprises: A noise adding unit, used for performing forward noise adding processing on the original ultrasonic image by using a diffusion model to generate a noisy image; A generator, used for generating a denoised image from the noisy image; A sampling module, used for sampling the denoised image to obtain a sampled image; The discriminator is used to discriminate the image at time t-1, the sampled image at the same time, and the tissue structure characteristics in the forward denoising process to obtain the denoised ultrasound image.

Images