Ultrasonic image enhancement method based on diffusion model

Abstract

The invention relates to an ultrasonic image enhancement method based on a diffusion model. According to the method, a dual-parallel branch network is adopted, a denoising diffusion model is constructed based on a low-frequency signal of wavelet transform, and the quality of an ultrasonic image is effectively improved by utilizing the multi-scale decomposition capability of the wavelet transform and the strong denoising performance of a conditional diffusion model. And the data calculation dimension of the diffusion model is reduced by using wavelet transform, so that the calculation efficiency is improved. And a residual network model for detail recovery is constructed based on the high-frequency signal of wavelet transform as an optional module, so that balance between optimization detail reconstruction and calculation efficiency is realized. The noise mode and important features of the image are learned through the double-frequency-domain hybrid enhancement model, noise suppression and image detail enhancement are realized, and the diagnostic value of the image is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of ultrasonic image enhancement, and in particular relates to an ultrasonic image enhancement method based on a diffusion model. Background Art

[0002] Ultrasound imaging technology is an important medical diagnostic tool that uses the high-frequency sound wave characteristics of ultrasound to obtain images of internal organs in the human body. Due to its non-invasive, real-time and cost-effective advantages, ultrasound imaging has been widely used in clinical diagnosis, such as assisting in kidney puncture and other operations. However, the quality of ultrasound images is affected by many factors, the most important of which is image noise. Ultrasound image noise mainly comes from the following aspects: Scattering noise: When ultrasound passes through human tissue, it will scatter due to the inhomogeneity of the tissue, resulting in scattering noise on the image. Reflection noise: When ultrasound is reflected at the interface of different media, the intensity difference of the reflected signal will produce noise on the image. Electronic noise: The electronic components in the ultrasound device will introduce electronic noise during signal reception and processing. Quantization noise: During the analog-to-digital conversion process, the continuous analog signal is quantized into a discrete digital signal, and this process may introduce quantization noise. Although a variety of denoising techniques have been developed to improve the quality of ultrasound images, these techniques have their own limitations: Spatial filters (such as mean filtering, median filtering, etc.), although they can remove noise, often blur image details, especially in edge areas. Frequency domain filters (such as low-pass filters, high-pass filters, etc.) can remove noise of specific frequencies in the frequency domain, but may lose some frequency components of the image, affecting image quality. Adaptive filters: can adjust the filter strength according to the local characteristics of the image, but the computational complexity is high and the performance is unstable in high-noise environments. Wavelet transform can use the multi-scale analysis capability of wavelet transform for denoising. Although it can better maintain image details, it has limited effect on denoising speckle noise in ultrasound images.

[0003] In order to improve the diagnostic value of ultrasound images, it is necessary to develop new image enhancement technologies that can effectively remove noise from images and preserve the detailed information in the images as much as possible, especially the edge and texture information that is crucial for diagnosis; can quickly process images to meet the needs of real-time clinical diagnosis; can adapt to different image qualities and noise levels, and have good versatility and robustness. Summary of the invention

[0004] In view of the deficiencies in the prior art, an object of the present invention is to provide an ultrasonic image enhancement method based on a diffusion model.

[0005] According to a first aspect of an embodiment of the present application, a method for ultrasonic image enhancement based on a diffusion model is provided, comprising:

[0006] Acquire an ultrasound image training set, wherein the ultrasound image training set includes an original ultrasound image and a corresponding target generated image;

[0007] Based on the ultrasound image training set, a dual-frequency domain ultrasound image enhancement diffusion model is trained, wherein the dual-frequency domain ultrasound image enhancement diffusion model converts an image into low-frequency coefficients and high-frequency coefficients through wavelet transform, and recursively reconstructs low-frequency sub-images and high-frequency sub-images through low-frequency diffusion generation and high-frequency residual calculation combined with inverse wavelet transform, thereby completing image enhancement;

[0008] An ultrasonic image to be processed is acquired, and the ultrasonic image to be processed is enhanced by using the trained dual-frequency domain image enhancement diffusion model.

[0009] Furthermore, the original ultrasound image is superimposed and fused with the corresponding organ segmentation mask to obtain a corresponding target generated image.

[0010] Furthermore, before training the dual-frequency domain ultrasound image enhancement diffusion model, the images in the ultrasound image training set are preprocessed, and the preprocessing includes adjusting the image size and performing grayscale processing on the image.

[0011] Furthermore, the dual-frequency domain ultrasound image enhancement diffusion model includes a two-dimensional discrete wavelet transform module, a low-frequency diffusion generation module, a high-frequency residual calculation module and a two-dimensional discrete wavelet inverse transform module;

[0012] The two-dimensional discrete wavelet transform module is used to transform the two-dimensional discrete wavelet transform of the ultrasound image to generate multi-level low-frequency coefficients and high-frequency coefficients;

[0013] The low-frequency diffusion generation module is used to denoise the low-frequency coefficients through a denoising diffusion probability model;

[0014] The high-frequency residual calculation module is used to enhance the high-frequency coefficients through a UNet type network;

[0015] The two-dimensional discrete wavelet inverse transform module is used to obtain an enhanced ultrasonic image through an inverse wavelet transform process based on the denoised low-frequency coefficients and the enhanced high-frequency coefficients.

[0016] Furthermore, the dual-frequency domain ultrasound image enhancement diffusion model is trained, including:

[0017] Using a two-dimensional discrete wavelet transform module, a two-dimensional discrete wavelet transform is performed on the target generated image to obtain multi-level target low-frequency coefficients and target high-frequency coefficients;

[0018] Using a two-dimensional discrete wavelet transform module, the original ultrasound image is subjected to a two-dimensional discrete wavelet transform to obtain multi-level original low-frequency coefficients and original high-frequency coefficients;

[0019] Training the low-frequency diffusion generation module based on the target low-frequency coefficient and the original low-frequency coefficient;

[0020] The high-frequency residual calculation module is trained based on the target high-frequency coefficients and the original high-frequency coefficients.

[0021] Furthermore, during the training process of the low-frequency diffusion generation module, the low-frequency coefficients of the N-1th step are reversely denoised to obtain the restoration coefficients Calculate the loss between the reference coefficient and the recovery coefficient, where the reference coefficient is the forward diffusion result of step N-1, and the recovery coefficient is the forward diffusion result for the Nth step The result obtained by reverse denoising.

[0022] Further, the trained dual-frequency domain image enhancement diffusion model is used to perform image enhancement on the ultrasound image to be processed, including:

[0023] Inputting the ultrasound image to be processed into the trained dual-frequency domain image enhancement diffusion model;

[0024] Multi-level low-frequency coefficients and high-frequency coefficients are obtained through a two-dimensional discrete wavelet transform module;

[0025] Processing the low-frequency coefficients using a low-frequency diffusion generation module to obtain reconstructed low-frequency coefficients;

[0026] Based on a predetermined high frequency coefficient threshold, determining whether high frequency coefficient enhancement is required;

[0027] If yes, the high-frequency coefficients of the high-frequency residual calculation module are enhanced by using a high-frequency residual calculation module to obtain enhanced high-frequency coefficients, and the reconstructed low-frequency coefficients and the enhanced high-frequency coefficients are processed by a two-dimensional discrete wavelet inverse transform module to obtain an enhanced image;

[0028] If not, the reconstructed low-frequency coefficients and high-frequency coefficients are processed by a two-dimensional discrete wavelet inverse transform module to obtain an enhanced image.

[0029] According to a second aspect of an embodiment of the present application, there is provided an ultrasonic image enhancement device based on a diffusion model, comprising:

[0030] A training set acquisition module, used to acquire an ultrasound image training set, wherein the ultrasound image training set includes an original ultrasound image and a corresponding target generated image;

[0031] A model training module, used for training a dual-frequency domain ultrasound image enhancement diffusion model based on the ultrasound image training set, wherein the dual-frequency domain ultrasound image enhancement diffusion model converts an image into low-frequency coefficients and high-frequency coefficients through wavelet transform, and recursively reconstructs low-frequency sub-images and high-frequency sub-images through low-frequency diffusion generation and high-frequency residual calculation combined with inverse wavelet transform, thereby completing image enhancement;

[0032] The image enhancement module is used to acquire the ultrasonic image to be processed and perform image enhancement on the ultrasonic image to be processed using the trained dual-frequency domain image enhancement diffusion model.

[0033] According to a third aspect of an embodiment of the present application, there is provided an electronic device, including:

[0034] one or more processors;

[0035] A memory for storing one or more programs;

[0036] When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in the first aspect.

[0037] According to a third aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which computer instructions are stored. When the instructions are executed by a processor, the steps of the method described in the first aspect are implemented.

[0038] The technical solution provided by the embodiments of the present application may have the following beneficial effects:

[0039] It can be seen from the above embodiments that the present application can effectively remove noise from ultrasound images while retaining important details of the image and improving the diagnostic value of the image. The present invention uses wavelet transform to reduce the data calculation dimension of the diffusion model, combines the image segmentation mask and the original image annotated by experts, and draws on low-light image processing technology to generate high-quality enhanced images through the diffusion process, thereby effectively improving the quality of ultrasound images. The wavelet transform can halve the spatial dimension after each transformation without sacrificing information, and the diffusion model training and reasoning are performed in the wavelet domain, with performance improved by dozens of times. In addition, the present invention does not rely on noise-free or low-noise standard label images, and is also applicable to other scenarios where it is difficult to obtain noise-free or low-noise images, and has broad application prospects.

[0040] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0042] Figure 1 The figure is a flow chart of a method for ultrasonic image enhancement based on a diffusion model according to an exemplary embodiment.

[0043] Figure 2 It is a model structure diagram of a dual-frequency domain ultrasound image enhancement diffusion model according to an exemplary embodiment.

[0044] Figure 3 The figure is a schematic diagram of a two-dimensional discrete wavelet change according to an exemplary embodiment.

[0045] Figure 4 The present invention is a flowchart of training a dual-frequency domain ultrasound image enhancement diffusion model according to an exemplary embodiment.

[0046] Figure 5 The present invention is a flowchart of a dual-frequency domain ultrasound image enhancement diffusion model according to an exemplary embodiment.

[0047] Figure 6 is a schematic diagram of the enhancement effect of a dual-frequency domain ultrasound image enhancement method based on a diffusion model according to an exemplary embodiment, wherein (a) is the original image, (b) is the enhanced image, and (c) is the actual organ annotated image.

[0048] Figure 7 It is a block diagram of an ultrasonic image enhancement device based on a diffusion model according to an exemplary embodiment.

[0049] Figure 8 The diagram is a schematic diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0050] Here, exemplary embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.

[0051] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0052] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0053] like Figure 1 As shown, the present application proposes an ultrasound image enhancement method based on a diffusion model, which may include:

[0054] S1: Acquire an ultrasound image training set, wherein the ultrasound image training set includes an original ultrasound image and a corresponding target generated image;

[0055] S2: Based on the ultrasound image training set, a dual-frequency domain ultrasound image enhancement diffusion model (DFDiff-UEM) is trained, wherein the dual-frequency domain ultrasound image enhancement diffusion model converts the image into low-frequency coefficients and high-frequency coefficients through wavelet transform, and recursively reconstructs low-frequency sub-images and high-frequency sub-images through low-frequency diffusion generation and high-frequency residual calculation combined with inverse wavelet transform, thereby completing image enhancement;

[0056] S3: Acquire an ultrasound image to be processed, and perform image enhancement on the ultrasound image to be processed using the trained dual-frequency domain image enhancement diffusion model.

[0057] It can be seen from the above embodiments that the present application can effectively remove noise from ultrasound images while retaining important details of the image and improving the diagnostic value of the image. The present invention uses wavelet transform to reduce the data calculation dimension of the diffusion model, combines the image segmentation mask and the original image annotated by experts, and draws on low-light image processing technology to generate high-quality enhanced images through the diffusion process, thereby effectively improving the quality of ultrasound images. The wavelet transform can halve the spatial dimension after each transformation without sacrificing information, and the diffusion model training and reasoning are performed in the wavelet domain, with performance improved by dozens of times. In addition, the present invention does not rely on noise-free or low-noise standard label images, and is also applicable to other scenarios where it is difficult to obtain noise-free or low-noise images, and has broad application prospects.

[0058] In a specific implementation of step S1, an ultrasound image training set is obtained, wherein the ultrasound image training set includes an original ultrasound image and a corresponding target generated image;

[0059] Specifically, the acquired original ultrasound image has a corresponding organ segmentation mask, and the segmentation mask is obtained by manual annotation by experts to ensure that each training image has a corresponding segmentation mask. The target generated image is obtained by superimposing and fusing the original ultrasound image and the corresponding organ segmentation mask with a predetermined fusion coefficient, wherein the fusion coefficient is set according to the actual situation. The original ultrasound image and the target generated image are used as training inputs for the diffusion model. The denoising effect based on the diffusion model is better, but a noise-free or low-noise reference image is required for training, and ultrasound images naturally lack noise-free or low-noise images. Therefore, the present application combines the organ segmentation mask annotated by experts and the original image to construct a target generated image, thereby forming an image pair.

[0060] Before training the dual-frequency domain ultrasound image enhancement diffusion model, the images in the ultrasound image training set need to be preprocessed, and the preprocessing includes: adjusting the image size to the model input size, performing grayscale processing on the image, etc.

[0061] In the specific implementation of step S2, a dual-frequency domain ultrasound image enhancement diffusion model (DFDiff-UEM) is trained based on the ultrasound image training set, wherein the dual-frequency domain ultrasound image enhancement diffusion model converts the image into low-frequency coefficients and high-frequency coefficients through wavelet transform, and recursively reconstructs low-frequency sub-images and high-frequency sub-images through low-frequency diffusion generation and high-frequency residual calculation combined with inverse wavelet transform, thereby completing image enhancement;

[0062] Specifically, after preprocessing, the ultrasound image to be enhanced is input into a dual-frequency domain ultrasound image enhancement diffusion model (DFDiff-UEM) for processing, such as Figure 2 As shown, the model includes a two-dimensional discrete wavelet transform module, a low-frequency diffusion generation module, a high-frequency residual calculation module and a two-dimensional discrete wavelet inverse transform module.

[0063] (1) The two-dimensional discrete wavelet transform (2D-DWT) module transforms the ultrasound image through N two-dimensional discrete wavelet transforms to generate N groups of low-frequency coefficients and N groups of high-frequency coefficients (horizontal high-frequency coefficients, vertical high-frequency coefficients and diagonal high-frequency coefficients):

[0064]

[0065] in, is the low-frequency coefficient after the Nth transformation, is the vertical high-frequency coefficient after the Nth transformation, is the horizontal high-frequency coefficient after the Nth transformation, is the diagonal high-frequency coefficient after the Nth transformation.

[0066] like Figure 3 As shown, Figure 3(a) in the figure represents the original image. Figure 3 (b) in the figure shows the result after the first-order wavelet transform. Figure 3 (c) in the figure indicates that the wavelet transform is continued on the low-frequency information after the first-level transform, where h represents the transform result in the horizontal direction, v represents the transform in the vertical direction, c represents the transform in the diagonal direction, and b represents the downsampled image.

[0067] Since the denoising efficiency of the diffusion model is not high, this application uses wavelet transform to reduce the data calculation dimension of the diffusion model. After the wavelet transform, the low-frequency coefficients are the global content information of the image, which effectively retains the overall background contour of the image. The three high-frequency coefficients contain sparse local detail descriptions and textures, etc., and the noise is also mainly in the high-frequency part. The images reconstructed by the three high-frequency coefficients through the ablation experiment are highly similar to the original images, while the images reconstructed by replacing the low-frequency coefficients cannot be restored well, and the information loss is serious. Therefore, the focus of ultrasonic image enhancement in the wavelet frequency domain is to fit the low-frequency coefficients of the corresponding target generated image. Therefore, the excellent generation ability of the diffusion model is used to reconstruct the low-frequency coefficients, and the remaining three high-frequency coefficients are reconstructed in combination with the high frequency to facilitate the recovery of local details.

[0068] (2) The low-frequency diffusion generation module is a denoising diffusion probability model (Denoising Diffusion Probabilistic Models, DDPM) for denoising based on low-frequency coefficients, including a forward diffusion process and a reverse denoising process.

[0069] The forward diffusion process includes obtaining the original ultrasound image Img_raw and the target generated image Img_fused obtained by fusion construction according to the set batch size, adjusting Img_raw and Img_fused to the input size specified by the model, adding step_num Gaussian noise to Img_fused according to the specified number of noise additions step_num, and obtaining the forward result Img_noised. The result of the nth noise addition is

[0070] The reverse denoising process is to use the denoising neural network to extract the noise from the above step_num times of denoising results. The target generated image Img_fused is restored in . The denoising neural network, a possible choice of U-Net network, namely DenUNet, applies a convolutional layer on the noisy image and calculates a temporal embedding for the noise level n, which enables the model to know at which time stage in the batch the noise is added when predicting the noise, and then composes the network through downsampling, upsampling and short-cut splicing.

[0071] (3) The high-frequency residual calculation module is implemented based on a UNet type network, such as U-Transformer, which integrates the self-attention mechanism and cross-attention mechanism in Transformer into UNet, overcoming the shortcomings of UNet that cannot model long-range relationships and spatial dependencies, thereby improving the connection with key contexts. The self-attention mechanism establishes global interactions between encoders during the downsampling process, and the cross-attention mechanism is added to the skip connection to restore spatial information in the UNet Decoder by filtering non-semantic features. The high-resolution features extracted by the encoder are reused through the skip connection to restore and make up for the lost spatial information.

[0072] (4) The two-dimensional inverse discrete wavelet transform (2D-IDWT) module reconstructs the processed coefficients into a complete image. This step involves the specific reorganization and superposition of low-frequency reconstruction coefficients, horizontal high-frequency recovery coefficients, vertical high-frequency recovery coefficients, and diagonal high-frequency recovery coefficients, and finally obtains an enhanced spatial domain image through the inverse wavelet transform process. It usually includes two one-dimensional inverse transforms in the vertical and horizontal directions. In each direction, the size and details of the original image are gradually restored through interpolation, filtering and other steps. This process ensures that the enhanced coefficients can be accurately mapped back to the original spatial domain, thereby producing ultrasound images with higher quality, less noise and richer details.

[0073]

[0074] in, is the low-frequency coefficient after the Nth inverse transform, is the vertical high-frequency coefficient after the Nth inverse transform, The horizontal high-frequency coefficients after the Nth inverse transform, is the diagonal high-frequency coefficient after the Nth inverse transform.

[0075] The inverse wavelet transform recursively reconstructs the low-frequency sub-image and the high-frequency sub-image by combining the two-dimensional discrete wavelet inverse transform to restore the image, establishes a recursive low-frequency denoising process, gradually removes the noise of each level of the image, performs 2D-DWT image restoration on the n-th level high- and low-frequency denoising results, performs image weighted fusion with the n-1-th level low-frequency denoised sub-image, and performs 2D-DWT image restoration on the n-1-th level high-frequency denoised sub-image. In the above process, the high-frequency and low-frequency denoising sub-networks of each level are repeated until the final clean image is restored.

[0076] Set the batch size for each iteration and input the prepared dataset into the model for iterative training, such as Figure 4 As shown, including:

[0077] S21: using a two-dimensional discrete wavelet transform module 2D-DWT, performing a two-dimensional discrete wavelet transform on the target generated image to obtain a multi-level target low-frequency coefficient and a target high-frequency coefficient, wherein the target high-frequency coefficient includes a target horizontal high-frequency coefficient, a target vertical high-frequency coefficient, and a target diagonal high-frequency coefficient;

[0078] S22: using a two-dimensional discrete wavelet transform module 2D-DWT, performing a two-dimensional discrete wavelet transform on the original ultrasonic image to obtain multi-level original low-frequency coefficients and original high-frequency coefficients, wherein the original high-frequency coefficients include original horizontal high-frequency coefficients, original vertical high-frequency coefficients, and original diagonal high-frequency coefficients;

[0079] S23: training the low-frequency diffusion generation module based on the target low-frequency coefficient and the original low-frequency coefficient;

[0080] Specifically, the loss function is calculated to update the parameters in the set denoising neural network and save the trained model parameters.

[0081] Specifically, the loss function calculation is as follows: the low-frequency coefficients of the N-1th step are reversely denoised, and the loss function is defined as the difference between the recovery coefficient and the reference coefficient, which can be but not limited to L1 and L2 losses. Taking L2 loss as an example, that is, The reference coefficient is the forward diffusion result of step N-1, and the recovery coefficient is the forward diffusion result for the Nth step The result of reverse denoising, i.e. the restoration coefficient

[0082] When minimizing the loss function through optimization, it is necessary to continuously update the convolution weight parameters of DenUNet's downsampling, upsampling and short-cut stages to complete the training.

[0083] S24: training the high-frequency residual calculation module based on the target high-frequency coefficient and the original high-frequency coefficient;

[0084] Specifically, based on the target high-frequency coefficients and the original high-frequency coefficients, the high-frequency residual calculation module is trained in a supervised training manner.

[0085] S25: Create a test set based on expert annotations to perform inference test on the dual-frequency domain ultrasound image enhancement diffusion model processed by steps S23 and S24, calculate the signal-to-noise ratio and structural similarity of the test set image, and determine whether the predetermined threshold condition is met. If not, return to change the coefficients when constructing the target generated image to regenerate the target generated image, use the new target generated image, and return to step S21 to continue training;

[0086] Specifically, by calculating the signal-to-noise ratio or structural similarity of the test set image, the signal-to-noise ratio threshold or structural similarity threshold is set for comparison. If the set threshold condition is not met, the model needs to be further trained. At this time, the target generated image needs to be updated. One possible method is to adjust the weight coefficient of the mask when the original image and the segmentation mask are fused when constructing the secondary target generated image. For example, the coefficient c is initially 0.5, and the coefficient is increased by (1-c) / 2 each time, that is, the coefficient is 0.5, 0.75, 0.875... approaching 1. The gradual noise addition of the simulated diffusion model can enhance the noise learning ability of the model.

[0087] In the specific implementation of step S3, an ultrasonic image to be processed is obtained, and the ultrasonic image to be processed is enhanced using the trained dual-frequency domain image enhancement diffusion model;

[0088] Specifically, Figure 5 As shown, the model inference process includes:

[0089] S31: inputting the ultrasound image to be processed into the trained dual-frequency domain image enhancement diffusion model;

[0090] S32: obtaining multi-level low-frequency coefficients and high-frequency coefficients through a two-dimensional discrete wavelet transform module;

[0091] S33: Processing the low-frequency coefficients using a low-frequency diffusion generation module to obtain reconstructed low-frequency coefficients;

[0092] S34: Based on a predetermined high frequency coefficient threshold, determining whether high frequency coefficient enhancement is required;

[0093] Specifically, an optional high frequency coefficient recovery is adopted, which is optional because ultrasound mainly focuses on the overall organ contour and detail reconstruction will reduce the computational efficiency.

[0094] S35: If yes, the high-frequency coefficients of the high-frequency residual calculation module are enhanced by using the high-frequency residual calculation module to obtain enhanced high-frequency coefficients, and the reconstructed low-frequency coefficients and the enhanced high-frequency coefficients are processed by a two-dimensional discrete wavelet inverse transform module to obtain an enhanced image;

[0095] S36: If not, the reconstructed low-frequency coefficients and high-frequency coefficients are processed by a two-dimensional discrete wavelet inverse transform module to obtain an enhanced image.

[0096] The enhanced image obtained is Figure 6 As shown, (a) is the original image, (b) is the enhanced image, and (c) is the actual organ labeled image. Compared with the original image, the image enhanced by the method of the present invention can show the organ information more clearly and has a higher application value.

[0097] In summary, the present invention provides an ultrasonic image enhancement method based on a diffusion model, aiming to solve the problem that the existing ultrasonic image denoising technology is difficult to effectively remove noise and maintain image details in the absence of a noise-free or low-noise original image reference. The present invention is implemented by a dual-frequency domain ultrasonic image enhancement model composed of a dual parallel branch network, and a denoising diffusion model is constructed based on a low-frequency signal of a two-dimensional discrete wavelet transform. The multi-scale decomposition capability of the wavelet transform and the powerful denoising performance of the conditional diffusion model are utilized to effectively improve the quality of the ultrasonic image. The data calculation dimension of the diffusion model is reduced by wavelet transform, thereby improving the computational efficiency. The residual network model for detail recovery based on the high-frequency signal of the wavelet transform is an optional module to achieve a balance between optimizing detail reconstruction and computational efficiency. The noise pattern and important features of the image are learned by the dual-frequency domain ultrasonic image enhancement model to achieve noise suppression and image detail enhancement. Combined with the image segmentation mask and the original image annotated by the expert, and drawing on the low-light image processing technology, the noise in the ultrasonic image is effectively reduced, while the important features and details of the image are maintained, thereby improving the diagnostic value of the image. The present invention can more effectively remove noise from ultrasonic images, improve the perceived quality of the image, and help doctors make more accurate diagnoses.

[0098] Corresponding to the aforementioned embodiment of the ultrasound image enhancement method based on the diffusion model, the present application also provides an embodiment of the ultrasound image enhancement device based on the diffusion model.

[0099] Figure 7 FIG. 1 is a block diagram of an ultrasonic image enhancement device based on a diffusion model according to an exemplary embodiment. Figure 7 , the device may include:

[0100] A training set acquisition module 21 is used to acquire an ultrasound image training set, wherein the ultrasound image training set includes an original ultrasound image and a corresponding target generated image;

[0101] A model training module 22 is used to train a dual-frequency domain ultrasound image enhancement diffusion model based on the ultrasound image training set, wherein the dual-frequency domain ultrasound image enhancement diffusion model converts an image into low-frequency coefficients and high-frequency coefficients through wavelet transform, and recursively reconstructs low-frequency sub-images and high-frequency sub-images through low-frequency diffusion generation and high-frequency residual calculation combined with inverse wavelet transform, thereby completing image enhancement;

[0102] The image enhancement module 23 is used to acquire the ultrasonic image to be processed and perform image enhancement on the ultrasonic image to be processed by using the trained dual-frequency domain image enhancement diffusion model.

[0103] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0104] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present application scheme. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0105] Accordingly, the present application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned ultrasound image enhancement method based on the diffusion model. Figure 8 As shown in FIG. 1 , a hardware structure diagram of any device with data processing capability in which an ultrasonic image enhancement method based on a diffusion model provided by an embodiment of the present invention is located, except that Figure 8 In addition to the processor, memory and network interface shown, any device with data processing capability in which the apparatus in the embodiment is located may also include other hardware, generally based on the actual functions of the device with data processing capability, which will not be described in detail.

[0106] Accordingly, the present application also provides a computer-readable storage medium on which computer instructions are stored, and when the instructions are executed by a processor, the ultrasonic image enhancement method based on the diffusion model as described above is implemented. The computer-readable storage medium can be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or a memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), an SD card, a flash card (Flash Card), etc. equipped on the device. Further, the computer-readable storage medium can also include both an internal storage unit of any device with data processing capabilities and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and can also be used to temporarily store data that has been output or is to be output.

[0107] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the contents disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

Claims

1. An ultrasonic image enhancement method based on a diffusion model, characterized in that: include: Acquire an ultrasound image training set, wherein the ultrasound image training set includes an original ultrasound image and a corresponding target generated image; Based on the ultrasound image training set, a dual-frequency domain ultrasound image enhancement diffusion model is trained, wherein the dual-frequency domain ultrasound image enhancement diffusion model converts an image into low-frequency coefficients and high-frequency coefficients through wavelet transform, and recursively reconstructs low-frequency sub-images and high-frequency sub-images through low-frequency diffusion generation and high-frequency residual calculation combined with inverse wavelet transform, thereby completing image enhancement; An ultrasonic image to be processed is acquired, and the ultrasonic image to be processed is enhanced by using the trained dual-frequency domain image enhancement diffusion model.

2. The method according to claim 1, characterized in that The corresponding target generated image is obtained by superimposing and fusing the original ultrasound image with the corresponding organ segmentation mask.

3. The method according to claim 1, characterized in that Before training the dual-frequency domain ultrasound image enhancement diffusion model, the images in the ultrasound image training set are preprocessed, and the preprocessing includes adjusting the image size and performing grayscale processing on the image.

4. The method according to claim 1, characterized in that: The dual-frequency domain ultrasound image enhancement diffusion model includes a two-dimensional discrete wavelet transform module, a low-frequency diffusion generation module, a high-frequency residual calculation module and a two-dimensional discrete wavelet inverse transform module; The two-dimensional discrete wavelet transform module is used to transform the two-dimensional discrete wavelet transform of the ultrasound image to generate multi-level low-frequency coefficients and high-frequency coefficients; The low-frequency diffusion generation module is used to denoise the low-frequency coefficients through a denoising diffusion probability model; The high-frequency residual calculation module is used to enhance the high-frequency coefficients through a UNet type network; The two-dimensional discrete wavelet inverse transform module is used to obtain an enhanced ultrasonic image through an inverse wavelet transform process based on the denoised low-frequency coefficients and the enhanced high-frequency coefficients.

5. The method according to claim 4, characterized in that Training of a dual-frequency domain ultrasound image enhancement diffusion model, including: Using a two-dimensional discrete wavelet transform module, a two-dimensional discrete wavelet transform is performed on the target generated image to obtain multi-level target low-frequency coefficients and target high-frequency coefficients; Using a two-dimensional discrete wavelet transform module, the original ultrasound image is subjected to a two-dimensional discrete wavelet transform to obtain multi-level original low-frequency coefficients and original high-frequency coefficients; Training the low-frequency diffusion generation module based on the target low-frequency coefficient and the original low-frequency coefficient; The high-frequency residual calculation module is trained based on the target high-frequency coefficients and the original high-frequency coefficients.

6. The method according to claim 5, characterized in that During the training process of the low-frequency diffusion generation module, the low-frequency coefficients of the N-1th step are reversely denoised to obtain the restoration coefficients Calculate the loss between the reference coefficient and the recovery coefficient, where the reference coefficient is the forward diffusion result of step N-1, and the recovery coefficient is the forward diffusion result for the Nth step The result obtained by reverse denoising.

7. The method according to claim 4, characterized in that Performing image enhancement on the ultrasonic image to be processed by using the trained dual-frequency domain image enhancement diffusion model includes: Inputting the ultrasound image to be processed into the trained dual-frequency domain image enhancement diffusion model; Multi-level low-frequency coefficients and high-frequency coefficients are obtained through a two-dimensional discrete wavelet transform module; Processing the low-frequency coefficients using a low-frequency diffusion generation module to obtain reconstructed low-frequency coefficients; Based on a predetermined high frequency coefficient threshold, determining whether high frequency coefficient enhancement is required; If yes, the high-frequency coefficients of the high-frequency residual calculation module are enhanced by using a high-frequency residual calculation module to obtain enhanced high-frequency coefficients, and the reconstructed low-frequency coefficients and the enhanced high-frequency coefficients are processed by a two-dimensional discrete wavelet inverse transform module to obtain an enhanced image; If not, the reconstructed low-frequency coefficients and high-frequency coefficients are processed by a two-dimensional discrete wavelet inverse transform module to obtain an enhanced image.

8. An ultrasonic image enhancement device based on a diffusion model, characterized in that: include: A training set acquisition module, used to acquire an ultrasound image training set, wherein the ultrasound image training set includes an original ultrasound image and a corresponding target generated image; A model training module, used for training a dual-frequency domain ultrasound image enhancement diffusion model based on the ultrasound image training set, wherein the dual-frequency domain ultrasound image enhancement diffusion model converts an image into low-frequency coefficients and high-frequency coefficients through wavelet transform, and recursively reconstructs low-frequency sub-images and high-frequency sub-images through low-frequency diffusion generation and high-frequency residual calculation combined with inverse wavelet transform, thereby completing image enhancement; The image enhancement module is used to acquire the ultrasonic image to be processed and perform image enhancement on the ultrasonic image to be processed using the trained dual-frequency domain image enhancement diffusion model.

9. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instruction is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images