Depth variation ultrasonic image speckle reduction method based on imaging model guidance

Through the depth variation framework guided by imaging model, combined with Bayesian model and deep learning, the problem of insufficient processing speed and adaptability of ultrasonic image denoising method is solved, and efficient speckle reduction and physical interpretability are achieved. It significantly improves the diagnostic efficiency of ultrasound imaging.

CN119027330APending Publication Date: 2024-11-26CHINA THREE GORGES UNIV
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410991708.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2024-11-26

Smart Images

  • Figure CN119027330A_ABST
    Figure CN119027330A_ABST
Patent Text Reader

Abstract

The invention discloses a depth variation ultrasonic image speckle reduction method based on imaging model guidance, which comprises the following steps of: (1) constructing a complete Bayesian model of ultrasonic image speckle reduction by utilizing a physical model of ultrasonic image imaging and statistical prior distribution of related parameters; and (2) on the basis of a variational inference principle, parameterizing noise variance and variational posteriori distribution of a noise-free image by using two deep convolutional networks respectively, and performing joint training optimization on the two deep networks through established evidence lower bound ELBO loss and image physical reconstruction loss, so that the pre-trained network can estimate the noise variance and infer a speckle subduction image at the same time. The method serves as a depth variation framework, has the physical interpretability of a model driving method, also has the universality and flexibility of a data driving method, and can obtain excellent ultrasonic speckle reduction performance.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of medical image denoising, and in particular to a deep variational ultrasound image speckle reduction method guided by an imaging model. Background Art

[0002] In ultrasound imaging technology, speckle, as an inherent interference factor, significantly reduces the visual quality and diagnostic value of the image. This noise originates from the complex scattering phenomenon when ultrasound waves propagate in human tissues, causing the image to present a random and granular texture, which poses a great challenge to doctors in interpreting the images. In order to overcome this problem, researchers have proposed a variety of speckle reduction techniques, including spatial filtering, frequency domain filtering, and methods based on physical models. Although these traditional technologies can improve image quality to a certain extent, they usually require fine-tuning of parameters for different scenarios, and the processing speed is limited, making it difficult to meet the needs of real-time imaging. In addition, due to their nature as being based on fixed mathematical models, these methods show certain limitations when faced with diverse tissue structures.

[0003] In recent years, the rise of deep learning technology has provided a new perspective for the processing of ultrasound speckle. With its excellent feature learning ability and nonlinear mapping function, deep learning models have shown great potential in the field of image denoising. However, its application in the field of ultrasound imaging also faces unique challenges. First, it is almost impossible to obtain completely speckle-free ultrasound images as training samples, which forces researchers to use synthetic data or preprocessed images as alternatives, but the data sets generated in this way often have gaps with actual ultrasound images, limiting the performance of the model. Secondly, although the deep learning model is powerful, its internal mechanism is like a black box and lacks intuitive physical explanation, which is an unacceptable shortcoming in the medical field that pursues accuracy and explainability.

[0004] In view of this, the present invention proposes an innovative solution, namely a deep variational framework that cleverly combines the flexibility of deep learning with the physical prior knowledge of ultrasound imaging. This framework uses the principle of variational inference to build a model that can both understand the physical basis of ultrasound imaging and learn from data. It not only improves the resolution and contrast of ultrasound images, but also maintains a deep understanding of the imaging process while ensuring the real-time performance of the algorithm. The present invention combines the dual advantages of model-driven and data-driven, and is expected to significantly improve the diagnostic efficiency of ultrasound imaging in clinical practice, provide doctors with clearer and more accurate image information, and thus improve the patient's diagnosis and treatment experience. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a deep variational ultrasound image speckle reduction method based on imaging model guidance, which overcomes the limitations of existing image denoising methods in the field of ultrasound imaging. For example, traditional speckle reduction methods have problems such as high parameter sensitivity, slow processing speed and insufficient adaptability to diverse tissues; while deep learning models face the challenges of data set construction and model interpretability.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is: The deep variational ultrasound image speckle reduction method based on imaging model guidance includes the following steps: Step 1: Using the physical model of ultrasound imaging and the statistical prior distribution of related parameters, a complete Bayesian model for ultrasound image speckle reduction is constructed; Step 2. Based on the variational inference principle, two deep convolutional networks are used to parameterize the noise variance and the variational posterior distribution of the noise-free image respectively. The two deep networks are jointly trained and optimized through the created evidence lower bound ELBO loss and image physical reconstruction loss. The pre-trained network can simultaneously estimate the noise variance and infer the speckle reduction image.

[0007] The above Step 1 includes the following steps: Step 1.1, define ultrasonic noisy images, noise-free images and multiplicative noise, build a physical model of ultrasonic imaging, and combine the imaging physical model and multiplicative noise The statistical distribution of noisy images can be derived Satisfies the mean , the variance is Gaussian distribution of Step 1.2, define the deep network for noisy images Predicted expected noise-free image, derived noise-free image The statistical prior distribution of .

[0008] The specific process of Step 1.1 above is: Step 1.1, define the spatial domain of ultrasound image as , whose pixel position is represented by , ;like , and Represent ultrasonic noisy image, noise-free image and multiplicative noise respectively, then the physical model of ultrasonic imaging can be expressed as: ; Among them, the multiplicative noise has zero mean and variance Gaussian noise, that is: ; in, It means the mean , the variance is Gaussian distribution of Combining imaging physics models with multiplicative noise The statistical distribution of noisy images can be derived Satisfies the mean , the variance is Gaussian distribution, that is: ; Due to the variance The value of the noise-free image Directly proportional changes, so it can also be equivalently described as changing with the spatial position Variance of change , that is, definition , then the above formula can be transformed into: .

[0009] The specific process of Step 1.2 above is: Step 1.2, Definition Represents the deep network for noisy images The predicted expected noise-free image is Represents the ultrasound image speckle reduction dataset A pair of ultrasonic noisy-expected noise-free image samples in; in creating a speckle reduction dataset When the noise-free image is expected The noisy image can be reduced by speckle reduction method. Filter to obtain; thus, the desired noise-free image Only approximately but not exactly equivalent to the noise-free image At the same time, since we expect a noise-free image Can provide noise-free images The rich prior information of The statistical prior distribution of can be modeled as the following Laplace distribution: ; in, Indicates that the position parameter is , the scale parameter is Laplace distribution of ; hyperparameters is the deviation position parameter The mean absolute deviation of , which is usually set to a small positive number; Furthermore, for the pixel position in Step 1.1 Variance of change , its statistical prior distribution is usually assumed to satisfy the inverse gamma distribution, that is: ; in, The shape parameter is , the scale parameter is The inverse gamma distribution of Controls the strength of the inverse gamma distribution, which is typically set to ,and is the pixel position The local neighborhood window size; Variance At pixel position The local neighborhood estimate of the noise variance of the entire image space domain is It can be calculated by the following formula: ; in, Indicates the window size is Gaussian filtering.

[0010] The above Step 1.1 and Step 1.2 are about noisy images , Noise Variance and noise-free image The prior distribution expression of constitutes a complete Bayesian model for ultrasonic image speckle reduction.

[0011] The above Step 2 includes the following steps: Step 2.1. Based on the Bayesian statistical point of view, create a variational distribution according to the variational inference principle To approximate the true posterior distribution ; The prediction function of the position parameters after variation is obtained through the SigmaSquareNet and DespecklingNet networks; Step 2.2. Network parameters of SigmaSquareNet and DespecklingNet and Optimize; by setting the loss function, the joint training of the two deep networks can benefit each other and achieve a common optimum; Step 2.3, reconstruct the ultrasonic noisy image, combine the ELBO loss in Step 2.2 and the noisy image reconstruction loss, and get the joint training optimization loss of SigmaSquareNet and DespecklingNet network; Step 2.4: Generate the desired noise-free image using Step 1.2 The ultrasonic noisy-expected noise-free image samples are constructed by the method, thereby creating an ultrasonic image speckle reduction dataset , and train the SigmaSquareNet and DespecklingNet networks; Step 2.5, in the testing phase after network training, the ultrasonic noisy image is first input into the pre-trained SigmaSquareNet network, and the output scale parameter image can be simply transformed to obtain the noise variance estimation image; then the noise variance estimation image and the ultrasonic noisy image are concatenated based on the feature dimension and input into the DespecklingNet network together to infer the ultrasonic speckle reduction image.

[0012] The specific steps of Step 2.1 above are: Step 2.1, based on the Bayesian statistical point of view, the speckle reduction of ultrasound images aims to reduce the noise of the image. Deriving Ultrasound Noise-Free Images and variance The posterior distribution of ; Based on the variational inference principle, consider creating a variational distribution To approximate the true posterior distribution ; Based on probability theory knowledge, variational posterior distribution Can be decomposed into: ; From Step 1.2 internal variance The prior distribution of satisfies the inverse gamma distribution, and its variational posterior distribution can be assumed to be Satisfies the following inverse gamma distribution: ; in, represents the variational posterior inverse gamma distribution based on pixel position The scale parameter prediction function is implemented by taking the parameter The SigmaSquareNet network is used to input ultrasonic noisy images. direct response to; Similarly, the noise-free image in Step 1.2 The prior distribution of , we can assume that its variational posterior distribution Satisfies the following Laplace distribution: ; in, represents the variational posterior Laplace distribution based on pixel positions The location parameter prediction function is implemented by taking the parameter DespecklingNet network for ultrasound noisy images and variance A direct response to the combined input.

[0013] The specific steps of Step 2.2 above are: Step 2.2: To achieve the optimal inference of the above variational posterior distribution related parameters, so that each variational posterior distribution is closer to the corresponding true posterior distribution, the network parameters of SigmaSquareNet and DespecklingNet need to be adjusted. and Therefore, a reasonable loss function must be created so that the joint training of the two deep networks can benefit each other and achieve a common optimum. For the convenience of expression, the scale parameter prediction function in Step 2.1 is and location parameter prediction function Respectively abbreviated as and ; Given an ultrasonic noisy image , whose log marginal probability Can be decomposed into: ; in, represents the KL divergence of two statistical distributions; due to the non-negativity of KL divergence, constitutes The lower bound of is called the Evidence Lower Bound, ELBO, and its calculation expression can be derived as follows: ; in, Represents a random variable Function The mathematical expectation of the random variable The probability density function of ; The analytical expression for the first term of the above ELBO is:

[0014] ; in, represents the digamma function; The analytical expression for the calculation of the last two KL divergences of ELBO can be derived as follows: ; ; The ELBO expression Transformed into the minimization objective function of network training, the following ELBO loss is formed: .

[0015] The specific steps of Step 2.3 above are: Step 2.3: The ultrasonic noisy image in Step 1.1 satisfies the Gaussian distribution , using the reparameterization technique, the ultrasound noisy image can be reconstructed for: ; in, and Represent the noise-free images and variance , which can be computed as the mode of their respective variational posterior distributions; From Step 2.1, the internal variation posterior distribution The Laplace distribution expression that satisfies the reconstruction estimate can be obtained : ; in, represents the mode of a random variable; Similarly, the variance reconstruction estimate Through Step 2.1 internal variation inverse gamma distribution Calculate the mode of , that is: ; In this way, the reconstructed noisy image can be re-expressed as: ; The physical reconstruction loss of the noisy image is created using the L1 norm, then: ; Combining the ELBO loss and noisy image reconstruction loss in Step 2.2, the joint training optimization loss of SigmaSquareNet and DespecklingNet network is obtained as: .

[0016] The present invention provides a deep variational ultrasound image speckle reduction method based on imaging model guidance, which utilizes the advantages of two types of image denoising algorithms, namely, classical methods and deep learning methods, while suppressing the shortcomings of the two types of methods, and proposes a deep variational framework based on imaging model guidance to achieve ultrasound image speckle reduction. A complete Bayesian model of ultrasound image speckle reduction is constructed by using the physical model of ultrasound imaging and the statistical prior distribution of related parameters, which effectively enhances its physical interpretability; based on the variational inference principle, two deep convolutional networks are used to parameterize the noise variance and the variational posterior distribution of the noise-free image respectively, and the two deep networks are jointly trained and optimized by creating the evidence lower bound (ELBO) loss and the image physical reconstruction loss, which effectively improves its versatility and flexibility. Comparative experimental results show that the method of the present invention has high performance and efficiency for ultrasound image speckle reduction, and has better clinical application value than the existing ultrasound image speckle reduction methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The present invention will be further described below in conjunction with the accompanying drawings and embodiments: Figure 1 It is a structural block diagram of the method of the present invention; Figure 2 is an ultrasonic noisy image; Figure 3 To expect a noise-free image; Figure 4 is the noise variance image after Gaussian filtering; Figure 5 Image of scale parameters inferred for the SigmaSquareNet network; Figure 6 Estimation image for noise variance; Figure 7 Ultrasonic speckle reduction image inferred by the DespecklingNet network; Figure 8 Comparison chart and objective evaluation of speckle reduction results using different methods. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of the present invention clearer, the following content will systematically and completely describe the specific technical solutions of the present invention in combination with the drawings provided according to the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0019] like Figure 1 As shown, the deep variational ultrasound image speckle reduction method based on imaging model guidance includes the following steps: Step 1: Using the physical model of ultrasound imaging and the statistical prior distribution of related parameters, a complete Bayesian model for ultrasound image speckle reduction is constructed. The detailed steps are as follows: Step 1.1, define the spatial domain of ultrasound image as , whose pixel position is represented by , ;like , and Represent ultrasonic noisy image, noise-free image and multiplicative noise respectively, then the physical model of ultrasonic imaging can be expressed as: ; Among them, the multiplicative noise has zero mean and variance Gaussian noise, that is: ; in, It means the mean , the variance is Gaussian distribution.

[0020] Combining imaging physics models with multiplicative noise The statistical distribution of noisy images can be derived Satisfies the mean , the variance is Gaussian distribution, that is: ; Due to the variance The value of the noise-free image Directly proportional changes, so it can also be equivalently described as changing with the spatial position Variance of change , that is, definition , then the above formula can be transformed into: ; Step 1.2, Definition Represents the deep network for noisy images The predicted expected noise-free image is Represents the ultrasound image speckle reduction dataset A pair of ultrasonic noisy-expected noise-free image samples in; in creating a speckle reduction dataset When the noise-free image is expected Noisy images can be treated with excellent speckle reduction methods (such as non-local low-rank algorithm framework) Filtering is performed to obtain (such as Figure 1 as shown); Figure 2The ultrasonic noisy image shown in FIG. 1 is filtered by the non-local low-rank algorithm framework (NLLR) to obtain the desired noise-free image. Figure 3 As shown; thus, a noise-free image is expected Only approximately but not exactly equivalent to the noise-free image At the same time, since we expect a noise-free image Can provide noise-free images The rich prior information of The statistical prior distribution of can be modeled as the following Laplace distribution: ; in, Indicates that the position parameter is , the scale parameter is Laplace distribution of ; hyperparameters is the deviation position parameter The mean absolute deviation of is usually set to a small positive number; in this example, the variance of the Laplace distribution is taken .

[0021] Furthermore, for the pixel position in Step 1.1 Variance of change , its statistical prior distribution is usually assumed to satisfy the inverse gamma distribution, that is: ; in, The shape parameter is , the scale parameter is The inverse gamma distribution of Controls the strength of the inverse gamma distribution, which is typically set to ,and is the pixel position The local neighborhood window size; Variance At pixel position The local neighborhood estimate of the noise variance of the entire image space domain is It can be calculated by the following formula: ; in, Indicates the window size is In this example, we use , .like Figure 4 Shown is the noise variance image obtained after Gaussian filtering.

[0022] The aforementioned noisy image , Noise Variance and noise-free image The prior distribution expression of constitutes a complete Bayesian model for ultrasonic image speckle reduction.

[0023] Step 2: Based on the variational inference principle, two deep convolutional networks are used to parameterize the noise variance and the variational posterior distribution of the noise-free image respectively. The two deep networks are jointly trained and optimized through the created evidence lower bound (ELBO) loss and image physical reconstruction loss. The pre-trained network can simultaneously estimate the noise variance and infer the speckle reduction image. The detailed steps are as follows: Step 2.1, based on the Bayesian statistical point of view, the speckle reduction of ultrasound images aims to reduce the noise of the image. Deriving Ultrasound Noise-Free Images and variance The posterior distribution of Based on the variational inference principle, consider creating a variational distribution To approximate the true posterior distribution . Based on the knowledge of probability theory, the variational posterior distribution Can be decomposed into: ; From Step 1.2 internal variance The prior distribution of satisfies the inverse gamma distribution, and its variational posterior distribution can be assumed to be Satisfies the following inverse gamma distribution: ; in, represents the variational posterior inverse gamma distribution based on pixel position The scale parameter prediction function is implemented by taking the parameter The SigmaSquareNet network is used to input ultrasonic noisy images. The specific structure of the SigmaSquareNet network in this example is as follows Figure 1 As shown in the figure, it contains 5 basic components, of which the first unit consists of a convolution layer + a ReLU activation layer, the three consecutive units in the middle are all composed of a convolution layer + a batch normalization layer + a ReLU activation layer, and the last unit is composed of a single convolution layer. The number of feature channels of the convolution layers of all components is set to 64.

[0024] Similarly, the noise-free image in Step 1.2 The prior distribution of , we can assume that its variational posterior distribution Satisfies the following Laplace distribution: ; in, represents the variational posterior Laplace distribution based on pixel positions The location parameter prediction function is implemented by taking the parameter DespecklingNet network for ultrasound noisy images and variance The direct response of the joint input. In this example, the DespecklingNet network uses a U-Net network with a depth of 4. The specific structure is as follows Figure 1 As shown in the figure, it contains 7 modules, of which the first module consists of two consecutive convolutional layers + ReLU activation layers; the second and third modules are composed of an average pooling layer connected to two consecutive convolutional layers + ReLU activation layers; the fourth module is composed of a transposed convolutional layer + ReLU activation layer connected to the structure of module 3; the fifth and sixth modules are composed of two consecutive convolutional layers + ReLU activation layers, connected to a transposed convolutional layer + ReLU activation layer; the last module is composed of two consecutive convolutional layers + ReLU activation layers, connected to a single convolutional layer. The three symmetrically scaled jump connections are spliced ​​based on the feature dimension.

[0025] Step 2.2: To achieve the optimal inference of the above variational posterior distribution related parameters, so that each variational posterior distribution is closer to the corresponding true posterior distribution, the network parameters of SigmaSquareNet and DespecklingNet need to be adjusted. and Therefore, a reasonable loss function must be created so that the joint training of the two deep networks can benefit each other and achieve a common optimum.

[0026] For the convenience of expression, the scale parameter prediction function in Step 2.1 is and location parameter prediction function Respectively abbreviated as and Given an ultrasonic noisy image , whose log marginal probability Can be decomposed into: ; in, represents the KL divergence of two statistical distributions; due to the non-negativity of KL divergence, constitutes The lower bound of is usually called the Evidence Lower Bound (ELBO), and its calculation expression can be derived as follows: ; in, Represents a random variable Function The mathematical expectation of the random variable The probability density function of .

[0027] The analytical expression for the first term of the above ELBO is:

[0028] ; in, Represents the digamma function.

[0029] The analytical expression for the calculation of the last two KL divergences of ELBO can be derived as follows: ; ; The ELBO expression Transformed into the minimization objective function of network training, the following ELBO loss is formed: ; Step 2.3. The physical model of ultrasound imaging is a priori constraint that must be met for the joint training optimization of SigmaSquareNet and DespecklingNet networks. Therefore, reconstructing noisy images based on the physical model of imaging and creating reconstruction losses are conducive to guiding the network's learning process and obtaining better network optimization performance.

[0030] The ultrasonic noisy image in Step 1.1 satisfies the Gaussian distribution. , using the reparameterization technique, the ultrasound noisy image can be reconstructed for: ; in, and Represent the noise-free images and variance , which can be computed as the mode of their respective variational posterior distributions.

[0031] From Step 2.1, the internal variation posterior distribution The Laplace distribution expression that satisfies the reconstruction estimate can be obtained : ; in, represents the mode of the random variable.

[0032] Similarly, the variance reconstruction estimate Through Step 2.1 internal variation inverse gamma distribution Calculate the mode of , that is: ; In this way, the reconstructed noisy image can be re-expressed as: ; The physical reconstruction loss of the noisy image is created using the L1 norm, then:

[0033] Combining the ELBO loss and noisy image reconstruction loss in Step 2.2, the joint training optimization loss of SigmaSquareNet and DespecklingNet network is obtained as: ; Step 2.4: Generate the desired noise-free image using Step 1.2 The ultrasonic noisy-expected noise-free image samples are constructed by the method, thereby creating an ultrasonic image speckle reduction dataset , and train the SigmaSquareNet and DespecklingNet networks. The detailed steps are as follows: Step 2.4.1. Create a speckle reduction dataset using two public breast tumor ultrasound image datasets BUID and BUSI BUID and BUSI contain 232 and 780 ultrasound images of benign and malignant breast tumors or normal breasts of varying sizes, respectively, which are divided into training and test sets in a ratio of 8:2. 185 images are randomly selected from BUID and 624 images are randomly selected from BUSI, totaling 809 images for the training set, and the remaining 47 images from BUID and 156 images from BUSI, totaling 203 images for the test set. As described in Step 1.2, all ultrasound noisy images The expected noise-free image All of them are obtained by filtering using the NLLR algorithm (such as Figure 1 As shown), therefore, the dataset created in this example It can be expressed as ,in Represents the NLLR algorithm filtering operation.

[0034] Step 2.4.2, based on the created data set When training the SigmaSquareNet and DespecklingNet networks, each sample image is randomly cut into 16 image slices of size 128X128 and input into the network. The mini-batch size is set to 32, the initial learning rate is 0.0001, and the piecewise constant strategy is used for decay. Random horizontal flipping is used for data enhancement of the input slice image, and the maximum round of network training is set to 100.

[0035] Step 2.5, in the testing phase after network training is completed, the ultrasonic noisy image is first input into the pre-trained SigmaSquareNet network, and the output scale parameter image can be obtained after a simple transformation to obtain the noise variance estimation image. Figure 5 This is the pre-trained SigmaSquareNet network input Figure 2 The scale parameter image inferred from the noisy image and the noise variance estimation image obtained after transformation are as follows: Figure 6 Then, the noise variance estimation image and the ultrasonic noisy image are concatenated based on the feature dimension and input into the DespecklingNet network to infer the ultrasonic speckle reduction image. Figure 7 is the pre-trained DespecklingNet network pair for this example Figure 2 Noisy images and Figure 6 The inferred speckle-reduced image after stitching the noise variance estimated image input.

[0036] like Figure 8 As shown in the figure, the bold values ​​are the best results, and the underlined values ​​are the suboptimal results. There are seven denoising algorithms in three categories. Figure 2The speckle reduction results of ultrasonic noisy images are respectively the classic method: NLLR (Non-Local Low-Rank framework); deep learning methods: RLN (Residual Learning Network) and US-Net (UltraSound Network); variational deep learning methods: VDN (Variational Denoising Network), VDIR (Variational Deep Image Restoration), VIRNet (Variational Image Restoration Network) and the method of the present invention (hereinafter referred to as VUID (Variational Ultrasound Image Despeckling)). From the speckle reduction results of this example, it can be seen that other comparison methods all have a certain degree of over-smooth areas, while the method of the present invention can effectively reduce speckle and retain more image details, with the best subjective visual effect.

[0037] Two no-reference image quality evaluation indicators, NIQE (Natural Image Quality Evaluator) and PIQE (Perceptual Image Quality Evaluator), are also used to evaluate the speckle reduction results of the contrast method. NIQE uses a set of statistical features extracted from natural images to build a multivariate Gaussian model. When evaluating image quality, NIQE calculates the same statistical features of the input image and compares them with the pre-trained Gaussian model. The quality of the image is determined by the difference between the input image features and the model. The smaller the difference, the better the image quality is considered. Because it mimics the human visual system's perception of image naturalness, it can better reflect the human eye's subjective perception of image quality. PIQE simulates the human visual system's perception of image quality. It evaluates the visual quality of an image by calculating a series of features related to the perceived quality of the image, including the image's noise level, blockiness, edge clarity, texture details, and other important factors that affect image quality judgment. The lower the PIQE score, the less distortion or artifacts there are in the image, the higher the image quality, and the more in line with human visual preferences.

[0038] Figure 8 The NIQE and PIQE indicators are calculated for both the ultrasonic noisy image and the speckle reduction result image. It can be seen that the VUID method of the present invention has the optimal NIQE value (bold value) and the suboptimal PIQE value (underlined value), which is consistent with the subjective visual evaluation results.

[0039] The objective index evaluation of the speckle reduction results for all 203 images in the test set is shown in Table 1. It can be seen that the method VUID of the present invention obtains the best speckle reduction performance (bold values) in the comparison of two different data sets BUID and BUSI and two different objective evaluation indicators.

[0040] Table 1 Objective index evaluation of ultrasound image speckle reduction using seven comparison methods

Claims

1. A deep variational ultrasound image speckle reduction method based on imaging model guidance, characterized in that: The following steps are involved: Step 1: Using the physical model of ultrasound imaging and the statistical prior distribution of related parameters, a complete Bayesian model for ultrasound image speckle reduction is constructed; Step 2. Based on the variational inference principle, two deep convolutional networks are used to parameterize the noise variance and the variational posterior distribution of the noise-free image respectively. The two deep networks are jointly trained and optimized through the created evidence lower bound ELBO loss and image physical reconstruction loss. The pre-trained network can simultaneously estimate the noise variance and infer the speckle reduction image.

2. The deep variational ultrasound image speckle reduction method based on imaging model guidance according to claim 1, characterized in that: The Step 1 includes the following steps: Step 1.1, define ultrasonic noisy images, noise-free images and multiplicative noise, build a physical model of ultrasonic imaging, and combine the imaging physical model and multiplicative noise The statistical distribution of noisy images can be derived Satisfying the mean , the variance is Gaussian distribution of Step 1.2, define the deep network for noisy images Predicted expected noise-free image, derived noise-free image The statistical prior distribution of .

3. The method for deep variational ultrasound image speckle reduction based on imaging model guidance according to claim 2, characterized in that: The specific process of Step 1.1 is as follows: Step 1.1, define the spatial domain of ultrasound image as , whose pixel position is represented by , ;like , and Represent ultrasonic noisy image, noise-free image and multiplicative noise respectively, then the physical model of ultrasonic imaging can be expressed as: ; Among them, the multiplicative noise has zero mean and variance Gaussian noise, that is: ; in, It means the mean , the variance is Gaussian distribution of Combining imaging physics models with multiplicative noise The statistical distribution of noisy images can be derived Satisfying the mean , the variance is Gaussian distribution, that is: ; Due to the variance The value of the noise-free image Directly proportional changes, so it can also be equivalently described as changing with the spatial position Variance of change , that is, definition , then the above formula can be transformed into: 。 4. The method for deep variational ultrasound image speckle reduction based on imaging model guidance according to claim 3, characterized in that: The specific process of Step 1.2 is as follows: Step 1.2, Definition Represents the deep network for noisy images The predicted expected noise-free image is Represents the ultrasound image speckle reduction dataset A pair of ultrasonic noisy-expected noise-free image samples in; in creating a speckle reduction dataset When the noise-free image is expected The noisy image can be reduced by speckle reduction method. Filter to obtain; Thus, a noise-free image is expected Only approximately but not exactly equivalent to the noise-free image At the same time, since we expect a noise-free image Can provide noise-free images The rich prior information of The statistical prior distribution of can be modeled as the following Laplace distribution: ; in, Indicates that the position parameter is , the scale parameter is Laplace distribution of ; hyperparameters is the deviation position parameter The mean absolute deviation of , which is usually set to a small positive number; Furthermore, for the pixel position in Step 1.1 Variance of change , its statistical prior distribution is usually assumed to satisfy the inverse gamma distribution, that is: ; in, The shape parameter is , the scale parameter is The inverse gamma distribution of controls the strength of the inverse gamma distribution, and is the pixel position The local neighborhood window size; Variance At pixel position The local neighborhood estimate of the noise variance of the entire image space domain is It can be calculated by the following formula: ; in, Indicates the window size is Gaussian filtering.

5. The method for deep variational ultrasound image speckle reduction based on imaging model guidance according to claim 4, characterized in that: The noisy image in Step 1.1 and Step 1.2 , Noise Variance and noise-free image The prior distribution expression of constitutes a complete Bayesian model for ultrasonic image speckle reduction.

6. The method for deep variational ultrasound image speckle reduction based on imaging model guidance according to claim 5, characterized in that: The Step 2 includes the following steps: Step 2.

1. Based on the Bayesian statistical point of view, create a variational distribution according to the variational inference principle To approximate the true posterior distribution ; The prediction function of the position parameters after variation is obtained through the SigmaSquareNet and DespecklingNet networks; Step 2.

2. Network parameters of SigmaSquareNet and DespecklingNet and Optimize; by setting the loss function, the joint training of the two deep networks can benefit each other and achieve a common optimum; Step 2.3, reconstruct the ultrasonic noisy image, combine the ELBO loss in Step 2.2 and the noisy image reconstruction loss, and get the joint training optimization loss of SigmaSquareNet and DespecklingNet network; Step 2.4: Generate the desired noise-free image using Step 1.2 The ultrasonic noisy-expected noise-free image samples are constructed by the method, thereby creating an ultrasonic image speckle reduction dataset , and train the SigmaSquareNet and DespecklingNet networks; Step 2.5, in the testing phase after network training, the ultrasonic noisy image is first input into the pre-trained SigmaSquareNet network, and the output scale parameter image can be simply transformed to obtain the noise variance estimation image; then the noise variance estimation image and the ultrasonic noisy image are concatenated based on the feature dimension and input into the DespecklingNet network together to infer the ultrasonic speckle reduction image.

7. The method for deep variational ultrasound image speckle reduction based on imaging model guidance according to claim 6, characterized in that: The specific steps of Step 2.1 are: Step 2.1, based on the Bayesian statistical point of view, the speckle reduction of ultrasound images aims to reduce the noisy images. Deriving Ultrasound Noise-Free Images and variance The posterior distribution of ; Create a variational distribution based on the variational inference principle To approximate the true posterior distribution ; Based on probability theory knowledge, variational posterior distribution Can be decomposed into: ; From Step 1.2 internal variance The prior distribution of satisfies the inverse gamma distribution, and its variational posterior distribution can be assumed to be Satisfies the following inverse gamma distribution: ; in, represents the variational posterior inverse gamma distribution based on pixel position The scale parameter prediction function is implemented by taking the parameter The SigmaSquareNet network is used to input ultrasonic noisy images. direct response to; Similarly, the noise-free image in Step 1.2 The prior distribution of , we can assume that its variational posterior distribution Satisfies the following Laplace distribution: ; in, represents the variational posterior Laplace distribution based on pixel positions The location parameter prediction function is implemented by taking the parameter DespecklingNet network for ultrasound noisy images and variance A direct response to the combined input.

8. The method for deep variational ultrasound image speckle reduction based on imaging model guidance according to claim 7, characterized in that: The specific steps of Step 2.2 are: Step 2.2: To achieve the optimal inference of the above variational posterior distribution related parameters, so that each variational posterior distribution is closer to the corresponding true posterior distribution, the network parameters of SigmaSquareNet and DespecklingNet need to be adjusted. and Therefore, a reasonable loss function must be created so that the joint training of the two deep networks can benefit each other and achieve a common optimum. For the convenience of expression, the scale parameter prediction function in Step 2.1 is and location parameter prediction function Respectively abbreviated as and ; Given an ultrasonic noisy image , whose log marginal probability Can be decomposed into: ; in, represents the KL divergence of two statistical distributions; due to the non-negativity of KL divergence, constitutes The lower bound of is called the Evidence Lower Bound, ELBO, and its calculation expression can be derived as follows: ; in, Represents a random variable Function The mathematical expectation of the random variable The probability density function of ; The analytical expression for the first term of the above ELBO is: ; in, represents the digamma function; The analytical expression for the calculation of the last two KL divergences of ELBO can be derived as follows: ; ; The ELBO expression Transformed into the minimization objective function of network training, the following ELBO loss is formed: 。 9. The imaging model-guided deep variational ultrasound image speckle reduction method according to claim 8, characterized in that: The specific steps of Step 2.3 are: Step 2.3: The ultrasonic noisy image in Step 1.1 satisfies the Gaussian distribution , using the reparameterization technique, the ultrasound noisy image can be reconstructed for: ; in, and Represent the noise-free images and variance , which can be computed as the mode of their respective variational posterior distributions; From Step 2.1, the posterior distribution of the internal variation The Laplace distribution expression that satisfies the reconstruction estimate can be obtained : ; in, represents the mode of a random variable; Similarly, the variance reconstruction estimate Through Step 2.1 internal variation inverse gamma distribution Calculate the mode of , that is: ; In this way, the reconstructed noisy image can be re-expressed as: ; The physical reconstruction loss of the noisy image is created using the L1 norm, then: ; Combining the ELBO loss and noisy image reconstruction loss in Step 2.2, the joint training optimization loss of SigmaSquareNet and DespecklingNet network is obtained as: 。

Citation Information

Cited By

  • Quality optimization method for aerial image of unmanned aerial vehicle and related equipment

    CN121073824A