A medical image denoising model training method and device

Abstract

This application discloses a method and device for training a medical image denoising model. The method comprises obtaining a noisy image and a clean image and performing preprocessing to obtain an input image, a label image, and a mask image. The input image is input into an encoder for feature extraction to obtain an original feature map. The Hessian matrix and matrix eigenvalues ​​of the original feature map are calculated. The Hessian response is calculated based on the matrix eigenvalues, and an edge feature map is extracted to obtain a deep Hessian attention feature map. The original feature map and the deep Hessian attention feature map are concatenated and input into a decoder for feature fusion to obtain an output image. The loss between the output image and the label image is calculated based on the mask image, and the weights are updated by backpropagation, thereby obtaining a trained medical image denoising model. This application facilitates tissue stratification and improves the quality of denoised images, making diagnostic results more accurate.

Description

Technical Field

[0001] The present application relates to the field of medical image processing technology, and in particular to a medical image denoising model training method and device. Background Art

[0002] Optical coherence tomography (OCT) is a low-coherence optical imaging modality. Due to its non-invasive, radiation-free, high-resolution, and real-time imaging characteristics, it is widely used in clinical imaging disciplines such as ophthalmology, dermatology, cardiology, and gastrointestinal tract. In particular, in ophthalmology, OCT is an effective tool for diagnosing a variety of eye diseases, including retinal diseases and glaucoma. However, due to the low coherence of light scattering in OCT, the inherent speckle noise reduces the signal-to-noise ratio, affecting image quality and accurate diagnosis. Classical algorithms such as filtering and non-local means can effectively remove speckle noise from OCT images. However, due to their high computational complexity, these methods are very time-consuming.

[0003] With the prevalence of deep learning technology in image processing, many classic convolutional neural network (CNN) architectures, such as U-Net and ResNet, have been applied to OCT image denoising, aiming to produce high-quality images. However, these approaches often suffer from unclear tissue boundary stratification in OCT images. Other approaches utilize GANs (Generative Adversarial Networks) to remove speckle noise from OCT images, such as those in patent applications 201910515611.2 and 202211149964.3. These approaches suffer from high model training complexity, a strong reliance on large amounts of data, and insufficient generalization across different OCT image types. Other approaches utilize three-dimensional convolutional neural networks for denoising, such as those in patent application 202311293845.X. However, these approaches consume significant computational resources, require long training times, and may face memory bottlenecks when processing large-scale images, resulting in poor denoising performance. Summary of the Invention

[0004] To this end, the present application provides a medical image denoising model training method and device to solve the problems of unclear tissue boundary stratification and poor denoising performance in existing OCT image denoising methods.

[0005] In order to achieve the above objectives, this application provides the following technical solutions:

[0006] In a first aspect, a medical image denoising model training method includes:

[0007] Step 1: Acquire a medical image of the same part; the medical image includes a noisy image and a clean image;

[0008] Step 2: Adjust the size of the noise image and the clean image, and perform normalization processing to obtain the input image and the label image;

[0009] Step 3: Create a mask image from the clean image;

[0010] Step 4: Input the input image into the encoder of the U-Net network for feature extraction to obtain the original feature map;

[0011] Step 5: Calculate the Hessian matrix of the original feature map, and calculate the matrix eigenvalues ​​of the Hessian matrix;

[0012] Step 6: Calculate the Hessian response according to the matrix eigenvalues, extract the edge feature map, and obtain the deep Hessian attention feature map;

[0013] Step 7: Concatenate the original feature map and the deep Hessian attention feature map, and input them into the decoder of the U-Net network for feature fusion to obtain the output image;

[0014] Step 8: Calculate the loss between the output image and the label image based on the mask image, and back-propagate and update the weights to obtain a trained medical image denoising model.

[0015] Preferably, the step 3 specifically includes: making an all-zero-value image with the same size as the clean image, generating a polygonal area in the target tissue area of ​​the all-zero-value image, and filling it with 1 value to obtain a mask image.

[0016] Preferably, in step 4, the U-Net network is a ResUNet network, an Attention U-Net network or a Mamba-UNet network.

[0017] Preferably, in step 5, the Hessian matrix calculation formula is:

[0018]

[0019] Among them, H i is the Hessian matrix, represents partial differential, x represents horizontal coordinate, y represents vertical coordinate, E i represents the original feature map, Indicates E i The second-order partial differential derivative in the x-direction, Indicates Ei The second-order partial differential derivative in the y direction, Indicates E i Mixed partial differential derivatives in the x and y directions.

[0020] Preferably, in step 6, the Hessian response is calculated using the Jerman method, the Frangi method or the Erdt method.

[0021] Preferably, in step 7, when the original feature map and the deep Hessian attention feature map are spliced, the ratio of the original feature map to the deep Hessian attention feature map is 1:2.

[0022] Preferably, in step 8, when calculating the loss between the output image and the label image based on the mask image, the loss function is any combination of mean square error loss, L1 loss, PSNR loss, and SSIM loss.

[0023] Preferably, when the loss function is a combination of L1 loss and SSIM loss, the loss function calculation formula is:

[0024]

[0025] in, represents the output image, b represents the label image, represents the SSIM loss, represents L1 loss, I mask represents the mask image, and represents the weight coefficient, Indicates the process of sharpening enhancement using a 3*3 convolution kernel.

[0026] Preferably, in step 8, the label image is a sharpened and enhanced label image.

[0027] In a second aspect, a medical image denoising model training device includes:

[0028] Medical image acquisition module: used to acquire medical images of the same part; the medical images include noisy images and clean images;

[0029] A data preprocessing module, configured to adjust the sizes of the noise image and the clean image and perform normalization processing to obtain an input image and a label image;

[0030] A mask image making module, configured to make a mask image from the clean image;

[0031] An original feature map extraction module is used to input the input image into the encoder of the U-Net network for feature extraction to obtain an original feature map;

[0032] A calculation module, used to calculate the Hessian matrix of the original feature map and calculate the matrix eigenvalues ​​of the Hessian matrix;

[0033] A deep Hessian attention feature map extraction module is used to calculate the Hessian response according to the matrix eigenvalues ​​and extract the edge feature map to obtain a deep Hessian attention feature map;

[0034] A feature fusion module is used to splice the original feature map and the deep Hessian attention feature map, and input them into the decoder of the U-Net network for feature fusion to obtain an output image;

[0035] A training module is used to calculate the loss between the output image and the label image based on the mask image, and back-propagate to update the weights, thereby obtaining a trained medical image denoising model.

[0036] Compared with the prior art, this application has at least the following beneficial effects:

[0037] The present application provides a medical image denoising model training method and device, which obtains a noisy image and a clean image of the same part and performs preprocessing to obtain an input image, a label image and a mask image, inputs the input image into the encoder of a U-Net network for feature extraction to obtain an original feature map; calculates the Hessian matrix of the original feature map, and calculates the matrix eigenvalues ​​of the Hessian matrix; calculates the Hessian response based on the matrix eigenvalues, and extracts the edge feature map to obtain a deep Hessian attention feature map; splices the original feature map and the deep Hessian attention feature map, and inputs them into the decoder of the U-Net network for feature fusion to obtain an output image; calculates the loss between the output image and the label image based on the mask image, and backpropagates to update the weights, thereby obtaining a trained medical image denoising model. This application enhances the U-Net network's attention to structural details by combining deep Hessian attention features. It can pay more attention to tissue boundary information and texture details in medical images when removing inherent speckle in medical images and reconstructing the original medical tissue structure, which helps tissue stratification and significantly improves the image quality after medical image denoising, making the diagnosis results more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] To more intuitively illustrate the prior art and the present application, exemplary drawings are provided below. It should be understood that the specific shapes and structures shown in the drawings should not generally be considered as limiting conditions for implementing the present application. For example, based on the technical concepts disclosed in this application and the exemplary drawings, those skilled in the art are capable of easily making routine adjustments or further optimizations to the addition / reduction / attribution division of certain units (components), the specific shapes, positional relationships, connection methods, and dimensional ratios.

[0039] Figure 1 A flowchart of a medical image denoising model training method provided in Example 1 of the present application;

[0040] Figure 2 A schematic diagram of the data preprocessing structure provided in Example 1 of the present application;

[0041] Figure 3 Schematic diagram of the ResUNet network structure based on the improved deep Hessian attention feature provided in Example 1 of the present application;

[0042] Figure 4 A schematic diagram of the improved deep Hessian attention feature supplementary connection structure provided in Example 1 of the present application;

[0043] Figure 5 This is a schematic diagram of the model training structure provided in Example 1 of this application. DETAILED DESCRIPTION

[0044] The present application will be further described below in detail through specific embodiments in conjunction with the accompanying drawings.

[0045] In the description of this application: unless otherwise specified, "plurality" means two or more. The terms "first," "second," "third," etc. in this application are intended to distinguish the objects referred to and do not have any special technical connotations (for example, they should not be understood as emphasizing importance or order). Expressions such as "including," "comprising," and "having" also mean "not limited to" (certain units, components, materials, steps, etc.).

[0046] The terms such as "upper", "lower", "left", "right", "middle", etc. cited in this application are usually used to indicate the general relative position relationship for the convenience of intuitive understanding by referring to the drawings, and are not absolute limitations on the position relationship in the actual product.

[0047] This application proposes a new medical image denoising method, namely a medical image denoising method based on an improved neural network based on deep Hessian attention features. The main idea is to enhance the U-Net network's attention to structural details by combining deep Hessian attention features. Deep Hessian attention features emphasize boundary information in OCT images, thereby injecting a kind of visual attention into the network. In addition, in order to effectively help recover more structural details, this application introduces a mask loss to improve the quality of OCT images, especially in areas of clinical importance. A mask is a binary image used to mark areas of interest, such as the retinal area in ophthalmic images. By incorporating the mask into the loss function, the denoising process is focused on these key areas, enhancing the overall image quality and diagnostic accuracy. This method effectively solves the challenges of maintaining tissue boundary integrity and enhancing texture features in OCT images, thereby significantly improving denoising performance and diagnostic accuracy.

[0048] This method also has promising application prospects in other imaging modalities, such as computed tomography (CT). Vascular structural details in CT images are crucial for diagnosing lung diseases, and the clarity of blood vessels during imaging directly impacts diagnostic accuracy. However, CT images often contain noise and artifacts, which can compromise image quality. By introducing deep Hessian attention features, this method can enhance attention to subtle structures in CT images, thereby improving the visibility and resolution of these structures, particularly vascular edges and bifurcations. Combined with a mask loss, this method ensures that the denoising process focuses more closely on key regions, such as blood vessels, in lung images, enhancing image quality in these areas and ultimately improving diagnostic accuracy and reliability. This multimodal image processing method not only improves the quality of OCT images but also significantly enhances the clarity of lung vascular structures in CT images. The promotion and application of this method could play a significant role in various medical imaging fields.

[0049] Example 1

[0050] See also Figure 1 This embodiment provides a medical image denoising model training method, including:

[0051] S1: Acquire medical images of the same part; the medical images include noise images I noisy and clean image I clean ;

[0052] S2: Adjust the size of the noisy image and the clean image, and perform normalization to obtain the input image and the label image;

[0053] See also Figure 2, this step requires the acquisition of the noise image I noisy and clean image I clean Perform data preprocessing, which includes: noisy and clean image I clean Normalize and adjust the data range to between 0 and 1. noisy is the input image I input , clean image I clean is the label image I abel .

[0054] S3: Make the clean image into a mask image;

[0055] Specifically, for each clean image I clean , this step is to make the clean image I clean A full-zero-value image of the same size is used to generate a polygonal area in the target tissue area of ​​the full-zero-value image and fill it with 1 value to obtain the mask image I make .

[0056] S4: Input the input image to the encoder of the U-Net network for feature extraction to obtain the original feature map;

[0057] Specifically, the U-Net network can be a UNet architecture network such as ResUNet network, Attention U-Net network or Mamba-UNet network, preferably ResUNet network. This embodiment is based on the ResUNet network, but only replaces the original skip connection of ResUNet with an improved deep Hessian attention feature supplementary connection. Therefore, the ResUNet network includes three parts: encoder, decoder and deep Hessian attention feature supplementary connection. Among them, the encoder and decoder both have residual structures, such as Figure 3 shown.

[0058] See also Figure 4 , assuming the encoder downsamples n times, the input image I input The feature map extracted by the encoder after the i-th downsampling is , that is, E i is the original feature map. In this step, the number of downsampling of the encoder can be increased or decreased, for example, it can be downsampled four times.

[0059] S5: Calculate the Hessian matrix of the original feature map and calculate the matrix eigenvalues ​​of the Hessian matrix;

[0060] Specifically, the Hessian matrix calculation formula is:

[0061]

[0062] Among them, H i is the Hessian matrix, represents partial differential, x represents horizontal coordinate, y represents vertical coordinate, E i represents the original feature map, Indicates E i The second-order partial differential derivative in the x-direction, Indicates E i The second-order partial differential derivative in the y direction, Indicates E i Mixed partial differential derivatives in the x and y directions.

[0063] S6: Calculate the Hessian response based on the matrix eigenvalues ​​and extract the edge feature map to obtain the deep Hessian attention feature map ;

[0064] Specifically, this step may use the Jerman method, the Frangi method, or the Erdt method to calculate the Hessian response.

[0065] S7: Concatenate the original feature map and the deep Hessian attention feature map and input them into the decoder of the U-Net network for feature fusion to obtain the output image;

[0066] Specifically, this step converts the original feature map E i and deep Hessian attention feature map After concatenation, it is used as the input of the jump connection, that is:

[0067]

[0068] The feature map D that enters the decoder for ni times of up-sample extraction n-i With F i Splice and transfer to n-i+1 times of the previous sample, that is:

[0069] .

[0070] In this step, the number of upsampling times of the decoder can be increased or decreased, for example: it can be upsampled four times;

[0071] In this step, when the original feature map and the extracted deep Hessian attention feature are concatenated as the skip connection input, the ratio between the original feature map and the deep Hessian attention feature can be changed. For example, the ratio between the original feature map and the deep Hessian attention feature map can be 1:2.

[0072] S8: Calculate the loss between the output image and the label image based on the mask image, and back-propagate and update the weights to obtain a trained medical image denoising model.

[0073] For details, please refer to Figure 5 During training, to improve the sharpness of the output image, the loss is calculated between the output image and the sharpened label image, and the weights are updated through backpropagation. When calculating the loss between the output image and the label image based on the mask image, the loss function can be any combination of mean squared error loss, L1 loss, PSNR loss, and SSIM loss. Other regularization terms or loss terms, such as contrastive loss and perceptual loss, can also be introduced to further improve denoising effects and image quality.

[0074] When the loss function is a combination of L1 loss and SSIM loss, the loss function f is calculated as:

[0075]

[0076] in, Represents the output image, that is, I output , b represents the label image, that is, I label , represents the SSIM loss, represents L1 loss, I mask represents the mask image, and represents the weight coefficient, + =1, ≥0, ≥0.

[0077] The SSIM loss calculation formula is:

[0078]

[0079] The L1 loss calculation formula is:

[0080]

[0081] in, Indicates the process of sharpening enhancement using a 3*3 convolution kernel [[0,-0.5,0], [-0.5,3, -0.5], [0, -0.5,0]], 、 、 and Respectively , the mean and standard deviation of b, c1 and c2 are two constants, n represents , the number of pixels in b, 、 express , the i-th pixel value in b.

[0082] It should be noted that the sharpening convolution kernel [[0,-0.5,0], [-0.5,3,-0.5], [0,-0.5,0]] can be replaced by other sharpening convolution kernels, such as [[0, -1, 0], [-1, 4, -1], [0, -1, 0]], etc.

[0083] When the medical image denoising model trained in this embodiment is used to denoise the OCT image, I input Input into the trained ResUNet network (i.e., medical image denoising model) based on deep Hessian attention feature improvement, and obtain I output , then, use the sharpening convolution kernel to output Enhanced ,Will The value range of is adjusted between 0 and 1, and denormalization is performed to obtain the final result.

[0084] This embodiment provides a medical image denoising model training method that, by incorporating deep Hessian attention features, enhances the U-Net network's focus on structural details. This allows for greater attention to tissue boundary information and texture details in OCT images when removing inherent speckle and reconstructing the original OCT tissue structure. This facilitates tissue stratification and significantly improves the image quality and signal-to-noise ratio of denoised OCT images, resulting in more accurate diagnostic results. Furthermore, this method is also applicable to other imaging modalities, such as CT images. For pulmonary vascular structures in CT images, this method can enhance their visibility, making vascular edges and details more distinct, thereby improving image quality and diagnostic accuracy.

[0085] Compared to the time-consuming nature of traditional denoising methods (such as filtering and non-local means), this embodiment utilizes a deep learning model to achieve more efficient computation. Even when processing large amounts of data, this method can quickly generate high-quality denoised images with high time efficiency. Furthermore, by introducing mask loss, this embodiment can focus on clinically important areas, such as the retina in OCT images and pulmonary vascular structures in CT images, thereby further improving image quality and diagnostic accuracy in these areas. This is of great significance for the precise location and diagnosis of lesions in clinical practice.

[0086] Example 2

[0087] This embodiment provides a medical image denoising model training device, including:

[0088] Medical image acquisition module: used to acquire medical images of the same part; the medical images include noisy images and clean images;

[0089] A data preprocessing module, configured to adjust the sizes of the noise image and the clean image and perform normalization processing to obtain an input image and a label image;

[0090] A mask image making module, configured to make a mask image from the clean image;

[0091] An original feature map extraction module is used to input the input image into the encoder of the U-Net network for feature extraction to obtain an original feature map;

[0092] A calculation module, used to calculate the Hessian matrix of the original feature map and calculate the matrix eigenvalues ​​of the Hessian matrix;

[0093] A deep Hessian attention feature map extraction module is used to calculate the Hessian response according to the matrix eigenvalues ​​and extract the edge feature map to obtain a deep Hessian attention feature map;

[0094] A feature fusion module is used to splice the original feature map and the deep Hessian attention feature map, and input them into the decoder of the U-Net network for feature fusion to obtain an output image;

[0095] A training module is used to calculate the loss between the output image and the label image based on the mask image, and back-propagate to update the weights, thereby obtaining a trained medical image denoising model.

[0096] For the specific implementation content of each module in a medical image denoising model training device, please refer to the above definition of a medical image denoising model training method, which will not be repeated here.

[0097] The technical features of the above embodiments can be combined arbitrarily (as long as there is no contradiction in the combination of these technical features). In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described; these embodiments that are not explicitly written should also be considered to be within the scope of this specification.

Claims

1. A medical image denoising model training method, characterized in that: include: Step 1: Acquire a medical image of the same part; the medical image includes a noisy image and a clean image; Step 2: Adjust the size of the noise image and the clean image, and perform normalization processing to obtain the input image and the label image; Step 3: Create a mask image from the clean image; Step 4: Input the input image into the encoder of the U-Net network for feature extraction to obtain the original feature map; Step 5: Calculate the Hessian matrix of the original feature map, and calculate the matrix eigenvalues ​​of the Hessian matrix; Step 6: Calculate the Hessian response according to the matrix eigenvalues, extract the edge feature map, and obtain the deep Hessian attention feature map; Step 7: Concatenate the original feature map and the deep Hessian attention feature map, and input them into the decoder of the U-Net network for feature fusion to obtain the output image; specifically: first, the original feature map E i and the deep Hessian attention feature map H′ i After splicing, it is used as the input of the jump connection and is expressed as: F i =cat(H′ i , E i ); Then, the feature map D is extracted by the decoder for ni times n-i With F i Splicing and transporting to n-i+1 times the previous sample, expressed by the formula: D n-i+1 =Decoder n-i+1 (cat(D n-i ,F i )); Step 8: Calculate the loss between the output image and the label image based on the mask image, and back-propagate and update the weights to obtain a trained medical image denoising model; wherein the loss function is a combination of L1 loss and SSIM loss, and the loss function calculation formula is: f(a,b)=γ*f ssim (a*I mask ,s(b)*I mask )+β*f L1 (a*I mask ,s(b)*I mask ) Among them, a represents the output image, b represents the label image, and f ssim represents the SSIM loss, f L1 represents L1 loss, I mask represents the mask image, γ and β represent weight coefficients, and s(·) represents the process of sharpening enhancement using a 3*3 convolution kernel.

2. The medical image denoising model training method according to claim 1, characterized in that: The step 3 specifically includes: making an all-zero-value image with the same size as the clean image, generating a polygonal area in the target tissue area of ​​the all-zero-value image, and filling it with 1 values ​​to obtain a mask image.

3. The medical image denoising model training method according to claim 1, characterized in that: In step 4, the U-Net network is a ResUNet network, an Attention U-Net network, or a Mamba-UNet network.

4. The medical image denoising model training method according to claim 1, characterized in that: In step 5, the Hessian matrix calculation formula is: Among them, H i is the Hessian matrix, represents partial differential, x represents horizontal coordinate, y represents vertical coordinate, E i represents the original feature map, Indicates E i The second-order partial differential derivative in the x-direction, Indicates E i The second-order partial differential derivative in the y direction, Indicates E i Mixed partial differential derivatives in the x and y directions.

5. The medical image denoising model training method according to claim 1, characterized in that: In step 6, the Hessian response is calculated using the Jerman method, the Frangi method, or the Erdt method.

6. The medical image denoising model training method according to claim 1, characterized in that: In step 7, when the original feature map and the deep Hessian attention feature map are spliced, the ratio of the original feature map to the deep Hessian attention feature map is 1:

2.

7. The medical image denoising model training method according to claim 1, characterized in that: In step 8, when calculating the loss between the output image and the label image based on the mask image, the loss function is any combination of mean square error loss, L1 loss, PSNR loss, and SSIM loss.

8. The medical image denoising model training method according to claim 1, characterized in that: In step 8, the label image is a sharpened and enhanced label image.

9. A medical image denoising model training device, characterized in that: include: Medical image acquisition module: used to acquire medical images of the same part; the medical images include noisy images and clean images; A data preprocessing module, configured to adjust the sizes of the noise image and the clean image and perform normalization processing to obtain an input image and a label image; A mask image making module, configured to make a mask image from the clean image; An original feature map extraction module is used to input the input image into the encoder of the U-Net network for feature extraction to obtain an original feature map; A calculation module, used to calculate the Hessian matrix of the original feature map and calculate the matrix eigenvalues ​​of the Hessian matrix; A deep Hessian attention feature map extraction module is used to calculate the Hessian response according to the matrix eigenvalues ​​and extract the edge feature map to obtain a deep Hessian attention feature map; The feature fusion module is used to splice the original feature map and the deep Hessian attention feature map, and input them into the decoder of the U-Net network for feature fusion to obtain the output image; specifically: first, the original feature map Ei and the deep Hessian attention feature map H′ are spliced ​​together. i After splicing, it is used as the input of the jump connection and is expressed as: F i =cat(H′ i , E i ); Then, the feature map Dn-i and Fi extracted from the previous sample of the decoder for the nith time are spliced ​​and transmitted to the n-i+1th time sample, which is expressed as: D n-i+1 =Decoder n-i+1 (cat(D n-i , F i )); A training module is used to calculate the loss between the output image and the label image based on the mask image, and back-propagate to update the weights, thereby obtaining a trained medical image denoising model; wherein the loss function is a combination of L1 loss and SSIM loss, and the loss function calculation formula is: f(a,b)=γ*f ssim (a*I mask ,s(b)*I mask )+β*f L1 (a*I mask ,s(b)*I mask ) Among them, a represents the output image, b represents the label image, and f ssim represents the SSIM loss, f L1 represents L1 loss, I mask represents the mask image, γ and β represent weight coefficients, and s(·) represents the process of sharpening enhancement using a 3*3 convolution kernel.

Images