An image deblurring method based on improved U-Net

By improving the multi-scale feature extraction and dynamic feature fusion of U-Net network, the lack of targeted attention and multimodal MRI compatibility in the existing methods is solved, and high-quality medical image defuzzing effect is achieved, which improves the accuracy and clarity of brain glioma diagnosis.

CN120451003BActive Publication Date: 2025-09-05SICHUAN UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202510936135.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-07-08
Publication Date
2025-09-05
Estimated Expiration
2045-07-08

AI Technical Summary

Technical Problem

The existing U-Net-based medical image defuzzing method lacks a targeted attention mechanism when dealing with brain glioma MRI, resulting in the loss of key anatomical details and is difficult to compatible with the heterogeneous texture characteristics of multimodal MRI, affecting diagnostic accuracy.

Method used

Using an improved U-Net network, through multi-scale feature extraction, channel attention mechanism and dynamic feature fusion, key anatomical structure features are enhanced and multimodal MRI feature fusion is optimized to achieve image reconstruction.

Benefits of technology

It significantly improves image clarity and diagnostic accuracy, especially the preoperative localization and grading diagnosis of brain glioma, and maintains the high fidelity of key imaging biomarkers.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120451003B_ABST
    Figure CN120451003B_ABST
Patent Text Reader

Abstract

This invention discloses an image deblurring method based on an improved U-Net, belonging to the field of medical image processing technology. The method comprises: S1, extracting features from a blurred medical image using three parallel convolution paths to obtain a multi-scale feature map; S2, enhancing the multi-scale feature map and fusing the enhanced multi-scale feature map at the same scale; S3, inputting the fused multi-scale feature map into a trained improved U-Net network for deep feature extraction and reconstruction, resulting in a clear medical image as output. Experimental verification demonstrates that this method not only effectively removes image blur and restores image details, but also demonstrates excellent generalization performance across a variety of blurry scenarios, suggesting great potential for application in fields such as medical image processing.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image processing, and in particular relates to an image deblurring method based on an improved U-Net (image segmentation network). Background Art

[0002] In the field of medical imaging, clear image quality is crucial for doctors to perform computer-assisted diagnosis, assist with interventional treatment, and determine surgical planning. Specifically, in the clinical diagnosis of gliomas, clear visualization based on MRI images plays a crucial role in preoperative localization, malignancy grading, and surgical navigation. However, during acquisition, MRI images are susceptible to multiple factors, including motion artifacts caused by vibration of the device's gradient coils, patient spontaneous breathing and neural tremors, body displacement caused by long scanning sequences, and radiofrequency field inhomogeneities. These artifacts can lead to blurring that significantly reduces spatial resolution and signal-to-noise ratio. This degradation effect not only obscures the infiltrative boundary features of gliomas (such as the abnormal signal transition zone in T2-FLAIR sequences) but also weakens the characterization of internal tumor heterogeneity (such as the necrotic and cystic areas in enhanced T1WI), directly impacting the imaging assessment of WHO grading criteria and molecular pathological features.

[0003] In recent years, deep learning-based U-Net and its variants have shown potential in medical image deblurring tasks, and their symmetric encoder-decoder structure can effectively capture multi-scale features. However, existing methods still have significant shortcomings in medical scenarios. Current methods face two challenges when processing brain glioma MRI: First, the traditional U-Net architecture fails to establish a targeted attention mechanism at different scales during the feature extraction stage. Key anatomical structures at different scales (such as tiny lesions, vascular branches, or tissue boundaries) may not be fully enhanced due to the lack of targeted attention mechanism, resulting in the loss of key diagnostic details in the deblurred image and a lack of targeted enhancement capability for the microstructural boundaries between the tumor core, edema zone, and normal white matter (such as peritumoral infiltration <2mm); Second, existing feature fusion strategies are difficult to be compatible with the heterogeneous texture features of multimodal MRI (such as T1WI and T2WI), and are prone to geometric distortion of key parameters such as the diffusion tensor anisotropy fraction (FA) during cross-modal reconstruction. In particular, for the differential diagnosis of IDH mutant and wild-type gliomas, existing methods have significant limitations in maintaining the magnetic susceptibility effect of microbleeds in T2* / SWI sequences. Therefore, there is an urgent need to develop MRI-specific deblurring methods that can adaptively focus on key image areas and optimize multi-scale feature fusion to meet the strict requirements of imaging biomarker fidelity for precise diagnosis and treatment of brain gliomas. Summary of the Invention

[0004] In response to the above-mentioned deficiencies in the existing technology, the present invention proposes an image deblurring method based on an improved U-Net to address the problems of the existing methods lacking a targeted attention mechanism at multiple scales in the feature extraction stage, making it difficult to directionally enhance the boundaries of peritumoral microstructures <2 mm; as well as the problems of being incompatible with the heterogeneous textures of multimodal MRI such as T1WI and T2WI and being unable to preserve the magnetic susceptibility effect of microbleeds in T2* / SWI sequences.

[0005] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is: an image deblurring method based on an improved U-Net, comprising the following steps:

[0006] S1. Feature extraction is performed on the medical blurred image through three parallel convolution paths to obtain a multi-scale feature map;

[0007] S2. Perform feature enhancement on the multi-scale feature map, and perform feature fusion on the enhanced multi-scale feature map at the same scale;

[0008] S3. Input the fused multi-scale feature map into the trained improved U-Net network to perform deep feature extraction and reconstruction, and output a clear medical image.

[0009] Among them, the improved U-Net network is an encoder-decoder structure. The encoder gradually expands the receptive field of view through downsampling operations to capture the global fuzzy features of the input feature map. The decoder gradually restores spatial details through upsampling operations and realizes image reconstruction by combining the fusion features output by the dynamic feature fusion mechanism to obtain clear medical images.

[0010] Furthermore, step S2 includes the following sub-steps:

[0011] S21, perform feature enhancement processing on each multi-scale feature map through the channel attention mechanism;

[0012] The multi-scale feature map includes a first feature map, a second feature map, and a third feature map obtained by extracting features based on different convolution kernels;

[0013] S22, the first feature map and the third feature map after feature enhancement are pooled and interpolated respectively to make their sizes consistent with the second feature map. Figure 1 To;

[0014] S23. Concatenate the first feature map, the second feature map, and the third feature map of the same size, reduce the number of channels through a 1×1 convolution operation, and fuse the features to obtain a fused multi-scale feature map.

[0015] Furthermore, in the improved U-Net network, the encoder includes four downsampling modules whose number of channels is gradually doubled, and the decoder includes four upsampling modules whose number of channels is gradually halved, corresponding one-to-one to the downsampling modules.

[0016] The output of each downsampling module is also connected through the input of the last upsampling module of the dynamic fusion module.

[0017] Furthermore, step S3 includes the following sub-steps:

[0018] S31. In the encoder, feature extraction is performed on the input fused multi-scale feature map in sequence through four sequentially connected downsampling modules to obtain a first intermediate feature, a second intermediate feature, a third intermediate feature, and a fourth intermediate feature, respectively;

[0019] S32, fusing the first intermediate feature, the second intermediate feature, the third intermediate feature, and the fourth intermediate feature through a dynamic feature fusion mechanism, outputting a fused feature, and inputting the fused feature into the last upsampling module in the decoder;

[0020] S33. In the decoder, the spatial details of the fused multi-scale feature map are gradually restored through the upsampling operation of four sequentially connected upsampling modules, and the fused features are combined to achieve image reconstruction and obtain a clear medical image.

[0021] Furthermore, in step S32, the method of performing feature fusion through the dynamic feature fusion mechanism is specifically as follows:

[0022] S32-1. Use a maximum pooling operation to unify the first intermediate feature, the second intermediate feature, the third intermediate feature, and the fourth intermediate feature to the scale of the minimum intermediate feature;

[0023] S32-2. Use a 1×1 convolution operation to unify the number of channels of the first to fourth intermediate features with unified scales to the number of channels of the intermediate feature with the largest scale, thereby obtaining the first to fourth intermediate features with unified dimensions.

[0024] S32-3, generating corresponding weight vectors from the first to fourth intermediate features with unified dimensions through a weight generator;

[0025] S32-4. According to the weight vector, the first to fourth intermediate features with unified dimensions are weightedly fused to obtain a fused feature.

[0026] Furthermore, the weight generator includes three convolutional layers and one fully connected layer.

[0027] Furthermore, the fusion feature is expressed as:

[0028]

[0029] Where, Represents the features after weighted fusion of multiple features in a unified dimension. represents the intermediate features of the i-th unified dimension, It represents the weight coefficient of the intermediate feature of the i-th unified dimension, and h, w, and c represent the three dimensions of the feature.

[0030] Furthermore, the improved U-Net network is trained in an end-to-end training manner, and the training loss function is:

[0031]

[0032] Where, 、 and Represent Charbonnier loss, perceptual loss and SSIM loss respectively, 、 and Respectively 、 and The weight coefficient of .

[0033] Furthermore, Charbonnier lost Expressed as:

[0034]

[0035] Where, Indicates a true and clear image. represents the deblurred image predicted by the improved U-Net network, represents the smoothing coefficient used to prevent gradient explosion;

[0036] Perceptual loss Expressed as:

[0037]

[0038] Where, represents the feature extractor, ‖·‖ represents the L1 norm calculation;

[0039] SSIM loss Expressed as:

[0040]

[0041] Where, Indicates the structural similarity between the image x before deblurring and the image y after deblurring, 、 Represents the average value of all pixels in the two images, represents the covariance of the two graphs, 、 represents the standard deviation of the two graphs, =(0.01L)², =(0.03L)², where L represents the pixel value range.

[0042] The beneficial effects of the present invention are:

[0043] 1. By introducing multi-scale feature extraction and a channel attention mechanism, this invention enables the model to adaptively enhance the features of key anatomical structures at different scales in medical images (such as tiny lesions, vascular branches, and tissue edges). This mechanism effectively avoids the loss of information about subtle lesions and tissue edges during the deblurring process, significantly improving the visualization clarity of tumor boundaries and surrounding structures, and providing a more reliable imaging basis for preoperative localization and graded diagnosis.

[0044] 2. This invention adopts a dynamic feature fusion mechanism in the improved U-Net decoding stage, which can adaptively adjust the fusion weights according to the texture heterogeneity of different MRI modalities (T1WI, T2WI, etc.). It not only retains the geometric accuracy of diffusion tensor anisotropy (FA) but also maintains the magnetic susceptibility effect of microbleeds in T2* / SWI sequences. Thus, it maintains the high fidelity of key imaging biomarkers during cross-modal reconstruction and improves the ability to remove artifacts of complex blur types.

[0045] 3. The present invention optimizes multi-scale feature extraction, channel attention mechanism, and dynamic feature fusion mechanism based on the particularity of medical images. Compared with existing technologies, it has significant advantages in maintaining the authenticity of tissue texture and obtaining the integrity of key diagnostic information. For blurring caused by typical medical imaging problems such as equipment jitter, patient movement, and low signal-to-noise ratio, this method can more effectively restore clear images with diagnostic value, significantly improving the clinical application value and analysis efficiency of medical images. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 The present invention provides an image deblurring method based on an improved U-Net.

[0047] Figure 2 This is the image deblurring framework provided by the present invention.

[0048] Figure 3 Schematic diagram of the multi-scale feature extraction and channel attention mechanism provided by the present invention.

[0049] Figure 4 This is the improved U-Net network structure diagram provided by the present invention.

[0050] Figure 5 Schematic diagram of the dynamic feature fusion mechanism provided by the present invention. DETAILED DESCRIPTION

[0051] The specific embodiments of the present invention are described below to facilitate understanding of the present invention by those skilled in the art. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations utilizing the concepts of the present invention are protected.

[0052] The embodiment of the present invention provides an image deblurring method based on an improved U-Net. Figures 1 and 2 As shown, the following steps are included:

[0053] S1. Feature extraction is performed on the medical blurred image through three parallel convolution paths to obtain a multi-scale feature map;

[0054] S2. Perform feature enhancement on the multi-scale feature map, and perform feature fusion on the enhanced multi-scale feature map at the same scale;

[0055] S3. Input the fused multi-scale feature map into the trained improved U-Net network to perform deep feature extraction and reconstruction, and output a clear medical image.

[0056] Among them, the improved U-Net network is an encoder-decoder structure. The encoder gradually expands the receptive field of view through downsampling operations to capture the global fuzzy features of the input feature map. The decoder gradually restores spatial details through upsampling operations and realizes image reconstruction by combining the fusion features output by the dynamic feature fusion mechanism to obtain clear medical images.

[0057] In step S1 of the embodiment of the present invention, for a blurred medical image X of size H×W×C, three convolution kernels of different sizes (3×3, 5×5, and 7×7) are used in parallel to perform convolution operations to extract features. These convolution kernels use the same padding to maintain spatial resolution, thereby obtaining multi-scale feature maps W1, W2, and W3 with different receptive fields, which are respectively expressed as: ×3 (X), W2 = Conv 5×5 (X), W3 = Conv 7×7 (X).

[0058] like Figure 3 As shown, step S2 of the embodiment of the present invention includes the following sub-steps:

[0059] S21, perform feature enhancement processing on each multi-scale feature map through the channel attention mechanism;

[0060] The multi-scale feature map includes a first feature map W1, a second feature map W2, and a third feature map W3 obtained by extracting features based on different convolution kernels;

[0061] S22, the first feature map W1 and the third feature map W3 after feature enhancement are pooled and interpolated respectively to make their sizes consistent with the second feature map Figure 1 To;

[0062] S23. Concatenate the first feature map, the second feature map, and the third feature map of the same size, reduce the number of channels through a 1×1 convolution operation, and fuse the features to obtain a fused multi-scale feature map.

[0063] In step S21 of this embodiment, the multi-scale feature maps W1, W2 and W3 are enhanced with important features and suppressed with unimportant features through the channel attention mechanism; specifically, global average pooling is first performed on each feature map to obtain the global feature information of each channel, and then this information is input into the MLP consisting of the first fully connected layer and the second fully connected layer. After the Swish activation function and the sigmoid activation function, a channel attention weight vector is generated. Finally, the weight vector is multiplied by the original feature map channel by channel to enhance important features and suppress unimportant features, thereby obtaining the first feature map, the second feature map and the third feature map.

[0064] Among them, the process of enhancing image features through the channel attention mechanism is expressed as:

[0065]

[0066] Where, represents the image features enhanced by the attention mechanism, represents the sigmoid activation function, represents channel-wise multiplication, represents the attention weight vector, represents a two-layer fully connected network, represents average pooling.

[0067] In the embodiment of the present invention, Figure 4 As shown in the figure, in the improved U-Net network, the encoder includes four downsampling modules with the number of channels gradually doubled, and the decoder includes four upsampling modules with the number of channels gradually halved, corresponding to the downsampling modules one by one; the output of each downsampling module is also connected to the input of the last upsampling module of the dynamic fusion module.

[0068] Based on the above improved U-Net network structure, step S3 of this embodiment includes the following sub-steps:

[0069] S31. In the encoder, feature extraction is performed on the input fused multi-scale feature map in sequence through four sequentially connected downsampling modules to obtain a first intermediate feature, a second intermediate feature, a third intermediate feature, and a fourth intermediate feature, respectively;

[0070] S32, fusing the first intermediate feature, the second intermediate feature, the third intermediate feature, and the fourth intermediate feature through a dynamic feature fusion mechanism, outputting a fused feature, and inputting the fused feature into the last upsampling module in the decoder;

[0071] S33. In the decoder, the spatial details of the fused multi-scale feature map are gradually restored through the upsampling operation of four sequentially connected upsampling modules, and the fused features are combined to achieve image reconstruction and obtain a clear medical image.

[0072] In step S31 of this embodiment, in the improved U-Net network, multiple downsampling modules in the encoder use convolution layers and maximum pooling layers to perform feature extraction and spatial resolution reduction, gradually expanding the receptive field to capture the global blur features of the image. This process generates intermediate features of four scales, namely, the first intermediate feature T1, the second intermediate feature T2, the third intermediate feature T3 and the fourth intermediate feature T4.

[0073] In step S32 of this embodiment, as Figure 5 As shown in FIG, the method for performing feature fusion through the dynamic feature fusion mechanism is as follows:

[0074] S32-1, use the maximum pooling operation to unify the first intermediate feature T1, the second intermediate feature T2, the third intermediate feature T3 and the fourth intermediate feature T4 to the scale of the minimum intermediate feature (H min ×W min );

[0075] S32-2, use 1×1 convolution operation to unify the number of channels of the first to fourth intermediate features of unified scale to the number of channels C of the largest scale intermediate feature max , we get the first to fourth intermediate features with unified dimensions;

[0076] S32-3, the first to fourth intermediate features with unified dimensions are used to generate corresponding weight vectors K through the weight generator T1 -K T4 ;

[0077] S32-4. According to the weight vector, the first to fourth intermediate features with unified dimensions are weightedly fused to obtain a fused feature.

[0078] In this embodiment, the weight generator includes three convolutional layers and one fully connected layer, which is expressed as:

[0079] KT =FC(Conv3(Conv2(Conv1(T))))

[0080] In the formula, T represents the middle layer feature, K T Represents its weight coefficient, Conv1, Conv2, Conv3 represent three-layer convolution operations, and FC represents the linear transformation of the fully connected layer.

[0081] In this embodiment, the fusion feature is expressed as:

[0082]

[0083] Where, Represents the features after weighted fusion of multiple features in a unified dimension. represents the intermediate features of the i-th unified dimension, It represents the weight coefficient of the intermediate feature of the i-th unified dimension, and h, w, and c represent the three dimensions of the feature.

[0084] In step S33 of this embodiment, in the decoder, corresponding to the encoder, in multiple upsampling modules, image spatial details are restored through upsampling operations such as deconvolution or interpolation, and the fusion features obtained by dynamically fusing the encoder output features are combined to achieve accurate image reconstruction.

[0085] In this embodiment of the present invention, a private dataset containing brain MRI images of glioma patients is used to train and test the improved U-Net network in an end-to-end manner, and the training loss function is:

[0086]

[0087] Where, 、 and Represent Charbonnier loss, perceptual loss and SSIM loss respectively, 、 and Respectively 、 and The weight coefficient of 、 and to balance the values ​​of different loss functions.

[0088] Among them, Charbonnier lost As the basic pixel-level reconstruction loss, it ensures the overall structural accuracy of the image, which is expressed as:

[0089]

[0090] Where, Indicates a true and clear image. represents the deblurred image predicted by the improved U-Net network, Represents the smoothing factor used to prevent gradient explosion.

[0091] Perceptual loss Ensure that the generated image is consistent with the real image in terms of high-level semantic features, thereby improving the visual quality, which can be expressed as:

[0092]

[0093] Where, represents the feature extractor, and ‖·‖1 represents the L1 norm calculation.

[0094] SSIM loss Improve the quality of generated images from the perspective of structural similarity, which is expressed as:

[0095]

[0096] Where, Indicates the structural similarity between the image x before deblurring and the image y after deblurring, 、 Represents the average value of all pixels in the two images, represents the covariance of the two graphs, 、 represents the standard deviation of the two graphs, =(0.01L)², =(0.03L)², where L represents the pixel value range.

[0097] In this embodiment of the present invention, when training the improved U-Net network, the batch size is set to 32, and the network is trained 100 times (epoch = 100) on the training dataset. Through multiple rounds of training, the network is encouraged to fully learn the data characteristics. The Adam optimizer is selected, which combines the advantages of Adagrad and RMSProp and performs well on non-convex optimization problems. β1 = 0.9, β2 = 0.999, and ε = 1e-8 are set as the exponential decay rates of the first-order and second-order moment estimates, respectively, and as a small constant to prevent division by zero errors. The initial learning rate is set to 1e-3. When the validation set loss decreases by less than 0.01 for five consecutive epochs, the learning rate is multiplied by 0.1 to decay. This ensures rapid parameter updates in the early stages of training and stable convergence in the later stages, preventing the model from oscillating around the optimal solution.

[0098] This embodiment of the present invention provides a test experiment for the above-mentioned image deblurring method to evaluate its performance compared to a traditional U-Net. First, a traditional U-Net was selected as the sole reference. The evaluation metrics focused on image quality and model performance. Image quality was evaluated using the Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Multi-Scale Perceptual Similarity (LPIPS). Higher PSNR and SSIM values ​​indicate higher image deblurring quality; lower LPIPS values ​​indicate a greater perceptual similarity between the generated image and the real image. Model performance was measured using inference time and parameter count. Inference time reflects model efficiency, while parameter count reflects model complexity.

[0099] The experimental results are shown in Table 1, which demonstrate that the present invention exhibits significant advantages. PSNR increased by 2.49dB from 28.76dB of the traditional U-Net to 31.25dB; SSIM increased by 0.028 from 0.923 to 0.951; and LPIPS decreased by 31.0% from 0.142 to 0.098. This demonstrates that the present method can effectively improve the quality of deblurred images, perceptually closer to true, clear images. In terms of model performance, the number of parameters in the present method increased from 34.2M to 38.3M, a 12.0% increase, and the inference time increased from 38ms to 42ms, a 10.5% increase. This demonstrates that the present method significantly improves the quality of image deblurring at the expense of a moderate increase in model complexity and inference time.

[0100] Table 1: Comparison between traditional U-Net and this method

[0101] index Traditional U-Net This method Improvement PSNR(dB) 28.76 31.25 +2.49 SSIM 0.923 0.951 +0.028 LPIPS 0.142 0.098 -31.0% Inference time (ms) 38 42 +10.5% Parameter quantity (M) 34.2 38.3 +12.0%

[0102] The present invention also provides an experimental comparison between the present invention and existing methods. This experimental comparison evaluates the performance of various image deblurring models, including DeblurGAN, SDWNet, and Restormer, which are widely used in these fields, as well as U-Net-based improvements, U-Net+(our) and U-Net++(our), to comprehensively examine the competitiveness of the present invention compared with similar technologies. U-Net+(our) represents an improved solution that uses only the multi-scale attention mechanism, while U-Net++(our) represents an improved solution that uses both multi-scale attention and dynamic feature fusion.

[0103] The experimental results are shown in Table 2. U-Net++(our) performs exceptionally well in terms of image quality, achieving a PSNR of 32.60dB and an SSIM of 0.951, surpassing most compared models. SDWNet and Restormer also perform well, achieving PSNRs of 31.26dB and 31.62dB, respectively, and SSIMs of 0.942 and 0.9103. Regarding model performance, the U-Net, U-Net+(our), and U-Net++(our) models each have a size of only 3.81MB, significantly smaller than the other models. U-Net has the shortest inference time, at only 0.03s, compared to SDWNet at 0.17s, DeblurGAN at 0.85s, and Restormer at 5.6s. Overall, U-Net++(our) strikes a good balance between image quality and model size.

[0104] Table 2: Performance comparison of various image deblurring models

[0105] Model Name PSNR (dB) SSIM Model size (M) Inference time (s) DeblurGAN 28.7 0.927 37.1 0.85 SDWNet 31.26 0.942 36.54 0.17 Restormer 31.62 0.9103 67.9 5.6 U-Net 28.76 0.824 3.81 0.03 U-Net+(our) 30.95 0.907 3.81 0.05 U-Net++(our) 32.60 0.951 3.81 0.05

[0106] Specific embodiments are used in the present invention to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present invention.

[0107] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and it should be understood that the scope of protection of the present invention is not limited to such specific descriptions and embodiments. Those skilled in the art can make various other specific variations and combinations based on the technical teachings disclosed in the present invention without departing from the essence of the present invention, and such variations and combinations are still within the scope of protection of the present invention.

Claims

1. An image deblurring method based on improved U-Net, characterized in that: The following steps are involved: S1. Feature extraction is performed on the medical blurred image through three parallel convolution paths to obtain a multi-scale feature map; S2. Perform feature enhancement on the multi-scale feature map, and perform feature fusion on the enhanced multi-scale feature map at the same scale; S3. Input the fused multi-scale feature map into the trained improved U-Net network to perform deep feature extraction and reconstruction, and output a clear medical image. The improved U-Net network is an encoder-decoder structure. The encoder gradually expands the perception field through downsampling operations to capture the global fuzzy features of the input feature map. The decoder gradually restores spatial details through upsampling operations and combines the fusion features output by the dynamic feature fusion mechanism to achieve image reconstruction and obtain clear medical images. In the improved U-Net network, the encoder includes four downsampling modules whose number of channels is gradually doubled, and the decoder includes four upsampling modules whose number of channels is gradually halved, corresponding to the downsampling modules one by one. The output of each downsampling module is also connected through the input of the last upsampling module of the dynamic fusion module; The step S3 includes the following sub-steps: S31. In the encoder, feature extraction is performed on the input fused multi-scale feature map in sequence through four sequentially connected downsampling modules to obtain a first intermediate feature, a second intermediate feature, a third intermediate feature, and a fourth intermediate feature, respectively; S32, fusing the first intermediate feature, the second intermediate feature, the third intermediate feature, and the fourth intermediate feature through a dynamic feature fusion mechanism, outputting a fused feature, and inputting the fused feature into the last upsampling module in the decoder; S33. In the decoder, the spatial details of the fused multi-scale feature map are gradually restored through the upsampling operation of four sequentially connected upsampling modules, and the fused features are combined to achieve image reconstruction to obtain a clear medical image; In step S32, the method of performing feature fusion through the dynamic feature fusion mechanism is specifically as follows: S32-1. Use a maximum pooling operation to unify the first intermediate feature, the second intermediate feature, the third intermediate feature, and the fourth intermediate feature to the scale of the minimum intermediate feature; S32-2. Use a 1×1 convolution operation to unify the number of channels of the first to fourth intermediate features with unified scales to the number of channels of the intermediate feature with the largest scale, thereby obtaining the first to fourth intermediate features with unified dimensions. S32-3, generating corresponding weight vectors from the first to fourth intermediate features with unified dimensions through a weight generator; S32-4. According to the weight vector, the first to fourth intermediate features with unified dimensions are weightedly fused to obtain a fused feature.

2. The image deblurring method based on improved U-Net according to claim 1, characterized in that The step S2 comprises the following sub-steps: S21, perform feature enhancement processing on each multi-scale feature map through the channel attention mechanism; The multi-scale feature map includes a first feature map, a second feature map, and a third feature map obtained by extracting features based on different convolution kernels; S22, performing pooling and interpolation processing on the first feature map and the third feature map after feature enhancement, respectively, so that their sizes are consistent with the second feature map; S23. Concatenate the first feature map, the second feature map, and the third feature map of the same size, reduce the number of channels through a 1×1 convolution operation, and fuse the features to obtain a fused multi-scale feature map.

3. The image deblurring method based on improved U-Net according to claim 1, characterized in that The weight generator includes three convolutional layers and one fully connected layer.

4. The image deblurring method based on improved U-Net according to claim 3, characterized in that The fusion feature is expressed as: ; Where, Represents the features after weighted fusion of multiple features in a unified dimension. represents the intermediate features of the i-th unified dimension, It represents the weight coefficient of the intermediate feature of the i-th unified dimension, and h, w, and c represent the three dimensions of the feature.

5. The image deblurring method based on improved U-Net according to claim 1, characterized in that The improved U-Net network is trained in an end-to-end training mode, and the training loss function is: ; Where, 、 and Represent Charbonnier loss, perceptual loss and SSIM loss respectively, 、 and Respectively 、 and The weight coefficient of .

6. The image deblurring method based on improved U-Net according to claim 5, characterized in that Charbonnier loss Expressed as: ; Where, Indicates a true and clear image. represents the deblurred image predicted by the improved U-Net network, represents the smoothing coefficient used to prevent gradient explosion; Perceptual loss Expressed as: S32, fusing the first intermediate feature, the second intermediate feature, the third intermediate feature, and the fourth intermediate feature through a dynamic feature fusion mechanism, outputting a fused feature, and inputting the fused feature into the last upsampling module in the decoder; ; Where, represents the feature extractor, ‖·‖ represents the L1 norm calculation; SSIM loss Expressed as: ; Where, Indicates the structural similarity between the image x before deblurring and the image y after deblurring, 、 Represents the average value of all pixels in the two images, represents the covariance of the two graphs, 、 represents the standard deviation of the two graphs, =(0.01L)², =(0.03L)², where L represents the pixel value range.

Citation Information

Patent Citations

  • Transform and multi-scale feature fusion-based image deblurring system and method

    CN119904386A

  • Image deblurring method based on U-Net network

    CN120070251A