Low-quality medical image unsupervised blind super-resolution denoising method based on NAFA-DIKNet

Abstract

The invention discloses a low-quality medical image unsupervised blind super-resolution denoising method based on NAFA-DIKNet, and aims to perform efficient blind super-resolution denoising processing on a low-quality medical image through unsupervised learning. According to the method, a DIP network, a DKP model and an NAFA module are combined to form a collaborative optimization framework. The DIP network generates a preliminary high-resolution image through random noise input and a simple encoder-decoder structure, generates a blurring kernel through a DKP model, and provides prior information of image reconstruction. The NAFA module introduces a gating mechanism between the DIP and the DKP, and dynamically optimizes the restoration of image details and the maintenance of a global structure by adaptively adjusting the fusion proportion of the image and the blurring kernel. According to the model, by alternately training the DIP model and the DKP model, local and global feature information can be effectively captured, and the stability and the reconstruction effect are improved by fusing the previous step and the current output information.

Description

Technical Field

[0001] The invention belongs to the fields of machine learning, medical image super-resolution and denoising, and in particular relates to an unsupervised blind super-resolution and denoising method for low-quality medical images based on NAFA-DIKNet. Background Art

[0002] Image super-resolution (SR) aims to restore high-resolution (HR) images from low-resolution (LR) images and is widely used in medical imaging, satellite imaging, video processing and other fields. In medical image processing, super-resolution technology is particularly important, as it can improve the detail resolution of images and enhance the accuracy of diagnosis. However, medical images are often affected by factors such as imaging equipment limitations, noise, and blur, which makes traditional super-resolution methods face many challenges. Therefore, in the absence of a large amount of prior information, how to restore high-quality images has become a research hotspot in super-resolution technology.

[0003] Traditional super-resolution methods usually rely on certain prior knowledge, such as blur kernels and downsampling factors. Classic interpolation methods (such as bilinear interpolation and cubic interpolation) reconstruct images through simple algorithms, but have limited effects in restoring image details. In recent years, deep learning methods (such as SRCNN and VDSR) have made significant progress in super-resolution reconstruction. These methods automatically extract features by learning a large amount of annotated data, thereby effectively improving image quality. However, these methods usually rely on a large amount of training data and require clear prior information such as blur kernels or downsampling factors.

[0004] Unsupervised super-resolution (USR) technology does not require paired low-resolution and high-resolution image data, and can restore high-resolution images by learning the features of low-resolution images. A typical representative of unsupervised super-resolution technology is DIP (Deep Image Prior), which generates high-resolution images by optimizing the input random noise. The core idea of ​​DIP is to use the structural prior of deep neural networks, so that it does not rely on a large amount of training data. In the DIP model, random noise is used as input, and high-resolution images are generated through a simple encoder-decoder network. The network parameters are gradually optimized by comparing with low-resolution images. The advantage of the DIP method is that it does not require high-quality training data, and can restore high-resolution images without supervision. It is widely used in scenarios where data is scarce. However, DIP also has some limitations, such as the large amount of computation in the training process, long optimization time, and easy overfitting.

[0005] Blind Super-Resolution (BSR) is a technique that aims to restore high-resolution images without explicit blur kernels or downsampling factors. In traditional super-resolution methods, it is usually assumed that the downsampling process or blur kernel of the image is known, but in practical applications, this information is often not available, especially in the fields of medical images and remote sensing images. Therefore, blind super-resolution has emerged, which can restore images without knowing the blur kernel or downsampling factor. DKP (Deep Kernel Prior) is a method in blind super-resolution that solves the image restoration problem by adaptively generating blur kernels. In DKP, the network first generates a blur kernel and combines the low-resolution image with the generated blur kernel to restore the high-resolution image. Unlike traditional methods, DKP does not require known downsampling factors or blur kernels, but automatically learns this information through deep neural networks. The core idea of ​​DKP is to generate the blur kernel of the image through a fully connected layer, and then use the blur kernel to reconstruct the image.

[0006] DIPDKP combines the unsupervised learning ability of DIP and the blind super-resolution ability of DKP. After adding the DKP model to the DIP network, DIPDKP can complete blind super-resolution reconstruction by automatically learning the features of low-resolution images and generating blur kernels without high-quality training data. This method effectively overcomes the problems of high computational complexity of DIP and inaccurate estimation of DKP blur kernels, and can adapt to different application scenarios while maintaining high image quality. However, DIPDKP still has some limitations. First, its computational cost is high, especially when processing high-resolution images, the training process requires a lot of computing resources and time. Second, DIP relies on random initialization during the generation process, and the blur kernel of DKP mainly focuses on the processing of blurred areas, so it has weak ability to recover high-frequency details in the image, especially texture and edge details are easily lost. In addition, DIP usually relies on the early stopping strategy to prevent overfitting, but the effect of this strategy is unstable and it is easy to produce inconsistent performance in different tasks. The DIP model may overfit to noise features during the update process, affecting image quality. Summary of the invention

[0007] Purpose of the invention: The purpose of the present invention is to provide an unsupervised blind super-resolution denoising method for low-quality medical images based on NAFA-DIKNet. The innovative NAFA (Nonlinear-FreeAdaptive Fusion) module optimizes detail recovery and global structure preservation by dynamically adjusting the fusion ratio of the image and blur kernel under the framework of unsupervised learning, thereby improving the quality of image reconstruction.

[0008] Technical solution: The present invention provides an unsupervised low-quality medical image blind super-resolution denoising algorithm based on NAFA-DIKNet, comprising the following steps:

[0009] S1: Collect and acquire low-resolution or noisy target medical MRI or CT image data;

[0010] S2: Standardize and enhance the original image data to improve the training effect of the model;

[0011] S3: The super-resolution denoising model is built based on the DIP network, NAFA module and DKP model;

[0012] During the model training process, the DIP network adopts a simple encoder-decoder structure, with low-resolution images and random noise as input and high-resolution images as output. The encoder part gradually extracts features through multi-layer convolution and downsampling, and adds non-local modules in deeper layers to enhance the global feature representation. The decoder part restores the image resolution through upsampling, and uses jump connections to fuse the encoder features with the current features to ensure that details are preserved. The DKP model consists of a fully connected layer for generating blur kernels. Its input is random noise and its output is the corresponding blur kernel feature. In addition, the NAFA module plays a key role between the DIP network and the DKP network. It further improves the feature expression capability by dynamically weighted fusion of the previous step output and the current output. This module uses a gating mechanism to adaptively adjust the feature fusion ratio during the high-resolution image generation process, thereby effectively capturing image details and structural information.

[0013] S4: input the pre-processed data into the super-resolution network for image super-resolution and denoising;

[0014] S5: Complete image processing and output the final super-resolution and denoising results.

[0015] Furthermore, the specific processes of S1 and S2 are as follows:

[0016] S1.1 Collect a private mouse cochlear dataset as the training and testing MRI dataset for this experiment;

[0017] S2.1 This private dataset consists of MRI data of cochlear slices of experimental mice;

[0018] S2.2 Data preprocessing stage, since the original medical images (such as MRI or CT images) have different resolutions, number of slices and other differences, the area of ​​interest in the image is first extracted through cropping operations and converted to the same resolution. Next, the data will be converted into a format suitable for the deep learning model (such as .npy and .pkl), and standardized processing (such as scaling pixel values ​​to [0, 1] or [-1, 1]) will be used to ensure the stability and accelerated convergence of the image during training. In order to further improve the training efficiency, the present invention adopts a high- and low-precision mixed training strategy to allocate data to high precision and low precision according to the requirements of the training stage, ensuring that the calculation process is accelerated while reducing memory consumption, thereby providing efficient data support for subsequent super-resolution image denoising and segmentation tasks;

[0019] Furthermore, the specific process of S3 is as follows:

[0020] The basic architecture of the S3.1 network is the DIP (Deep Image Prior) network:

[0021] S3.1.1The specific structure of the DIP (Deep Image Prior) network adopts a simple encoder-decoder architecture. The input layer receives a low-resolution image and random noise and outputs a high-resolution image. The encoder part consists of multiple downsampling layers with 16, 32, 64 and 128 channels, respectively, and extracts features through a 3x3 convolution kernel. In order to enhance the global dependency of features, NONLocalBlock2D is introduced in the second and deeper layers. The decoder part gradually restores the image resolution through upsampling layers. The channel configuration is consistent with the encoder, and jump connections are used to fuse the encoder features with the current features. Finally, a high-resolution image is output through a 1x1 convolution layer, and a Sigmoid activation function is used to ensure that the output range is between 0 and 1. In addition, the network also contains a fully connected network (FCN) for kernel prior generation, with 1000 units in its hidden layer, and the probability distribution of the blur kernel is generated through a Softmax layer. The overall architecture is designed to keep simplicity so that it can converge quickly and achieve high-quality image reconstruction.

[0022] S3.1.2 The design principle of DIP lies in its unsupervised learning ability, which can directly utilize the network's prior knowledge without relying on a large amount of training data. The simplified structure helps to accelerate convergence and generate high-quality images in a limited number of training iterations. In addition, random noise input enables the model to learn the diversity and structural characteristics of images, thereby improving the quality and realism of generated images.

[0023] S3.2 DKP (DeepKernelPrior) model:

[0024] S3.2.1The DKP model adopts a fully connected network structure and is designed to generate blur kernels for image reconstruction. The specific parameter configurations are as follows: the number of input channels is set to 2, representing a combination of low-resolution images and random noise, and the number of output channels is 3, corresponding to the generated blur kernel features. There is one hidden layer in the model with 1000 units, and the ReLU6 activation function is used to enhance the nonlinear expression ability of the network. The last layer uses the Softmax function to convert the output of the blur kernel into a probability distribution to ensure that the generated blur kernel conforms to physical properties. In addition, the DKP model is implemented in conjunction with the MCMC sampling mechanism to provide a variety of blur kernel samples and enhance the robustness of image reconstruction.

[0025] S3.2.2 The collaborative training of DKP and DIP is achieved by alternately optimizing the parameters of the two. During the training process, the parameters of the DIP network are first fixed and the DKP model is updated. At this time, DKP generates a blur kernel through the fully connected layer, and combines the current input noise and the high-resolution image generated by the DIP network to calculate the blur kernel loss (MSE) and image loss. Next, the parameters of the DKP network are fixed, the DIP network is updated, the data consistency loss is calculated through the current blur kernel (such as using SSIM and MSE in turn according to the iteration period), and the gradient loss is introduced as a regularization term to ensure the details and quality of the generated image. This alternating optimization mechanism promotes the effective collaboration between DIP and DKP, allowing the two to jointly improve the quality and accuracy of image reconstruction.

[0026] S3.3 In order to improve the quality of image reconstruction and enhance the feature fusion capability, we proposed the NAFA module (Nonlinear-Free Adaptive Fusion) and the INAFA module (Iterative Nonlinear-Free Adaptive Fusion), which aims to effectively integrate the features generated by the DIP network and the DKP model. The NAFA module uses a gating mechanism to adaptively adjust the fusion ratio of the high-resolution image and the blur kernel, and makes full use of the information of each scale of the generated image to optimize the feature expression:

[0027] S3.3.1Overall description of NAFA module:

[0028] Compared with traditional feature fusion methods, the NAFA module reduces the complexity of the model by not using nonlinear activation functions, adapts to the characteristics of DIP unsupervised super-resolution, and avoids the risk of information loss. In the process of high-resolution image generation, the NAFA module dynamically adjusts the fusion ratio of global features and local features through a gating mechanism to enhance the ability to capture image details. In particular, the NAFA module uses the current high-resolution image and blur kernel output information, and generates adaptive weighting coefficients through a gating mechanism, so that the fusion ratio of the high-resolution image and the blur kernel can be dynamically adjusted with the image content. This design realizes the adaptive fusion of features, thereby enhancing the effect of detail restoration and global structure preservation. In addition, the NAFA module makes full use of the output information of the previous iteration and the current generation information, so that each update process can integrate the previously generated high-resolution image features and the current blur kernel output to form a stable and detailed feature expression.

[0029] The input feature map is processed in two ways: one is the global path:

[0030] Global average pooling: x gp =GlobalPool(x);

[0031] Among them, x is the input feature map, dimension R B×C×H×W ;x gp is the feature map after global pooling, dimension R B×C×1×1 .

[0032] Channel reduction convolution: x' gp =W gp *x gp ;

[0033] Among them, W gp is the weight matrix, dimension R B×M×1×1 , M is the number of intermediate channels; x' gp is the feature map after channel reduction, dimension R B×M×1×1 .

[0034] Upsample to original space:

[0035] Among them, W up is a learnable upsampling weight matrix with dimension R M×M×3×3 ; It is a feature map upsampled to the original spatial size, dimension R B×M×H×W .

[0036] The other is a pixel-by-pixel path:

[0037] 1*1 convolution: x pw =W pw *x;

[0038] Among them, W pw is the pixel-by-pixel convolution weight matrix, dimension R M×C×1×1 ;x pw It is the feature map after pixel-by-pixel convolution, dimension R B ×M×H×W .

[0039] Then adaptive weighted fusion:

[0040]

[0041] Among them, ∝ is a learnable weight parameter, with an initial value of 0.5 and a range of [0,1]; x combined is the fused feature map, dimension R B×M×H×W .

[0042] S3.3.2: The NAFA module generates adaptive weighting coefficients through a gating mechanism, so that the fusion ratio of the high-resolution image and the blur kernel can be dynamically adjusted according to the image content; the NAFA module uses the output information of the previous iteration and the current generation information, so that each update process can integrate the previously generated high-resolution image features and the current blur kernel output to form a stable and detailed feature expression;

[0043] Application gating mechanism:

[0044] Channel splitting: x1,x2=Split(x combined , dim=1);

[0045] Among them, x1, x2 are the feature maps after splitting, and the dimension is R B×M / 2×H×W .

[0046] Element-wise multiplication: x gate =x1⊙x2;

[0047] Among them, x gate is the gated feature map after element-by-element multiplication, dimension R B×M / 2×H×W .

[0048] Channel projection: x' gate =W proj *x gate ;

[0049] Among them, W proj is the projection matrix, dimension R C×M / 2×1×1 ; x' gate is the gated feature after projection, dimension R B×C×H×W .

[0050] Final output:

[0051] out t =x⊙x′ gate

[0052] Among them, out t is the final output feature map, dimension R B×C×H×W .

[0053] S3.3.3: Iterative Nonlinear-Free Adaptive Fusion Module INAFA: Based on the NAFA module, the iterative nonlinear free adaptive fusion module improves the effect of feature fusion by combining the features of the previous and next iterations, and uses the gating mechanism and blur kernel for adaptive fusion to further enhance the detail retention and structural consistency of the reconstructed image.

[0054] Furthermore, in step S3.3.3, the Iterative Nonlinear-Free Adaptive Fusion module INAFA is specifically implemented as follows:

[0055] DIP output before and after fusion:

[0056] x add =out t-1 +out t ;

[0057] Among them, out t-1 ,out t It is the feature map of the previous iteration and the feature map of the current iteration, dimension R B×C×H×W ;

[0058] Calculate the attention weights:

[0059] attention weights =NAFA(x add );

[0060] Among them, attention weights is the attention weight, dimension R B×C×H×W .

[0061] Weighting the features:

[0062] weighted t-1 =attention weights ⊙out t-1 ;

[0063] weighted t =(1-attention weights )⊙out t ;

[0064] out′=weighted t-1 +weightedt ;

[0065] Among them, weighted t-1 , weighted t is the weighted previous feature and current feature, out' is the output feature after weighted fusion, dimension R B×C×H×W ;

[0066] Concatenate fusion features and blur kernel:

[0067] combined features =Concat(out′,K,dim=1);

[0068] Among them, K is the blur kernel feature map, dimension R B×C×H×W ; combined features is the concatenation result of fusion features and blur kernel, dimension R B×2C×H×W .

[0069] Calculate the fusion weight through SimpleGate:

[0070] f1,f2=Split(combined features , dim=1);

[0071] fusion weights =f1⊙f2;

[0072] Among them, fusion weights is the fusion weight, dimension R B×C×H×W , f1,f2 are the features after channel splitting, dimension R B ×C×H×W .

[0073] Adaptive fusion of blur kernel and fusion features: Use the generated fusion weight to adjust the ratio of blur kernel and fusion features to achieve a balance between details and structures to obtain the final output:

[0074] out final =dusion weights ⊙out′+(1-fusion weights )⊙K

[0075] Among them, out final is the final fusion output image, dimension R B×C×H×W .

[0076] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0077] The present invention forms an efficient image reconstruction framework by combining the innovatively designed NAFA module and INAFA module with the DIP network. The NAFA module extracts multi-scale features through global and pixel-by-pixel paths, and dynamically adjusts the fusion ratio of global and local features through an adaptive weighting mechanism to ensure a balance between detail recovery and global structure preservation. Its gating mechanism avoids excessive nonlinear operations, significantly reduces computational complexity, and improves the flexibility of feature expression and the quality of image reconstruction.

[0078] Based on NAFA, the INAFA module combines the features of the previous and next iterations, performs weighted fusion through attention weights, captures long-term features, and improves output stability. At the same time, the blur kernel features and gating mechanism are used to dynamically adjust the ratio of blur kernel and fusion features, so that the model can perform adaptive processing for different blur regions, enhancing the image reconstruction capability in complex scenes. The DKP model dynamically generates blur kernel features and works in conjunction with the DIP network to improve the restoration capability of blur regions, making the generated images smoother and richer in details.

[0079] Compared with the existing technology, the present invention achieves significant improvement in detail restoration, global consistency and model stability through multi-module collaborative optimization. The innovative dynamic feature fusion and blur kernel adjustment strategy enable the present invention to show higher clarity, stability and robustness in image reconstruction tasks, while greatly improving computational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 The overall flow chart of brain tumor segmentation based on NAFA-DIKNet;

[0081] Figure 2 NAFA-DIKNet network architecture diagram;

[0082] Figure 3 NAFA module architecture diagram;

[0083] Figure 4 INAFA module architecture. DETAILED DESCRIPTION

[0084] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0085] See also Figure 1 and Figure 2 The present invention provides a technical solution based on the NAFA-DIKNet medical image super-resolution algorithm, comprising the following steps:

[0086] Step 1, the present invention uses a private dataset, whose images have quality problems such as high noise and low resolution. Other CT or MRI images that need to be improved and do not have high-quality similar samples can try this model;

[0087] Step 2. In the data preprocessing stage, since the original medical images (such as MRI or CT images) have different resolutions, number of slices and other differences, the area of ​​interest in the image is first extracted through a cropping operation and converted to the same resolution. Next, the data will be converted into a format suitable for the deep learning model (such as .npy and .pkl), and standardized (such as scaling pixel values ​​to [0, 1] or [-1, 1]) to ensure the stability and accelerated convergence of the image during training. In order to further improve the training efficiency, the present invention adopts a high- and low-precision mixed training strategy to allocate data to high precision and low precision according to the requirements of the training stage, ensuring that the calculation process is accelerated while reducing memory consumption, thereby providing efficient data support for subsequent super-resolution image denoising and segmentation tasks;

[0088] Step 3, the present invention is an unsupervised super-resolution denoising algorithm based on NAFA-DIKNet, which integrates the DIP network and DKP model of the NAFA module:

[0089] like Figure 2 As shown in the figure, the overall architecture is based on the U-shaped network architecture of DIP, combined with the NAFA module and the DKP model. The NAFA module is integrated in the DIP network for multi-scale feature extraction, which enhances the image detail recovery and global structure preservation capabilities. In the feature fusion block, the INAFA module receives the output of the previous and current DIP iterations, performs gating processing with the blur kernel after feature fusion, and adaptively adjusts the fusion ratio to ensure full fusion of information and restoration of details during image reconstruction.

[0090] After one iteration of the DIP network, the NAFA module extracts and fuses image features at multiple scales instead of directly calculating the fusion coefficients of the image and blur kernel. The NAFA module extracts features using both global and local pathways and adaptively fuses these features through a learnable parameter α. The gating mechanism dynamically adjusts the fusion ratio of global and local features, enhancing the model's ability to capture image details and maintain structural consistency.

[0091] Specifically:

[0092] The input feature map is processed in two ways: one is the global path:

[0093] Global average pooling: x gp =GlobalPool(x);

[0094] Among them, x is the input feature map, dimension R B×C×H×W ;x gp is the feature map after global pooling, dimension R B×C×1×1 .

[0095] Channel reduction convolution: x' gp =W gp *x gp ;

[0096] Among them, W gp is the weight matrix, dimension R B×M×1×1 , M is the number of intermediate channels; x' gp is the feature map after channel reduction, dimension R B×M×1×1 .

[0097] Upsample to original space:

[0098] Among them, W up is a learnable upsampling weight matrix with dimension R M×M×3×3 ; It is a feature map upsampled to the original spatial size, dimension R B×M×H×W .

[0099] The other is a pixel-by-pixel path:

[0100] 1*1 convolution: x pw =W pw *x;

[0101] Among them, W pw is the pixel-by-pixel convolution weight matrix, dimension R M×C×1×1 ;x pw It is the feature map after pixel-by-pixel convolution, dimension R B ×M×H×W .

[0102] Then adaptive weighted fusion:

[0103]

[0104] Among them, ∝ is a learnable weight parameter, with an initial value of 0.5 and a range of [0,1]; x combined is the fused feature map, dimension R B×M×H×W .

[0105] Application gating mechanism:

[0106] Channel splitting: x1,x2=Split(x combined , dim=1);

[0107] Among them, x1, x2 are the feature maps after splitting, and the dimension is R B×M / 2×H×W .

[0108] Element-wise multiplication: x gate =x1⊙x2;

[0109] Among them, xgate is the gated feature map after element-by-element multiplication, dimension R B×M / 2×H×W .

[0110] Channel projection: x' gate =W proj *x gate ;

[0111] Among them, W proj is the projection matrix, dimension R C×M / 2×1×1 ; x' gate为 The gated features after projection, dimension R B×C×H×W .

[0112] Final output:

[0113] out t =x⊙x′ gate

[0114] Among them, out t is the final output feature map, dimension R B×C×H×W .

[0115] Through the above steps, the NAFA module realizes the extraction and adaptive fusion of multi-scale features, enhancing the model's ability to capture image details and global information.

[0116] Based on the NAFA module, the INAFA module further combines the output features of the previous and subsequent DIP iterations, and improves the quality and stability of image reconstruction through gated fusion with the blur kernel.

[0117] The specific implementation is as follows:

[0118] DIP output before and after fusion:

[0119] x add =out t-1 +out t ;

[0120] Among them, out t-1 ,out t It is the feature map of the previous iteration and the feature map of the current iteration, dimension R B×C×H×w ;

[0121] Calculate the attention weights:

[0122] attention weights =NAFA(x add );

[0123] Among them, attention weights is the attention weight, dimension R B×C×H×W ;

[0124] Weighting the features:

[0125] weighted t-1 =attention weights ⊙out t-1 ;

[0126] weighted t =(1-attention weights )⊙out t ;

[0127] out′=weighted t-1 +weighted t ;

[0128] Among them, weighted t-1 ,weithted t is the weighted previous feature and current feature, out' is the output feature after weighted fusion, dimension R B×C×H×W ;

[0129] Concatenate fusion features and blur kernel:

[0130] combined features =Concat(out′,K,dim=1);

[0131] Among them, K is the blur kernel feature map, dimension R B×C×H×W ; combined features is the concatenation result of fusion features and blur kernel, dimension R B×2C×H×W .

[0132] Calculate the fusion weight through SimpleGate:

[0133] f1,f2=Split(combined features , dim=1);

[0134] fusion weights =f1⊙f2;

[0135] Among them, fusion weights is the fusion weight, dimension R B×C×H×W , f1,f2 are the features after channel splitting, dimension R B ×C×H×W .

[0136] Adaptive fusion of blur kernel and fusion features: Use the generated fusion weight to adjust the ratio of blur kernel and fusion features to achieve a balance between details and structures to obtain the final output:

[0137] outfinal =dusion weights ⊙out′+(1-fusion weghts )⊙K

[0138] Among them, out final is the final fusion output image, dimension R B×C×H×W .

[0139] In this way, the INAFA module effectively adjusts the fusion ratio of fusion features and blur kernels, improving the detail recovery and global structure preservation capabilities of image reconstruction.

[0140] In the actual training process, the NAFA and INAFA modules gradually improve the quality of image reconstruction by continuously updating and optimizing the adaptive fusion coefficients and gating mechanisms. The model uses multi-scale feature extraction, adaptive feature fusion, and the integration of previous and next iteration information to enhance the ability to capture long-term image features and stabilize the output of image reconstruction.

[0141] The design of NAFA and INAFA modules not only reduces the complexity of the network (because too many nonlinear activation functions are avoided), but also significantly improves the performance of super-resolution reconstruction tasks. Through adaptive feature fusion and information guidance mechanisms, these modules further improve the clarity and structural consistency of the image. Its unique gating mechanism and information fusion strategy enable the model to provide more stable and higher-quality image reconstruction results in practical applications.

[0142] Step 4: Put the preprocessed data batches into the network for network training, and apply the early stopping mechanism to obtain the super-resolution and denoised images;

[0143] Step 5: Get the super-resolution results for data analysis, and connect to the segmentation network to compare the network model effect.

[0144] Table 1. Network model performance comparison table

[0145]

[0146]

Claims

1. An unsupervised blind super-resolution denoising method for low-quality medical images based on NAFA-DIKNet, comprising the following steps: S1: Collect and obtain low-resolution or noisy raw image data, including target medical MRI and CT image data; S2: normalize and enhance the original image data to obtain processed image data; S3: Construct a super-resolution denoising model based on the Deep Image Prior network DIP, the Nonlinear-FreeAdaptive Fusion module NAFA and the Deep Kernel Prior model DKP; in, The DIP network adopts an encoder-decoder structure. The encoder part gradually extracts features through multi-layer convolution and downsampling, and adds non-local modules in deeper layers to enhance the global feature representation. The decoder part restores the image resolution through upsampling, and uses jump connections to fuse the encoder features with the current features to retain details. The DKP model consists of a fully connected layer for generating blur kernels. Its input is random noise and its output is the corresponding blur kernel features. The NAFA module uses a gating mechanism to adaptively adjust the feature fusion ratio during the high-resolution image generation process to capture image details and structural information. S4: inputting the processed image data into the super-resolution and denoising model for image super-resolution and denoising; S5: Complete image processing and output the final super-resolution and denoising results.

2. According to the unsupervised blind super-resolution denoising method for low-quality medical images based on NAFA-DIKNet according to claim 1, the specific processes of S1 and S2 are as follows: S1.1: Collect a private mouse cochlear dataset as a training and testing MRI dataset; S2.1: The private dataset consists of MRI data of cochlear slices of experimental mice; S2.2: In the data preprocessing stage, since the original medical images have different resolutions, number of slices and differences, the area of ​​interest in the image is first extracted through cropping operations and converted to the same resolution; then, the data will be converted into formats suitable for deep learning models, including .npy and .pkl, and the stability and accelerated convergence of the image during training are ensured by scaling the pixel values ​​to [0, 1] or [-1, 1]; a high- and low-precision mixed training strategy is adopted to allocate the data to high precision and low precision according to the requirements of the training stage.

3. According to the unsupervised blind super-resolution denoising method for low-quality medical images based on NAFA-DIKNet according to claim 1, step S3 specifically comprises the following steps: S3.1: The basic architecture of the network is Deep Image Prior network DIP; S3.2: Use the connection network structure to build the DeepKernelPrior model DKP; S3.3: Construct the Nonlinear-Free Adaptive Fusion module NAFA and the Iterative Nonlinear-FreeAdaptive Fusion module INAFA to improve the quality of image reconstruction and enhance the feature fusion capability.

4. According to the unsupervised blind super-resolution denoising method for low-quality medical images based on NAFA-DIKNet according to claim 3, step S3.1 specifically comprises the following steps: S3.1.1: The specific structure of the Deep Image Prior network DIP adopts a simple encoder-decoder architecture. The input layer receives a low-resolution image and random noise and outputs a high-resolution image. The encoder part consists of multiple downsampling layers with the number of channels being 16, 32, 64, and 128, respectively. Features are extracted through a 3x3 convolution kernel. S3.1.2: To enhance the global dependency of features, NONLocalBlock2D is introduced in the second and deeper layers; the decoder gradually restores the image resolution through upsampling layers, the channel configuration is consistent with the encoder, and skip connections are used to fuse the encoder features with the current features; S3.1.3: Finally, a high-resolution image is output through a 1x1 convolutional layer, and a Sigmoid activation function is used to ensure that the output range is between 0 and 1; in addition, the network also contains a fully connected network FCN for kernel prior generation, whose hidden layer has 1000 units, and the probability distribution of the blur kernel is generated through the Softmax layer.

5. According to the unsupervised blind super-resolution denoising method for low-quality medical images based on NAFA-DIKNet according to claim 3, step S3.2 specifically comprises the following steps: S3.2.1: The DKP model uses a fully connected network structure to generate blur kernels for image reconstruction. The parameters are configured as follows: the number of input channels is set to 2, representing the combination of low-resolution images and random noise, and the number of output channels is 3, corresponding to the generated blur kernel features; there is one hidden layer in the model with 1000 units, and the ReLU6 activation function is used to enhance the nonlinear expression ability of the network; the last layer uses the Softmax function to convert the output of the blur kernel into a probability distribution to ensure that the generated blur kernel conforms to physical properties; in addition, the DKP model is implemented in conjunction with the MCMC sampling mechanism to provide a variety of blur kernel samples to enhance the robustness of image reconstruction; S3.2.2: The collaborative training of DKP and DIP is achieved by alternately optimizing the parameters of the two. During the training process, the parameters of the DIP network are first fixed and the DKP model is updated. At this time, DKP generates a blur kernel through the fully connected layer, and combines the current input noise and the high-resolution image generated by the DIP network to calculate the blur kernel loss MSE and image loss; next, the parameters of the DKP network are fixed, the DIP network is updated, the data consistency loss is calculated through the current blur kernel, and the gradient loss is introduced as a regularization term to ensure the details and quality of the generated image.

6. According to the unsupervised blind super-resolution denoising method for low-quality medical images based on NAFA-DIKNet according to claim 3, step S3.3 specifically comprises the following steps: S3.3.1: NAFA module adaptive weighted fusion: The NAFA module does not use nonlinear activation functions to reduce the complexity of the model, adapt to the characteristics of DIP unsupervised super-resolution, and avoid the risk of information loss. During the high-resolution image generation process, the NAFA module dynamically adjusts the fusion ratio of global features and local features through a gating mechanism to enhance the ability to capture image details; the NAFA module uses the current high-resolution image and blur kernel to output information; The input feature map is processed in two ways: one is the global path: Global average pooling: x gp =GlobalPool(x); in, x is the input feature map, dimension R B×C×H×W ;x gp is the feature map after global pooling, dimension R B×C×1×1 ; Channel reduction convolution: x' gp =W gp *x gp ; Among them, W gp is the weight matrix, dimension R B×M×1×1 , M is the number of intermediate channels; x' gp is the feature map after channel reduction, dimension R B×M×1×1 ; Upsample to original space: Among them, W up is a learnable upsampling weight matrix with dimension R M×M×3×3 ; It is a feature map upsampled to the original spatial size, dimension R B×M×H×W ; The other is a pixel-by-pixel path: 1*1 convolution: x pw =W pw *x; Among them, W pw is the pixel-by-pixel convolution weight matrix, dimension R M×C×1×1 ;x pw It is the feature map after pixel-by-pixel convolution, dimension R B ×M×H×W ; Then adaptive weighted fusion: Among them, ∝ is a learnable weight parameter, with an initial value of 0.5 and a range of [0,1]; x combined is the fused feature map, dimension R B×M×H×W ; S3.3.2: The NAFA module generates adaptive weighting coefficients through a gating mechanism, so that the fusion ratio of the high-resolution image and the blur kernel can be dynamically adjusted according to the image content; the NAFA module uses the output information of the previous iteration and the current generation information, so that each update process can integrate the previously generated high-resolution image features and the current blur kernel output to form a stable and detailed feature expression; Application gating mechanism: Channel splitting: x1,x2=Split(x combined , dim=1); Among them, x1, x2 are the feature maps after splitting, and the dimension is R B×M / 2×H×W ; Element-wise multiplication: x gate =x1⊙x2; Among them, x gate is the gated feature map after element-by-element multiplication, dimension R B×M / 2×H×W ; Channel projection: x' gate =W proj *x gate ; Among them, W proj is the projection matrix, dimension R C×M / 2×1×1 ; x' gate is the gated feature after projection, dimension R B×C×H×W ; Final output: out t =x⊙x' gate Among them, out t is the final output feature map, dimension R B×C×H×W ; S3.3.3: Iterative Nonlinear-Free Adaptive Fusion Module INAFA: Based on the NAFA module, the iterative nonlinear free adaptive fusion module improves the effect of feature fusion by combining the features of the previous and next iterations, and uses the gating mechanism and blur kernel for adaptive fusion to further enhance the detail retention and structural consistency of the reconstructed image.

7. According to the unsupervised blind super-resolution denoising method for low-quality medical images based on NAFA-DIKNet in claim 6, in step S3.3.3, the IterativeNonlinear-FreeAdaptive Fusion module INAFA is specifically implemented as follows: DIP output before and after fusion: x add =out t-1 +out t ; in, out t-1 ,out t It is the feature map of the previous iteration and the feature map of the current iteration, dimension R B×C×H×W ; Calculate the attention weights: attention weights =NAFA(x add ); Among them, attention weights is the attention weight, dimension R B×C×H×W ; Weighting the features: weighted t-1 =attention weights ⊙out t-1 ; weighted t =(1-attention weights )⊙out t ; out'=weighted t-1 +weighted t ; Among them, weighted t-1 , weighted t is the weighted previous feature and current feature, out' is the output feature after weighted fusion, dimension R B×C×H×W ; Concatenate fusion features and blur kernel: combined features =Concat(out',K,dim=1); Among them, K is the blur kernel feature map, dimension R B×C×H×W ; combined features is the concatenation result of fusion features and blur kernel, dimension R B×2C×H×W ; Calculate the fusion weights through SimpleGate: f1,f2=Split(combined features ,dim=1); fusion weights =f1⊙f2; Among them, fusion weights is the fusion weight, dimension R B×C×H×W , f1,f2 are the features after channel splitting, dimension R B×C×H×W ; Adaptive fusion of blur kernel and fusion features: Use the generated fusion weight to adjust the ratio of blur kernel and fusion features to achieve a balance between details and structures to obtain the final output: out final =fusion weights ⊙out'+(1-fusion weights )⊙K Among them, out final is the final fusion output image, dimension R B×C×H×W .

Images