Medical image super-resolution reconstruction method and system based on recursive attention mechanism

Abstract

The invention belongs to the technical field of medical diagnosis, and provides a medical image super-resolution reconstruction method and system based on a recursive attention mechanism, and the method comprises the following steps: S1, inputting a low-resolution medical image; s2, performing preliminary feature extraction on the input image through a convolutional neural network; s3, performing multi-layer recursive processing on the extracted features by using a recursive attention mechanism, wherein each layer of recursive processing comprises a local-global attention module and a recursive global attention module; s4, performing residual connection on the recursive processed features and the preliminarily extracted features to form fusion features; s5, carrying out image pixel position disturbance operation on the fusion features; s6, reconstructing the processed features into a high-resolution medical image through a convolutional neural network; by introducing a recursive attention mechanism and a multi-layer jump connection structure, the problems of insufficient feature fusion and generation of artifacts in the medical image super-resolution field are effectively solved, the quality of the reconstructed image is remarkably improved, and the detail reconstruction capability is enhanced.

Description

Technical Field

[0001] The present invention belongs to the field of medical diagnosis technology, and specifically to a medical image super-resolution reconstruction method and system based on a recursive attention mechanism. Background Art

[0002] With the rapid development of modern medical diagnostic technology, high-quality medical images play a vital role in clinical diagnosis. However, obtaining high-quality medical images still faces numerous challenges due to the uneven distribution of medical equipment, long imaging times, and the low diagnostic accuracy of low-resolution images. Medical image super-resolution technology offers a new approach to addressing these issues. By reconstructing high-resolution images from low-resolution images through algorithmic means, it improves the quality and diagnostic value of medical images without increasing hardware costs or patient burden.

[0003] Existing medical imaging super-resolution technologies face challenges such as insufficient feature fusion and artifact generation, which severely impact the clinical value of reconstructed images. Traditional methods, such as interpolation and reconstruction-based approaches, have limited effectiveness when dealing with complex medical anatomical structures. While deep learning-based approaches have improved reconstruction performance to some extent, they still face challenges such as insufficient hierarchical feature fusion, artifact generation from detailed details, and high computational resource consumption. Furthermore, the specificity of medical scenarios (such as sensitivity to lesion regions, high edge sharpness requirements, and modality specificity) also places higher demands on super-resolution technology.

[0004] To address the above challenges, the present invention proposes a medical image super-resolution reconstruction method and system based on a recursive attention mechanism. This method aims to effectively solve the problems of insufficient feature fusion and artifact generation in the field of medical image super-resolution through a recursive attention mechanism and a multi-layer skip connection structure, thereby improving the quality of reconstructed images and their clinical application value. Summary of the Invention

[0005] To address the aforementioned technical issues, the present invention provides a medical image super-resolution reconstruction method and system based on a recursive attention mechanism. While existing technologies have improved reconstruction performance to a certain extent, they still suffer from issues such as insufficient hierarchical feature fusion, detail generation artifacts, and high computational resource consumption. Furthermore, the particularities of medical scenarios (such as sensitivity to lesion areas, high edge sharpness requirements, and modality specificity) also place higher demands on super-resolution technology.

[0006] The medical image super-resolution reconstruction method based on the recursive attention mechanism includes the following steps:

[0007] S1, input low-resolution medical images;

[0008] S2, perform preliminary feature extraction on the input image through convolutional neural network;

[0009] S3. Use the recursive attention mechanism to perform multi-layer recursive processing on the extracted features. Each layer of recursive processing includes the local-global attention module (L-CSA) and the recursive global attention module (RG-CSA);

[0010] S4, perform residual connection on the recursively processed features and the initially extracted features to form fusion features;

[0011] S5, performing image pixel position perturbation operation on the fusion feature;

[0012] S6. Reconstruct the processed features into high-resolution medical images through convolutional neural networks.

[0013] Preferably, the recursive attention mechanism obtains global features through a multi-layer recursive convolutional network, and the recursive convolution operation includes multi-layer convolution and attention score calculation, which is used to enhance cross-layer transmission and fusion of features.

[0014] Preferably, the local-global attention module (L-CSA) and the recursive global attention module (RG-CSA) both include an attention module and a feedforward attention module, and the attention module includes local self-attention (L-SA) and convolution operations for extracting local and global features.

[0015] Preferably, the residual connection is used for cross-layer feature transfer, alleviating the gradient vanishing problem, and promoting the fusion of low-level texture information and high-level semantic features, thereby enhancing detail reconstruction capabilities.

[0016] Preferably, the image pixel position perturbation operation is used to enhance the model's ability to restore image details, and the quality of the reconstructed image is improved by disrupting the pixel positions and combining feature dimensionality increase and dimensionality reduction operations.

[0017] The medical image super-resolution reconstruction system based on the recursive attention mechanism is applicable to the above-mentioned medical image super-resolution reconstruction method based on the recursive attention mechanism, including:

[0018] An input module for receiving low-resolution medical images;

[0019] Feature extraction module, including convolutional neural network, is used to perform preliminary feature extraction on the input image;

[0020] The recursive attention processing module includes a multi-layer recursive attention mechanism for recursively processing the extracted features;

[0021] The residual connection module is used to perform residual connection between the recursively processed features and the initially extracted features;

[0022] Image reconstruction module, including convolutional neural network, is used to reconstruct the processed features into high-resolution medical images;

[0023] The output module is used to output the reconstructed high-resolution medical images.

[0024] A processor is configured to execute the medical image super-resolution reconstruction method based on the recursive attention mechanism.

[0025] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned medical image super-resolution reconstruction method based on the recursive attention mechanism.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] By introducing a recursive attention mechanism and a multi-layer skip connection structure, this invention effectively solves the problems of insufficient feature fusion and artifact generation in the field of medical image super-resolution, significantly improves the quality of reconstructed images, enhances the ability to reconstruct details, and reduces computing resource consumption. At the same time, it meets the high requirements of medical scenarios for sensitivity to lesion areas, high edge sharpness, and modality specificity, providing more reliable and efficient image processing support for the development of telemedicine and precision medicine. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a model framework diagram of the present invention;

[0029] Figure 2 This is one of the diagrams showing the effect of the medical field dataset of the present invention;

[0030] Figure 3 This is the second diagram showing the effect of the medical field dataset of the present invention;

[0031] Figure 4 This is the third diagram showing the effect of the medical field dataset of the present invention;

[0032] Figure 5 This is a framework diagram of the medical image super-resolution reconstruction system based on the recursive attention mechanism of the present invention. DETAILED DESCRIPTION

[0033] The following embodiments of the present invention are described in further detail with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention but are not intended to limit the scope of the present invention.

[0034] Example 1: The present invention provides a medical image super-resolution reconstruction method based on a recursive attention mechanism, such as Figure 1 As shown, the super-resolution reconstruction of CT images based on the recursive attention mechanism includes the following steps:

[0035] S1. Input low-resolution CT image:

[0036] Obtain low-resolution CT images from medical equipment (such as CT scanners) as input;

[0037] S2. Feature extraction:

[0038] A convolutional neural network (CNN) is used to perform preliminary feature extraction on the input low-resolution CT image. The CNN network contains multiple convolutional layers and pooling layers to extract low-level texture features and preliminary high-level semantic features of the image.

[0039] S3, recursive attention processing:

[0040] The extracted features are input into the recursive attention processing module; this module contains a multi-layer recursive attention mechanism, and each layer of recursive processing includes a local-global attention module (L-CSA) and a recursive global attention module (RG-CSA);

[0041] In the L-CSA module, local self-attention (L-SA) and convolution operations are used to extract local features and focus on key areas in the feature map;

[0042] In the RG-CSA module, global features are acquired through recursive convolution operations. The recursive convolution operation includes multi-layer convolution and attention score calculation to enhance cross-layer transmission and fusion of features. Recursive convolution achieves deep processing with fewer parameters by sharing parameters, avoiding overfitting.

[0043] S4, residual connection:

[0044] The recursively processed features are residually connected with the initially extracted features. The residual connection alleviates the gradient vanishing problem by transferring features across layers, promotes the fusion of low-level texture information and high-level semantic features, and enhances the ability to reconstruct details.

[0045] S5. Image pixel position disturbance operation:

[0046] Perform an image pixel position perturbation operation on the features after residual connection. This operation enhances the model's ability to recover image details by disrupting the pixel positions and combining feature dimension increase and dimension reduction operations.

[0047] S6. Image reconstruction:

[0048] The processed features are fed into an image reconstruction module, which contains a convolutional neural network and is used to reconstruct the features into a high-resolution CT image. During the reconstruction process, a loss function (such as L1 loss) is optimized to make the reconstructed image as close as possible to the original high-resolution image.

[0049] S7. Output high-resolution CT images:

[0050] The reconstructed high-resolution CT images are output for doctors to make diagnoses.

[0051] Embodiment 2: The present invention provides a medical image super-resolution reconstruction method based on a recursive attention mechanism, specifically including MRI image super-resolution reconstruction based on a recursive attention mechanism, comprising the following steps:

[0052] S1. Input low-resolution MRI image:

[0053] Obtain low-resolution MRI images from medical equipment (such as MRI scanners) as input;

[0054] S2. Feature extraction:

[0055] Perform preliminary feature extraction on the input low-resolution MRI image using a convolutional neural network similar to that in Example 1;

[0056] S3, recursive attention processing:

[0057] The extracted features are input into the recursive attention processing module, which processes the features recursively through a multi-layer recursive attention mechanism. Particular attention is paid to the subtle structures and lesion areas in the MRI images, and multi-scale features are extracted and fused through the L-CSA and RG-CSA modules.

[0058] S4, residual connection and image pixel position disturbance operation:

[0059] Similar to the first embodiment, residual connection and image pixel position perturbation operations are performed to enhance the detail reconstruction capability of the MRI image;

[0060] S5. Image reconstruction:

[0061] The processed features are input into the image reconstruction module to reconstruct high-resolution MRI images;

[0062] S6. Output high-resolution MRI images:

[0063] The reconstructed high-resolution MRI images are output for doctors to make more accurate diagnoses.

[0064] Example 3: The present invention provides a medical image super-resolution reconstruction method based on a recursive attention mechanism, specifically including ultrasound image super-resolution reconstruction based on a recursive attention mechanism, comprising the following steps:

[0065] S1. Input low-resolution ultrasound image:

[0066] Obtain low-resolution ultrasound images from medical equipment (such as ultrasound diagnostic equipment) as input;

[0067] S2. Feature extraction:

[0068] Use convolutional neural networks to perform preliminary feature extraction on the input low-resolution ultrasound images;

[0069] S3, recursive attention processing:

[0070] The extracted features are input into the recursive attention processing module, which processes the features recursively through a multi-layer recursive attention mechanism. Special attention is paid to blood flow signals and microlesions in ultrasound images, and multi-scale features are extracted and fused through the L-CSA and RG-CSA modules.

[0071] S4, residual connection and image pixel position disturbance operation:

[0072] Similar to the first embodiment, residual connection and image pixel position perturbation operations are performed to enhance the detail reconstruction capability of the ultrasound image;

[0073] S5. Image reconstruction:

[0074] The processed features are input into the image reconstruction module to reconstruct high-resolution ultrasound images;

[0075] S6. Output high-resolution ultrasound images:

[0076] The reconstructed high-resolution ultrasound images are output for doctors to make more accurate diagnoses.

[0077] From the above, it can be seen that the medical image super-resolution reconstruction method and system based on the recursive attention mechanism proposed in the present invention can effectively solve the problems of insufficient feature fusion and artifact generation in the field of medical image super-resolution, and improve the quality and diagnostic value of the reconstructed image; this method and system have broad application prospects and provide strong support for the development of telemedicine and precision medicine.

[0078] In order to make the purpose, features and advantages of the present invention more intuitive and easy to understand, the present invention will be further explained in detail below with reference to the accompanying drawings and specific implementation methods.

[0079] like Figure 1 As shown, Figure 1 This is a model framework diagram of the present invention, which includes:

[0080] Low-resolution input image x i After passing through the convolutional network, it enters the recursive attention (RG) main module for feature extraction:

[0081] e i =Conv2D1(x i );

[0082]

[0083] where e i is the encoded feature, is the output feature of the jth layer in the recursive attention layer, Conv2D1 is the first convolutional network, RG j is the recursive attention network of the jth layer.

[0084] In each layer of recursive attention module, the input features are combined through multiple L-CSA and RG-CSA modules in sequence:

[0085]

[0086] in, is the result after the L-CSA module acts on the recursive attention in the jth layer, where is the result after the RG-CSA module acts on the recursive attention in the jth layer, is the output feature of the j-1th layer in the recursive attention, L-CSA j is the L-CSA module in the recursive attention of the jth layer, RG-CSA j is the RG-CSA module in the recursive attention of the jth layer, Conv2D J is the convolutional module in the recursive attention at layer j.

[0087] The structures of the L-CSA and RG-CSA modules are similar to the Transformer Encoder structure, both consisting of an attention module and a feedforward attention module. The attention module in the L-CSA of the present invention consists of two parts: L-SA and CNN. Each part first passes through a pre-normalization module:

[0088]

[0089] The attention module in RG-CSA consists of two parts: RG-SA and CNN. Each part first passes through a pre-normalization module:

[0090]

[0091] in, For each feature in the L-CSA module after passing the L-SA module, For each feature after passing through the convolution module in the L-CSA module, For each L-CSA module, the features processed by residual connection and linear layer are: For each feature in the RG-CSA module after passing through the RG-SA module, For each feature after passing through the convolution module in the RG-CSA module, is the feature after residual connection and linear layer processing in each RG-CSA module.

[0092] RG-SA is the core module of the recursive attention mechanism. Compared with the commonly used attention mechanism, the recursive attention mechanism obtains global features through a multi-layer recursive convolution network:

[0093]

[0094] wherein, and are the three input features q, k and v in the recursive attention, and are the result features of the recursive convolution operation, and S is the attention score. W q , W k and W v are the auxiliary transformation matrices of q, k and v, W r is the recursive convolution operation, and W1 and W2 are the two convolution operations after the recursive convolution.

[0095] After the image features pass through all the recursive attention modules, they are connected in residual connection with the features before entering the recursive attention module, to ensure the richness of high-latitude features and low-latitude features in the result:

[0096] m i = g i + e i ;

[0097] wherein, m i is the feature after residual connection.

[0098] After the feature extracted by the main part of the recursive attention is processed through a convolution network, an image pixel position disturbance operation is performed. Reference is made to the operation of the Adapter and the low-latitude vector for long-distance connection, and then a final output image is obtained after a convolution network:

[0099] p i = Conv2D3(m i );

[0100] q i = Pixel Shuffle(p i );

[0101] t i = Conv2D up (ReLU(Conv2D down (m i ));

[0102] o i=Conv2D4(q i +t i );

[0103] Among them, p i is the feature after convolution operation, q i is the feature after the image pixel position disturbance operation, t i is the jump connection branch feature, o i is the final output feature, Conv2D3 and Conv2D4 are the third and fourth convolution operations respectively, Pixel Shuffle is the pixel position perturbation operation, Conv2D up and Conv2D down They are feature dimension increase and dimension reduction operations respectively, and ReLU is a nonlinear activation operation.

[0104] After obtaining the final feature vector, calculate the L1 loss with the original high-resolution image:

[0105]

[0106] At this point, the forward propagation process of the training phase is completed, and the overall update and iteration of the RTSR network is achieved by optimizing the final loss function until the entire training process is completed.

[0107] Specifically, the dataset of the present invention is divided into two parts: a public dataset and a medical field dataset. The model is first trained and tested on the public dataset, and then migrated to the medical field dataset to verify the model effect.

[0108] 1) Public datasets

[0109] In terms of public data sets, the present invention uses five public data sets: DIV2K, Set5, Set14, B100, and Urban100. Among them, DIV2K is a high-quality benchmark data set widely used in the field of image super-resolution. It consists of 1,000 2K resolution (approximately 2040×1080 pixels) images, covering a variety of scenes such as natural landscapes, people, animals and plants, with rich textures and clear details. The data set is divided into 800 training images, 100 verification images, and 100 test images (the true values ​​of the test images are not disclosed to objectively evaluate the algorithm). Its advantages lie in high image diversity and uniform resolution. Low-resolution samples are often generated through degradation models such as bicubic interpolation for model learning to reconstruct high-resolution images. Since it was proposed by Timofte et al. in 2017, DIV2K has become the core data of the NTIRE super-resolution competition, promoting the development of classic models such as EDSR and RCAN. It is still an important benchmark for evaluating the generalization and detail restoration capabilities of models, and has a profound impact on academic research and industrial applications.

[0110] Set5 is a commonly used benchmark dataset in the field of image super-resolution, containing 5 classic images (such as baby, bird, butterfly, etc.), providing high-resolution original images and corresponding low-resolution versions, for evaluating the reconstruction quality of algorithms. Although the data volume is small, due to its early established standardized position, it is still widely cited to compare the performance of different models. This dataset has the convenience and historical continuity, but is limited by the sample size, often needs to be used with larger datasets such as Set14 to enhance statistical significance.

[0111] Set14 is a classic test dataset in the field of image super-resolution, containing 14 high-quality images, covering people, animals, natural scenes, and other diverse content, with rich texture and edge details. Its image resolution is high (such as 512x512), often down-sampled into low-resolution input, for evaluating the performance of algorithms in detail reconstruction, noise suppression, etc. This dataset is used as a benchmark for super-resolution models along with Set5, BSD100, etc., quantifying and comparing the restoration effects of different methods through PSNR, SSIM, etc., especially having important reference value in testing the generalization ability of models.

[0112] B100 is a commonly used benchmark dataset in the field of image super-resolution, containing 100 natural scene images, covering a wide range of content (such as people, animals, buildings, etc.), for evaluating the reconstruction ability of models in diverse scenarios. Its image texture complexity is high, challenging detail recovery and edge preservation, often used in conjunction with Set5, Set14, etc. datasets, becoming an important standard for measuring the generalization performance of algorithms, especially having reference value in processing real-world low-resolution images.

[0113] Ur ban100 is a widely used benchmark dataset in the field of image super-resolution, containing 100 high-resolution urban landscape images (buildings, streets, etc.), with rich details and complex structures (such as repeated textures, geometric symmetry), specifically for testing the reconstruction ability of algorithms in real complex scenarios. Its high challenge drives the development of edge sharpening, detail recovery, etc. technologies, often used as an important standard for measuring model performance, especially having significant evaluation value in handling high-frequency information and maintaining structural consistency.

[0114] 2) Medical field dataset

[0115] In the medical field dataset, we select 3000 high-definition chest CT photos from the 3MAD dataset, and construct low-resolution photos with scaling factors of 2, 4, and 8 by adding random noise. All the pictures are combined together as a medical field dataset, for image super-resolution model transfer training and effect evaluation.

[0116] Specifically, the training process of this method is as follows:

[0117] First, the experimental environment used in our experiment is introduced. The related hardware and software environment used in the experiment is shown in Table 1:

[0118] Table 1 Experimental hardware and software environment

[0119]

[0120]

[0121] Next, the parameter settings in model training are introduced, as shown in Table 2:

[0122] Table 2 Model training parameters

[0123] Parameter name Parameter value Learning rate 0.001 Optimizer Adam Loss function Cross-entropy Batch size 16 Number of training epochs 200 Weight decay ratio 0.0005

[0124] As can be seen from the above, the model framework of the present application uses the DIV2K training set in the training process, and uses 3500 pictures as the model input. The batch_size used in the training process is 8, the optimizer is Adam, the initial learning rate is set to 2e -4 , the exponential decay rate β1 is initialized to 0.9, and the β2 is initialized to 0.999. The total training rounds are 2253 rounds, the total iteration steps are 500000, the learning rate is kept unchanged for the first 100 rounds in the training process, and then it is decayed by 20% every 100 rounds until 2e -5 , and then the whole training process lasts about 130 hours. Save the trained model and load the model to test a small amount of samples to ensure that the model effect is normal. After verifying that the model effect is normal, complete the model deployment, and test the model in batches.

[0125] Specifically, the present application gives a formal description of the RTSR algorithm and the training process, and the algorithm is as follows:

[0126]

[0127] Specifically, the evaluation indicators of the present application include:

[0128] The present application mainly uses the classical evaluation indicators in the field of image super-resolution, peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) for effect evaluation. Among them, (PSNR) is a common evaluation indicator in the field of image super-resolution. PSNR is the ratio of the maximum power of the signal to the power of the signal noise, which measures the quality of the low-resolution image reconstructed into a high-resolution image, and is usually expressed in decibels (dB). The higher the PSNR index, the better the quality of image reconstruction. Its formula includes:

[0129]

[0130] Wherein, MaxValue represents the maximum value in the pixel value of the image, and MSE represents the mean value of the square difference between the corresponding pixels of the two images.

[0131] SSI M is an index for evaluating the similarity of two images, which is commonly used to measure the similarity of images before and after distortion, and is also used to measure the authenticity of model generated images, such as image rain removal, image defogging, image harmonization, etc. The calculation of SSIM is based on a sliding window, that is, each time a window with a size of N×N is taken from the picture, the SSIM index is calculated based on the window, and after traversing the entire image, the average value of all window values is taken as the SSIM index of the entire image. Its formula includes:

[0132]

[0133] Wherein, μ represents the feature mean value, and σ represents the feature standard deviation.

[0134] Wherein, when the scaling factor is set to 2, the experimental results of the series base model and the RTSR model of the application are shown in Table 3:

[0135] Table 3 Experimental results of scaling factor 2

[0136]

[0137] From the above table, overall, the RTSR model exhibits the best comprehensive performance under a 2-fold scaling factor, with its PSNR and SSIM indicators leading significantly on all four test sets. On the most challenging Urban100 dataset, RTSR surpasses SwinIR's 33.81 / 0.9427 with 34.49 PSNR / 0.9473 SSIM, indicating that its attention mechanism is more effective for complex texture reconstruction. Notably, models based on the Transformer architecture (SwinIR, CAT-A, RTSR) perform significantly better in complex scenarios such as Set14 and Urban100, with CAT-A's PSNR reaching 34.78 in Set14, a 0.62 improvement over the traditional HAN model. Cross-dataset comparisons show that all models perform best in the simple scenarios of Set5, while their performance in complex urban scenarios such as Urban100 decays by about 4 to 5, reflecting the existing methods' bottleneck in high-frequency detail recovery. The SSIM index is positively correlated with PSNR, but RTSR's SSIM advantage in Urban100 is more significant than its PSNR improvement, indicating its outstanding structural similarity restoration capability. At the same time, it also proves the feasibility of the combination of global attention and local attention in the RTSR model in the field of image super-resolution.

[0138] When the scaling factor is set to 3, the experimental results of the series base model and the RTSR model of the application are shown in Table 4:

[0139] Table 4 Experimental results of scaling factor 3

[0140]

[0141] From the above table, overall, the RTSR model shows significant advantages in the super-resolution task: it achieves the current optimal indicators on the four benchmark data sets of Set5, Set14, B100 and Urban100. Specifically, compared with the baseline model CAT-A, the PSNR of RTSR on the Urban100 data set is improved by 0.23, and the SSIM is improved by 0.31%, indicating that it is particularly good at processing high-frequency detail reconstruction of complex urban scenes. From the perspective of technical updates, the model performance gradually improves with the optimization of network architecture (such as the improvement of Transformer from SwinIR to CAT-A, and then to the recursive attention of RTSR), which shows that the introduction of more efficient attention mechanism or feature fusion module design can have a positive effect on the model itself. It is worth noting that the performance gap of all models on Urban100 is the most significant, while the improvement of B100 is smaller, which reflects that the difference in the modeling ability of complex textures of the model is enlarged as the scaling factor increases. The advantage of the recursive attention mechanism model of the model of the application is further manifested.

[0142] When the scaling factor is set to 4, the experimental results of the series base model and the RTSR model of the application are shown in Table 5:

[0143] Table 5 Experimental results of scaling factor 4

[0144]

[0145] As can be seen from the table above, the proposed RTSR model demonstrates comprehensive and leading performance advantages in super-resolution tasks. Its PSNR and SSIM indicators are significantly better than those of the comparison models on the four mainstream test sets Set5, Set14, B100, and Urban100. In particular, on the complex scene dataset Urban100, RTSR achieves PSNR and SSIM of 28.06 and 0.8378, respectively, which are 0.17 and 0.39% higher than CAT-A, demonstrating its outstanding ability to handle high-frequency details and complex textures. A horizontal comparison shows that RTSR improves PSNR by an average of approximately 0.4 to 0.6 compared to traditional attention models (such as HAN and NLSA), validating the effectiveness of its multi-way skip connection and recurrent attention structure design. Compared with the current mainstream model SwinIR, the PSNR on Set5 / Set14 / B100 / Urban100 is improved by 0.21 / 0.21 / 0.11 / 0.56, respectively, demonstrating the universal applicability of the improved strategy. Particularly noteworthy is the significant improvement of 0.61% on Urban100 compared to CAT-A. We speculate that RTSR effectively addresses the challenge of reconstructing complex architectural structures through innovative cross-scale feature fusion or dynamic sensing mechanisms. From a technological perspective, model performance shows a continuous upward trend from HAN to RTSR, with the SSIM metric increasing by a cumulative 1.33% on Set5, with the most significant improvement on Urban100, reaching 1.99%. This demonstrates the significant impact of technological innovations based on the Transformer architecture on image super-resolution results.

[0146] In order to verify the necessity of the two approaches of recursive global attention and increasing feature richness through skip connections in the present invention, the present invention conducts ablation experiments on these two points. The test datasets are still the four test sets Set5, Set14, B100 and Urban100, and the scaling factor is set to 4. The recursive attention mechanism module and the two-step residual connection module in the RGSR model are respectively removed as the new version of the ablation model, and the model training hyperparameters are consistent with RGSR.

[0147] The ablation experiment results are shown in Table 3-6:

[0148] Table 6 Ablation experiment results

[0149]

[0150]

[0151] As can be seen from the above table, overall, the recursive attention and skip connections of the model of the present invention have played a positive role and are necessary modules in the model. Specifically, when the model of the present invention removes the residual connection, the final high-dimensional vector produces a large amount of forgetting of the original input features after multiple feature extractions. The two skip connections can well supplement the original feature content in the high-dimensional features, and then directly act on the subsequent high-resolution pixel restoration module, implicitly increase the constraints on the final high-definition image, and improve the stability of the final result. When the model of the present invention removes the recursive attention mechanism, the model itself is greatly restricted in extracting global features, and it is impossible to extract and restore features from a global perspective. The correlation between pixel features that are far apart is reduced, and the resulting image is prone to overall incoordination and other problems. Therefore, the global features based on the recursive attention mechanism are also necessary modules for the image super-resolution model of the present invention. In addition, from the perspective of evaluation indicators, the effect of the model of the present invention after removing the global attention mechanism is more reduced, which can reflect from the side that global attention is more important to the model itself and is also the key direction of subsequent optimization.

[0152] Specifically, the model training of the invention:

[0153] Through the evaluation results on standard data, the present invention has completed the preliminary screening of the target model and the verification of the necessity of the innovative points. Therefore, the present invention migrates the optimal model in the general field to the medical field. First, the medical data set is divided into a training set, a validation set, and a test set according to a ratio of 7:2:1, and then merged with the public data set DIV2K for model migration training. The hyperparameter selection in the training phase is consistent with that in the public data set training phase. After the training is completed, the trained model is saved and loaded for a small number of sample tests to ensure that the model effect is normal. After verifying that the model effect is normal, the model deployment is completed and the test data is collected in batches.

[0154] The evaluation metrics for the medical dataset are consistent with those for the public dataset. PSNR and SSIM are used to evaluate the model performance. The SwinIR and CAT-A models, which perform well in the public dataset, are selected for comparison with the baseline model. The scaling factors are 2 and 4, respectively. The specific model performance is shown in Table 7.

[0155] Table 7 Experimental results in the medical field

[0156]

[0157] From the above table, overall, the RTSR model of the application achieves the optimal effect when the scaling factor is 2 and 4. Specifically, when the scaling factor is 2, RTSR improves 0.21% compared with SwinIR and 0.12% compared with CAT-A in the PSNR index; and improves 0.3% compared with SwinIR and 0.17% compared with CAT-A in the SSIM index. From the experimental results, the recursive attention of the application has certain advantages in obtaining global attention and local attention features, and can guarantee the stability of the model itself under the addition of multi-layer jump connection, so as to achieve better results.

[0158] When the scaling factor is 4, RTSR improves 0.49% compared with SwinIR and 0.25% compared with CAT-A in the PSNR index; and improves 0.45% compared with SwinIR and 0.27% compared with CAT-A in the SSIM index. When the scaling factor is 4, the overall difficulty of the image super-resolution task increases, and the effects of the three models decrease, but RTSR still realizes the smallest effect decrease amplitude by virtue of the two technical points of the application, which also proves the robustness and stability of the model structure of the application from the side.

[0159] As can be seen from the above, the application randomly selects several groups of photos using the optimal model for prediction in the public data set test set and the medical field test set, and compares the effects of the baseline model, ELAN, SwinIR and CAT-A. Among them, the prediction effect of the medical field data set is as shown in Figures 2 to 4 From left to right are the original image, ELAN, SwinIR, CAT-A and RTSR model actual effect.

[0160] As can be seen from the above, overall, the RTSR of the application can restore the original low-resolution image more clearly, and compared with the three control models, the clarity improvement effect can also be observed. At the same time, the RTSR model also has good effect in detail processing, and no local jagged or phantom phenomenon is found in the display image. Therefore, the case display shows the feasibility of the model of the application in the medical field image super-resolution task from the subjective level.

[0161] As described above, the technical scheme proposed by the application effectively solves the problems of insufficient feature fusion and generated artifacts in the medical image super-resolution field by introducing the recursive attention mechanism and the multi-layer jump connection structure, significantly improves the quality and diagnostic value of the reconstructed image, and provides strong support for the development of telemedicine and precision medicine. Especially under the premise of not increasing the hardware cost and the burden of patients, the resolution and clinical application value of medical images are improved.

[0162] Working principle: First, low-resolution medical images are input, and preliminary feature extraction is performed through a convolutional neural network. Then, the extracted features are recursively processed in multiple layers using a recursive attention mechanism, including a local-global attention module and a recursive global attention module, to enhance cross-layer transmission and fusion of features. The recursively processed features are then residually connected with the preliminary extracted features to form fused features, and the image pixel position is perturbed to enhance detail recovery capabilities. Finally, the processed features are reconstructed into high-resolution medical images through a convolutional neural network.

[0163] Medical image super-resolution reconstruction system based on recursive attention mechanism, such as Figure 5 As shown, the medical image super-resolution reconstruction method based on the recursive attention mechanism mentioned above includes:

[0164] An input module for receiving low-resolution medical images;

[0165] Feature extraction module, including convolutional neural network, is used to perform preliminary feature extraction on the input image;

[0166] The recursive attention processing module includes a multi-layer recursive attention mechanism for recursively processing the extracted features;

[0167] The residual connection module is used to perform residual connection between the recursively processed features and the initially extracted features;

[0168] Image reconstruction module, including convolutional neural network, is used to reconstruct the processed features into high-resolution medical images;

[0169] The output module is used to output the reconstructed high-resolution medical images.

[0170] From the above, we can see that the medical image super-resolution reconstruction system based on the recursive attention mechanism achieves accurate reconstruction from low-resolution medical images to high-resolution images by integrating the input module, feature extraction module, recursive attention processing module, residual connection module, image reconstruction module and output module. Its beneficial effect is to effectively improve the feature fusion efficiency and reconstructed image quality, reduce artifact generation, and enhance the ability to express details, providing clearer and more accurate image support for medical diagnosis.

[0171] The present application provides an electronic device applicable to the above-mentioned medical image super-resolution reconstruction method based on the recursive attention mechanism, including:

[0172] Memory, used to protect computer programs and data;

[0173] Processor, used to run system programs.

[0174] An embodiment of the present application provides a computer storage medium suitable for the above-mentioned medical image super-resolution reconstruction method based on the recursive attention mechanism, and performs hierarchical confidentiality management on the above-mentioned system and data in accordance with confidentiality management requirements.

[0175] Those skilled in the art will appreciate that the embodiments of the present application can be provided as a system or a computer program product. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0176] The present application is described with reference to the flowcharts and / or block diagrams of the devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0177] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0178] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0179] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0180] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0181] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology for information storage. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0182] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, commodity, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, commodity, or apparatus comprising the element.

[0183] The embodiments of the present invention are provided for the purpose of illustration and description. Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations of the present invention. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A medical image super-resolution reconstruction method based on a recursive attention mechanism, characterized in that: The following steps are involved: S1, input low-resolution medical images; S2, perform preliminary feature extraction on the input image through convolutional neural network; S3. Use the recursive attention mechanism to perform multi-layer recursive processing on the extracted features. Each layer of recursive processing includes a local-global attention module and a recursive global attention module. S4, perform residual connection on the recursively processed features and the initially extracted features to form fusion features; S5, performing image pixel position perturbation operation on the fusion feature; S6. Reconstruct the processed features into high-resolution medical images through convolutional neural networks.

2. The medical image super-resolution reconstruction method based on the recursive attention mechanism according to claim 1, characterized in that: The recursive attention mechanism obtains global features through a multi-layer recursive convolutional network. The recursive convolution operation includes multi-layer convolution and attention score calculation, which is used to enhance the cross-layer transmission and fusion of features.

3. The medical image super-resolution reconstruction method based on the recursive attention mechanism according to claim 1, characterized in that: The local-global attention module and the recursive global attention module both include an attention module and a feedforward attention module. The attention module includes local self-attention and convolution operations for extracting local and global features.

4. The medical image super-resolution reconstruction method based on the recursive attention mechanism according to claim 1, characterized in that: The residual connection is used for cross-layer feature transfer, alleviating the gradient vanishing problem, and promoting the fusion of low-level texture information and high-level semantic features, thereby enhancing detail reconstruction capabilities.

5. The medical image super-resolution reconstruction method based on the recursive attention mechanism according to claim 1, characterized in that: The image pixel position perturbation operation is used to enhance the model's ability to restore image details. By disrupting the pixel positions and combining feature dimensionality increase and dimensionality reduction operations, the quality of the reconstructed image is improved.

6. A medical image super-resolution reconstruction system based on recursive attention mechanism, characterized by: A medical image super-resolution reconstruction method based on a recursive attention mechanism according to any one of claims 1 to 5, comprising: An input module for receiving low-resolution medical images; Feature extraction module, including convolutional neural network, is used to perform preliminary feature extraction on the input image; The recursive attention processing module includes a multi-layer recursive attention mechanism for recursively processing the extracted features; The residual connection module is used to perform residual connection between the recursively processed features and the initially extracted features; Image reconstruction module, including convolutional neural network, is used to reconstruct the processed features into high-resolution medical images; The output module is used to output the reconstructed high-resolution medical images.

7. A processor, characterized in that: The method is configured to execute the medical image super-resolution reconstruction method based on the recursive attention mechanism according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the medical image super-resolution reconstruction method based on the recursive attention mechanism described in any one of claims 1 to 5 is implemented.

Images