Medical image super-resolution reconstruction method and system based on external image fusion

Abstract

The application discloses a medical image super-resolution reconstruction method and system based on external image fusion, and relates to the field of medical image processing; the core of the application is to introduce an external high-resolution reference image associated with a target low-resolution medical image as an auxiliary information source, to extract multi-scale features of the target and reference images through a multi-branch convolution network, and to design an adaptive feature fusion module; the module is based on an attention mechanism, dynamically generates modulation coefficients and attention weights and other parameters, selectively weights the features of the reference image, and then deeply fuses the features with the features of the target image, to finally reconstruct a high-quality high-resolution image; the application breaks through the limitation of relying only on its own information, fully utilizes external priori, significantly reduces the model parameter quantity, significantly improves the detail fidelity and structural consistency of the reconstructed image, and has excellent clinical practicability and deployment feasibility.

Description

Technical Field

[0001] This invention relates to the field of medical image processing, and in particular to a method and system for medical image super-resolution reconstruction based on external image fusion. Background Technology

[0002] Medical imaging technologies, such as ultrasound (US), magnetic resonance imaging (MRI), computed tomography (CT), and X-ray imaging, are indispensable tools for modern clinical diagnosis. However, due to limitations in hardware performance, patient radiation dose safety control, and imaging time, clinically acquired medical images often have limited resolution. These low-resolution images are frequently accompanied by problems such as blurred details, noise interference, and unclear edges, severely restricting doctors' ability to identify minute lesions and fine anatomical structures, affecting the accuracy of early diagnosis and the precise planning of treatment plans. Therefore, how to reconstruct high-quality high-resolution (HR) images from existing low-resolution (LR) medical images, i.e., medical image super-resolution (SR) reconstruction technology, has become an important research direction in the field of medical image processing.

[0003] Early super-resolution methods were mainly based on interpolation (such as bicubic interpolation) or reconstruction theory. These methods only utilize the low-frequency information of a single image, have limited reconstruction capabilities, and are prone to producing overly smoothed textures, failing to recover true high-frequency details. With breakthroughs in deep learning technology, data-driven super-resolution methods have made revolutionary progress.

[0004] Specifically, methods based on convolutional neural networks (CNNs) (such as SRCNN) have for the first time achieved an end-to-end nonlinear mapping from LR to HR; methods based on generative adversarial networks (GANs) (such as SRGAN) have significantly improved the visual perception quality of reconstructed images by introducing adversarial loss; and models based on Transformers (such as SwinIR) utilize self-attention mechanisms to better model long-range dependencies in images. These methods have demonstrated superior performance on natural image datasets and some medical image datasets.

[0005] Nevertheless, current mainstream deep super-resolution methods generally suffer from a fundamental limitation: they are essentially "single-image" super-resolution methods, meaning the reconstruction process relies solely on the information contained within the single low-resolution input image. When image quality is severely degraded (e.g., highly blurred, noisy, or lacking texture), these methods struggle to reconstruct convincing details and structures due to a lack of sufficient information constraints, easily resulting in artifacts or structural distortions. This is particularly true in medical imaging, where images of different patients, or even the same anatomical location from different times within the same patient, often exhibit high structural similarity. Existing deep learning methods have failed to effectively mine and utilize this rich prior knowledge of external high-resolution images, leading to a bottleneck in performance improvement.

[0006] In recent years, a few studies have attempted to incorporate external information. For example, some works have used databases of the same modality for reference, but these are mostly limited to style transfer or simple feature stitching, failing to achieve adaptive and refined fusion of external reference images and target images at the deep feature level. In addition, existing fusion methods are often complex in model and have a huge number of parameters, which is not conducive to deployment in real-world clinical environments.

[0007] Therefore, how to design a super-resolution reconstruction framework that can intelligently retrieve and deeply integrate prior information from external related images while maintaining the model's lightweight nature, thereby overcoming the limitations of existing technologies that rely on single image information and significantly improving the reconstruction quality and structural fidelity of medical images, especially ultrasound images that are susceptible to noise and resolution limitations, is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method and system for medical image super-resolution reconstruction based on external image fusion. This aims to reduce the negative impact of network dynamic defense actions on network performance, thereby achieving a balance between the security of network dynamic defense and system performance. Through steps such as dynamic defense strategy generation and fusion, hybrid flow table generation and preloading, tag generation and flow table matching, and a strategy feedback mechanism for dynamic performance monitoring, efficient network communication and stronger network dynamic security protection can be achieved.

[0009] To achieve the above objectives, this application proposes a medical image super-resolution reconstruction method based on external image fusion, comprising the following steps: Step S1: Obtain the target medical image to be processed and the reference medical image associated with the target medical image, wherein the target medical image is a low-resolution image and the reference medical image is a high-resolution image; Step S2: Extract features from the target medical image and the reference medical image using a multi-branch convolutional network to obtain a target feature map and a reference feature map, wherein different branches use convolutional kernels of different sizes to extract multi-scale features; Step S3: Input the target feature map and the reference feature map into the feature fusion module, dynamically generate fusion parameters based on the attention mechanism, and use the parameters to perform weighted fusion of the target feature map and the reference feature map to obtain an enhanced feature map; Step S4: Based on the enhanced feature map, reconstruct a high-resolution reconstructed image corresponding to the target medical image.

[0010] As a further solution, in step S2, the feature extraction is achieved through a multi-branch convolutional network, which includes at least two parallel convolutional branches. Different convolutional branches use convolutional kernels of different sizes to extract features of the image at different scales.

[0011] As a further solution, the multi-branch convolutional network includes four parallel convolutional branches with kernel sizes of 1×1, 3×3, 5×5 and 7×7, respectively. The features extracted by each branch are concatenated in the channel dimension to form the target feature map or reference feature map.

[0012] As a further solution, the processing flow of each convolutional branch in the multi-branch convolutional network is as follows: sequentially perform 1×1 convolution, first PReLU activation function, convolution of the corresponding size, and second PReLU activation function on the input features.

[0013] As a further solution, in step S3, the feature fusion module performs the following operations: Step S31: Perform layer normalization processing on the target feature map to obtain normalized features; Step S32: Based on the target feature map and the reference feature map, generate parameters including at least modulation coefficients and attention weights through convolution operations; Step S33: Perform an affine transformation on the normalized features using the modulation coefficients and bias terms to obtain the first path features; Step S34: After mapping the attention weights to activation functions, multiply them element-wise with the normalized features to obtain the second feature path; Step S35: Fuse the first feature path with the second feature path to obtain the enhanced feature map.

[0014] As a further solution, in step S32, the target feature map and the reference feature map are concatenated, and then three sets of parameters are generated through a 1×1 convolutional layer, which are used as the original parameters of the modulation coefficient, the bias term and the attention weight, respectively.

[0015] As a further solution, in step S1, the reference medical image is obtained through an image retrieval mechanism, the retrieval condition being that it is similar to the target medical image in terms of anatomical structure or pathological features.

[0016] As a further solution, the Adam optimizer is used during training, and the learning rate is adjusted using a cosine annealing strategy.

[0017] As a further solution, the target medical image and the reference medical image are medical images of the same patient at different times; wherein, the medical image is one of ultrasound image, MRI image, CT image or X-ray image.

[0018] On the other hand, the present invention also provides a medical image super-resolution reconstruction system based on external image fusion, comprising: Image acquisition module: used to acquire the target medical image to be processed and the reference medical image associated with the target medical image, wherein the target medical image is a low-resolution image and the reference medical image is a high-resolution image; Feature extraction module: used to extract features from the target medical image and the reference medical image respectively through a multi-branch convolutional network to obtain target feature map and reference feature map, wherein different branches use convolutional kernels of different sizes to extract multi-scale features; Feature fusion module: used to input the target feature map and the reference feature map into the feature fusion module, dynamically generate fusion parameters based on the attention mechanism, and use the parameters to perform weighted fusion of the target feature map and the reference feature map to obtain an enhanced feature map; Image reconstruction module: used to reconstruct and generate a high-resolution reconstructed image corresponding to the target medical image based on the enhanced feature map.

[0019] Compared with related technologies, the medical image super-resolution reconstruction method and system based on external image fusion provided by this invention has the following advantages: 1. This invention innovatively introduces an external high-resolution reference image as an auxiliary information source, overcoming the limitation of traditional single-image super-resolution techniques that rely solely on the target image's own information. Utilizing attention-guided adaptive feature fusion, clear and relevant structural priors and texture details from the reference image can be accurately incorporated into the target image reconstruction process, significantly improving the objective quality and visual fidelity of the reconstructed image.

[0020] Experimental results show that, on the 4x super-resolution task, our method outperforms mainstream deep learning methods (such as VDSR and IDMN) in both the peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM), achieving higher detail restoration and better structural consistency, thus providing clearer and more reliable image evidence for clinical diagnosis.

[0021] 2. The multi-branch feature extraction and adaptive fusion network architecture designed in this invention achieves an excellent balance between parameter efficiency and performance, maintaining the lightweight and efficient nature of the model while achieving high performance.

[0022] Through a carefully designed parallel multi-scale convolutional branch and a parameter generation and fusion strategy based on 1×1 convolution, the model requires only about 306K parameters, significantly fewer than some mainstream models (e.g., about 64% fewer than IDMN), yet achieves superior reconstruction results. This lightweight design greatly reduces the model's storage overhead and computational complexity, improving the practical deployment feasibility of the method and making it easier to integrate into clinical imaging workstations or mobile diagnostic devices.

[0023] 3. The fusion mechanism constructed in this invention can effectively utilize external prior information as strong guidance in the reconstruction process, enabling the model to converge quickly and achieve leading performance even under conditions of relatively limited training resources (e.g., only one-twentieth of the conventional number of training iterations). This indicates that the method has superior learning efficiency and robustness. In clinical scenarios where data acquisition is difficult or paired data is scarce, this ability to utilize external similar priors helps improve the model's generalization performance and cope with more diverse cases.

[0024] 4. This invention fully considers the structural repeatability and noise characteristics of medical images (especially ultrasound images), as well as the stringent requirements for fidelity in key lesion areas in clinical diagnosis. The multi-branch structure comprehensively captures features from global contours to local details; the dynamic weight allocation mechanism ensures that, when fusing external information, unique, diagnostically relevant lesion information in the target image is preferentially preserved and enhanced, avoiding the loss or overwriting of beneficial clinical information. This method directly addresses the practical need to improve the diagnostic value of medical images and has promising prospects for clinical translation and application.

[0025] In summary, this invention not only innovatively utilizes external image information in principle and achieves high-quality and high-efficiency super-resolution reconstruction at the technical level, but also closely aligns with the actual needs of medical imaging diagnosis at the application level, providing an effective technical tool for advancing precision medicine. Attached Figure Description

[0026] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0027] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0028] Figure 1 A schematic diagram illustrating the steps of a medical image super-resolution reconstruction method based on external image fusion provided by the present invention; Figure 2 A schematic diagram of a medical image super-resolution reconstruction system based on external image fusion provided by the present invention; Figure 3 This is a schematic diagram of feature fusion and reconstruction provided by the present invention; wherein, a is a low-resolution target image; b is an external image; and c is the image reconstructed through feature fusion. Figure 4 This is a schematic diagram of the model architecture provided by the present invention; Figure 5 A comparison diagram of test results provided by this invention. Figure 1 ; Figure 6 A comparison diagram of test results provided by this invention. Figure 2 ; The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0030] Example 1 Please see Figure 1 This embodiment provides a medical image super-resolution reconstruction method based on external image fusion, including the following steps: Step S1: Obtain the target medical image to be processed and the reference medical image associated with the target medical image, wherein the target medical image is a low-resolution image and the reference medical image is a high-resolution image; Step S2: Extract features from the target medical image and the reference medical image using a multi-branch convolutional network to obtain a target feature map and a reference feature map, wherein different branches use convolutional kernels of different sizes to extract multi-scale features; Step S3: Input the target feature map and the reference feature map into the feature fusion module, dynamically generate fusion parameters based on the attention mechanism, and use the parameters to perform weighted fusion of the target feature map and the reference feature map to obtain an enhanced feature map; Step S4: Based on the enhanced feature map, reconstruct a high-resolution reconstructed image corresponding to the target medical image.

[0031] It should be noted that the medical image super-resolution reconstruction method provided in this embodiment introduces potential external image resources to improve the effectiveness of the medical image super-resolution reconstruction method and provides technical support for obtaining clear medical images in clinical practice. By retrieving high-resolution external medical images with similar structural features to the input image and combining attention mechanism and feature alignment module, the reference image is effectively fused, guiding the super-resolution reconstruction network to improve image quality.

[0032] This method achieves PSNR / SSIM of 37.27 / 0.9351 in the ×4 magnification task. Compared with mainstream deep learning methods, our method improves PSNR by 0.21 dB over VDSR and 0.52 dB over IDMN, while maintaining the best performance in SSIM. Regarding model parameters, our method uses only 306 K parameters, a reduction of approximately 64% compared to IDMN, yet still achieves superior performance. Introducing potential external image resources can provide structural prior information, effectively improving the quality and efficiency of medical image super-resolution reconstruction, and has promising clinical application prospects. Furthermore, in step S2, the feature extraction is achieved through a multi-branch convolutional network, which includes at least two parallel convolutional branches. Different convolutional branches use convolutional kernels of different sizes to extract features of the image at different scales.

[0033] Furthermore, the multi-branch convolutional network includes four parallel convolutional branches with kernel sizes of 1×1, 3×3, 5×5, and 7×7, respectively. The features extracted by each branch are concatenated along the channel dimension to form the target feature map or reference feature map.

[0034] Specifically, the network in this embodiment contains four parallel convolutional branches with kernel sizes of 1×1, 3×3, 5×5, and 7×7, respectively. This specific design is a key technical means to achieve the function of "extracting multi-scale features": the 1×1 convolution is mainly used for cross-channel information integration and dimensionality reduction; the 3×3 and 5×5 convolutions focus on capturing medium to local scale tissue structures and texture details; while the large 7×7 convolutional kernel is responsible for perceiving a larger range of anatomical contours and global spatial context. By concatenating the feature maps representing different scales extracted by these four branches along the channel dimension, the final target or reference feature map becomes a highly information-rich composite feature representation that integrates everything from fine textures to overall morphology, providing a solid and comprehensive information foundation for subsequent deep fusion.

[0035] Furthermore, the processing flow of each convolutional branch in the multi-branch convolutional network is as follows: sequentially perform 1×1 convolution, first PReLU activation function, convolution of the corresponding size, and second PReLU activation function on the input features.

[0036] Specifically, this processing flow design embodies efficient and modular feature extraction logic. First, 1×1 convolutions are used to compress or adjust the channel dimension of the input features. This reduces the computational cost of subsequent large convolutional kernel operations and also achieves preliminary fusion of cross-channel information. The subsequent PReLU activation function introduces a parameterized nonlinear transformation, enhancing the model's expressive power and helping to alleviate the gradient vanishing problem.

[0037] Then, the core features are extracted using a convolutional kernel of the corresponding size (e.g., 3×3, 5×5, etc.) from this branch. This is a crucial step in capturing the specific scale-specific spatial information that this branch is responsible for. Finally, the PReLU activation function is used again to further non-linearly map the extracted features, enhancing their discriminative power. This sequential design of "adjustment-activation-extraction-reactivation" ensures that each branch can efficiently and layer-by-layer extract compact and expressive features from the input at its target scale.

[0038] Furthermore, in step S3, the feature fusion module performs the following operations: Step S31: Perform layer normalization processing on the target feature map to obtain normalized features; Step S32: Based on the target feature map and the reference feature map, generate parameters including at least modulation coefficients and attention weights through convolution operations; Step S33: Perform an affine transformation on the normalized features using the modulation coefficients and bias terms to obtain the first path features; Step S34: After mapping the attention weights to activation functions, multiply them element-wise with the normalized features to obtain the second feature path; Step S35: Fuse the first feature path with the second feature path to obtain the enhanced feature map.

[0039] Specifically, the feature fusion module is designed to achieve adaptive and refined fusion of reference image information. The entire process begins with layer normalization of the target feature map (S31), which aims to stabilize the data distribution and provide a standardized benchmark for subsequent calculations. Subsequently, the module dynamically generates control parameters (S32) by analyzing the target and reference feature maps, including modulation coefficients and bias terms for affine transformation, as well as weights for attention weighting. These parameters are not pre-set but are learned in real time by the network based on the input content, which is key to achieving "adaptation".

[0040] Based on these parameters, the module performs two complementary fusion operations in parallel: one path uses affine transformation (S33) to globally scale and translate the normalized features; the other path uses an attention mechanism (S34) to selectively enhance the features in channels or space. Finally, the features from these two paths, which have undergone "global adjustment" and "selective enhancement" respectively, are fused (S35) to output an enhanced feature map that fully utilizes external reference priors while closely incorporating the characteristics of the target image itself.

[0041] Furthermore, in step S32, after concatenating the target feature map and the reference feature map, three sets of parameters are generated through a 1×1 convolutional layer, which are used as the original parameters of the modulation coefficient, the bias term and the attention weight, respectively.

[0042] Specifically, this step defines the generation mechanism for the control parameters required for adaptive fusion, which is the core operation for realizing dynamic weight allocation. First, the target feature map and the reference feature map are concatenated along the channel dimension. This operation places the feature information from the two sources in the same representation space, laying the foundation for subsequent analysis of their interrelationships.

[0043] Subsequently, a 1×1 convolutional layer is used to process the concatenated fused features. This lightweight convolution learns and extracts three key control signals from the mixed information. These three parameters—modulation coefficients (α), bias terms (β), and original attention weights (γ)—are not pre-set but generated in real-time by the network based on the current input features. They are used to precisely control the intensity of subsequent affine transformations, translation amounts, and channel attention weights, respectively. This design makes the fusion process completely data-driven, enabling dynamic adjustments to the fusion strategy for each specific "target-reference" image pair, thereby achieving highly adaptive and precise feature enhancement.

[0044] Furthermore, in step S1, the reference medical image is obtained through an image retrieval mechanism, and the retrieval condition is that it is similar to the target medical image in terms of anatomical structure or pathological features.

[0045] Specifically, this step clarifies the specific sources and selection criteria for external reference images in this invention, solving the key prerequisite problem of "how to obtain effective auxiliary information." By introducing an "image retrieval mechanism," this method can proactively and intelligently search for high-quality images that best match the target image in key visual features from large-scale medical image databases (such as hospital PACS systems or standardized public datasets) as references. Its core retrieval condition is set as "similarity of anatomical structures or pathological features," which ensures that the selected reference images share the same organ tissue morphology or lesion patterns as the target images, thus making the high-resolution prior information they contain highly relevant and portable. This content-based retrieval is the fundamental guarantee that external information can effectively assist in reconstruction rather than introduce interference.

[0046] In practice, retrieval can be based on the patient's own imaging history (the optimal choice, ensuring subject consistency) or on anonymized databases across patients. This definition not only gives the method flexibility in data utilization but also provides an operational path for its clinical implementation. It emphasizes that this approach does not rely on arbitrary external images but on a purposeful, intelligent data acquisition process aimed at enhancing reconstruction results, thereby transforming external information from "potentially available resources" into "reliable and usable inputs."

[0047] Furthermore, the Adam optimizer is used during training, and the learning rate is adjusted using a cosine annealing strategy.

[0048] Specifically, this invention employs the Adam optimizer in conjunction with a cosine annealing learning rate scheduling strategy during the model training phase. This combination optimizes both the efficiency and stability of the training process. By combining the advantages of momentum and adaptive learning rates (such as RMSProp), the Adam optimizer can automatically adjust the learning step size of each parameter, achieving fast and stable convergence in the early stages of training. It is particularly suitable for handling complex networks like this invention, which have multi-branch structures and adaptive fusion modules.

[0049] Meanwhile, the cosine annealing strategy dynamically adjusts the trajectory of the global learning rate, causing it to smoothly decay from its initial value to near zero following a cosine function. This arrangement allows the model to explore the parameter space with larger step sizes in the early stages of training, while fine-tuning is performed with extremely small step sizes in the later stages. This effectively avoids getting trapped in local optima and helps improve the generalization performance and convergence quality of the final model. The synergistic effect of these two approaches ensures that the lightweight network proposed in this invention can fully exploit its performance potential within a relatively limited number of training iterations. This is a crucial technical guarantee for the model to achieve superior reconstruction results with fewer parameters.

[0050] Furthermore, the target medical image and the reference medical image are medical images of the same patient at different times; wherein, the medical image is one of ultrasound image, MRI image, CT image or X-ray image.

[0051] Example 2 Please see Figure 2 The present invention also provides a medical image super-resolution reconstruction system based on external image fusion, comprising: Image acquisition module: used to acquire the target medical image to be processed and the reference medical image associated with the target medical image, wherein the target medical image is a low-resolution image and the reference medical image is a high-resolution image; Feature extraction module: used to extract features from the target medical image and the reference medical image respectively through a multi-branch convolutional network to obtain target feature map and reference feature map, wherein different branches use convolutional kernels of different sizes to extract multi-scale features; Feature fusion module: used to input the target feature map and the reference feature map into the feature fusion module, dynamically generate fusion parameters based on the attention mechanism, and use the parameters to perform weighted fusion of the target feature map and the reference feature map to obtain an enhanced feature map; Image reconstruction module: used to reconstruct and generate a high-resolution reconstructed image corresponding to the target medical image based on the enhanced feature map.

[0052] It should be noted that this embodiment proposes an improved scheme combining external image fusion with super-resolution reconstruction algorithms, aiming to fully utilize the complementary information of external images to improve the effect of medical image super-resolution reconstruction. Specifically, this scheme first fuses the medical image with the external image using a multimodal image fusion method based on an attention mechanism to obtain a fused image. The attention mechanism can adaptively allocate weights to different modalities of the image, highlighting important information and suppressing noise and interference, thereby improving the quality of the fused image. When fusing the external image and the target image, the attention mechanism can automatically focus on the information of the lesion area, enhancing the representation of the lesion area in the fused image, thereby effectively improving the detail restoration capability of the lesion area and the overall reconstruction quality of the image, providing support for subsequent diagnosis. Figure 3 The feature fusion diagram is shown.

[0053] As the core component for achieving auxiliary functions using external images, the key to the feature fusion module lies in accurately controlling the fusion scale of the two image features. To this end, an attention mechanism is specifically introduced to enable the model to autonomously learn the importance weight allocation logic for the two image features. Specifically, regions in medical images with clear features and high information density, such as areas with well-defined lesion boundaries and intact tissue structures, are assigned higher weights to ensure that crucial diagnostic information is not weakened. Conversely, regions in natural images that strongly complement the features of medical images, such as areas that supplement missing information in medical images through texture details or edge transitions, are also given appropriate weights.

[0054] Through such weighting adjustments, the two features can be efficiently fused, ultimately forming a more comprehensive and representative fused feature. This fusion mode has significant advantages: it can both leverage the rich detail information of natural images to fill the gaps in certain dimensions of medical image data, and firmly maintain the dominant position of key diagnostic information in medical images through attention mechanisms, avoiding the dilution of core diagnostic information due to fusion. Building upon this foundation, a super-resolution reconstruction network model based on external image fusion is constructed. This model primarily consists of three closely interconnected modules: feature extraction, feature fusion, and image reconstruction. In the feature extraction module, deep feature mining is performed on both the input low-resolution medical image and the preprocessed (e.g., denoising and normalization) natural image. A convolutional neural network (CNN) is used as the core architecture, capturing key elements of the image layer by layer through the progressive stacking of multiple convolutional layers and the nonlinear mapping of activation functions. For medical images, the extraction process focuses on morphological features of the lesion area and hierarchical structure of the tissue, information with direct diagnostic significance, to ensure that subsequent reconstruction can accurately restore diagnostic details. For natural images, the focus is on extracting rich edge transition patterns, subtle texture distribution, and other general visual features to provide effective material for supplementing the details of medical images. In medical image super-resolution reconstruction, target image features (Y), external image features (β), and normalized features (Y, β, β) are used. The dynamic fusion of these parameters is shown in Table 1. It needs to accurately adapt to the clinical diagnostic requirements for detail fidelity, lesion enhancement, and modal compatibility.

[0055] Table 1. Medical significance and function of different feature maps Target images: Low-resolution medical images acquired clinically, resulting in low clarity due to equipment limitations, radiation dose control, scanning time constraints, and other factors. These images often suffer from blurred details and unclear structures, requiring super-resolution reconstruction to improve resolution and aid in clinical diagnosis.

[0056] External reference images: The patient's previous examination data, extracted from the hospital's imaging system or electronic medical records with informed consent, is the best reference source. It can reflect the long-term morphological characteristics of the same area, such as the stability of the lesion, and can minimize interference from external structures.

[0057] By referring to Y The DiMoSR model proposed by lmaz et al., and the model architecture we designed are shown in [link to model architecture]. Figure 4 As shown The design logic of multi-branch convolution is crucial for understanding the complex structure of medical images, which include global contours, mesoscale structures, and local details. To fully capture this information, a four-branch parallel convolutional structure is designed, covering different receptive fields, as shown in Table 2. Unlike explicit optical flow estimation, this method utilizes the natural spatial correlation of multi-scale features to achieve alignment.

[0058] Table 2 Comparison of branch kernels of different sizes Features extracted by large convolutional kernels (7×7) correspond to the macroscopic structural location of the image and the overall orientation of organs.

[0059] Features extracted by small convolutional kernels (3×3 / 5×5) correspond to local detail locations. After multi-branch extraction and fusion of dual-path images (target image and external image), features are naturally aligned in the channel dimension, with similar semantic / structural information corresponding to the same channel. No additional displacement calculation is required, simplifying the process and adapting to the high repetitiveness of medical image structures. For the same organ in different scans, structural and semantic channels can be matched. In each branch, features are fused through interactive branching. First, 1×1 convolution is used to further compress channels, reducing computation, while fusing cross-channel information. Then, the PReLU activation function is applied. The parameterized ReLU addresses neuron death and enhances nonlinear expression. Finally, features are extracted using convolutional kernels of the corresponding size (3×3 / 5×5 / 7×7) and activated again by PReLU. Taking the 3×3 branch as an example, the process is: Input features → 1×1 convolution → PReLU → 3×3 convolution → PReLU.

[0060] 1.4.3 Unlike traditional methods that rely solely on a single ultrasound image for super-resolution reconstruction, this model introduces external images as auxiliary information sources, broadening the information input dimension. It utilizes potential prior knowledge from external images to guide the super-resolution reconstruction of ultrasound images, representing an innovative approach. The feature interactions between different branches (indicated by green arrows) and the multi-stage feature fusion method break away from the traditional single-branch or simple feature fusion pattern, enabling more flexible and effective integration of features from different sources and scales, providing a more powerful feature representation capability for super-resolution reconstruction. The feature enhancement strategy, which generates modulation coefficients and attention weights through 1×1 convolutions and combines affine transformation with attention mechanisms, is a novel feature processing strategy. It adaptively adjusts the enhancement method according to the input features, offering greater flexibility and specificity compared to traditional fixed enhancement methods.

[0061] In medical image super-resolution tasks, the core logic of target-external image cross-fusion is to break down the independent processing boundaries between the two images. Through feature flow across images and multiple rounds of interaction, the information depth of the target image and the external image is deeply complementary. Unlike the traditional single-path-dominated mode, cross-fusion emphasizes dynamic, bidirectional iterative optimization of features, using the high-definition structure of the external image to repair the loss of detail in the target image. It preserves the unique lesion information of the target image and avoids over-referencing the external image, which could lead to lesion obscuration.

[0062] In a specific test instance, we used Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) to measure reconstruction quality. All metrics were calculated on the Y channel of the YCbCr color space. To enhance the model's generalization ability, we implemented data augmentation strategies, specifically random horizontal mirroring and rotations of 90°, 180°, and 270°. During data preprocessing, unlike the traditional method of randomly cropping 64×64 pixel image patches from low-resolution (LR) images, we directly used 157×113 pixel low-resolution image patches. Considering the 4x super-resolution upscaling requirement, these LR image patches correspond to a 628×452 pixel region in the high-resolution HR space. During model training, we used the Adam optimizer for parameter updates, with the momentum parameter β1 set to 0.9 and the second-order moment estimation parameter β2 set to 0.99. The entire training process lasted for 30,000 iterations, processing 8 pairs of LR-HR image pairs per batch. The learning rate scheduling uses a cosine annealing strategy, with an initial learning rate set to 1×10. - ³, gradually decaying to 1×10 as training progresses. -5 This balances the need for rapid convergence in the early stages of training with the need for fine-grained optimization in the later stages. The test results are shown in Table 3, and the image results are compared as follows: Figure 5 and Figure 6 As shown.

[0063] Table 3 Benchmark results of super-resolution networks Note: The last two metrics are Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM), both calculated based on the Y channel only.

[0064] As shown in Table 3, the method of this invention achieves a PSNR of 37.27 dB and an SSIM of 0.9351 on the classic benchmark set, achieving the best performance among similar methods. It should be noted that in single-image super-resolution tasks, existing research generally shows that the performance differences between methods are usually small. For example, the PSNR improvement between representative methods such as VDSR, FSRCNN, and IDMN is generally only 0.05–0.08 dB, with extremely limited room for improvement. Against this backdrop, the method of this invention still achieves a 0.2 dB increase, far exceeding the common level, demonstrating the effectiveness of the proposed network in terms of structural design and feature utilization. At the same time, the SSIM index also reaches 0.9351, which is 0.0018 higher than the second-best method, even though improvement becomes increasingly difficult as SSIM approaches 1 (in contrast, the second-best method only improves by 0.0011 compared to the third-best method), further demonstrating the advantages of the method of this invention in terms of detail preservation and structural consistency. In addition to the performance improvements, the method of this invention also demonstrates outstanding model efficiency. Compared to IDMN with its 852K parameters, the method of this invention uses only about 306K parameters, yet still achieves superior reconstruction results. This demonstrates that the method not only outperforms mainstream models in performance but also possesses higher computational efficiency and deployability. More notably, due to limitations in computational resources, the training iterations required by this invention are only about 1 / 20th of those in conventional experimental settings. Even under these extremely limited conditions, the proposed method still leads in PSNR and SSIM, exhibiting faster convergence and stronger generalization ability. Therefore, the current results can be considered a conservative estimate of the model's potential; with more abundant training resources, this method is expected to further improve performance. In summary, the method of this invention, through the novel idea of ​​introducing external images, not only surpasses the common levels of previous work in this task in terms of performance improvement but also possesses advantages in parameter efficiency and convergence, demonstrating its potential value in practical applications.

[0065] The proposed medical image super-resolution reconstruction method based on external image fusion demonstrates significant advantages in multi-scale feature extraction, cross-fusion, and dynamic weight allocation, providing clearer and more valuable image support for medical imaging diagnosis, such as ultrasound. The multi-branch convolutional structure constructed in this invention, through synergistic action, can detect lesions earlier, overcoming the shortcomings of traditional single-scale feature extraction methods that easily miss key details, significantly improving diagnostic sensitivity, and achieving a precise response to the needs of medical image diagnosis. The cross-fusion mechanism breaks the limitations of independent image processing, enabling bidirectional empowerment of information from both the target image and the external image.

[0066] The above are only some embodiments of this application and do not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for super-resolution reconstruction of medical images based on external image fusion, characterized in that, Includes the following steps: Step S1: Obtain the target medical image to be processed and the reference medical image associated with the target medical image, wherein the target medical image is a low-resolution image and the reference medical image is a high-resolution image; Step S2: Extract features from the target medical image and the reference medical image using a multi-branch convolutional network to obtain a target feature map and a reference feature map, wherein different branches use convolutional kernels of different sizes to extract multi-scale features; Step S3: Input the target feature map and the reference feature map into the feature fusion module, dynamically generate fusion parameters based on the attention mechanism, and use the parameters to perform weighted fusion of the target feature map and the reference feature map to obtain an enhanced feature map; Step S4: Based on the enhanced feature map, reconstruct a high-resolution reconstructed image corresponding to the target medical image.

2. The method for medical image super-resolution reconstruction based on external image fusion according to claim 1, characterized in that, In step S2, the feature extraction is achieved through a multi-branch convolutional network, which includes at least two parallel convolutional branches. Different convolutional branches use convolutional kernels of different sizes to extract features of the image at different scales.

3. The method for medical image super-resolution reconstruction based on external image fusion according to claim 2, characterized in that, The multi-branch convolutional network includes four parallel convolutional branches with kernel sizes of 1×1, 3×3, 5×5, and 7×7. The features extracted by each branch are concatenated along the channel dimension to form the target feature map or reference feature map.

4. A method for medical image super-resolution reconstruction based on external image fusion according to claim 2 or claim 3, characterized in that, The processing flow of each convolutional branch in the multi-branch convolutional network is as follows: 1×1 convolution, first PReLU activation function, convolution of the corresponding size, and second PReLU activation function are performed sequentially on the input features.

5. The method for medical image super-resolution reconstruction based on external image fusion according to claim 1, characterized in that, In step S3, the feature fusion module performs the following operations: Step S31: Perform layer normalization processing on the target feature map to obtain normalized features; Step S32: Based on the target feature map and the reference feature map, generate parameters including at least modulation coefficients and attention weights through convolution operations; Step S33: Perform an affine transformation on the normalized features using the modulation coefficients and bias terms to obtain the first path features; Step S34: After mapping the attention weights to activation functions, multiply them element-wise with the normalized features to obtain the second feature path; Step S35: Fuse the first feature path with the second feature path to obtain the enhanced feature map.

6. The method for medical image super-resolution reconstruction based on external image fusion according to claim 5, characterized in that, In step S32, after concatenating the target feature map and the reference feature map, three sets of parameters are generated through a 1×1 convolutional layer, which are used as the original parameters of the modulation coefficient, the bias term and the attention weight, respectively.

7. The method for medical image super-resolution reconstruction based on external image fusion according to claim 1, characterized in that, In step S1, the reference medical image is obtained through an image retrieval mechanism, and the retrieval condition is that it is similar to the target medical image in terms of anatomical structure or pathological features.

8. The method for medical image super-resolution reconstruction based on external image fusion according to claim 1, characterized in that, The Adam optimizer was used during training, and the learning rate was adjusted using a cosine annealing strategy.

9. The method for medical image super-resolution reconstruction based on external image fusion according to claim 1, characterized in that, The target medical image and the reference medical image are medical images of the same patient at different times; wherein, the medical image is one of ultrasound image, MRI image, CT image or X-ray image.

10. A medical image super-resolution reconstruction system based on external image fusion, characterized in that, include: Image acquisition module: used to acquire the target medical image to be processed and the reference medical image associated with the target medical image, wherein the target medical image is a low-resolution image and the reference medical image is a high-resolution image; Feature extraction module: used to extract features from the target medical image and the reference medical image respectively through a multi-branch convolutional network to obtain target feature map and reference feature map, wherein different branches use convolutional kernels of different sizes to extract multi-scale features; Feature fusion module: used to input the target feature map and the reference feature map into the feature fusion module, dynamically generate fusion parameters based on the attention mechanism, and use the parameters to perform weighted fusion of the target feature map and the reference feature map to obtain an enhanced feature map; Image reconstruction module: used to reconstruct a high-resolution reconstructed image corresponding to the target medical image based on the enhanced feature map.

Images