Medical image super-resolution reconstruction method, device, equipment, medium and product

Abstract

The invention discloses a medical image super-resolution reconstruction method and device, equipment, a medium and a product, and relates to the technical field of computer image processing, and the method comprises the steps: carrying out the preprocessing of the original data of brain nuclear magnetic resonance of a target object, and obtaining a high-resolution reference image and a low-resolution target image; respectively extracting multi-scale features of the target image and the reference image to obtain a multi-scale target feature map and a multi-scale reference feature map; based on the multi-scale target feature map and the multi-scale reference feature map, screening out key features and performing texture enhancement on the key features to obtain a multi-scale texture feature map; aligning and fusing the multi-scale texture feature map and the multi-scale target feature map to obtain a multi-scale fused feature map; and generating a high-resolution target image based on the multi-scale fusion feature map. According to the method, the core anatomical structure of the reconstructed image is clearer, the noise is lower, and the requirements for anatomical information of different scales in clinical diagnosis are met.

Description

Methods, devices, equipment, media and products for super-resolution reconstruction of medical images Technical Field

[0001] This application relates to the field of computer image processing technology, and in particular to a method, apparatus, device, medium and product for super-resolution reconstruction of medical images. Background Technology

[0002] Magnetic resonance imaging (MRI) of the brain is the core basis for the accurate diagnosis of diseases such as brain tumors and hydrocephalus. However, due to the high cost of scanning equipment, limited scanning time, insufficient patient cooperation, and noise interference and undersampling effect during image transmission and storage, it is difficult to obtain high-quality brain MRI images, which directly restricts the accuracy of clinical diagnosis.

[0003] Currently, multi-contrast super-resolution (MCSR) technology is the mainstream approach to restore clarity from low-resolution brain MRI images. Its core principle is to enhance the restoration effect by leveraging complementary information from MRI images of different contrasts. However, existing MCSR technologies are prone to introducing redundant features from irrelevant regions of the reference image during feature extraction and fusion. This makes the feature interaction and aggregation processes susceptible to latent noise, thereby reducing the accuracy of super-resolution restoration and failing to meet the stringent requirements of clinical diagnosis for image detail. Summary of the Invention

[0004] In view of the above-mentioned defects or deficiencies in the related technologies, the purpose of this application is to provide a method, device, equipment, medium and product for super-resolution reconstruction of medical images, which can make the core anatomical structure of the reconstructed image clearer and the noise lower, so as to meet the needs of clinical diagnosis for anatomical information at different scales.

[0005] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a medical image super-resolution reconstruction method, comprising: preprocessing the raw data of a target object's brain magnetic resonance imaging (MRI) to obtain a high-resolution reference image and a low-resolution target image; the reference image and the target image are images of different contrasts and resolutions of the target object's core brain region; extracting multi-scale features from the target image and the reference image respectively to obtain a multi-scale target feature map and a multi-scale reference feature map; based on the multi-scale target feature map and the multi-scale reference feature map, selecting key features and performing texture enhancement on the key features to obtain a multi-scale texture feature map; aligning and fusing the multi-scale texture feature map with the multi-scale target feature map to obtain a multi-scale fused feature map; and generating a high-resolution target image based on the multi-scale fused feature map.

[0006] Optionally, the preprocessing of the raw brain MRI data of the target object to obtain a low-resolution target image includes: splitting the raw data according to the slice dimension, extracting all axial, coronal and / or sagittal slices to obtain raw images; retaining a preset number of raw images, and performing spatial center cropping on each of the retained raw images to remove non-brain regions at the edges of each slice to obtain raw brain images; cropping the raw brain images according to the dataset type and determining the target image to be restored and a high-resolution parametric image; converting each target image to be restored from the spatial domain to the frequency domain using Fourier transform to obtain a frequency domain image; cropping a low-frequency region of corresponding size from the central region of the frequency domain image according to the target scaling factor to obtain a low-frequency image; and converting the low-frequency image from the frequency domain to the spatial domain using inverse Fourier transform to obtain the low-resolution target image.

[0007] Optionally, before extracting multi-scale features from the target image and the reference image respectively to obtain a multi-scale target feature map and a multi-scale reference feature map, the method further includes: interpolating and upsampling the target image based on the spatial resolution of the reference image to obtain an interpolated target image; and converting both the interpolated target image and the parameter image into a single-channel tensor format to obtain a target image and a reference image of the same format.

[0008] Optionally, the step of extracting multi-scale features from the target image and the reference image to obtain multi-scale target feature maps and multi-scale reference feature maps includes: obtaining feature maps of the target image and the reference image through a 1×1 convolution, followed by initial feature extraction and nonlinear activation through a residual block to obtain a first multi-scale target feature map and a first multi-scale reference feature map; performing downsampling operations on the first multi-scale target feature map and the first multi-scale reference feature map, and obtaining a second multi-scale target feature map and a second multi-scale reference image feature map through a residual block; and performing a downsampling operation on the second multi-scale target feature map and the first multi-scale reference image feature map. The target feature map and the second multi-scale reference feature map are downsampled, and a third multi-scale target feature map and a third multi-scale reference feature map are obtained through residual blocks. The third multi-scale target feature map and the third multi-scale reference feature map are downsampled and residual block feature extraction is performed on the third multi-scale target feature map and the third multi-scale reference feature map until the last layer of the encoder is reached, resulting in the Nth multi-scale target feature map and the Nth multi-scale reference feature map; where N is a positive integer and is the same as the number of layers of the encoder. The multi-scale target feature maps and multi-scale reference feature maps extracted from each layer of the encoder are integrated in sequence to obtain the multi-scale target feature map and the multi-scale reference feature map.

[0009] Optionally, the step of selecting key features and performing texture enhancement based on the multi-scale target feature map and the multi-scale reference feature map to obtain a multi-scale texture feature map includes: encoding the channel context of the multi-scale target feature map and the multi-scale reference feature map to generate a target query feature Q and a reference key feature K, respectively; calculating the similarity score of pixel pairs between the target query feature Q and the reference key feature K, and selecting the top k highest similarity scores using a top-k algorithm to obtain the index values ​​of the top k highest similarity scores; setting the mask value of the index position corresponding to each index value to 1 and keeping the non-index positions 0 through a scatter operation to obtain a binary sparse mask; replacing the similarity score of the non-mask positions with negative infinity based on the binary sparse mask to obtain the key features of the current scale; performing multi-scale texture enhancement on the key features of the current scale to obtain the texture-enhanced features of the current scale; and integrating the texture-enhanced features of the current scale from all scales to obtain the multi-scale texture feature map.

[0010] Optionally, aligning and fusing the multi-scale texture feature map with the multi-scale target feature map to obtain a multi-scale fused feature map includes: mapping the distribution of the multi-scale texture feature map to a distribution consistent with that of the multi-scale target feature map, obtaining an aligned multi-scale texture feature map and an aligned multi-scale target feature map; concatenating the aligned multi-scale texture feature map and the aligned multi-scale target feature map along the channel dimension to obtain a concatenated feature map; fusing the concatenated feature map using a sparse selection cross-attention transformer to obtain a fused feature map for each scale; and fusing the fused feature maps and target feature maps of different scales through the upsampling and feature adaptation module of the decoder to obtain the multi-scale fused feature map.

[0011] Secondly, this application provides a medical image super-resolution reconstruction device, comprising: a preprocessing module for preprocessing the raw data of a target object's brain magnetic resonance imaging (MRI) to obtain a high-resolution reference image and a low-resolution target image; wherein the reference image and the target image are images of different contrasts and resolutions of the target object's brain core region; an extraction module for extracting multi-scale features from the target image and the reference image respectively to obtain a multi-scale target feature map and a multi-scale reference feature map; a filtering module for filtering key features based on the multi-scale target feature map and the multi-scale reference feature map and performing texture enhancement on the key features to obtain a multi-scale texture feature map; an alignment and fusion module for aligning and fusing the multi-scale texture feature map with the multi-scale target feature map to obtain a multi-scale fused feature map; and a generation module for generating a high-resolution target image based on the multi-scale fused feature map.

[0012] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the medical image super-resolution reconstruction method described in any one of the above.

[0013] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the medical image super-resolution reconstruction method described above.

[0014] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the medical image super-resolution reconstruction method described above.

[0015] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application provides a method, apparatus, device, medium, and product for super-resolution reconstruction of medical images. By preprocessing the raw data of the target object's brain MRI, the core brain region can be screened out and a low-resolution target image that fits the clinical real scene can be generated, providing a high-quality data foundation for subsequent image reconstruction and effectively eliminating interference from non-brain tissues; by extracting multi-scale features from the target image and reference image, the global anatomical structure and local detail information of the target image and reference image can be captured simultaneously, providing comprehensive support for feature fusion; by using multi-scale target feature maps and multi-scale reference feature maps, key features are screened out and texture enhancement is performed on the key features, which can more accurately preserve... This method retains the core information of high similarity between the target image and the reference image, masks low-correlation redundancy and implicit noise, and enhances texture details such as small blood vessels and lesion edges to improve feature quality. By aligning and fusing multi-scale texture feature maps with multi-scale target feature maps, the problem of feature distribution differences between images with different contrasts can be solved, achieving deep complementarity and efficient integration of multi-scale information and avoiding fusion conflicts. High-resolution target images can be generated through multi-scale fusion feature maps. High-resolution target images have the characteristics of clear anatomical structures and precise details, which can provide reliable imaging support for the accurate diagnosis of diseases such as brain tumors and hydrocephalus. At the same time, reference images can be used to shorten clinical scanning time, reduce equipment dependence and medical costs, and have both important technological innovation value and clinical application prospects. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 is a flowchart illustrating a medical image super-resolution reconstruction method according to an embodiment of this application; Figure 2 is a schematic diagram illustrating target images and parameter images of different datasets according to an embodiment of this application; Figure 3 is a structural framework diagram of a sparse selective cross-attention Transformer network according to an embodiment of this application; Figure 4 is a structural schematic diagram of a sparse cross-attention module and a multi-scale feedforward module according to an embodiment of this application; Figure 5 is a visualization schematic diagram of the IXI dataset magnified 4 times according to an embodiment of this application; Figure 6 is a visualization schematic diagram of the BraTs dataset magnified 4 times according to an embodiment of this application; Figure 7 is a visualization diagram of the BraTs dataset magnified 4 times according to an embodiment of this application; Figure 8 is a schematic diagram of ablation analysis of k-value sorting on different datasets according to an embodiment of this application; Figure 9 is a schematic diagram of ablation analysis of different k-values ​​in the sparse cross-attention module on the IXI dataset according to an embodiment of this application; Figure 10 is a functional module schematic diagram of a medical image super-resolution reconstruction device according to an embodiment of this application; Figure 11 is a structural schematic diagram of a computer device according to an embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0020] In an exemplary embodiment, as shown in FIG1, a medical image super-resolution reconstruction method is provided. This method is executed by a computer device, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method includes the following steps S10 to S50. Specifically: Step S10 involves preprocessing the raw data of the target object's brain MRI to obtain a high-resolution reference image and a low-resolution target image. In the exemplary embodiment, as shown in FIG2, the multi-scale reference image and the target image are images of the core brain region of the target object at different resolutions.

[0021] Step S20: Extract multi-scale features from the target image and the reference image respectively to obtain a multi-scale target feature map and a multi-scale reference feature map; Step S30: Based on the multi-scale target feature map and the multi-scale reference feature map, select key features and perform texture enhancement on the key features to obtain a multi-scale texture feature map; Step S40: Align and fuse the multi-scale texture feature map and the multi-scale target feature map to obtain a multi-scale fused feature map; Step S50: Generate a high-resolution target image based on the multi-scale fused feature map.

[0022] Specifically, this application embodiment restores the sharpness of the target image by constructing a Sparse Selective Cross-Attention Transformer Network (SCANSR). The SCANSR network includes an encoder, a Sparse Selective Cross-Attention Transformer (SCAT), and a decoder. For example, as shown in Figure 3, given a low-resolution target image... Given a high-resolution reference image ,in, This represents the spatial resolution of the feature map. This represents the magnification factor of the resolution. In the Sparse Selective Cross-Attention Transformer network, firstly, for the target image I... lr Perform interpolation to match reference image I ref The resolution is then determined by using an encoder to upsample the target image I. lr and reference image I ref Extracting multi-scale features from F yields F. lr and F ref feature.

[0023] Secondly, the sparse selective cross-attention transformer includes a sparse selective cross-attention module (SCA) and a multi-scale feed-forward model (MSFM). The sparse cross-attention module and the multi-scale feed-forward model are used to obtain the F... lr and F ref The texture features are searched in the middle. Then, the Texture Adaptation Module (TAM) in the decoder is used to align each scale with the texture features F. lr The features are then fused. Finally, a 1×1 convolution is used to generate I from the fused features. SREach stage of the encoder-decoder architecture encompasses its own specific spatial resolution and channel dimension. Pixel-unshuffle and pixel-shuffle operations are applied for downsampling and upsampling.

[0024] As a preferred embodiment, the solution of this application is implemented as follows: In the preprocessing stage, the composite image of each slice is reshaped into a 256×256 matrix and a 240×240 matrix by spatial center cropping. The 256×256 matrix conforms to the IXI dataset type, and the 240×240 matrix conforms to the BraTs dataset type. Only 60 slices in the center of the brain region are retained. Then, Fourier transform is applied to the frequency domain, and low-frequency regions of the original size (×2 and ×4 downsampled) are cropped from the center of the spectrum. Finally, inverse Fourier transform is applied to return to the image domain to obtain low-resolution slices of the corresponding resolution.

[0025] In the sparse selection cross-attention transformer feature extraction stage, the sparse selection cross-attention transformer is used as the feature extraction unit, which is formally given l-1 blocks of input features. and The characteristic, the process can be defined as follows (1), that is: (1) Wherein, LN is the normalization layer; and These represent the outputs from the sparse cross-attention module and the multi-scale feedforward module, respectively.

[0026] The sparse cross-attention module, as shown in the left image of Figure 4, specifically encodes the channel context using 1×1 convolutions and 3×3 depthwise convolutions after normalization. A cross-attention mechanism is applied along the channel direction. Next, the similarity scores of pixel pairs between the qurey of the reshaped target image and the key of the reference image are calculated, and the top k similarity scores are selected and their index values ​​are returned. Here, k is an adjustable parameter used to dynamically control the sparsity, with values ​​being appropriate scores (e.g., 1 / 2). Then, a scatter operation is used to set the Mask values ​​(initially all 0) at these indexed positions from 0 to 1, thus constructing a binary sparse mask that retains only the most important features. Finally, the attention scores at non-masked positions are replaced with negative infinity to sparsify the attention map, ensuring that subsequent softmax operations only apply to the selected key feature pairs. This masks unnecessary features with low-similarity attention weights, achieving adaptive selection of the top k similarity attention weights, retaining the most relevant parts of the reference and target images, and removing irrelevant or only slightly relevant parts. This dynamic selection causes attention to shift from dense to sparse, as shown in equation (2): (2) Where T k (.) represents the learnable top-k selection operator, as shown in equation (3): (3) Multi-scale feedforward module: As shown in the right figure of Figure 4, given the input tensor, after the normalization layer, the channel dimension is first expanded by 1×1 convolution, and then it is sent to two parallel branches (using 3×3 and 5×5 depth convolution respectively), and the extraction of multi-scale information is enhanced by feature transformation cross.

[0027] In the feature fusion stage, in real-world scenarios, although MRI images of the brain with different contrasts have similar anatomical structures, their color and brightness distributions are different. Therefore, the distribution of the extracted reference contrast image features is inconsistent with the distribution of the target contrast image features. Thus, simply connecting or stitching together the features of the reference image and the target image is not the optimal solution. As shown in Figure 5, this embodiment provides a method whereby, firstly, the target image features and the extracted reference image features are connected, and then input into a convolutional layer to obtain two parameters γ and β with the same size as the target image features. Then, instance normalization is applied to the reference image features, followed by affine transformation to update the features, as shown in the following equation (4): (4) Then the mapped F ref The features are concatenated with those of the target image and fused using Resblock. Finally, a high-resolution target image is generated based on the fused feature map.

[0028] By performing steps S10 to S50 above, preprocessing the raw brain MRI data of the target object allows for the selection of the core brain region and the generation of low-resolution target images that closely resemble real clinical scenarios. This provides a high-quality data foundation for subsequent image reconstruction and effectively eliminates interference from non-brain tissues. Extracting multi-scale features from the target and reference images simultaneously captures both global anatomical structures and local details, providing comprehensive support for feature fusion. Through multi-scale target feature maps and multi-scale reference feature maps, key features are selected and texture enhancement is applied to these features, allowing for the more accurate preservation of high-similarity core information between the target and reference images. It masks correlation redundancy and latent noise while enhancing texture details such as small blood vessels and lesion edges, thus improving feature quality. By aligning and fusing multi-scale texture feature maps with multi-scale target feature maps, it can solve the problem of feature distribution differences between images with different contrasts, achieving deep complementarity and efficient integration of multi-scale information and avoiding fusion conflicts. High-resolution target images can be generated through multi-scale fusion feature maps. High-resolution target images have the characteristics of clear anatomical structures and precise details, which can provide reliable imaging support for the accurate diagnosis of diseases such as brain tumors and hydrocephalus. At the same time, reference images can be used to shorten clinical scanning time, reduce equipment dependence and medical costs, and have both important technological innovation value and clinical application prospects.

[0029] In another exemplary embodiment of this application, step S10 is replaced by steps S101 to S106, namely: step S101, splitting the original data according to the slice dimension, extracting all axial, coronal and / or sagittal slices to obtain the original image; it should be noted that the original data involved in this application is a three-dimensional medical image, such as three-dimensional brain MRI volume data. The original data is cut into two-dimensional planar images (slices) according to the slice dimension. For example, the three-dimensional brain MRI is split into two-dimensional slices along the horizontal cutting direction of the vertical axis. Slicing the original data not only eliminates interference from non-brain tissues, but also provides standardized two-dimensional input data for subsequent preprocessing and feature extraction.

[0030] Step S102: Retain a preset number of original images, and perform spatial center cropping on each of the retained original images to remove non-brain regions from the edges of each slice, obtaining the original brain image; Step S103: Crop the original brain image according to the dataset type and determine the target image to be restored and the high-resolution parameter image; Step S104: Use Fourier transform to convert each target image to be restored from the spatial domain to the frequency domain, obtaining the frequency domain image; Step S105: Crop the corresponding target image from the center region of the frequency domain image according to the target scaling factor. The low-frequency region of the size is used to obtain a low-frequency image; in the example embodiment, the target scaling factor is ×2 or ×4. If the scaling factor is ×2, the cropping size is half of the original cropping size (e.g., 256×256 in the IXI dataset → 128×128, 240×240 in the BraTs dataset → 120×120); if the scaling factor is ×4, the cropping size is one-quarter of the original cropping size, for example, 256×256 in the IXI dataset → 64×64, 240×240 in the BraTs dataset → 60×60.

[0031] Step S106: Use inverse Fourier transform to convert the low-frequency image from the frequency domain to the spatial domain to obtain a low-resolution target image.

[0032] It should be noted that the low-frequency components of the original image correspond to the overall anatomical structure (such as the brain outline and large blood vessels), while the high-frequency components correspond to details (such as small lesions and tissue edges). The core problem of clinical low-resolution MRI is that "undersampling leads to the loss of high-frequency components," so frequency domain processing is closer to the real scene.

[0033] A two-dimensional Fourier transform is performed on the cropped original high-resolution target image to transform the original image from the spatial domain (pixel distribution) to the frequency domain (frequency component distribution). According to the target scaling factor (×2 or ×4), a low-frequency region of the corresponding size is cropped from the central region of the frequency domain image, discarding the high-frequency region and retaining the low-frequency component in the center of the frequency domain, simulating the resolution reduction caused by clinical undersampling. An inverse two-dimensional Fourier transform is performed on the cropped low-frequency frequency domain image to transform it back to the spatial domain, obtaining the final low-resolution target image. The blurring characteristics of the target image are consistent with those of low-quality clinical MRI (without additional artificial distortion).

[0034] In another exemplary embodiment of this application, before extracting multi-scale features from the target image and the reference image respectively to obtain a multi-scale target feature map and a multi-scale reference feature map, the method further includes: interpolating and upsampling the target image based on the spatial resolution of the reference image to obtain an interpolated target image; and converting both the interpolated target image and the parameter image into a single-channel tensor format to obtain a target image and a reference image of the same format.

[0035] It should be noted that interpolation upsampling and format conversion of the target image provide support for subsequent feature extraction, alignment, and fusion.

[0036] In another exemplary embodiment of this application, step S20 is replaced by steps S201 to S205, namely: Step S201, the target image and the reference image are convolved with a 1×1 to obtain feature maps of the target image and the reference image, and then subjected to initial feature extraction and nonlinear activation through a residual block to obtain a first multi-scale target feature map and a first multi-scale reference feature map; Step S202, the first multi-scale target feature map and the first multi-scale reference feature map are downsampled, and a second multi-scale target feature map and a second multi-scale reference image feature map are obtained through a residual block; Step S203, the second multi-scale target image feature map is downsampled... Step S204: Downsampling operation is performed on the target feature map and the second multi-scale reference feature map, and the third multi-scale target feature map and the third multi-scale reference feature map are obtained through residual blocks; Step S205: Downsampling operation and residual block feature extraction are performed on the third multi-scale target feature map and the third multi-scale reference feature map, until the last layer of the encoder is reached, to obtain the Nth multi-scale target feature map and the Nth multi-scale reference feature map; where N is a positive integer and is the same as the number of layers of the encoder; Step S206: The multi-scale target feature maps and multi-scale reference feature maps extracted from each layer of the encoder are integrated in order to obtain the multi-scale target feature map and the multi-scale reference feature map.

[0037] In another exemplary embodiment of this application, step S30 is replaced by steps S301 to S305, namely: Step S301, encoding the channel context of the multi-scale target feature map and the multi-scale reference feature map to generate target query feature Q and reference key feature K respectively; Step S301, calculating the similarity score of pixel pairs between target query feature Q and reference key feature K, and filtering out the top k highest similarity scores using the top-k algorithm to obtain the index values ​​of the top k highest similarity scores; Step S302, setting the mask value of the index position corresponding to each index value to 1 and keeping the non-index position 0 through a scatter operation to obtain a binary sparse mask; Step S303, based on the binary sparse mask, replacing the similarity score of the non-mask position with negative infinity to obtain the key feature of the current scale; Step S304, performing multi-scale texture enhancement on the key feature of the current scale to obtain the texture-enhanced feature of the current scale; Step S305, integrating the texture-enhanced features of the current scale from all scales to obtain a multi-scale texture feature map.

[0038] It should be noted that the sparse selective cross-attention mechanism, through a learnable top-k selection operator, dynamically filters and retains the top k high-similarity features between the target and reference images, shielding low-relevance redundant information and implicit noise. This fundamentally solves the problem of traditional global dense attention amplifying interference from irrelevant regions, significantly improving the accuracy of feature aggregation and making the core anatomical structures of the reconstructed image clearer and with lower noise. The feature adaptation module innovatively achieves adaptive alignment of feature distributions in MRI images of different contrasts by generating parameters γ and β to perform affine transformations on the features of the reference image. This effectively solves the feature fusion conflict problem caused by brightness and color differences in different weighted images such as T1WI and T2WI, making feature interaction smoother and the fusion effect more natural.

[0039] In another exemplary embodiment of this application, step S40 is replaced by steps S401 to S404, namely: step S401, mapping the distribution of the multi-scale texture feature map to a distribution consistent with the multi-scale target feature map, to obtain an aligned multi-scale texture feature map and an aligned multi-scale target feature map; step S402, stitching the aligned multi-scale texture feature map and the aligned multi-scale target feature map along the channel dimension to obtain a stitched feature map; step S403, fusing the stitched feature map using a sparse selection cross-attention transformer to obtain a fused feature map for each scale; step S404, fusing the fused feature map and the target feature map of different scales through the upsampling and feature adaptation module of the decoder to obtain a multi-scale fused feature map.

[0040] It should be noted that the Multi-Scale Feedforward Network (MSFM) adopts a 3×3 and 5×5 parallel deep convolution design, which takes into account both the capture of microscopic details such as small blood vessels and lesion edges in the brain and macroscopic structures such as the overall outline of the brain and tissue partitions. Then, feature cross-linking is used to strengthen the association of multi-scale information, so that the reconstructed image retains both accurate local details and maintains the complete global structure, fully meeting the clinical diagnostic needs for anatomical information at different scales.

[0041] In other embodiments, a validation process for the sparse selective cross-attention Transformer network is also provided, as follows: Datasets: Two datasets are used to evaluate the performance of the sparse selective cross-attention Transformer network involved in the above embodiments, as shown in Table 1. These are two announced datasets: the IXI dataset (Figure 6) and the BraTs dataset (Figure 7). The IXI dataset includes registered T2-weighted images and PD-weighted MR images containing 578 patients. PD-weighted images are used as reference images to guide the recovery of T2-weighted images. The BraTs dataset contains 258 pairs of multimodal brain images, including aligned T1-weighted images, T1ce-weighted images, T2-weighted images, and PD-weighted images. T1-weighted images are used as reference images to guide the recovery of T2-weighted images.

[0042] It should be noted that the image shown in Figure 6 is the qualitative result of the IXI dataset at 4x magnification. The first row shows the super-resolution result, and the second row presents the error plot. The first column, from bottom to top, shows the reference image (bottom) and the ground truth (top). In the PSNR and SSIM metrics at the bottom, red indicates the best results. The image shown in Figure 7 is the qualitative result of the BraTs dataset at 4x magnification; the first row shows the super-resolution result, and the second row presents the error plot. The first column, from bottom to top, shows the reference image (bottom) and the ground truth (top). In the PSNR and SSIM metrics at the bottom, red indicates the best results.

[0043] Training details: If the initial channel C is 64, the expansion ratio is set to 2; regarding sparse selection across the attention transformer, the sparsity in the sparse selection across the attention transformer... Set as .

[0044] During training, a batch size of 24 was used, employing the AdamW optimizer for a total of 100 epochs, with an initial learning rate fixed at e⁻⁴. The entire framework was executed on PyTorch using four NVIDIA GeForce RTX 3090 GPUs, operating in an end-to-end learning manner.

[0045] Training details for the comparison methods: To be fair, all comparison methods were also trained for 100 epochs on an NVIDIA GeForce RTX 3090 GPU with a batch size of 24, using the AdamW optimizer and a learning rate of e-4.

[0046] Results Evaluation: On the IXI and Brats datasets, the performance of the Sparse Selective Cross-Attention Transformer network was compared with state-of-the-art multi-contrast MRI super-resolution methods (MCHA, MGDUN, WavTans, DCAMSR, ECFNet, SANet, A2-CDic, DCANSR). The comparison results are shown in Tables 2 and 3. The Sparse Selective Cross-Attention Transformer network achieved the best results in both PSNR and SSIM on the IXI dataset (×2PSNR / SSIM: 38.3998 dB / 0.9828; ×4PSNR / SSIM: 37.1465 dB / 0.9728) and the BraTs dataset (×2PSNR / SSIM: 36.7013 dB / 0.9881; ×4PSNR / SSIM: 33.9066 dB / 0.9695).

[0047] Ablation Study: (1) Effect of top-k selection: To examine the effect of top-k selection on sparse cross-attention modules, the training results of sparse cross-attention modules without top-k are shown in Figure 8. Here, w / top-k represents the method using top-k selection, and w / o top-k represents the method without top-k selection. It can be seen that the peak signal-to-noise ratio (PSNR) of the target image obtained using the top-k selection method is higher than that obtained without the top-k selection method.

[0048] (2) The impact of the number of k: The key parameter of the proposed sparse cross-attention module is k, and its impact is studied in Figure 9. Note that the optimal choice of k determines the boundary control of the sparsity rate. If k is manually set to a single value, such as 1 / 2, it is noted that its performance is very sensitive to k. To avoid exhaustive search, a controllable range is set for k to dynamically learn the score that contributes the most. When k is too small, it is found that its performance will undoubtedly drop sharply due to the loss of details and textures and insufficient extraction of global information caused by excessive sparsity.

[0049] When sparse cross-attention module for The optimal result is achieved when k is constant. As k increases, the final performance gradually declines due to the introduction of irrelevant and useless features.

[0050] (3) Effectiveness of the multi-scale feedforward module and the feature adaptation module: To evaluate the effectiveness of the multi-scale feedforward module and the feature adaptation module, two variants of the Sparse Selective Cross-Attention Transformer network were proposed for experiments, as shown in Table 4. In the latter, w / o MCFM, with FNN replaces the proposed multi-scale feedforward module in the Sparse Selective Cross-Attention Transformer network with a regular FNN; w / o TAM removes the feature adaptation module in the Sparse Selective Cross-Attention Transformer network and replaces it with a concat operation. The results in the table show that the Sparse Selective Cross-Attention Transformer network performs better in PSNR and SSIM than the two variants (w / o MCFM, with FNN, and w / o TAM). This indicates that each design strategy considered contributes to the final performance of the Sparse Selective Cross-Attention Transformer network.

[0051] Table 1 Table 2 Table 3 Table 4 It should be noted that, after thorough validation using the IXI and BraTs datasets, the sparse selective cross-attention Transformer network provided in this application comprehensively surpasses existing methods in terms of peak signal-to-noise ratio and structural similarity index (SSIM), a core evaluation metric. It can stably output high-resolution, high-confidence MRI images, providing reliable imaging support for the early screening and precise localization of brain diseases such as brain tumors and hydrocephalus. Furthermore, this sparse selective cross-attention Transformer network can shorten the target image scanning time by using reference images, reducing patient discomfort during examination, decreasing reliance on equipment for high-resolution scanning, and indirectly controlling medical costs. It possesses significant clinical practical value and broad application prospects.

[0052] Based on the same inventive concept, this application also provides a medical image super-resolution reconstruction apparatus for implementing the aforementioned medical image super-resolution reconstruction method. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the medical image super-resolution reconstruction apparatus provided below can be found in the limitations of the medical image super-resolution reconstruction method described above, and will not be repeated here.

[0053] In an exemplary embodiment, as shown in FIG10, a medical image super-resolution reconstruction device is provided. The medical image super-resolution reconstruction device 100 includes: a preprocessing module 1001, an extraction module 1002, a filtering module 1003, an alignment and fusion module 1004, and a generation module 1005. The preprocessing module 1001 is used to preprocess the raw data of the target object's brain MRI to obtain a high-resolution reference image and a low-resolution target image. The multi-scale reference image and the target image are images of different contrasts and resolutions of the target object's brain core region. The extraction module 1002 is used to extract multi-scale features from the target image and the reference image respectively to obtain a multi-scale target feature map and a multi-scale reference feature map. The filtering module 1003 is used to filter out key features based on the multi-scale target feature map and the multi-scale reference feature map and perform texture enhancement on the key features to obtain a multi-scale texture feature map. The alignment and fusion module 1004 is used to align and fuse the multi-scale texture feature map and the multi-scale target feature map to obtain a multi-scale fused feature map. The generation module 1005 is used to generate a high-resolution target image based on the multi-scale fused feature map.

[0054] As an optional implementation, the preprocessing module 1001 is specifically used to: split the original data according to the slice dimension, extract all axial, coronal and / or sagittal slices to obtain the original image; retain a preset number of original images, and perform spatial center cropping on each of the retained original images to remove non-brain regions at the edges of each slice to obtain the original brain image; crop the original brain image according to the dataset type and determine the target image to be restored and the high-resolution parametric image; use Fourier transform to convert each target image to be restored from the spatial domain to the frequency domain to obtain the frequency domain image; according to the target scaling factor, crop the low-frequency region of the corresponding size in the central region of the frequency domain image to obtain the low-frequency image; use inverse Fourier transform to convert the low-frequency image from the frequency domain to the spatial domain to obtain the low-resolution target image.

[0055] As an optional implementation, the above-mentioned medical image super-resolution reconstruction device 100 further includes an interpolation conversion module, which is used to perform interpolation upsampling on the target image based on the spatial resolution of the reference image to obtain the interpolated target image; and convert both the interpolated target image and the parameter image into a single-channel tensor format to obtain a target image and a reference image of the same format.

[0056] As an optional implementation, the extraction module 1002 is used to: obtain feature maps of the target image and the reference image through 1×1 convolution; then perform initial feature extraction and nonlinear activation through a residual block to obtain a first multi-scale target feature map and a first multi-scale reference feature map; perform downsampling operation on the first multi-scale target feature map and the first multi-scale reference feature map, and obtain a second multi-scale target feature map and a second multi-scale reference image feature map through the residual block; perform downsampling operation on the second multi-scale target feature map and the second multi-scale reference feature map, and obtain a third multi-scale target feature map and a third multi-scale reference feature map through the residual block; perform downsampling operation and residual block feature extraction on the third multi-scale target feature map and the third multi-scale reference feature map, until the last layer of the encoder is reached, to obtain the Nth multi-scale target feature map and the Nth multi-scale reference feature map; where N is a positive integer and is the same as the number of layers of the encoder; and integrate the multi-scale target feature maps and multi-scale reference feature maps extracted from each layer of the encoder in sequence to obtain a multi-scale target feature map and a multi-scale reference feature map.

[0057] As an optional implementation, the filtering module 1003 is specifically used to: encode the channel context of the multi-scale target feature map and the multi-scale reference feature map to generate target query feature Q and reference key feature K respectively; calculate the similarity score of pixel pairs between target query feature Q and reference key feature K, and filter out the top k highest similarity scores using the top-k algorithm to obtain the index values ​​of the top k highest similarity scores; set the mask value of the index position corresponding to each index value to 1 and keep the non-index position 0 through a scatter operation to obtain a binary sparse mask; based on the binary sparse mask, replace the similarity score of the non-mask position with negative infinity to obtain the key feature of the current scale; perform multi-scale texture enhancement on the key feature of the current scale to obtain the texture-enhanced feature of the current scale; integrate the texture-enhanced features of the current scale from all scales to obtain a multi-scale texture feature map.

[0058] As an optional implementation, the alignment and fusion module 1004 is specifically used to: map the distribution of the multi-scale texture feature map to a distribution consistent with that of the multi-scale target feature map, to obtain an aligned multi-scale texture feature map and an aligned multi-scale target feature map; concatenate the aligned multi-scale texture feature map and the aligned multi-scale target feature map along the channel dimension to obtain a concatenated feature map; fuse the concatenated feature map using a sparse selection cross-attention transformer to obtain a fused feature map for each scale; and fuse the fused feature maps and target feature maps of different scales through the upsampling and feature adaptation module of the decoder to obtain a multi-scale fused feature map.

[0059] This implementation method, by preprocessing the raw brain MRI data of the target object, can screen out the core brain regions and generate low-resolution target images that closely resemble real clinical scenarios, providing a high-quality data foundation for subsequent image reconstruction and effectively eliminating interference from non-brain tissues. By extracting multi-scale features from the target and reference images, it can simultaneously capture the global anatomical structure and local detail information of both images, providing comprehensive support for feature fusion. Through multi-scale target feature maps and multi-scale reference feature maps, key features are screened out and texture enhancement is applied to these key features, which can more accurately preserve the high-similarity core information between the target and reference images while masking low-similarity features. This method eliminates redundancy and latent noise while enhancing texture details such as fine blood vessels and lesion edges, thus improving feature quality. By aligning and fusing multi-scale texture feature maps with multi-scale target feature maps, it can solve the problem of feature distribution differences between images with different contrasts, achieving deep complementarity and efficient integration of multi-scale information and avoiding fusion conflicts. High-resolution target images can be generated through multi-scale fusion feature maps. High-resolution target images are characterized by clear anatomical structures and precise details, which can provide reliable imaging support for the accurate diagnosis of diseases such as brain tumors and hydrocephalus. At the same time, reference images can be used to shorten clinical scanning time, reduce equipment dependence and medical costs, and have both important technological innovation value and clinical application prospects.

[0060] In an exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram is shown in Figure 11. The computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is connected to the system bus via the I / O interfaces. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device stores medical image super-resolution reconstruction data. The input / output interfaces of the computer device are used for exchanging information between the processor and external devices. The communication interface of the computer device is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a medical image super-resolution reconstruction method.

[0061] Those skilled in the art will understand that the structure shown in Figure 11 is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0062] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0063] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0064] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0065] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0066] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0067] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0068] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0069] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for super-resolution reconstruction of medical images, characterized in that, The medical image super-resolution reconstruction method includes: preprocessing the raw data of the target object's brain MRI to obtain a high-resolution reference image and a low-resolution target image; the reference image and the target image are images of different contrasts and resolutions of the target object's core brain region; extracting multi-scale features from the target image and the reference image respectively to obtain a multi-scale target feature map and a multi-scale reference feature map; based on the multi-scale target feature map and the multi-scale reference feature map, selecting key features and performing texture enhancement on the key features to obtain a multi-scale texture feature map; aligning and fusing the multi-scale texture feature map with the multi-scale target feature map to obtain a multi-scale fused feature map; and generating a high-resolution target image based on the multi-scale fused feature map.

2. The medical image super-resolution reconstruction method according to claim 1, characterized in that, The preprocessing of the raw brain MRI data of the target object to obtain a low-resolution target image includes: splitting the raw data according to the slice dimension, extracting all axial, coronal and / or sagittal slices to obtain raw images; retaining a preset number of raw images, and performing spatial center cropping on each of the retained raw images to remove non-brain regions at the edges of each slice to obtain raw brain images; cropping the raw brain images according to the dataset type and determining the target image to be restored and a high-resolution parametric image; converting each target image to be restored from the spatial domain to the frequency domain using Fourier transform to obtain a frequency domain image; cropping a low-frequency region of corresponding size from the central region of the frequency domain image according to the target scaling factor to obtain a low-frequency image; and converting the low-frequency image from the frequency domain to the spatial domain using inverse Fourier transform to obtain the low-resolution target image.

3. The medical image super-resolution reconstruction method according to claim 1, characterized in that, Before extracting multi-scale features from the target image and the reference image respectively to obtain a multi-scale target feature map and a multi-scale reference feature map, the method further includes: interpolating and upsampling the target image based on the spatial resolution of the reference image to obtain an interpolated target image; and converting both the interpolated target image and the parameter image into a single-channel tensor format to obtain a target image and a reference image of the same format.

4. The medical image super-resolution reconstruction method according to claim 3, characterized in that, The step of extracting multi-scale features from the target image and the reference image to obtain multi-scale target feature maps and multi-scale reference feature maps includes: performing a 1×1 convolution on the target image and the reference image to obtain feature maps of the target image and the reference image, followed by initial feature extraction and nonlinear activation through a residual block to obtain a first multi-scale target feature map and a first multi-scale reference feature map; performing a downsampling operation on the first multi-scale target feature map and the first multi-scale reference feature map, and obtaining a second multi-scale target feature map and a second multi-scale reference image feature map through a residual block; and performing a downsampling operation on the second multi-scale target feature map and the first multi-scale reference image feature map. The feature map and the second multi-scale reference feature map are downsampled, and a third multi-scale target feature map and a third multi-scale reference feature map are obtained through residual blocks. The third multi-scale target feature map and the third multi-scale reference feature map are downsampled and residual block feature extraction is performed on the third multi-scale target feature map and the third multi-scale reference feature map until the last layer of the encoder is reached, to obtain the Nth multi-scale target feature map and the Nth multi-scale reference feature map; where N is a positive integer and is the same as the number of layers of the encoder. The multi-scale target feature maps and multi-scale reference feature maps extracted from each layer of the encoder are integrated in order to obtain the multi-scale target feature map and the multi-scale reference feature map.

5. The medical image super-resolution reconstruction method according to claim 1, characterized in that, The process of selecting key features and performing texture enhancement based on the multi-scale target feature map and the multi-scale reference feature map to obtain a multi-scale texture feature map includes: encoding the channel context of the multi-scale target feature map and the multi-scale reference feature map to generate a target query feature Q and a reference key feature K, respectively; calculating the similarity score of pixel pairs between the target query feature Q and the reference key feature K, and selecting the top k highest similarity scores using a top-k algorithm to obtain the index values ​​of the top k highest similarity scores; setting the mask value of the index position corresponding to each index value to 1 and keeping the non-index positions 0 through a scatter operation to obtain a binary sparse mask; replacing the similarity score of the non-mask positions with negative infinity based on the binary sparse mask to obtain the key features of the current scale; performing multi-scale texture enhancement on the key features of the current scale to obtain the texture-enhanced features of the current scale; and integrating the texture-enhanced features of the current scale from all scales to obtain the multi-scale texture feature map.

6. The medical image super-resolution reconstruction method according to claim 1, characterized in that, The step of aligning and fusing the multi-scale texture feature map with the multi-scale target feature map to obtain a multi-scale fused feature map includes: mapping the distribution of the multi-scale texture feature map to a distribution consistent with that of the multi-scale target feature map, obtaining an aligned multi-scale texture feature map and an aligned multi-scale target feature map; concatenating the aligned multi-scale texture feature map and the aligned multi-scale target feature map along the channel dimension to obtain a concatenated feature map; fusing the concatenated feature map using a sparse selection cross-attention transformer to obtain a fused feature map for each scale; and fusing the fused feature maps and target feature maps of different scales through an upsampling and feature adaptation module of the decoder to obtain the multi-scale fused feature map.

7. A medical image super-resolution reconstruction device, characterized in that, The medical image super-resolution reconstruction device includes: a preprocessing module for preprocessing the raw data of the target object's brain MRI to obtain a high-resolution reference image and a low-resolution target image; the reference image and the target image are images of different contrasts and resolutions of the target object's core brain region; an extraction module for extracting multi-scale features from the target image and the reference image respectively to obtain a multi-scale target feature map and a multi-scale reference feature map; a filtering module for filtering key features based on the multi-scale target feature map and the multi-scale reference feature map and performing texture enhancement on the key features to obtain a multi-scale texture feature map; an alignment and fusion module for aligning and fusing the multi-scale texture feature map and the multi-scale target feature map to obtain a multi-scale fused feature map; and a generation module for generating a high-resolution target image based on the multi-scale fused feature map.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the steps of the medical image super-resolution reconstruction method according to any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the medical image super-resolution reconstruction method according to any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the medical image super-resolution reconstruction method according to any one of claims 1-6.

Images