A medical image super-resolution reconstruction method and system

Abstract

This invention relates to the field of image processing technology and discloses a method and system for super-resolution reconstruction of medical images. The method includes: constructing a multimodal medical image dataset comprising high-resolution images and corresponding low-resolution image pairs; decoupling and modeling organ structure and noise features using frequency domain analysis and Gaussian mixture models to establish a multimodal organ template library; performing organ identification and frequency domain feature matching on the input image, and retrieving the optimal noise prior from the multimodal organ template library; and injecting a noise prior guiding term during the reverse denoising process of the diffusion model to achieve anatomically adaptive super-resolution image reconstruction. This invention, through the deep integration of multimodal data-driven approaches, frequency domain statistical modeling, and diffusion models, achieves end-to-end optimization of medical image super-resolution reconstruction from noise decoupling to structurally adaptive super-resolution image reconstruction, improving the accuracy, efficiency, and robustness of medical image super-resolution reconstruction, and is applicable to complex clinical scenarios.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to a method and system for super-resolution reconstruction of medical images. Background Technology

[0002] Medical imaging, such as computed tomography (CT) and magnetic resonance imaging (MRI), plays a crucial role in modern clinical diagnosis as non-invasive diagnostic tools. High-resolution medical images are of great significance for early disease detection and accurate diagnosis. However, acquiring high-resolution medical images faces numerous challenges due to limitations in imaging equipment, radiation dose control, cost factors, and patient movement during image acquisition. Medical image super-resolution reconstruction technology, as an important solution, is facing a fundamental contradiction between accurate reconstruction and computational efficiency.

[0003] In recent years, deep learning technology has demonstrated outstanding performance in image super-resolution (SR) tasks, especially in the field of natural graphics super-resolution. However, existing technologies rarely make specific adjustments for the characteristics of medical images, and existing methods are insufficient to meet the requirements of medical ultrasound images for reconstruction accuracy and efficiency. Summary of the Invention

[0004] In view of this, the present invention provides a method and system for super-resolution reconstruction of medical images to solve the shortcomings of low accuracy and efficiency in the existing technology for super-resolution reconstruction of medical images.

[0005] In a first aspect, the present invention provides a method for super-resolution reconstruction of medical images, comprising:

[0006] Construct a multimodal medical image dataset that includes high-resolution images and corresponding low-resolution image pairs;

[0007] Frequency domain analysis and Gaussian mixture model were used to decouple organ structure and noise characteristics, and a multimodal organ template library was established.

[0008] The input image is subjected to organ identification and frequency domain feature matching, and the optimal noise prior is retrieved from the multimodal organ template library;

[0009] By injecting a noise prior guiding term into the inverse denoising process of the diffusion model, adaptive super-resolution image reconstruction of anatomical structures can be achieved.

[0010] The medical image super-resolution reconstruction method provided in this invention achieves full-process optimization from noise decoupling to adaptive structural reconstruction through the deep integration of multimodal data-driven, frequency domain-statistical modeling and diffusion model. Its core advantage lies in combining prior knowledge of anatomical structure noise with data-driven methods, which not only improves the accuracy, efficiency and robustness of medical image super-resolution, but also provides technical support for clinical low-dose imaging, early diagnosis and cross-modal analysis, and has significant scientific value and clinical translation potential.

[0011] In one optional implementation, the construction of a multimodal medical image dataset comprising high-resolution images and corresponding low-resolution image pairs includes:

[0012] By adjusting the equipment parameters of medical imaging equipment, real high- and low-resolution image pairs of multiple modalities can be obtained.

[0013] When real-world data is insufficient, simulated data is generated using a physics-driven degradation model. This model generates low-resolution images by applying point spread function blurring, noise addition, and downsampling operations to high-resolution images. The data generated in this way can cover noise patterns rarely seen in real-world scenes, expanding the diversity of training data and improving the model's robustness under extreme imaging conditions.

[0014] This invention obtains real high- and low-resolution image pairs by directly adjusting the parameters of medical imaging equipment, ensuring that the data strictly conforms to the physical characteristics of clinical imaging. When real data is insufficient, the data generated by the physical-driven degradation model can cover noise patterns rarely seen in real scenes, expand the diversity of training data, and improve the robustness of the model under extreme imaging conditions.

[0015] In one optional implementation, the step of decoupling and modeling organ structure and noise characteristics using frequency domain analysis and Gaussian mixture models to establish a multimodal organ template library includes:

[0016] By modeling the dependency between organ structure and noise using a joint probability model, the decoupling of structure and noise can be achieved.

[0017] A pre-trained multimodal organ segmentation network is used to identify organs and generate organ masks from input images, and Fourier transform is performed on each image and its organ mask to extract the image spectrum.

[0018] The extracted image spectrum is divided into multiple frequency bands according to the frequency range, and frequency domain noise features are extracted.

[0019] The frequency domain noise characteristics are statistically analyzed using mean vector, covariance matrix, and power spectrum histogram, respectively, to obtain three types of statistics. These three statistics together constitute a frequency band-level noise prior template unit.

[0020] Based on the noise prior template unit, a conditional Gaussian mixture model is used to generate sub-noise prior templates for different tissue regions inside the organ.

[0021] This invention utilizes a joint probability model to separate organ structure from noise distribution, and combines it with frequency domain multi-scale analysis to achieve layered protection from the overall organ outline to fine textures, thereby improving the clarity of structural reconstruction and increasing the noise suppression rate. It uses mean vectors, covariance matrices, and power spectrum histograms to construct frequency band-level template units, combined with a conditional Gaussian mixture model, to capture the noise heterogeneity within organs, reducing tissue boundary noise artifacts and improving contrast. It uniformly represents the frequency domain noise characteristics of multiple modalities, supporting cross-modal prior transfer. Furthermore, it automatically constructs templates using a pre-trained segmentation network, adapting to the needs of rapid clinical deployment.

[0022] In one optional implementation, the step of performing organ identification and frequency domain feature matching on the input image, and retrieving the optimal noise prior from the multimodal organ template library, includes:

[0023] The pre-trained organ segmentation network performs organ recognition on the input image patch and outputs the corresponding organ mask map for organ region localization.

[0024] The image patch is transformed from the spatial domain to the frequency domain and divided into multiple frequency bands. The optimal noise prior is obtained by calculating the similarity between the feature vector of the current image patch in each frequency band and the low-resolution frequency domain features of the corresponding organ in the multimodal organ template library.

[0025] This invention employs a pre-trained organ segmentation network to accurately locate organ regions and output masks. After frequency domain transformation and multi-band segmentation, the similarity between the frequency domain features of image patches and the corresponding organ features in the template library is calculated. The optimal noise prior is then retrieved. This process achieves region-specific noise modeling, avoids global assumption bias, and enhances the ability to handle small organs and complex structures. Simultaneously, it integrates the advantages of frequency domain decoupling and cross-modal feature generalization to improve reconstruction accuracy and robustness, optimize computational efficiency to meet real-time clinical needs, and adaptively repair low-quality images, providing strong support for medical image super-resolution reconstruction.

[0026] In one optional implementation, the step of injecting a noise prior guiding term during the inverse denoising process of the diffusion model to achieve anatomical structure-adaptive super-resolution image reconstruction includes:

[0027] Using the basic structure of the standard diffusion model, a noise prior guiding term is injected into the reverse denoising process of the diffusion model. The noise prior guiding term estimates its noise components based on the current intermediate state and performs Mahalanobis distance calculation with the noise template corresponding to the organ to generate a frequency-domain driven spatial guiding signal for super-resolution image reconstruction.

[0028] In this embodiment of the invention, a noise prior term based on Mahalanobis distance is injected during the reverse denoising of the diffusion model. By measuring the difference between the current noise and the organ noise template, a spatial guidance signal is generated. This can accurately match the noise characteristics of different organs for denoising, adaptively preserve anatomical structural details, effectively suppress complex noise, enhance the model's generalization ability to multimodal images, accelerate model convergence, improve the efficiency and quality of super-resolution image reconstruction, and provide clearer and more accurate images for medical diagnosis.

[0029] In one alternative implementation, the space guidance signal is expressed by the following formula:

[0030]

[0031] Among them, z t For the intermediate state at time step t, ∈ θ For noise prediction networks, G N (z t (k,t) is a noise prior guide term based on organ category k, used to dynamically suppress unstructured noise components that match the noise pattern of that organ, β t The a priori strength coefficient follows an annealing curve as t decreases, α t This is the decay factor for time step t, used to control the gradual addition or removal of noise. T represents the total number of generation steps;

[0032] The noise prediction network adopts the U-Net model, which includes a preset number of downsampling encoder layers, a preset number of upsampling decoder layers, a temporal embedding layer, and an attention layer. The model training uses a weighted loss function:

[0033]

[0034] Where w(t) is the time-dependent weighting function; L perceptual For perceptual loss, the distance is based on the features of the pre-trained U-Net network, λ is the perceptual loss weight, z0 is the original image, and z pred The predicted restored image is denoted as ∈, where ∈ represents added noise.

[0035] This invention employs a spatially guided signal formula based on noise priors, combined with prior intensity coefficients derived from annealing variations, to dynamically and accurately suppress unstructured noise matching organ noise patterns, avoiding structural loss due to excessive denoising. A noise prediction network using the U-Net architecture, with its downsampling encoder, upsampling decoder, temporal embedding layer, and attention layer, effectively extracts and fuses multi-scale image features, enhancing its ability to model complex anatomical structures. A weighted loss function, by assigning weights to early time steps, accelerates the model's learning efficiency in critical denoising stages. Combined with perceptual loss constraining high-level semantic consistency, the reconstructed image not only more closely resembles the real image at the pixel level but also preserves important anatomical structures and texture features in medical images, ultimately significantly improving the quality, reconstruction efficiency, and clinical applicability of super-resolution images.

[0036] In one optional implementation, the noise prior guiding term is:

[0037]

[0038] Among them, F, F -1 These are the Fourier transform and its inverse transform, μ kb ,∑ kb These are the mean vector and covariance matrix of organ k in frequency band b, stored in the organ template library, respectively. b (t) is the frequency band weighting function.

[0039] This invention utilizes Fourier transform to convert images to the frequency domain, accurately separating the frequency domain features of anatomical structures and noise. By using the mean vector and covariance matrix of the corresponding frequency bands of organs in the template library, the difference between the current image noise and the prior noise pattern can be quantified, and unstructured noise can be suppressed in a targeted manner. The frequency band weighting function can dynamically allocate weights according to the importance of different frequency bands to organ structures, focusing on protecting key features such as low-frequency contours and high-frequency textures, avoiding excessive smoothing or loss of details during denoising. The inverse Fourier transform maps the frequency domain guiding signal back to the spatial domain, achieving spatially specific suppression of noise. This allows the reconstructed image to retain the true anatomical structure while effectively removing interference components that match the organ noise pattern, improving the signal-to-noise ratio and structural fidelity of the super-resolution image.

[0040] In a second aspect, the present invention provides a medical image super-resolution reconstruction system, the system comprising:

[0041] The image dataset construction module is used to construct a multimodal medical image dataset that includes high-resolution images and corresponding low-resolution image pairs.

[0042] The organ template library construction module is used to decouple and model organ structure and noise characteristics using frequency domain analysis and Gaussian mixture model to establish a multimodal organ template library;

[0043] An optimal noise prior matching module is used to perform organ identification and frequency domain feature matching on the input image and to retrieve the optimal noise prior from the multimodal organ template library.

[0044] The image reconstruction module is used to inject noise prior guiding terms into the inverse denoising process of the diffusion model to achieve adaptive super-resolution image reconstruction of anatomical structures.

[0045] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the medical image super-resolution reconstruction method of the first aspect or any corresponding embodiment described above.

[0046] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the medical image super-resolution reconstruction method of the first aspect or any corresponding embodiment described above.

[0047] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute the medical image super-resolution reconstruction method of the first aspect or any corresponding embodiment described above. Attached Figure Description

[0048] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0049] Figure 1 This is a schematic flowchart of a medical image super-resolution reconstruction method according to an embodiment of the present invention;

[0050] Figure 2 This is a structural block diagram of a medical image super-resolution reconstruction system according to an embodiment of the present invention;

[0051] Figure 3 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] Existing medical image super-resolution reconstruction methods generally suffer from the following problems:

[0054] 1. Ignoring the unique anatomical characteristics of medical images can lead to reconstruction results that, while possessing high-resolution statistical features, may violate anatomical rationality. Existing deep learning-based medical image reconstruction methods often perform super-resolution reconstruction on one or more medical images, primarily by collecting and analyzing the overall noise features of the image, neglecting the inconsistencies in noise distribution caused by the characteristics of the patient's anatomical structure. Traditional medical image super-resolution methods select specific locations in the image as sampling points for feature statistics, failing to utilize anatomical priors and easily introducing biases.

[0055] 2. The interaction is rigid and limited, supporting only basic operations such as rectangular selection, which differs significantly from the objective reality that medical image noise changes based on anatomy. Specifically, taking CT as an example, existing ultrasound methods often perform super-resolution processing based on the region of interest (ROI, which is mostly a rectangular window) selected by the operator. More complex methods may utilize the statistical analysis of noise features within a cube formed by multiple preceding and following CT layers to optimize the ROI region of a particular layer, ignoring the operator's selection of targets across ROIs based on anatomical understanding and the statistical analysis of noise characteristics within organs. Furthermore, due to the lack of anatomical decomposition, hierarchical super-resolution reconstruction is impossible, naturally lacking multi-target collaborative enhancement and priority control mechanisms, easily leading to visual confusion between critical lesion areas and non-areas of interest.

[0056] 3. The computational complexity is not matched with diagnostic needs, especially in portable devices such as ultrasound and mobile DR, where real-time processing is difficult to achieve. This mainly refers to super-resolution methods based on deep learning, which emphasize super-resolution effects but give little consideration to the need for cross-device super-resolution reconstruction and do not take into account the time requirements for super-resolution reconstruction in clinical practice, resulting in clinicians having to wait a long time when they need high-quality reconstructed images.

[0057] Therefore, this embodiment provides a medical image super-resolution reconstruction method, which achieves efficient and accurate medical image super-resolution reconstruction within a diffusion model framework by injecting anatomical prior knowledge, thus optimizing the clinical environment with limited computing resources. Figure 1This is a flowchart of a medical image super-resolution reconstruction method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0058] Step S101: Construct a multimodal medical image dataset including high-resolution images and corresponding low-resolution image pairs.

[0059] Specifically, in medical image super-resolution reconstruction tasks, high-quality datasets are fundamental for model training. To construct datasets suitable for super-resolution tasks, this invention, after authorization, collected real medical images of various modalities (such as CT, MRI, and ultrasound) from multiple hospitals or imaging devices. These data include high-resolution (HR) and low-resolution (LR) images, and sub-datasets are constructed according to different modalities. The construction of high-resolution and low-resolution image pairs can cover the anatomical features of different imaging modalities, forming complementary cross-modal information, providing more comprehensive anatomical priors for subsequent reconstruction, and is particularly suitable for multi-dimensional analysis of complex organs (such as the heart and brain).

[0060] Real-world data refers to high-resolution and low-resolution image pairs acquired through actual imaging equipment. For example, in CT imaging, high-resolution and low-resolution image pairs can be generated by adjusting equipment parameters. To generate these pairs, embodiments of the present invention employ a dual-parameter imaging model: high-resolution images are obtained by setting equipment parameters to achieve the clearest possible image. For example, in CT imaging, a smaller slice thickness (e.g., ≤2.5 mm), a moderate tube current, and a tube voltage of 100-120 kVp are selected. Low-resolution images are generated by adjusting equipment parameters or by degrading the high-resolution image. For example, in CT imaging, a larger slice thickness (e.g., ≥5 mm), a lower tube current, and a higher tube voltage are selected.

[0061] In constructing multimodal medical image datasets, when real data is insufficient, this embodiment of the invention generates simulated data through a physically driven degradation model, which can supplement the lack of real data while ensuring the diversity of training data. Specifically, point spread function (PSF) blurring, noise addition, and downsampling operations can be applied to high-resolution images through software simulation to further simulate realistic low-resolution images.

[0062]

[0063] Among them, I k The original high-resolution (HR) image; k is the index, representing the k-th HR image; For convolution operations, F -1 and F-1 These represent the Fourier transform and the inverse Fourier transform, respectively; PSF m The point spread function for imaging modality m (e.g., m = C for CT images, m = MRI for MRI images); ↓ s For downsampling operation; η m (·) represents the noise distribution simulation under different modes.

[0064] Point spread function (PSF) m An anisotropic 3D Gaussian kernel can be used as follows:

[0065]

[0066] Where σ is the standard deviation of the Gaussian kernel, θ(z) is the feature convolution function representing the hardware characteristics, and (x,y,z) are the three-dimensional spatial coordinates.

[0067] Therefore, the multimodal medical image dataset is represented as follows:

[0068]

[0069] Among them, D real For a real dataset, D degraded For simulation datasets, This is a data merging operation.

[0070] Step S102: Frequency domain analysis and Gaussian mixture model are used to decouple and model the organ structure and noise characteristics to establish a multimodal organ template library.

[0071] Specifically, the goal of constructing a multimodal organ template library in this invention is to characterize the typical noise distribution characteristics of different organs under different medical imaging modalities, and to achieve accurate description and reuse of noise in organ regions through decoupled modeling of structure and noise. This invention decomposes images into low-frequency structural information (organ contours, major anatomical structures) and high-frequency detail information (texture, edges) using Fourier transform, and combines this with a Gaussian mixture model to parametrically model the noise distribution in different frequency bands (e.g., mean, covariance), thus separating the frequency domain features of physiological structures and imaging noise. For example, the low-frequency contour of the liver and quantum noise in CT scans have different energy distributions in the frequency domain. The model can accurately capture this difference, and the mixture model can fit multi-peak noise distributions (e.g., noise mixing under different CT doses), more closely reflecting actual clinical imaging scenarios (e.g., low-dose scanning, rapid imaging sequences), and improving the representation ability of noise priors. The specific construction process of the multimodal organ template library is as follows:

[0072] A1. By modeling the dependency relationship between organ structure and noise through a joint probability model, the structure and noise can be decoupled.

[0073] Specifically, to separate organ structures and noise components from medical images, this embodiment of the invention first introduces a joint probability model p(S, N|M) to express the statistical relationship between organ structures and their corresponding noise. Here, S represents the structural information of the organ (e.g., organ boundaries, shape), N represents the noise distribution within the organ region, and M represents the medical imaging modality used (e.g., CT, MRI). The significance of this joint probability model lies in modeling the dependency between organ structures and noise given the imaging modality, thereby providing prior knowledge support for subsequent noise estimation and structure reconstruction.

[0074] Furthermore, assuming a certain degree of independence between organ structure and noise, the joint distribution can be decomposed into:

[0075] p(S,N|M)=p(S|M)·p(N|S,M) (4)

[0076] This shows that the prior distribution p(S|M) of organ structure and the conditional distribution p(N|S,M) of noise under a specific structure can be modeled separately, thereby achieving decoupling between structure and noise.

[0077] A2. A pre-trained multimodal organ segmentation network is used to perform organ recognition and organ mask generation on the input image, and Fourier transform is performed on each image and its organ mask to extract the image spectrum.

[0078] Specifically, in this embodiment of the invention, a pre-trained multimodal organ segmentation network (e.g., nnU-Net) is used to perform organ recognition and mask generation on the input image. This network, trained on a large medical image dataset, can accurately identify and segment the boundaries of different organs, outputting a binary mask image M corresponding to each organ. k , where k is the organ number. Then, a two-dimensional Fourier transform is performed on each image and its organ mask: F high =F(I high ), F low =F(I low Extract the image spectrum.

[0079] A3. Divide the extracted image spectrum into multiple frequency bands according to the frequency range and extract frequency domain noise features;

[0080] Specifically, the extracted image spectrum is converted into multiple frequency bands divided according to frequency range as follows:

[0081] Φ k ={φ k1 , φ k2 , …φ kB} (5)

[0082] Where b = 1,...,B represents the frequency band index, φkb Let b be the frequency domain feature of organ k in the b-th frequency band. In order to better capture the low-frequency changes commonly found in medical images (low-frequency information is richer and has a greater impact on super-resolution reconstruction of images), this embodiment of the invention adopts a logarithmic scale when dividing the frequency bands, that is, allocating more frequency bands in the low-frequency region.

[0083] The noise characteristics of an organ region can be expressed as:

[0084] N f =F(I high )-F(I low (6)

[0085] Formula (6) extracts the noise spectral components of unstructured parts of an image by comparing the spectrum of a low-resolution image modulated by a point spread function with the spectrum of a high-resolution image. This modeling method does not require clean structural images or noisy images as input and is suitable for paired image data commonly found in clinical practice.

[0086] A4. The frequency domain noise characteristics are statistically analyzed using the mean vector, covariance matrix, and power spectrum histogram, respectively, to obtain the corresponding three types of statistics. These three types of statistics together constitute a frequency band-level noise prior template unit.

[0087] Specifically, based on the decomposition of the noise characteristics of each organ into multiple frequency bands in the frequency domain, this embodiment of the invention uses the following three statistical measures for modeling:

[0088] 1. Mean vector μ kb : represents the overall noise intensity level within organ k region of this frequency band;

[0089] 2. Covariance matrix Σ kb : Characterizes the spatial correlation of noise, reflecting the noise dependency between adjacent pixels / voxels;

[0090] 3. Power Spectrum Histogram H kb Further capture non-Gaussian noise characteristics (such as skewness, heavy tails, etc.) to describe the statistical laws of energy distribution in the frequency domain.

[0091] These three types of statistics together constitute a frequency band-level noise prior template unit T. kb :

[0092] T kb ={μ kb , ∑ kb H kb} (7)

[0093] The above three statistical measures can be used to characterize noise from multiple perspectives, such as spatial distribution, frequency characteristics, and non-Gaussianity, thereby enhancing the expressive power and adaptability of the model.

[0094] A5. Based on the noise prior template unit, a conditional Gaussian mixture model is used to generate sub-noise prior templates for different tissue regions inside the organ.

[0095] Specifically, even though the three statistics mentioned above describe the overall noise distribution of an organ well, in actual medical images, there may be multiple local noises within the same organ, and the different tissues in different regions within the organ will exhibit different noise characteristics. Therefore, it is difficult to accurately model the noise heterogeneity within an organ using only a single distribution. To address this, this embodiment of the invention introduces a conditional Gaussian mixture model to generate sub-noise prior templates for different tissue regions within the organ, thereby improving the accuracy and applicability of the template library in modeling the noise heterogeneity within organs.

[0096] For the noise characteristics N of organ k in the b-th frequency band kb The probability distribution modeled in this embodiment of the invention is as follows:

[0097]

[0098] Where M is the number of mixed components in the Gaussian mixture, w m The weights of the m-th mixture component satisfy the normalization constraint. μ kbm , ∑ kbm H kbm Let N(·) represent the mean vector, covariance matrix, and power spectrum histogram of the m-th component, respectively, and let N(·) be the multivariate Gaussian distribution function.

[0099] To solve the above hybrid model, this embodiment of the invention uses the expectation-maximization (EM) algorithm for iterative optimization, which can control the noise modeling time of large-scale medical images to the minute level, meeting the needs of rapid clinical library construction.

[0100] Step S103: Perform organ identification and frequency domain feature matching on the input image, and retrieve the optimal noise prior from the multimodal organ template library.

[0101] In the process of image reconstruction or denoising, this invention, in order to achieve efficient and accurate super-resolution reconstruction of noisy high-resolution medical images, introduces an anatomically structure-guided frequency domain analysis mechanism to quickly retrieve the prior noise information of the organ that best matches the image. Specifically, the following steps are included:

[0102] B1 uses a pre-trained organ segmentation network to identify organs in the input image patch and outputs the corresponding organ mask map for organ region localization.

[0103] Specifically, a pre-trained organ segmentation network (e.g., nnU-Net) outputs a corresponding organ mask map to determine the organ category to which the current image patch belongs. In this embodiment of the invention, the pre-trained organ segmentation network performs pixel-level organ mask segmentation (e.g., liver, lung) on ​​the input image patch, and can match specific noise priors for different organ anatomical characteristics (e.g., the high noise characteristics of air-containing tissues in the lungs, and the uniform texture of liver parenchyma). For example, the difference in noise distribution between lung nodule regions and normal lung parenchyma can be modeled separately after mask localization, avoiding optimization bias caused by global noise assumptions. Local noise analysis guided by organ masks can reduce interference from noise features of adjacent organs (e.g., motion artifact noise from heartbeats on chest CT), making the noise prior more closely match the true imaging characteristics of the target region.

[0104] B2 transforms the image patch from the spatial domain to the frequency domain and divides it into multiple frequency bands. By calculating the similarity between the feature vector of the current image patch in each frequency band and the low-resolution frequency domain features of the corresponding organ in the multimodal organ template library, the optimal noise prior is obtained.

[0105] Specifically, the image patch R is transformed from the spatial domain to the frequency domain, obtaining its spectrum F(R), and then divided into multiple frequency bands in a manner consistent with the template library, denoted as... Where B represents the total number of frequency bands. This invention converts image blocks to the frequency domain and divides them into multiple frequency bands, such as low-frequency (contour) and mid-to-high-frequency (texture edges), allowing for the analysis of noise energy distribution within each band. For example, high-frequency noise at bone edges and low-frequency noise in soft tissue regions of CT images have different power spectrum characteristics in the frequency domain. By matching template library features across frequency bands, synergistic optimization of "structural protection + noise suppression" can be achieved.

[0106] For each frequency band b, calculate the feature vector F of the current image patch in that frequency band. b (R) Low-resolution frequency domain features F of the image corresponding to organ k in the organ template library b (I low Similarity matching is performed to obtain the corresponding noise prior. In this embodiment of the invention, Euclidean distance is used as the similarity metric function, that is:

[0107]

[0108] Where i is the i-th sample in the template library, and the noise spectrum reflects the distribution of unstructured noise in the image patch under a specific frequency band.

[0109] Furthermore, after calculating the most similar Then, directly obtain its corresponding noise spectrum. The optimal noise prior is obtained through rapid retrieval of the noise spectrum. This prior is then used in the denoising process of the diffusion model, guiding the model to preserve organ-specific texture and structural details during reconstruction while effectively suppressing unstructured noise. In one embodiment, the template library employs indexed storage (such as KD-trees or hash tables), combined with an image block processing mechanism, which can control the noise prior retrieval time for a single medical image to the millisecond level, meeting the needs of real-time clinical reconstruction (such as intraoperative ultrasound guidance and emergency CT).

[0110] The multimodal template library stores the frequency domain features of the same organ under different modalities (such as CT and PET). When the input image is a low-dose CT, the low-frequency structural features of the corresponding organ in the PET template can be retrieved as a priori to make up for the lack of soft tissue resolution of CT and achieve cross-modal noise suppression and structural restoration.

[0111] Step S104: In the reverse denoising process of the diffusion model, a noise prior guiding term is injected to achieve adaptive super-resolution image reconstruction of anatomical structures.

[0112] The present invention integrates a diffusion model guidance mechanism that incorporates prior information on anatomical structure noise. By introducing frequency-domain matched noise priors, the diffusion model enhances its ability to identify and suppress unstructured noise during medical image super-resolution reconstruction, thereby improving the structural fidelity and clinical usability of the reconstructed images.

[0113] This invention employs the basic structure of a standard diffusion model, including a forward noise addition process and a backward noise reduction process. The forward process progressively adds Gaussian noise to the high-resolution image until it is completely degraded into pure noise. The backward process learns to progressively remove noise and restore the original image information.

[0114] Specifically, in order to effectively suppress unstructured noise, this embodiment of the invention introduces an organ-specific noise prior guiding term G in the reverse process of the diffusion model. N (z t (k, t). This guiding term is based on the current intermediate state z. t Estimate its noise components Then, the Mahalanobis distance is calculated between the signal and the noise template corresponding to organ k, thereby generating a frequency-domain driven spatial guidance signal:

[0115]

[0116] Among them, z t For the intermediate state at time step t, ∈ θ For noise prediction networks, G N (z t (k,t) is a noise prior guide term based on organ category k, used to dynamically suppress unstructured noise components that match the noise pattern of that organ, βt The a priori strength coefficient follows an annealing curve as t decreases; α t This is the decay factor for time step t, used to control the gradual addition or removal of noise. T represents the total number of generation steps.

[0117] This invention utilizes Mahalanobis distance calculation, fully considering the covariance structure of noise data, to accurately measure the similarity between the noise in the current intermediate state of the image and the noise template corresponding to the organ. The resulting spatial guidance signal can guide the diffusion model during reverse denoising, performing more realistic denoising operations tailored to the noise characteristics of different organs and regions, avoiding blind denoising, and making the reconstructed image closer to the real high-resolution image in terms of detail and overall structure, significantly improving reconstruction accuracy. The guidance signal generated using prior noise information can specifically identify and suppress noise in the image. Compared with traditional diffusion models, this method can more effectively handle complex noise distributions, especially non-Gaussian noise and salt-and-pepper noise commonly found in multimodal medical images, significantly reducing noise levels in the image and improving the signal-to-noise ratio and visual quality.

[0118] The noise prior guiding term directly utilizes the constructed organ noise prior template library, which is represented as follows:

[0119]

[0120] Among them, F, F -1 These are the Fourier transform and its inverse transform, μ kb ,∑ kb These are the mean vector and covariance matrix of organ k in frequency band b, stored in the organ template library, respectively. b (t) is the frequency band weighting function.

[0121] The core idea of ​​this guiding term is to identify components similar to organ noise distribution in the frequency domain and measure their deviation using Mahalanobis distance, thereby dynamically suppressing these noise components. In the denoising stage of the diffusion model, by introducing organ-specific noise priors, the model can identify and suppress unstructured noise components highly similar to typical organ noise patterns, thus preserving more real structural information, improving the detail quality and clinical reliability of the reconstructed image. Simultaneously, the noise prior guiding term provides clear direction for the reverse denoising process of the diffusion model, reducing blind exploration during denoising and helping the model find the optimal solution faster, thereby accelerating the model's convergence speed. This not only shortens the time required for image reconstruction but also reduces computational resource consumption, improving overall computational efficiency and facilitating the rapid generation of high-quality super-resolution images in real-world clinical environments.

[0122] Furthermore, the noise prediction network in this embodiment of the invention adopts a U-Net structure, specifically including:

[0123] 1. Encoder: 4 downsampling blocks, each block contains two residual convolutional layers;

[0124] 2. Decoder: 4 upsampling blocks, each block contains two residual convolutional layers;

[0125] 3. Temporal Embedding Layer: Positional encoding is used to convert time step t into a feature vector and inject it into each residual block;

[0126] 4. Attention Layer: Apply self-attention mechanism to intermediate and deep features to enhance global information acquisition.

[0127] 5. A weighted loss function is used during model training:

[0128]

[0129] Where w(t) is the time-dependent weighting function; L perceptual For perceptual loss, the distance is based on the features of the pre-trained U-Net network, λ is the perceptual loss weight, for example, set to 0.1, z0 is the original image, z pred The predicted restored image is denoted as ∈, where ∈ represents added noise.

[0130] This invention employs a noise prediction network based on the U-Net architecture. By utilizing a downsampling encoder, an upsampling decoder, a temporal embedding layer, and an attention layer, it effectively extracts and fuses multi-scale image features, enhancing the modeling ability for complex anatomical structures. The weighted loss function accelerates the model's learning efficiency in the critical denoising stage by assigning weights to early temporal steps. Combined with perceptual loss to constrain the high-level semantic consistency of the image, the reconstructed image not only closely resembles the real image at the pixel level but also preserves important anatomical structures and texture features in medical images, ultimately significantly improving the quality, reconstruction efficiency, and clinical applicability of super-resolution images.

[0131] This embodiment also provides a medical image super-resolution reconstruction system, which is used to implement the above embodiments and preferred embodiments, and will not be repeated as already described. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the system described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0132] This embodiment provides a medical image super-resolution reconstruction system, such as Figure 2 As shown, it includes:

[0133] Image dataset construction module 201 is used to construct a multimodal medical image dataset that includes high-resolution images and corresponding low-resolution image pairs;

[0134] The organ template library construction module 202 is used to decouple and model organ structure and noise characteristics using frequency domain analysis and Gaussian mixture model to establish a multimodal organ template library;

[0135] The optimal noise prior matching module 203 is used to perform organ identification and frequency domain feature matching on the input image and retrieve the optimal noise prior from the multimodal organ template library.

[0136] Image reconstruction module 204 is used to inject noise prior guiding terms into the inverse denoising process of the diffusion model to achieve adaptive super-resolution image reconstruction of anatomical structures.

[0137] In some alternative implementations, the image dataset construction module 201 includes:

[0138] The real data acquisition unit is used to acquire real high- and low-resolution image pairs of multiple modalities by adjusting the equipment parameters of the medical imaging equipment.

[0139] The simulation data acquisition unit is used to generate simulation data through a physics-driven degradation model when real data is insufficient. The degradation model generates a low-resolution image by applying point spread function blurring, noise addition, and downsampling operations to the high-resolution image.

[0140] In some alternative implementations, the organ template library construction module 202 includes:

[0141] The decoupling unit is used to model the dependency between organ structure and noise through a joint probability model, thereby achieving decoupling between structure and noise.

[0142] The image spectrum acquisition unit is used to perform organ recognition and organ mask generation on the input image using a pre-trained multimodal organ segmentation network, and to extract the image spectrum by performing Fourier transform on each image and its organ mask.

[0143] The noise feature extraction unit is used to divide the extracted image spectrum into multiple frequency bands according to the frequency range and extract frequency domain noise features;

[0144] The first noise prior template production unit is used to statistically analyze the frequency domain noise characteristics using mean vector, covariance matrix and power spectrum histogram respectively, to obtain the corresponding three types of statistics. The three types of statistics together constitute a frequency band-level noise prior template unit.

[0145] The second noise prior template production unit generates sub-noise prior templates for different tissue regions within an organ by using a conditional Gaussian mixture model based on the noise prior template unit.

[0146] In some alternative implementations, the optimal noise prior matching module 203 includes:

[0147] The organ recognition and localization unit is used to identify organs in the input image block based on a pre-trained organ segmentation network and output the corresponding organ mask map for organ region localization.

[0148] The noise prior matching unit is used to transform the image patch from the spatial domain to the frequency domain and divide it into multiple frequency bands. By calculating the similarity between the feature vector of the current image patch in each frequency band and the low-resolution frequency domain features of the corresponding organ in the multimodal organ template library, the optimal noise prior is obtained.

[0149] In some alternative implementations, the image reconstruction module 204 includes:

[0150] The noise prior term injection unit is used to inject noise prior terms into the reverse denoising process of the diffusion model using the basic structure of the standard diffusion model. The noise prior term estimates its noise components based on the current intermediate state.

[0151] The image reconstruction unit is used to calculate the Mahalanobis distance to the noise template corresponding to the organ and generate a frequency-domain driven spatial guidance signal for super-resolution image reconstruction.

[0152] In one alternative implementation, the space guidance signal is expressed by the following formula:

[0153]

[0154] Among them, z t For the intermediate state at time step t, ∈ θ For noise prediction networks, G N (z t (k,t) is a noise prior guide term based on organ category k, used to dynamically suppress unstructured noise components that match the noise pattern of that organ, β t The a priori strength coefficient follows an annealing curve as t decreases; α t This is the decay factor for time step t, used to control the gradual addition or removal of noise. T represents the total number of generation steps.

[0155] The noise prediction network adopts the U-Net model, which includes a preset number of downsampling encoder layers, a preset number of upsampling decoder layers, a temporal embedding layer, and an attention layer. The model training uses a weighted loss function:

[0156]

[0157] Where w(t) is the time-dependent weighting function; L perceptual For perceptual loss, the distance is based on the features of the pre-trained U-Net network, λ is the perceptual loss weight, z0 is the original image, and z pred The predicted restored image is denoted as ∈, where ∈ represents added noise.

[0158] In one optional implementation, the noise prior guiding term is:

[0159]

[0160] Among them, F, F -1 These are the Fourier transform and its inverse transform, μ kb ,∑ kb These are the mean vector and covariance matrix of organ k in frequency band b, stored in the organ template library, respectively. b (t) is the frequency band weighting function.

[0161] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0162] The medical image super-resolution reconstruction system in this embodiment is presented in the form of functional units. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0163] This invention also provides a computer device having the above-described features. Figure 2 The medical image super-resolution reconstruction system shown.

[0164] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 3As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 3 Take a processor 10 as an example.

[0165] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0166] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0167] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0168] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0169] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0170] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0171] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0172] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and all such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A method for super-resolution reconstruction of medical images, characterized in that, include: Construct a multimodal medical image dataset that includes high-resolution images and corresponding low-resolution image pairs; Frequency domain analysis and Gaussian mixture model were used to decouple organ structure and noise characteristics, and a multimodal organ template library was established. Performing organ identification and frequency domain feature matching on the input image, and retrieving the optimal noise prior from the multimodal organ template library, includes: The pre-trained organ segmentation network performs organ recognition on the input image patch and outputs the corresponding organ mask map for organ region localization. The image patch is transformed from the spatial domain to the frequency domain and divided into multiple frequency bands. The optimal noise prior is obtained by calculating the similarity between the feature vector of the current image patch in each frequency band and the low-resolution frequency domain features of the corresponding organ in the multimodal organ template library. In the inverse denoising process of the diffusion model, a noise prior guiding term is injected to achieve adaptive super-resolution image reconstruction of anatomical structures, including: Using the basic structure of the standard diffusion model, a noise prior guiding term is injected into the inverse denoising process of the diffusion model. This noise prior guiding term estimates its noise components based on the current intermediate state and calculates the Mahalanobis distance between it and the noise template corresponding to the organ, generating a frequency-domain driven spatial guiding signal for super-resolution image reconstruction. The spatial guiding signal is expressed by the following formula: in, This represents the intermediate state at time step t. For noise prediction networks, Based on organ category The noise prior guide is used to dynamically suppress unstructured noise components that match the noise pattern of the organ. The a priori strength coefficient follows an annealing curve as t decreases; This is the decay factor for time step t, used to control the gradual addition or removal of noise. T represents the total number of generation steps.

2. The method according to claim 1, characterized in that, The construction of the multimodal medical image dataset includes high-resolution images and corresponding low-resolution image pairs, comprising: By adjusting the equipment parameters of medical imaging equipment, real high- and low-resolution image pairs of multiple modalities can be obtained. When real data is insufficient, simulation data is generated through a physics-driven degradation model. This degradation model generates low-resolution images by applying point spread function blur noise to high-resolution images and performing downsampling operations.

3. The method according to claim 1, characterized in that, The method employs frequency domain analysis and Gaussian mixture modeling to decouple organ structure and noise characteristics, establishing a multimodal organ template library, including: By modeling the dependency between organ structure and noise using a joint probability model, the decoupling of structure and noise can be achieved. A pre-trained multimodal organ segmentation network is used to identify organs and generate organ masks from input images, and Fourier transform is performed on each image and its organ mask to extract the image spectrum. The extracted image spectrum is divided into multiple frequency bands according to the frequency range, and frequency domain noise features are extracted. The frequency domain noise characteristics are statistically analyzed using mean vector, covariance matrix, and power spectrum histogram, respectively, to obtain three types of statistics. These three statistics together constitute a frequency band-level noise prior template unit. Based on the noise prior template unit, a conditional Gaussian mixture model is used to generate sub-noise prior templates for different tissue regions inside the organ.

4. The method according to claim 1, characterized in that, The noise prediction network adopts the U-Net model, which includes a preset number of downsampling encoder layers, a preset number of upsampling decoder layers, a temporal embedding layer, and an attention layer. The model training uses a weighted loss function: ， in, A time-dependent weighting function; To perceive loss, distance is based on features from the pre-trained U-Net network. To perceive loss weights, For the original image, For the predicted restored image, This is added noise.

5. The method according to claim 4, characterized in that, The noise prior guiding term is: in, These are the Fourier transform and its inverse transform, respectively. These are the mean vector and covariance matrix of organ k in frequency band b, stored in the organ template library. It is a frequency band weighting function.

6. A medical image super-resolution reconstruction system, characterized in that, include: The image dataset construction module is used to construct a multimodal medical image dataset that includes high-resolution images and corresponding low-resolution image pairs. The organ template library construction module is used to decouple and model organ structure and noise characteristics using frequency domain analysis and Gaussian mixture model to establish a multimodal organ template library; An optimal noise prior matching module is used to perform organ identification and frequency domain feature matching on the input image, and to retrieve the optimal noise prior from the multimodal organ template library, including: The pre-trained organ segmentation network performs organ recognition on the input image patch and outputs the corresponding organ mask map for organ region localization. The image patch is transformed from the spatial domain to the frequency domain and divided into multiple frequency bands. The optimal noise prior is obtained by calculating the similarity between the feature vector of the current image patch in each frequency band and the low-resolution frequency domain features of the corresponding organ in the multimodal organ template library. The image reconstruction module is used to inject a noise prior guiding term into the inverse denoising process of the diffusion model to achieve adaptive super-resolution image reconstruction of anatomical structures, including: Using the basic structure of the standard diffusion model, a noise prior guiding term is injected into the inverse denoising process of the diffusion model. This noise prior guiding term estimates its noise components based on the current intermediate state and calculates the Mahalanobis distance between it and the noise template corresponding to the organ, generating a frequency-domain driven spatial guiding signal for super-resolution image reconstruction. The spatial guiding signal is expressed by the following formula: in, This represents the intermediate state at time step t. For noise prediction networks, Based on organ category The noise prior guide is used to dynamically suppress unstructured noise components that match the noise pattern of the organ. The a priori strength coefficient follows an annealing curve as t decreases; This is the decay factor for time step t, used to control the gradual addition or removal of noise. T represents the total number of generation steps.

7. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the medical image super-resolution reconstruction method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the medical image super-resolution reconstruction method according to any one of claims 1 to 5.

Images