A medical image super-resolution method and device guided by high-frequency information based on diffusion model

Abstract

The present invention discloses a medical image super-resolution method and device guided by high-frequency information based on a diffusion model. The method first obtains medical image data and performs preprocessing to construct a data set; then constructs a high-frequency generation network, further constructs a super-resolution algorithm combining discrete wavelet transform and diffusion model, and trains the model based on the data set; then uses the data set to evaluate the performance of the trained super-resolution algorithm through evaluation indicators, and further adjusts the model parameters according to the evaluation results to obtain the optimal super-resolution algorithm model; finally, the optimal super-resolution algorithm model is obtained, and the low-resolution image is input into the optimal super-resolution algorithm model to obtain a high-resolution image. The present invention includes a super-resolution algorithm model that uses a high-frequency generation network, combines discrete wavelet transform and diffusion model, and avoids the problem of the diffusion model noise adding process destroying the high-frequency information.

Description

Technical Field

[0001] The present invention relates to the cross-technical field of computer technology and medical image cross-modality registration, and in particular to a medical image super-resolution method and device guided by high-frequency information based on a diffusion model. Background Art

[0002] Medical imaging is a powerful diagnostic and research tool that creates visual representations of anatomical structures and has been widely used for disease diagnosis and surgical planning. In clinical practice, commonly used medical imaging such as magnetic resonance imaging (MR) is widely used to assist in patient diagnosis, but it is often of poor quality and noisy. High-quality, high-resolution images help medicine make more informed decisions. Therefore, it is very important to use super-resolution algorithms to improve image resolution.

[0003] Diffusion models are emerging as a powerful solution for high-fidelity image generation, even surpassing the quality of adversarial generative networks in many cases. However, their slow training and inference speed is a huge bottleneck that hinders their use in real-time applications. In recent research work, a variety of methods have addressed this problem from different angles. For example, the DiffusionGAN method significantly reduces the running time of the model by reducing the number of sampling steps from thousands to a few. Phung et al. combined the discrete wavelet transform with the denoising diffusion probability model, extracted low-frequency and high-frequency components from the image and feature levels through wavelet decomposition, and adaptively processed these components to speed up processing while maintaining good generation quality. However, the problem of the diffusion model's denoising process destroying high-frequency information still exists. Summary of the invention

[0004] The purpose of the present invention is to address the deficiencies of the prior art and propose a medical image super-resolution method and device guided by high-frequency information based on a diffusion model. The diffusion model network used in the present invention includes an additional high-frequency generation network, which transforms the noise addition space of the diffusion model from the original image space to the residual space, thereby avoiding the problem of the diffusion model noise addition process destroying the high-frequency information.

[0005] The objective of the present invention is achieved through the following technical solutions: In a first aspect, the present invention provides a medical image super-resolution method guided by high-frequency information based on a diffusion model, the method comprising the following steps:

[0006] (1) Obtain medical imaging data, process it to obtain low-resolution images, and construct a data set;

[0007] (2) Perform wavelet transform on the images in the data set, concatenate the images containing high-frequency information and predict them through the high-frequency generation network to obtain the predicted high-frequency image, connect an image with the same shape as the image containing low-frequency information after wavelet transform, but with a value of 0, and subtract the concatenated result from the original high-resolution image to obtain the residual image;

[0008] (3) Input the residual image into the diffusion model, calculate the pixel loss with the wavelet transformed image after adding Gaussian noise, and optimize the parameters of the high-frequency generation network and the diffusion model;

[0009] (4) Obtain a low-resolution image of a medical image, and obtain a high-resolution image by inputting the optimized high-frequency generation network and diffusion model through wavelet transform.

[0010] Furthermore, in step (1), the medical imaging data includes magnetic resonance images and computed tomography images.

[0011] Furthermore, in step (1), the preprocessing of the medical image data includes adjusting the window width and window position for denoising, resampling to adjust the voxel spacing, normalizing the pixel values ​​and downsampling the original high-resolution image to obtain a low-resolution image.

[0012] Furthermore, in step (2), the high-frequency generation network includes three depthwise separable convolutions, two cross-attention layers, a progressive dilation module, and three depthwise separable convolutions; the images HH, LH, and HL containing high-frequency information obtained by wavelet transform are concatenated and then features are extracted through three depthwise separable convolutions; the cross-attention layer uses the information in HH and LH to supplement the details in HL; the progressive dilation module uses dilated convolution to extract local information for local restoration; and finally, three depthwise separable convolutions are used to reduce the number of channels to obtain the prediction result.

[0013] Furthermore, in step (3), the residual image is used as the input of the diffusion model. First, Gaussian noise is continuously added through the forward process to obtain a completely noisy image. The four images obtained by wavelet transforming the low-resolution image are used as conditional inputs and connected with the completely noisy image through the channel dimension as the input of the generator in the diffusion module. The pixel loss of the denoised image and the residual image is calculated.

[0014] Furthermore, the generator in the diffusion module receives the noisy image samples and their time indexes as source image inputs at time t, and outputs denoised image samples as target modalities at time tk which is k time points back; wherein, based on the image samples, t different moments are set, and Gaussian noise is continuously added through the diffusion model to obtain the noisy image samples and their time indexes.

[0015] Furthermore, in step (4), the low-resolution image is transformed by wavelet transform to obtain a low-frequency image and three high-frequency images; the three high-frequency images are merged through channels and enter the high-frequency generation network to obtain a predicted high-frequency image; the low-frequency image and the three high-frequency images are merged through channels to obtain a wavelet domain image, the wavelet domain image and the noise image of the same size are merged through channels and enter the diffusion model to obtain a predicted wavelet domain image, and the predicted wavelet domain image and the predicted high-frequency image are added to obtain the final high-resolution image.

[0016] The second aspect of the present invention provides a medical image super-resolution device guided by high-frequency information based on a diffusion model, comprising a memory and one or more processors, wherein the memory stores executable code, and when the processor executes the executable code, it implements the medical image super-resolution method guided by high-frequency information based on a diffusion model.

[0017] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, and when the program is executed by a processor, the medical image super-resolution method guided by high-frequency information based on a diffusion model is implemented.

[0018] A fourth aspect of the present invention provides a computer program product, including a computer program, which, when executed by a processor, implements the medical image super-resolution method guided by high-frequency information based on a diffusion model.

[0019] Beneficial effects of the present invention: The present invention designs a new super-resolution method that utilizes a high-frequency generation network, combines wavelet transform and a diffusion network model, and solves the problem of low sampling speed of the diffusion model and high consumption of computing resources, while also avoiding the problem of the diffusion model noise adding process destroying high-frequency information. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 It is a flowchart of a medical image super-resolution method using a high-frequency generation network combined with wavelet transform and diffusion model in an embodiment of the present invention.

[0022] Figure 2 It is a schematic diagram of a network structure using a high frequency generation network combined with a wavelet transform and a diffusion model provided by an embodiment of the present invention.

[0023] Figure 3It is a schematic diagram of the network structure of a high-frequency generation network provided by an embodiment of the present invention.

[0024] Figure 4 It is a schematic diagram of the network structure of the residual module in the generator network of the diffusion model in an embodiment of the present invention.

[0025] Figure 5 This is the MR and CT image preprocessing process in the embodiment of the present invention.

[0026] Figure 6 It is a structural schematic diagram of a medical image super-resolution device of the present invention that utilizes a high-frequency generation network and combines wavelet transform and diffusion model. DETAILED DESCRIPTION

[0027] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0028] The terms used in the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "the" and "the" used in the present invention and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0029] It should be understood that although the terms first, second, third, etc. may be used in the present invention to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0030] The present invention is described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the features of the following embodiments and implementations can be combined with each other.

[0031] See also Figure 1The present invention designs a medical image super-resolution method using a high-frequency generation network, combined with discrete wavelet transform and diffusion model. First, low-frequency and high-frequency components are extracted from the image and feature levels through discrete wavelet transform, and the high-frequency information of the wavelet space is predicted using a pre-trained high-frequency generation network. The diffusion model is allowed to learn the residual space between the prediction results of the high-frequency generation network and the actual results, avoiding the problem of the diffusion model adding noise to destroy the high-frequency information. Specifically, it includes the following steps:

[0032] (1)Obtain the patient’s medical imaging data.

[0033] In this embodiment, the private medical image data of the hospital is used, and the medical image data includes magnetic resonance (MR) images or computed tomography (CT) images of 305 patients. The real CT image and MR image both include: plain scan phase (NC), arterial phase (ART), portal phase (PV), and delayed phase (DL). The data format of MR image and CT image is nii.

[0034] (2) Preprocess the medical imaging data, construct a dataset based on the preprocessed medical imaging data, and divide the dataset into a training set, a validation set, and a test set.

[0035] In this embodiment, the data set can be divided into a training set, a validation set, and a test set in a certain ratio according to actual needs, such as 6:2:2, etc.; data can also be randomly selected to construct a training set, a validation set, and a test set.

[0036] Among them, Figure 5 As shown, the specific process of preprocessing includes the following steps:

[0037] (2.1) Adjusting the window width and window position: Adjust the window width and window position of medical image data for denoising.

[0038] In this embodiment, if the medical image data is a CT image, the window width can be set to (-110, 190) according to the doctor's prior knowledge, and the corresponding window level is (40, 300), and usually a numerical value can be selected for setting; the truncated method can be used to obtain the denoised CT image, such as the np.clip algorithm in the numpy library. If the medical image data is an MR image, since the MR image cannot select a fixed value to adjust the window width and window level, an image denoising method can be used to obtain the denoised MR image, such as the estimate_sigma algorithm and the nlmeans algorithm in the dipy library.

[0039] (2.2) Resampling: The medical image data is resampled using the resample_image method in the nilearn library to adjust the voxel spacing of the image.

[0040] (2.3) Normalization: Normalize the pixel values ​​of medical imaging data.

[0041] Specifically, if it is a CT image, its pixel value can be directly normalized to [-1,1] using linear normalization. If it is an MR image, the pixel value can be initially normalized using z-score, and then normalized using linear normalization, for example, it can be normalized to [-1,1] to facilitate subsequent calculations. It should be understood that z-score normalization is a common method of data processing, through which data of different magnitudes can be converted into a uniformly measured Z-Score for comparison; of course, other normalization methods can also be used to normalize pixel values, such as zero-mean normalization.

[0042] (2.4) Downsampling: Downsample the original high-resolution image to obtain a low-resolution image.

[0043] Specifically, for high-resolution images of original medical imaging data, the algorithm in the OpenCV library is used to assist in downsampling. First, the image is Gaussian blurred using the GaussianBlue function; second, the image is downsampled four times using the resize function, and INTER_CUBIC is selected as the interpolation method.

[0044] (3) Using the high-frequency generation network and combining it with wavelet transform, a diffusion model based on the diffusion model is constructed. The diffusion model is trained using the training set to obtain a trained diffusion model.

[0045] (3.1) A super-resolution algorithm model using a high-frequency generation network combined with a wavelet transform and a diffusion model, wherein the super-resolution algorithm model includes: a high-frequency generation module, a wavelet transform module, and a diffusion module.

[0046] In this embodiment, the high frequency generation module is as follows: Figure 3 As shown in the figure, it includes three depth-wise separable convolutions, two cross-attention layers, a progressive dilation module and three depth-wise separable convolutions. The depth-wise separable convolution is used to extract the features of the input coefficients to improve the effect; the cross-attention layer uses the information in HH and LH to supplement the details in HL. The progressive dilation module uses dilated convolution for local recovery, where the first and last convolutions are used to extract local information, and the middle dilated convolution is used to improve the receptive field and better utilize long-distance information. The last three depth-wise separable convolutions are used to reduce the number of channels to obtain the predicted results.

[0047] Furthermore, the depthwise separable convolution includes two convolutional layers. The kernel size of the first convolutional layer is 3, the stride is 1, and the padding parameter is 1. The kernel size of the second convolutional layer is 1, the stride is 1, and the padding parameter is 0. The cross attention layer performs query, key and value conversion on the input data through the depthwise separable convolutional layer, and then performs attention calculation and normalization operations, which can effectively encode and represent the input data. In deep learning tasks, this module can be used as a component of the attention mechanism to help the model capture important information of the input data and improve the performance of the model in various tasks. The progressive expansion module includes five convolutional layers and four activation functions. The kernel size of the first convolutional layer is 3, the stride is 1, the padding parameter is 1, and the dilation is 1. The kernel size of the second convolutional layer is 3, the stride is 1, the padding parameter is 2, and the dilation is 2. The kernel size of the third convolutional layer is 3, the stride is 1, the padding parameter is 3, and the dilation is 3. The kernel size of the fourth convolutional layer is 3, the stride is 1, the padding parameter is 2, and the dilation is 2. The kernel size of the fifth convolutional layer is 3, the stride is 1, the padding parameter is 1, and the dilation is 1. The activation function uses LeakyReLU.

[0048] In this embodiment, the wavelet transform module uses the classic "Haar" wavelet. The decomposition divides the input into four sub-bands, representing the average image (LL) and high-frequency details in the horizontal, vertical and diagonal directions (LH, HL, HH).

[0049] In this embodiment, the idea of ​​the diffusion module is: for data distribution, forward construct Gaussian noise probability distributions of T different time periods (representing different noise levels). After that, the distribution of the previous moment is gradually reversely predicted until it is traced back to the initial moment, so that the predicted distribution is close to the real distribution. The entire time series has the properties of a Markov chain. It should be understood that the diffusion model is divided into a diffusion stage and an inverse diffusion stage. In the diffusion stage, by continuously adding noise to the residual modal data between the generated image output by the high-frequency generation network and the target domain image, the data is changed from the original distribution to the desired distribution, for example, by continuously adding Gaussian noise to change the original data distribution to a normal distribution; in the inverse diffusion stage, a neural network is used to restore the data from a normal distribution to the original data distribution; wherein each point on the normal distribution is a mapping of the real data, and the diffusion model has better interpretability.

[0050] The diffusion module includes a generator that receives a noisy image sample and its time index as a source image input at time t, and outputs a denoised image sample for the target modality at time point tk after going back k moments. Based on the image sample, t different moments are set, and Gaussian noise is continuously added through the diffusion model to obtain the noisy image sample and its time index.

[0051] The generator of the diffusion module uses UNet as the backbone network, and the UNet backbone network uses six downsampling blocks and six upsampling blocks. Each downsampling module uses two residual modules, followed by a convolutional layer to halve the feature map resolution, and the channel dimension is doubled every other downsampling block; then the 32-dimensional sinusoidal position encoding is projected through two layers of multilayer perceptron (MLP) to calculate the time embedding of each time index, and then the time embedding is added to the feature map of the residual sub-block. Each upsampling block uses two residual modules, followed by a convolutional layer to double the feature map resolution, and the channel dimension is halved every other upsampling block.

[0052] Furthermore, the kernel size of the convolution layer is 3, that is, the convolution kernel is 3*3, and the padding is 1.

[0053] In this embodiment, the structure of the residual module is as follows: Figure 4 As shown in the figure, the residual module includes image boundary padding, convolution layer (Conv), normalization layer (BatchNorm) and activation function (ReLU activation function).

[0054] (3.2) Use the training set to train the super-resolution model, and update the parameters of the super-resolution model according to the loss of the super-resolution model to obtain a trained super-resolution model.

[0055] When using the training set to train the super-resolution model, set the number of iterations and the learning rate, use the optimizer to train and update the weight parameters of the super-resolution model, set the number of samples selected for each training (batch size), for example, you can set the batch size to 4, input the low-resolution images and high-resolution images in the training set into the super-resolution model for iterative training, and update the weight parameters according to the loss of the super-resolution model to obtain a trained model.

[0056] Preferably, the number of iterations is set to 80 and the learning rate is set to 0.0001.

[0057] Furthermore, the optimizer includes the Adaptive Moment Estimation (Adam) optimizer, the AdaGrad optimizer, and the RMSProp optimizer. It should be understood that a suitable optimizer can be selected to train the model according to actual requirements. The Adam optimizer uses the first-order moment estimation and the second-order moment estimation of the gradient to dynamically adjust the learning rate of each parameter. Its main advantage is that after bias correction, each iteration learning rate will have a certain range, making the parameters relatively stable; the AdaGrad optimizer can independently adjust the learning rate of the model parameters, can make large updates to sparse parameters and small updates to frequent parameters, and is suitable for processing sparse data; the RMSProp optimizer uses an exponentially weighted moving average instead of the gradient square sum to address the problem of increasing cumulative gradient square sums.

[0058] like Figure 2 As shown in the figure, the super-resolution model includes three modules. Specifically, the low-resolution image LR is first transformed by discrete wavelet transform to obtain four images LL, LH, HL, and HH. Among them, the three high-frequency images LH, HL, and HH are concatenated together in the channel dimension as the input of the high-frequency generation network to obtain the predicted high-frequency image. The predicted high-frequency image is connected to an image with the same shape as the input image LL, but with a value of 0 in the channel dimension. The result of the connection is subtracted from the high-resolution image after discrete wavelet transform to obtain the residual image. The residual image is used as the input of the diffusion model. First, Gaussian noise is continuously added through the forward process to obtain a completely noisy image. The four images LL, LH, HL, and HH obtained by discrete wavelet transform of the low-resolution image are used as conditional inputs and concatenated with the completely noisy image through the channel dimension as the input of the generator in the diffusion module. After the backward process, the pixel loss of the denoised image and the residual image is calculated to achieve optimization.

[0059] In this embodiment, the loss of the super-resolution model includes the pixel-level loss in the diffusion model.

[0060] Furthermore, the expression of pixel-level loss is:

[0061] L = MSE(xy)

[0062] Among them, x represents the result of the diffusion module generator input, y represents the residual image input to the forward process of the diffusion module, and MSE represents the mean square error.

[0063] (4) Obtain a trained super-resolution algorithm model, use the validation set to evaluate the performance of the trained model using evaluation indicators, and adjust the model parameters based on the evaluation results to obtain the optimal model.

[0064] In this embodiment, after the trained super-resolution model is obtained in step (3), the low-resolution image in the validation set is first subjected to discrete wavelet transform to obtain a low-frequency image and three high-frequency images. The three high-frequency images are connected through the channel dimension to enter the high-frequency generation network to obtain the predicted high-frequency image. The low-frequency image after discrete wavelet transform and the three high-frequency images are connected through the channel dimension to obtain a wavelet domain image. The wavelet domain image and the randomly generated noise image of the same size are connected through the channel dimension to enter the diffusion model to obtain the predicted wavelet domain image. The predicted wavelet domain image and the predicted high-frequency image are added to obtain the final high-resolution image. The predicted high-resolution image and the actual high-resolution image are used to evaluate the model through evaluation indicators, and the parameters of the model are adjusted according to the evaluation results to obtain the optimal model.

[0065] In this embodiment, the evaluation indicators include learning-based image perceptual similarity (LPIPS) and Frechet Inception distance (FID), and the performance of the trained model is evaluated by these two evaluation indicators. The smaller the value of learning-based image perceptual similarity, the better the performance of the generator, and the smaller the Frechet Inception distance, the better the performance of the generator.

[0066] It should be understood that LPIPS is a commonly used image similarity metric that learns the reverse mapping of generated images to real images, forces the generator to learn the reverse mapping of reconstructing real images from fake images, and prioritizes the perceptual similarity between them. The smaller the LPIPS value, the more similar the two images are, and vice versa, the greater the difference. FID can calculate the distance between the feature vectors of the real image and the generated image. The similarity of the two groups of images is measured based on this evaluation metric. The greater the similarity, the higher the quality of the generated image; the smaller the FID value, the greater the similarity between the real image and the generated image.

[0067] Illustratively, in this embodiment, sample data containing 60 patients is used to generate high-resolution images from low-resolution images, and the above two evaluation indicators are used. The results are shown in Table 1.

[0068] Table 1: Model evaluation results

[0069]

[0070] It can be seen from Table 1 that, compared with the existing DiWa and the super-resolution model constructed by the present invention, the performance of the model constructed by the present invention is better.

[0071] It should be understood that other evaluation indicators can also be selected to evaluate the performance of the trained generator, such as the peak signal-to-noise ratio (PSNR) of the image. The larger the PSNR value, the less distortion, indicating that the quality of the generated target domain image is better. Another example is structural similarity (SSIM, Structural Similarity Index Measurement). SSIM is an indicator that measures the similarity between two images. It can evaluate the quality of the generated target domain image. The larger the SSIM value, the more similar the two images are, indicating that the generated target domain image is closer to the real target domain image.

[0072] In summary, the super-resolution algorithm used in the present invention utilizes a high-frequency generation network, combined with wavelet transform and diffusion network model, to solve the problem of low sampling speed of the diffusion model and high consumption of computing resources, while also avoiding the problem of the diffusion model noise adding process destroying high-frequency information.

[0073] (5) Input the low-resolution image or the low-resolution image in the test set into the optimal algorithm model to obtain a high-resolution image.

[0074] Specifically, a high-resolution image can be obtained by inputting a low-resolution image or a low-resolution image in a test set into a super-resolution algorithm model.

[0075] In summary, the current mainstream super-resolution algorithm uses a diffusion model, combined with wavelet transform and other methods to speed up the processing speed while maintaining good generation quality. However, the problem of the diffusion model noise adding process destroying high-frequency information still exists. The super-resolution algorithm used in the present invention uses a high-frequency generation network, combined with wavelet transform and diffusion network model, to solve the problem of the diffusion model noise adding process destroying high-frequency information while solving the problem of the diffusion model noise adding process destroying high-frequency information under the condition of low sampling speed of the diffusion model and high consumption of computing resources.

[0076] Corresponding to the aforementioned embodiment of a medical image super-resolution method guided by high-frequency information based on a diffusion model, the present invention also provides an embodiment of a medical image super-resolution device guided by high-frequency information based on a diffusion model.

[0077] See also Figure 6A medical image super-resolution device guided by high-frequency information based on a diffusion model provided in an embodiment of the present invention includes a memory and one or more processors, wherein the memory stores executable code, and when the processor executes the executable code, it is used to implement a medical image super-resolution method guided by high-frequency information based on a diffusion model in the above embodiment.

[0078] An embodiment of a medical image super-resolution device guided by high-frequency information based on a diffusion model provided by the present invention can be applied to any device with data processing capabilities, and the device with data processing capabilities can be a device or apparatus such as a computer. The device embodiment can be implemented through software, or through hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capabilities in which it is located reading the corresponding computer program instructions in the non-volatile memory into the internal memory for execution. From a hardware perspective, if Figure 6 As shown, it is a hardware structure diagram of a medical image super-resolution device guided by high-frequency information based on a diffusion model provided by the present invention, in which any device with data processing capability is located, except Figure 6 In addition to the processor, memory, network interface, and non-volatile memory shown, any device with data processing capabilities in which the apparatus in the embodiments is located may also include other hardware, generally based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0079] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0080] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.

[0081] An embodiment of the present invention further provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, a medical image super-resolution method guided by high-frequency information based on a diffusion model in the above embodiment is implemented.

[0082] The computer-readable storage medium may be an internal storage unit of any device with data processing capability described in any of the aforementioned embodiments, such as a hard disk or a memory. The computer-readable storage medium may also be an external storage device of any device with data processing capability, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit and an external storage device of any device with data processing capability. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capability, and may also be used to temporarily store data that has been output or is to be output.

[0083] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the medical image super-resolution method guided by high-frequency information based on a diffusion model.

[0084] The above embodiments are used to illustrate the present invention rather than to limit the present invention. Any modification and change made to the present invention within the spirit of the present invention and the protection scope of the claims shall fall within the protection scope of the present invention.

Claims

1. A medical image super-resolution method guided by high-frequency information based on a diffusion model, characterized in that: The method comprises the following steps: (1) Obtain medical imaging data, process it to obtain low-resolution images, and construct a data set; (2) Perform wavelet transform on the images in the dataset, concatenate the images containing high-frequency information, and predict them through the high-frequency generation network to obtain the predicted high-frequency image. Connect an image with the same shape as the image containing low-frequency information after wavelet transform, but with a value of 0. The result of the connection is subtracted from the original high-resolution image to obtain a residual image. The high-frequency generation network includes three depth-wise separable convolutions, two cross-attention layers, a progressive dilation module, and three depth-wise separable convolutions. Concatenate the images HH, LH, and HL containing high-frequency information obtained by wavelet transform and extract features through three depth-wise separable convolutions. The cross-attention layer uses the information in HH and LH to supplement the details in HL. The progressive dilation module uses dilated convolution to extract local information for local restoration. Finally, three depth-wise separable convolutions are used to reduce the number of channels to obtain the predicted result. (3) Input the residual image into the diffusion model, calculate the pixel loss with the wavelet transformed image after adding Gaussian noise, and optimize the parameters of the high-frequency generation network and the diffusion model; (4) Obtain a low-resolution image of a medical image, and obtain a high-resolution image by inputting the optimized high-frequency generation network and diffusion model through wavelet transform.

2. According to claim 1, a medical image super-resolution method guided by high-frequency information based on a diffusion model is characterized in that: In step (1), the medical imaging data includes magnetic resonance images and computed tomography images.

3. The medical image super-resolution method guided by high-frequency information based on a diffusion model according to claim 1, characterized in that: In step (1), the preprocessing of the medical image data includes adjusting the window width and window position for denoising, resampling to adjust the voxel spacing, normalizing the pixel values ​​and downsampling the original high-resolution image to obtain a low-resolution image.

4. The medical image super-resolution method guided by high-frequency information based on a diffusion model according to claim 1, characterized in that: In step (3), the residual image is used as the input of the diffusion model. First, Gaussian noise is continuously added through the forward process to obtain a completely noisy image. The four images obtained by wavelet transforming the low-resolution image are used as conditional inputs and connected to the completely noisy image through the channel dimension as the input of the generator in the diffusion module. The pixel loss of the denoised image and the residual image is calculated.

5. The medical image super-resolution method guided by high-frequency information based on a diffusion model according to claim 4, characterized in that: The generator in the diffusion module receives a noisy image sample and its time index as a source image input at time t, and outputs a denoised image sample as a target modality at time tk, which is k time points back. Based on the image sample, t different moments are set, and Gaussian noise is continuously added through the diffusion model to obtain the noisy image sample and its time index.

6. The medical image super-resolution method guided by high-frequency information based on a diffusion model according to claim 1, characterized in that: In step (4), the low-resolution image is transformed by wavelet transform to obtain a low-frequency image and three high-frequency images; the three high-frequency images are merged through channels and enter the high-frequency generation network to obtain the predicted high-frequency image; the low-frequency image and the three high-frequency images are merged through channels to obtain a wavelet domain image, the wavelet domain image and the noise image of the same size are merged through channels and enter the diffusion model to obtain the predicted wavelet domain image, and the predicted wavelet domain image and the predicted high-frequency image are added to obtain the final high-resolution image.

7. A medical image super-resolution device guided by high-frequency information based on a diffusion model, comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that: When the processor executes the executable code, a medical image super-resolution method guided by high-frequency information based on a diffusion model as described in any one of claims 1 to 6 is implemented.

8. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, a medical image super-resolution method guided by high-frequency information based on a diffusion model as described in any one of claims 1 to 6 is implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the medical image super-resolution method guided by high-frequency information based on a diffusion model as described in any one of claims 1 to 6 is implemented.

Images