Systems and methods for deblurring and denoising medical images

Abstract

This application includes systems and methods for deblurring and denoising medical images. Deblurring and denoising medical images, such as x-ray fluoroscopic images, can be challenging and can be addressed using deep learning based techniques. An artificial neural network (ANN) can be trained using training images with synthetic noise and training images with real noise. During training, parameters of the ANN can be adjusted based at least on a first loss designed to maintain continuity between successive medical images generated by the ANN and a second loss designed to maintain similarity of patches within medical images generated by the ANN. The parameters of the ANN can also be adjusted based on a third loss that can be computed from a gold standard associated with the synthetic training images. These techniques can be used to enable transfer learning between synthetic images and real images.

Description

Technical Field

[0001] This application relates to the field of medical imaging, and in particular to the denoising of medical images. Background Technology

[0002] Achieving medical image restoration involving both denoising and deblurring can be challenging because these tasks may have opposing objectives: denoising aims to suppress high-frequency components in an image (e.g., since high-frequency components may be dominated by noise), while deblurring aims to enhance high-frequency components to sharpen the image. Furthermore, real-world noisy medical images can be difficult to denoise because the noise in these images may not follow a constant distribution (e.g., the noise may vary with time and / or space). Therefore, techniques that work well for synthetic noisy medical images may not transfer well when processing images with real-world noise. Thus, systems, methods, and apparatuses capable of deblurring and denoising real-world medical images such as X-ray fluoroscopy images are desirable. Summary of the Invention

[0003] This document describes systems, methods, and apparatuses associated with deblurring and denoising medical images, such as X-ray fluoroscopy images included in X-ray fluoroscopy videos. According to one or more embodiments of this disclosure, an apparatus configured to perform a deblurring and denoising task may include at least one processor configured to: obtain a sequence of input medical images; process the sequence of input medical images through an artificial neural network (ANN); and generate a sequence of output medical images corresponding to the sequence of input medical images based on the processing, wherein the characteristics of each output medical image may be reduced blur and reduced noise compared to a corresponding input medical image. The ANN may be trained using at least a first training dataset and a second training dataset, the first training dataset comprising medical images with synthetic noise and the second training dataset comprising medical images with real noise. During training, the parameters of the ANN may be adjusted based at least on a first loss and a second loss, the first loss being designed to maintain continuity between consecutive medical images generated by the ANN, and the second loss being designed to maintain similarity between two or more blocks (e.g., assumed to be similar blocks in feature space) within a medical image generated by the ANN.

[0004] In the example, the medical images in the first training dataset described herein can be associated with corresponding deblurred and denoised gold standard images, and the ANN can be configured to predict corresponding deblurred and denoised medical images based on each medical image in the first training dataset during training, and further adjust the parameters of the ANN based on a third loss, which indicates the difference between the deblurred and denoised medical images predicted by the ANN and the corresponding deblurred and denoised gold standard images.

[0005] In the example, the ANN may include multiple cascaded subnetworks, each of which may include a deblurring module and a denoising module. Training the ANN may include: generating a first output medical image and a second output medical image using the ANN based on two consecutive medical images from a second training dataset, determining a first set of robust features for the first output medical image, determining a second set of robust features for the second output medical image, calculating a first loss (e.g., the first loss may indicate the difference between the first and second robust features) based at least on the first and second robust features, and adjusting the parameters of the ANN to reduce the first loss. In the example, the first set of robust features for the first output medical image may be determined by: extracting multiple first features from the first output medical image (e.g., using a pre-trained feature extraction neural network), adding noise to the first output medical image to obtain a first noisy output medical image, extracting multiple second features from the first noisy output medical image, and selecting features that remain substantially unchanged before and after adding noise to the first output medical image as the first set of robust features. Similarly, the second noise-resistant feature set of the second output medical image can be determined by: extracting multiple third features from the second output medical image (e.g., using a pre-trained feature extraction neural network), adding noise to the second output medical image to obtain a second noisy output medical image, extracting multiple fourth features from the second noisy output medical image, and selecting features that remain substantially unchanged before and after adding noise to the second output medical image as the second noise-resistant feature set.

[0006] In the example, the training operation of the ANN may include: generating a deblurred and denoised medical image using the ANN based on medical images from a second training dataset; determining a first set of robust features for a first block within the deblurred and denoised medical image; determining a second set of robust features for a second block within the deblurred and denoised medical image (e.g., the second block may include pixel values ​​or features similar to the first block); calculating a second loss based at least on the first and second robust features (e.g., the second loss may indicate the difference between the first and second robust features); and adjusting the parameters of the ANN to reduce the second loss. The first set of robust features for the first block may be determined by: extracting multiple first features from the first block; adding noise to the first block to obtain a first noisy block; extracting multiple second features from the first noisy block; and selecting features that remain substantially unchanged before and after adding noise to the first block as the first set of robust features. Similarly, the second noise-resistant feature set of the second block can be determined by: extracting multiple third features from the second block, adding noise to the second block to obtain a second noisy block, extracting multiple fourth features from the second block, and selecting features that remain approximately unchanged before and after adding noise to the second block as the second noise-resistant feature set.

[0007] In the example, the training operation of the ANN may include generating a first deblurred and denoised medical image using the ANN based on a first medical image from a second training dataset, wherein the first deblurred and denoised medical image can be generated by first deblurring the first medical image and then denoising it. The training operation may also include generating a second deblurred and denoised medical image using the ANN based on the first medical image from the second training dataset, wherein the second deblurred and denoised medical image can be generated by first denoising the first medical image and then deblurring it. The parameters of the ANN can then be tuned with the goal of reducing the difference between the first and second deblurred and denoised medical images. Attached Figure Description

[0008] The examples disclosed herein can be understood in more detail from the following description, which is given by way of example in conjunction with the accompanying drawings.

[0009] Figure 1 This is a diagram illustrating an example of deblurring and denoising a medical image sequence using deep learning (DL) techniques according to one or more embodiments of the present disclosure.

[0010] Figure 2 This is a diagram illustrating an example of an artificial neural network (ANN) according to one or more embodiments of the present disclosure.

[0011] Figure 3This is a diagram illustrating an example process for training an ANN to perform deblurring and denoising tasks according to one or more embodiments of the present disclosure.

[0012] Figure 4 This is a flowchart illustrating example operations that can be associated with training a neural network according to one or more embodiments of the present disclosure.

[0013] Figure 5 This is a block diagram illustrating example components of a device that can be configured to perform tasks described according to one or more embodiments of the present disclosure. Detailed Implementation

[0014] The present disclosure is illustrated by way of example rather than limitation in the figures.

[0015] Figure 1 An example of deblurring and denoising medical images using deep learning (DL) based techniques is illustrated. As shown, a sequence of medical images 102, such as an X-ray fluoroscopy image sequence (e.g., from an X-ray fluoroscopy video), can be obtained, where each medical image 102 may include a certain amount of blur and noise. This blur and noise may be undesirable because, for example, they may hinder the ability to identify small features, perhaps only a few pixels in size, for disease detection and treatment. Mathematically, the degradation process that can produce blur and noise in medical image 102 can be expressed as y = x * k + n, where x can represent a high-quality, clean image (e.g., without blur and noise), k can represent a blur kernel (e.g., such as a Gaussian kernel), n can represent the added noise, and y can represent the resulting degraded image (e.g., ...). Figure 1 (See medical image 102). Therefore, removing blur and noise from medical image 102 (e.g., deblurring and denoising) may be an ill-posed problem, as it may involve recovering a clean image x from an observed degraded image y without prior knowledge of the blur kernel k and / or noise n. Existing techniques for deblurring (e.g., removing or reducing its blur) of medical image 102 rely on estimating and calculating the point spread function of the imaging system used to capture the medical image; however, these techniques may add artifacts and / or increase noise in the medical image. Similarly, currently available denoising techniques (e.g., removing or reducing noise) work by estimating the optical flow associated with the medical image, which may require future image frames that are not available in real-time applications.

[0016] To address these issues and recover (e.g., deblurred and denoised) medical images 102 in a manner that meets actual clinical needs, deep learning-based techniques can be applied to train a machine learning (ML) model to predict (e.g., generate) a sequence of output medical images 104 that can correspond to a sequence of input medical images 102, wherein each output medical image 104 can be a deblurred and denoised version of the corresponding input medical image (e.g., the output image 104 may be characterized by reduced blur and noise compared to the input image 102). Such an ML model can be learned and implemented via an artificial neural network (ANN) 106, which can be trained using at least a first training dataset and a second training dataset to reduce blur and noise in the sequence of input medical images 102, the first training dataset including medical images with synthetic noise and the second training dataset including medical images with real noise. As will be described in more detail below, the ANN 106 can be trained to learn to transfer knowledge and / or abilities acquired from processing synthetic images to processing real images at inference time. An ANN can be trained to do this, for example, based at least on a first loss and a second loss, the first loss being designed to maintain continuity between consecutive medical images generated by the ANN, and the second loss being designed to maintain similarity between two or more blocks (e.g., blocks similar to each other in feature space) within a medical image generated by the ANN. ANN 106 can further utilize a third loss computed based on images generated by the ANN and corresponding gold-standard images to further improve the network's transfer learning capabilities.

[0017] ANN 106 can be implemented using various neural network architectures. For example, the ANN can be implemented as an expanded neural network comprising multiple subnetworks (e.g., six subnetworks), wherein each subnetwork may include a deblurring module and a denoising module, and the subnetworks can be configured to iteratively deblur and denoise the input image sequence. Figure 2 An example of such a neural network 200 (e.g., Figure 1Examples of ANNs (Analog Networks) can include multiple subnetworks 202, 204, etc. Each of these subnetworks can include a deblurring module (e.g., 202a, 204a, etc.) and a denoising module (e.g., 202b, 204b, etc.), and can be configured to receive an input image (e.g., from a medical image sequence 206 or the output of a previous subnetwork) and produce an output image (e.g., 208) characterized by reduced blur and noise compared to the input image. In the example, the deblurring module can be configured to implement programming logic for applying one or more Fast Fourier Transform (FFT) and Inverse FFT operations (e.g., utilizing a prior blur kernel such as a Gaussian kernel) to the input image, and the denoising module can include a Convolutional Neural Network (CNN) (e.g., with a U-net architecture based on residual blocks), which can be trained to extract features associated with underlying anatomical structures from the input image and predict the denoised image based on the extracted features.

[0018] The CNN described herein may include an input layer and one or more convolutional layers, pooling layers, and / or fully connected layers. The input layer may be configured to receive an input image, while each convolutional layer may include multiple convolutional kernels or filters with corresponding weights for extracting features from the input image associated with underlying anatomical structures. Following the convolutional layers may be batch normalization and / or linear or non-linear activations (e.g., such as the Corrected Linear Unit (ReLU) activation function), and the features extracted through the convolutional operations may be downsampled by one or more pooling layers to obtain a representation of the features, e.g., in the form of feature vectors or feature maps. The CNN may also include one or more up-pooling layers and one or more transposed convolutional layers. Through the up-pooling layers, the features extracted by the above operations may be upsampled, and the upsampled features may be further processed by one or more transposed convolutional layers (e.g., via multiple deconvolutional operations) to derive a magnified or dense feature map or feature vector. Then, before the denoised image is passed to the next sub-network for further deblurring and denoising, the dense feature map or vector may be used to predict the denoised image (e.g., output image 208).

[0019] It should be noted that although the term "subnetwork" is used to describe neural network 200, those skilled in the art will understand that "subnetwork" may also include components (e.g., hardware and / or software components) that are not conventionally considered as part (e.g., layers) of a neural network. Those skilled in the art will also understand that neural network 200 can be trained to perform additional functions (e.g., in addition to deblurring and denoising), including, for example, enhancing the contrast of deblurred and denoised images generated by the neural network for better visualization purposes.

[0020] Figure 3 An example is shown for training an ANN 300 (e.g., Figure 1ANN 106 or Figure 2 A neural network 200 is used to perform the deblurring and denoising operations described herein. As shown in the figure, training can use both synthetic medical images 302 and real medical images 304 and is based on multiple loss functions (e.g., L...). TC L N and / or L GT The design aims to improve training results, such as ensuring that knowledge learned from the synthetic medical image 302 can be transferred (e.g., used) to process the real medical image 304. The synthetic medical image 302 (e.g., in the first training dataset) may be generated based on a high-quality (e.g., unblurred and noise-free) gold-standard (GT) medical image 306 and includes synthetic noise added by a noise synthesis module 308, which may be part of a device configured to perform the deblurring and denoising tasks described herein. The noise synthesis module 308 may be configured to determine the distribution of noise from the real medical image 304 and generate synthetic noise with a similar distribution to be added to the synthetic medical image 302. The noise synthesis module 308 may, for example, extract the noise variance associated with the real medical image 304 as a function of the pixel intensity of the real medical image 304 (e.g., by applying a high-pass filter to the real medical image 304) and fit a power-law regression to that function to generate synthetic noise that can resemble the noise in the real medical image 304.

[0021] Although the noise generated by the noise synthesis module 308 and added to the synthetic medical image 302 can resemble the noise in the real medical image 304, the precise distribution of the real noise may be difficult to replicate because, for example, the statistics of the real noise may not follow a constant distribution, but may vary with time and space. Therefore, both the synthetic medical image 302 and the real medical image 304 can be used for training the ANN 300. For example, as Figure 3 As illustrated in the upper branch example, ANN 300 can be configured during training to receive medical images from a training dataset including synthetic medical images 302 and generate (e.g., predict) a deblurred and denoised output image 310. ANN 300 can then compare the output image 310 with a corresponding gold standard image 306 and determine a loss (e.g., L) that indicates the difference between the output image 310 predicted by the ANN and the gold standard image 306. GT Loss (L) GT The loss can be computed based on various loss functions, including, for example, loss functions based on mean squared error (MSE), loss functions based on L1 or L2 norm, etc., and the ANN 300 can be configured to adjust its parameters (e.g., the weights associated with each layer of the ANN) to minimize or reduce the loss (e.g., by backpropagating the gradient descent of the loss through the ANN).

[0022] Training the ANN 300 may also include obtaining medical images from a training dataset including real medical images 304, and processing the medical images through the ANN to obtain another deblurred and denoised output image 310 (e.g., 310 may be used herein to represent an output image generated from a synthetic medical image 302 or a real medical image 304). For example, in response to obtaining the input medical image 304, the ANN 300 may be configured to process the input medical image 304 through multiple sub-networks of the ANN (e.g., 300a, 300b, etc.) to iteratively deblur and denoise the input medical image (e.g., each sub-network of the ANN may include a deblurring module and a denoising module), thereby generating a deblurred and denoised image 312 as the output of each sub-network (e.g., 300a, 300b, etc.). When the ANN 300 processes consecutive images from the real medical image dataset 304, a loss (e.g., L...) can be calculated. TC This is used to force the ANN to adjust its parameters, enabling consistency between the ANN's successive outputs. For example, ANN 300 can process a first medical image (e.g., with a timestamp T-1) from a real medical image dataset 304 and generate a first deblurred and denoised output image I. T-1 ANN 300 can also process second medical images (e.g., with timestamp T) that are temporally consecutive to the first medical image from a real medical image dataset 304, and generate a second deblurred and denoised output image I. T Then, based on the first output image I... T-1 First noise resistance feature F T-1 Set and second output medical image I T The second noise resistance feature F T Set to calculate loss L TC The loss can indicate the first noise resistance feature F. T-1 Set and second noise resistance feature F T The differences between sets, and ANN 300 can be configured to further tune its parameters to reduce the loss. This loss (e.g., L...) TC This can help improve the training of ANN 300 because the real medical images 304 can include consecutive image sequences from medical videos (e.g., X-ray fluoroscopy videos), where it can be expected that the features of the underlying anatomical structures in the images remain consistent (e.g., with small variations) between consecutive time stamps. Therefore, if properly trained, ANN 300 can also be expected to produce consistent results between consecutive time stamps (e.g., pixels with higher feature similarity should be more similar in pixel values).

[0023] In the example, LTC It can be calculated based on the following equation:

[0024]

[0025] L TC =||S·I T -S·I T -1|| 2)

[0026] Among them, I T and I T-1 The output image generated by ANN 300 can be represented, which can be associated with timestamps T and T-1 respectively, and F T and F T-1 They can be respectively represented from I T and I T-1 Extracted noise-resistant features.

[0027] Various techniques can be used to derive robust features from the individual output images generated by the ANN 300. For example, a pre-trained feature extraction neural network (e.g., a CNN with multiple convolutional and / or pooling layers) can be used to extract multiple first features from the output image. Noise (e.g., synthetic noise) can then be added to the output image to obtain a corresponding noisy output image, and the pre-trained feature extraction neural network can again be used to extract multiple second features from the noisy output image. From the multiple first and multiple second features, features that remain substantially unchanged before and after adding noise to the output image can be selected as robust features, which can represent the inherent features of the underlying anatomical structures in the output image.

[0028] During the training of an ANN 300, an additional loss can be determined based on blocks within the output image generated by the ANN. Such a loss (e.g., Figure 3 The L shown N ) can be used with L TC A similar approach can be used to compute and enforce similarity between blocks (e.g., assuming similar blocks in the feature space). For example, based on an input image from a real medical image dataset 304, ANN 300 can generate a deblurred and denoised output image, and can be based on the first set of denoising features Fi of the first block i in the output image and the second set of denoising features F of the second block j in the output image. j Set to calculate L N The loss can indicate the first noise resistance feature F. i Set and second noise resistance feature F j The differences between the sets. Then, ANN 300 can be configured to further tune its parameters to reduce the loss L. N L NIt can be calculated, for example, based on the following equation:

[0029]

[0030] L N =||S ij ·P i -S ij ·P j || 4)

[0031] Among them, P i and P j Two blocks in the output image generated by ANN 300 can be represented, and F i and F j They can be respectively represented from P i and P j Extracted noise-resistant features.

[0032] Noise-resistant features for blocks can be extracted in a manner similar to those for continuous images described above. For example, a pre-trained feature extraction neural network (e.g., the same feature extraction network described above or a different feature extraction network) can be used to extract noise-resistant features from blocks (e.g., P). i or P j Multiple first features are extracted. Then, noise (e.g., synthetic noise) can be added to the block to obtain a corresponding noisy block, and a pre-trained feature extraction neural network can again be used to extract multiple second features from the noisy block. From the multiple first features and multiple second features, features that remain approximately unchanged before and after adding noise can be selected as noise-resistant features.

[0033] The loss described in this article (e.g., L) GT L N L TC These can be used individually or combined (e.g., combined into a single loss) to facilitate the training of an ANN 300. For example, the losses can be weighted according to the following equation and combined into a single loss L:

[0034] L = L GT +αL N +βL TC 5)

[0035] During training, the various weights of the loss (such as α, β, etc.) can be adjusted to achieve the optimal result.

[0036] By alternating the order of deblurring and denoising operations during ANN training, the ANN300's ability to deblur and denoise realistic noisy medical images can be further improved. For example, during ANN 300 training, the ANN can be configured to generate a first deblurred and denoised output image based on an input training image by first deblurring the input training image and then denoising it. The ANN can also be configured to generate a second deblurred and denoised medical image based on the same input training image by first denoising it and then deblurring it. The ANN can then be tuned to reduce the difference between the first and second deblurred and denoised images.

[0037] Figure 4 Examples are shown that can be used with training neural networks (e.g., Figure 1 ANN 106 Figure 2 Neural networks 200 and / or Figure 3 The example operation 400 is associated with training an ANN (300) to perform one or more tasks described herein. As shown, training operation 400 may include: at 402 initializing the execution parameters of the neural network (e.g., weights associated with the individual layers of the neural network), for example by sampling from a probability distribution or by copying the parameters of another neural network with a similar structure. Training operation 400 may also include: at 404 processing the input (e.g., a blurred and noisy training image) using the currently assigned parameters of the neural network; and at 406 predicting the desired result (e.g., a deblurred and denoised image). At 408, the prediction result may be compared with a gold standard (e.g., if training is supervised, such as...). Figure 3 In the case of a synthetic medical image dataset 302, the loss associated with the prediction can be determined, for example, based on a loss function (such as the mean squared error between the prediction and the gold standard). The prediction can also be used to calculate another loss (e.g., if the training is unsupervised, such as...). Figure 3 (Regarding the case of real medical image dataset 304), such as regarding Figure 3 The description of L TC and / or L N At step 410, the loss can be evaluated to determine if one or more training termination criteria are met. For example, if the loss is below a threshold or if the change in loss between two training iterations is below a threshold, the training termination criterion can be determined to be met. If the termination criterion is determined to be met at step 410, training can end; otherwise, at step 412, for example, before training returns to step 406, the currently assigned network parameters can be adjusted by backpropagating the gradient descent of the loss function through the network.

[0038] For the sake of simplicity, training operations are depicted and described in a specific order throughout this document. However, it should be understood that training operations can occur in various orders, simultaneously, and / or with other operations not presented or described herein. Furthermore, it should be noted that not all operations that may be included in the training methods are depicted and described herein, and not all exemplified operations need to be performed.

[0039] The systems, methods, and / or apparatuses described herein may be implemented using one or more processors, one or more storage devices, and / or other suitable auxiliary devices (such as display devices, communication devices, input / output devices, etc.). Figure 5 An example device 500 is illustrated that can be configured to perform the deblurring and denoising tasks described herein. As shown, device 500 may include a processor (e.g., one or more processors) 502, which may be a central processing unit (CPU), graphics processing unit (GPU), microcontroller, reduced instruction set computer (RISC) processor, application-specific integrated circuit (ASIC), application-specific instruction set processor (ASIP), physical processing unit (PPU), digital signal processor (DSP), field-programmable gate array (FPGA), or any other circuitry or processor capable of performing the functions described herein. Device 500 may also include communication circuitry 504, memory 506, mass storage device 508, input device 510, and / or communication link 512 (e.g., communication bus) through which one or more components shown in the figures exchange information.

[0040] Communication circuitry 504 can be configured to send and receive information using one or more communication protocols (e.g., TCP / IP) and one or more communication networks, including local area networks (LANs), wide area networks (WANs), the Internet, and wireless data networks (e.g., Wi-Fi, 3G, 4G / LTE, or 5G networks). Memory 506 can include a storage medium (e.g., a non-transitory storage medium) configured to store machine-readable instructions that, when executed, cause processor 502 to perform one or more functions described herein. Examples of machine-readable media can include volatile or non-volatile memory, including but not limited to semiconductor memory (e.g., electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)), flash memory, etc.). Mass storage device 508 can include one or more disks, such as one or more internal hard disks, one or more removable disks, one or more magneto-optical disks, one or more CD-ROMs or DVD-ROMs, etc., on which instructions and / or data can be stored for operation of processor 502. Input device 510 may include a keyboard, mouse, voice-controlled input device, touch-sensitive input device (e.g., touch screen), etc., for receiving user input from device 500.

[0041] It should be noted that device 500 can operate as a standalone device or can be connected to other computing devices (e.g., networked or grouped) to perform the functions described herein. And even in Figure 5 Only one example of each component is shown in the figure, and those skilled in the art will understand that the device 500 may include multiple instances of one or more components shown in the figure.

[0042] Although this disclosure has been described according to certain embodiments and generally associated methods, changes and variations of the embodiments and methods will be apparent to those skilled in the art. Therefore, the above description of exemplary embodiments does not limit this disclosure. Other changes, substitutions, and modifications are possible without departing from the spirit and scope of this disclosure. Furthermore, unless specifically stated otherwise, discussions using terms such as “analyze,” “determine,” “enable,” “identify,” and “modify” refer to the actions and processes of a computer system or similar electronic computing device that manipulate and transform data representing physical (e.g., electronic) quantities within the registers and memories of the computer system into other data representing physical quantities within the computer system's memory or other such information storage, transmission, or display devices.

[0043] It should be understood that the above description is intended to be illustrative and not restrictive. Many other embodiments will become apparent to those skilled in the art upon reading and understanding the above description. Therefore, the scope of this disclosure should be determined by reference to the appended claims and the full scope of their equivalents.

Claims

1. A method for processing medical images, the method comprising: Obtain the sequence of the input medical image; The sequence of input medical images is processed by an artificial neural network (ANN), wherein the ANN is trained to reduce both blur and noise in the sequence of input medical images; and Based on the processing, a sequence of output medical images corresponding to the sequence of input medical images is generated, wherein, Each of the output medical images is characterized by both reduced blur and reduced noise compared to a corresponding input medical image; The ANN is trained using at least a first training dataset and a second training dataset, the first training dataset comprising medical images with synthetic noise, and the second training dataset comprising medical images with real noise; and During the training, the parameters of the ANN are adjusted based on at least a first loss and a second loss, the first loss being designed to maintain continuity between consecutive medical images generated by the ANN, and the second loss being designed to maintain similarity between two or more blocks within a medical image generated by the ANN; the medical images in the first training dataset are associated with corresponding deblurred and denoised gold standard images, and wherein, during the training of the ANN, the ANN is configured to predict corresponding deblurred and denoised medical images based on each of the medical images in the first training dataset, and the parameters of the ANN are further adjusted based on a third loss, the third loss indicating the difference between the deblurred and denoised medical images predicted by the ANN and the corresponding deblurred and denoised gold standard images.

2. The method according to claim 1, wherein, The training of the ANN includes: The ANN is used to generate a first output medical image and a second output medical image based on two consecutive medical images from the second training dataset. Determine the first set of noise-resistant features for the first output medical image; Determine the second noise-resistant feature set of the second output medical image; The first loss is calculated based at least on the first noise-resistant feature set and the second noise-resistant feature set, wherein the first loss indicates the difference between the first noise-resistant feature set and the second noise-resistant feature set; and Adjust the parameters of the ANN to reduce the first loss.

3. The method according to claim 2, wherein, The first set of noise-resistant features for determining the first output medical image includes: Extract multiple first features from the first output medical image; Add noise to the first output medical image to obtain a first noisy output medical image; Extract multiple second features from the first noisy output medical image; and The features that remain substantially unchanged before and after the noise is added to the first output medical image are selected as the first set of noise-reducing features.

4. The method according to claim 3, wherein, The plurality of first features and the plurality of second features are extracted using a pre-trained feature extraction neural network.

5. The method according to claim 1, wherein, The training of the ANN includes: The ANN is used to generate deblurred and denoised medical images based on medical images from the second training dataset; Determine a first set of noise-resistant features for a first block within the deblurred and denoised medical image; Determine the second set of noise-resistant features for the second block within the deblurred and denoised medical image; The second loss is calculated based at least on the first noise-resistant feature set and the second noise-resistant feature set, wherein the second loss indicates the difference between the first noise-resistant feature set and the second noise-resistant feature set; and Adjust the parameters of the ANN to reduce the second loss.

6. The method according to claim 1, wherein, The training of the ANN includes: The ANN is used to generate a first deblurred and denoised medical image based on a first medical image in the second training dataset, wherein the first deblurred and denoised medical image is generated by first deblurring the first medical image and then denoising the first medical image. The ANN generates a second deblurred and denoised medical image based on the first medical image in the second training dataset, wherein the second deblurred and denoised medical image is generated by first denoising the first medical image and then deblurring it; and The parameters of the ANN are adjusted to reduce the difference between the first deblurred and denoised medical image and the second deblurred and denoised medical image.

7. The method according to claim 1, wherein, The ANN includes multiple serially coupled subnetworks, and each of the serially coupled subnetworks includes a deblurring module and a denoising module.

8. The method according to claim 1, wherein, The sequence of input medical images was obtained from X-ray fluoroscopy videos.

9. A computer program product comprising instructions that, when run on a computer, cause the computer to perform the method as described in any one of claims 1-8.

Images