Method and apparatus for performing diffusion-based image restoration using stratified sampling step sharing

Abstract

A method and apparatus for reducing the processing time for image restoration while maintaining the quality of a restored image generated from an input image. [Solution] The method involves obtaining a diffusion-based probabilistic model trained to perform image restoration over T steps, dividing the initial T1 steps into multiple layers, grouping multiple input images into several groups for each layer, determining an initial representative image for each group, and using the obtained model to perform a sequence of sampling steps starting from the initial representative image of the group to generate a corresponding sequence of representative images for the group, and for each group in the last layer, for each input image in the group, T=T1+T2, and using the obtained model to perform a second sequence of T2 sampling steps starting from the last one of the sequence of representative images for the group to generate a final image.

Description

[Technical Field]

[0001] This application relates to the restoration of a sequence of images. [Background technology]

[0002] Deep learning-based image restoration models typically use a series of steps to modify an input image to a desired degree. Once a sequence of images is acquired, the deep learning model can process the sequence by repeating the series of steps for each image in the sequence. The number of steps used to process the sequence can affect the processing time and capacity required by the deep learning model.

[0003] It is desirable to reduce the processing time for image restoration while maintaining the quality of the restored image.

[0004] The foregoing "Background" discussion is intended to generally present the context of this application. The inventors' work to the extent described in this Background section, as well as aspects of the description that may not be admitted as prior art at the time of filing, are not admitted expressly or impliedly as prior art to this application. Summary of the Invention [Means for solving the problem]

[0005] In one embodiment, the present application relates to a method for performing diffusion-based image restoration on a plurality of input images, the method including: obtaining a diffusion-based probabilistic model trained to perform image restoration over T steps, where T is an integer greater than or equal to three; dividing the initial T1 steps into a plurality of layers, where T1 is an integer greater than or equal to two; for each layer of the plurality of layers, grouping the plurality of input images into a number of groups; determining an initial representative image for each group of the groups; and performing, for each group, a sequence of sampling steps starting from the initial representative image of the group using the obtained model to generate a corresponding sequence of representative images of the group; and for each group in a last layer of the plurality of layers, for each input image in the group, where T2 is an integer greater than or equal to one, where T=T1+T2, to generate a final image, the restored image corresponding to the input image.

[0006] In one embodiment, the present application relates to an apparatus for performing diffusion-based image restoration on a plurality of input images, including a processing circuit configured to: obtain a diffusion-based probabilistic model trained to perform image restoration over T steps, where T is an integer greater than or equal to three; divide the initial T steps into a plurality of layers, where T is an integer greater than or equal to two; group the plurality of input images into a number of groups; determine an initial representative image for each group of the groups; and perform, for each group, a sequence of sampling steps starting from the initial representative image of the group using the obtained model to generate a corresponding sequence of representative images for the group; and, for each group in a last layer of the plurality of groups in the plurality of layers, perform, for each input image in the group, a second sequence of T sampling steps starting from the last one of the sequence of representative images of the group using the obtained model, where T is an integer greater than or equal to one, where T=T+T, to generate a final image, the restored image corresponding to the input image.

[0007] In one embodiment, the present application relates to a non-transitory computer-readable storage medium for storing computer-readable instructions that, when executed by a computer, cause the computer to perform the above-described method of performing diffusion-based image restoration on a plurality of input images. [Brief explanation of the drawings]

[0008] A more complete appreciation of the present invention and many of its attendant advantages will be readily obtained as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings, in which: [Figure 1A] It shows an independent sampling process performed on a series of images acquired sequentially across the time or space dimension. [Figure 1B]Figure 1 shows a schematic diagram of the process for denoising a set of N images (Z0, Z1, Z2, Z3, Z4, Z5, ... ZN) via trained DDPM. [Figure 1C] We show a sharing mechanism performed on a series of images acquired sequentially across time or spatial dimensions, where multiple (e.g., four) groups are implemented for sampling step sharing, followed by independent processing. [Figure 1D] Figure 1 shows a schematic of the process for denoising a set of N images (Z0, Z1, Z2, Z3, Z4, Z5, … ZN) via trained DDPM using a shared initial diffusion step. [Figure 2] 1 illustrates a layered sharing mechanism performed on a series of images acquired sequentially across a time or space dimension, according to an embodiment of the present application. [Figure 3] FIG. 1 shows a schematic diagram of a bulk diffusion-based image restoration workflow according to one embodiment of the present disclosure. [Figure 4] According to one embodiment of the present application, it is shown that the condition image is applied according to the layer and the number of groups, or applied individually. [Figure 5] 1 illustrates a DDPM restoration procedure for a four-phase scan using a multi-phase stratified sampling step-sharing mechanism according to an embodiment of the present application. [Figure 6] 1 illustrates an exemplary stratified sampling step shared restoration workflow according to an embodiment of the present application. [Figure 7] 10 shows a comparison of DDPM recovery results between regular DDPM without sampling step sharing, sampling step sharing applied to each individual phase, and multi-phase stratified sampling step sharing, according to one embodiment of the present application. [Figure 8] Using regular DDPM as a comparison baseline, we present a quantitative comparison of DDPM reconstruction results between sampling step sharing applied to each individual phase and multi-phase stratified sampling step sharing. [Figure 9]A comparison is shown between the regular DDPM, the DDPM with sampling step sharing using the hierarchical sharing approach, and the DDPM with sampling step sharing without the hierarchical sharing approach. [Figure 10] 1 is a schematic diagram of a hardware system for performing a method according to an embodiment of the present application; [Figure 11] FIG. 1 is a schematic diagram of an imaging system according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0009] Referring now to the drawings, like reference numerals designate identical or corresponding parts throughout the several views.

[0010] As used herein, the terms "a" or "an" are defined as one or more than one. As used herein, the term "plurality" is defined as two or more than two. As used herein, the term "another" is defined as at least a second or more. As used herein, the terms "including" and / or "having" are defined as comprising (i.e., open language). Throughout this specification, references to "one embodiment," "particular implementation," "embodiment," "implementation," "example," or similar terms mean that a particular feature, structure, or characteristic described in connection with this embodiment is included in at least one embodiment of the present application. Thus, the appearances of such phrases or in various places throughout the specification are not necessarily all referring to the same embodiment. Furthermore, particular characteristics, structures, or characteristics may be combined in any suitable manner in one or more embodiments without limitation.

[0011] The present application relates to systems and methods for image restoration using deep learning-based models. Image restoration techniques include, but are not limited to, noise removal, deblurring, resolution enhancement (e.g., super-resolution imaging), and image / signal reconstruction (e.g., compressed sensing). Each of these techniques can be used independently or in combination to improve the visibility of features in an image. Image restoration has important applications in medical imaging modalities such as computed tomography (CT) scanning and magnetic resonance imaging (MRI), which are often affected by noise due to physical interactions within the imaging system. It will be appreciated that the systems and methods described herein are not limited to medical imaging applications and can be used for a variety of imaging types and techniques. In particular, the methods of the present application are useful for processing any volumetric image data (e.g., a series of images or image slices) acquired over a spatial or temporal span.

[0012] Generative deep learning-based models can be used to degrade noise and similar artifacts in acquired images and generate restored images of higher quality than the acquired images. For example, a generative model can be used to denoise an image by transforming a first data distribution (noisy image data) into a second data distribution (restored image data). For example, such a generative model can be a denoising diffusion probabilistic model (DDPM). DDPM is described herein as an illustrative example of a class of generative models, and it can be understood that other types of probabilistic models, particularly diffusion-based probabilistic models for image restoration, are also compatible with the methods of the present application.

[0013] Typically, DDPM can be used to denoise images in a series of diffusion steps. DDPM can be trained to denoise images in an iterative process, with each diffusion step producing an increasingly denoised image. DDPM can be trained to denoise images by transforming a first probability distribution corresponding to an input image (e.g., a noisy image) into a second probability distribution corresponding to an output image (e.g., a denoised image). For example, the first probability distribution can be a normal distribution corresponding to normal (Gaussian) noise present in the acquired image. DDPM can be trained to denoise images by transforming the normal probability distribution into a predictive distribution corresponding to a denoised and restored image.

[0014] DDPM can be trained using a set or sequence of training images. The set of training images can include a target image, which can be a clean or denoised image, and noisy images generated from the target image. The noisy training images can be generated by applying modeled noise (e.g., Gaussian noise) to the target image in one or more steps. The set of training images can include images with increasing amounts of noise. The set of training images can further include purely noisy images generated from the target image. The modeled noise can be similar to or based on the type of noise expected in the acquired images that DDPM will be used to restore. The target images can be similar to or based on the type of images that DDPM will be used to recover. Each training image can be input to DDPM. DDPM can be trained to denoise the input training images and output a restored image at each step in a series of diffusion steps. The series of diffusion steps can correspond to one or more steps used to apply noise to the target image. In this way, DDPM can be trained to "reverse" the step-by-step process of applying noise to an image to remove noise from the image.

[0015] At each diffusion step, the DDPM can be trained to minimize a loss function, where the loss function corresponds to the difference between the predicted output image for a given diffusion step and the noise in the training images. Thus, the DDPM can be trained to accurately predict and model the difference between the noise in each input image and the output image at each diffusion step. Training the DDPM can include setting one or more weights of the model. The one or more weights of the model can be varied for each diffusion step in a series of diffusion steps or for at least one diffusion step in the series. In one embodiment, conditioned images can be input to the DDPM during the training process to guide the generation of the output image. For example, the conditioned images can be target images. The target images used to train the DDPM can be at least one target image or can include multiple target images. For example, the at least one target image can include a low-resolution medical image (e.g., a CT image) and an edge-detected medical image (e.g., a CT image) or an otherwise processed medical image. Similarly, the conditioned images used to train the DDPM can be at least one conditioned image or can include multiple conditioned images. In one example, the at least one conditioned image can include a low-resolution medical image (e.g., a CT image) and an edge-detected or otherwise processed medical image (e.g., a CT image). In one example, the at least one conditioned image or the at least one target image can include three consecutive conditioned images for a multidimensional (e.g., 2.5D) process.

[0016] The inputs to the trained DDPM can be a noisy image, a conditioned image, and a diffusion step (also referred to as a time step or sampling step). The DDPM can predict a second probability distribution corresponding to the restored image using the conditioned image as a known condition. The DDPM can denoise a pure noise image in a series of diffusion steps to generate a final restored image. The pure noise image can be an initial input image to the DDPM. The DDPM can output a denoised image (also referred to herein as a restored image) for each diffusion step. The denoised image output from each diffusion step in the series of diffusion steps can be input to the next diffusion step to iteratively denoise the pure noise image. The DDPM can include one or more learned weights used to output the restored image, and the values ​​of the one or more learned weights can depend on the diffusion step. Details regarding the training and use of DDPM can be found in Ho, J. et. al, (2020) "Denoising diffusion probabilistic models." Advances in neural information processing systems, 33, 6840-6851, and Xia, W. et. al, (2022) "Low-Dose CT Using Denoising Diffusion Probabilistic Model for 20x times Speedup." arXiv preprint arXiv:2209.15136, each of which is incorporated herein by reference in its entirety for all purposes.

[0017] DDPM can be used to denoise a series of images. The series can be acquired sequentially across time or space. For example, a renal scan can acquire a series of images over time to assess kidney function. In the denoising process (also known as the inference or sampling process), DDPM can denoise in an initial denoising (diffusion) step to identify larger or more generalized features. These larger features are typically consistent across a series of images. For example, the general shape and location of the kidney and its structures can be initially identified in the scan images and are unlikely to change within a single renal scan. DDPM can then denoise and identify smaller features or details in a later diffusion step. In the example of a renal scan, DDPM can identify the shape and size of renal structures and finer details of the location of contrast agents within the kidney. These details may change across a series of images as the renal system processes fluid in the body. Changes across a series of images are likely gradual and continuous over time. Therefore, adjacent images in a series may be similar to each other.

[0018] In another example, a series of images can be acquired by scanning one or more cross sections of the body along one or more directions. In a similar manner, DDPM can first remove noise and identify larger features, such as the general shape of the cross section of the body and the organs within it. DDPM can then remove noise and identify smaller features and / or finer details of the organs. Adjacent images in the series also depict parts of the body that are close to each other. Therefore, as the scan progresses along the body, adjacent images in the series are also likely to be similar to each other and share major features.

[0019] In another example, a series of images can be acquired sequentially while the contrast agent is processed within the patient's body. Multiphase imaging, which can be used in CT and MRI, is a technique that involves acquiring scans at different time points after injecting an intravenous contrast agent. Multiphase imaging can be performed to optimize the visualization of different structures or lesions with different contrast enhancement patterns. Multiphase imaging can be useful for detecting and characterizing vascular lesions, tumors, ischemia, inflammation, or trauma in various organs. By comparing images acquired in different phases, such as the non-contrast phase, arterial phase, (portal) venous phase, and / or delayed phase, a radiologist or similar operator can evaluate blood flow, perfusion, and contrast agent excretion in tissues of interest. For multiphase imaging datasets, which typically contain three to four times the total image volume compared to single-phase imaging, the total processing time for diffusion-based reconstruction is significantly longer.

[0020] Figure 1A shows 24 independent diffusion processes performed on a series of 24 images acquired sequentially across the time or spatial dimension using the regular DDPM method. Figure 1B shows the results of the trained DDPM performed on a series of N images (Z0, Z1, Z2, Z3, Z4, Z5, … Z). N ) is a schematic diagram of a typical process for denoising a sequence of N images. A sequence of N images can be collected over a period of time or along a spatial direction. For each image in the sequence, a pure noise image can be input to the DDPM. The pure noise image can be generated using a probability distribution model such as a Gaussian distribution. A conditioned image can also be input to the DDPM for denoising processing. For example, a conditioned image can be the image to be denoised (a sequence of images (Z0, Z1, Z2, Z3, Z4, Z5, ... Z N The conditioned image may be one of the following: A conditioned image may include multiple conditioned images, such as edge-detected or other processed images.

[0021] DDPM calculates the diffusion time for each image (Z0, Z1, Z2, Z3, Z4, Z5, … Z) in a series of diffusion steps. N ) can be used to denoise the pure noise image x in a series of T diffusion steps. T and for each image in the sequence, we denoise the restored image x T-1 , x T-2 At each diffusion step, the restored image from the previous diffusion step can be input to the DDPM along with the conditioned image and time step (T-1, T-2, etc.). For example, the DDPM generates a pure noise image x based on the conditioned image Z0. T and restore the image x at time step T-1. T-1 DDPM then generates the image x based on the conditional image Z0. T-1 and further restored image x at time step T-2. T-2 At time step T, DDPM can output an image x0, which is a denoised version of the conditional image Z0. Each image (Z0, Z1, Z2, Z3, Z4, Z5, ... Z N After T time steps for x0, x1, x2, x3, x4, x5, … x N ) can be output. In this way, DDPM performs N*T diffusion steps to denoise all images in the sequence.

[0022] DDPM restores images through a stepwise de-diffusion process, which requires repeatedly applying a neural network trained over hundreds to thousands of diffusion steps, making it significantly slower than supervised learning. To address this issue, diffusion acceleration techniques have been proposed, such as denoising diffusion implicit models, early stopping, and latent space diffusion. However, these techniques typically focus on accelerating inference for a single instance.

[0023] For bulk processing of large volumes of images, such as volumetric CT scans, U.S. Application No. 18 / 403,170 proposes an early "sampling step sharing" processing mechanism for continuous DDPM reconstruction workflows. The proposed method can significantly reduce the processing time of, for example, a whole-body CT scan by over 90%. This workflow acceleration approach exploits the inherent characteristics of the data, resulting in potential performance improvements.

[0024] Specific concepts and ideas from U.S. Application No. 18 / 403,170 ("the '170 Application") are illustrated in Figures 1C-1D. As an example, to accelerate processing flow, the '170 Application utilizes a sharing technique (grouping) along z. For example, as shown in Figure 1C, four groups are implemented for sampling step sharing, followed by 24 independent processes. The processing mechanism of the '170 Application can exploit similarities between adjacent images in a sequence of images to achieve faster denoising of the sequence of images using a diffusion-based probabilistic model. The method can reduce the number of diffusion steps required to denoise a sequence of images while maintaining inference accuracy. Reducing the number of diffusion steps can result in faster denoising as well as reduced computational power usage.

[0025] According to the '170 application, the method can include grouping images in a sequence and performing initial batch denoising on the image groups using the trained DDPM. An image group can be a subset of adjacent images in a sequence of images collected over a period of time or across a spatial direction. The sequence of images can be divided into one or more groups, each of which can contain one or more images. Each of the one or more image groups can have the same or different number of images. For example, a first image group G0 can include the first n images in the sequence, a second image group G1 can include the subsequent n+1 through n+m images in the sequence, and so on. The number of images in a group can be referred to as the group thickness. The group thickness can be adjusted based on the type of image acquisition or the subject being imaged in the sequence. For example, the number of images in a group can be set so that one or more features or feature types (e.g., features of a certain size) are constant in each image in the group.

[0026] According to the '170 application, the method can include determining a representative image for a group of n images. In one example, the representative image can be generated by calculating the average value of each pixel across the n images. In one embodiment, the representative image can be an image of the group of images (e.g., the first image, the nth image, the n / 2th image). In one embodiment, the representative image can be a preprocessed image, such as an image processed and weighted in the frequency domain. The representative image can be generated by any combination of image calculations and processing, and is not limited to the examples provided herein. The representative image can be input to DDPM as a conditioned image at each diffusion step of a series of initial diffusion steps. The number of initial diffusion steps can be represented by the quantity T1. DDPM can denoise a pure noise image over T1 initial diffusion steps using the representative image as a conditioned image. Features identified by DDPM in the initial diffusion steps are likely to be consistent across each image in the group of images. Therefore, denoising conditioned on the representative image, rather than on each image in the group, is sufficient for every image in the group at the initial diffusion step. The number of initial diffusion steps in which representative images are used as conditioned images can be adjusted based on the total number of diffusion steps (T), the type of scan, the features expected in the series of images, etc. For example, the number of initial diffusion steps can be set so that DDPM can identify features present in each image in the group of images within the initial diffusion steps.

[0027] According to the '170 application, DDPM can output a representative reconstructed image after the final (T1) step in the series of initial diffusion steps. The representative reconstructed image can be generated by DDPM from a pure noise image using a representative image of the group of images as a conditional image. In one embodiment, the representative reconstructed image can include one or more features shared across each image in the group of images. After the T1 initial diffusion steps, the appearance of each image in the group of images may diverge. For example, the appearance of finer details and smaller features may differ in each image in the group of images. These features may be indistinguishable by DDPM until the T1 initial diffusion steps are complete. The T1 initial diffusion steps can denoise larger, less detailed features common to the series of images, resulting in a visible representative reconstructed image. Thus, the T1 initial diffusion steps can be shared among each image in the group of images and do not need to be repeated by DDPM for each image.

[0028] According to the '170 application, the representative reconstructed image generated from the T1 initial diffusion steps can be used as an input image for a further diffusion step conditioned on each image in the group of images. The DDPM can denoise the representative reconstructed image using each image in the group of n images as a conditioned image to generate a reconstructed image for each image in the group of n images. The denoising of the representative reconstructed image conditioned on each image can be performed over T2 diffusion steps. The DDPM can then output n final images corresponding to the n images in the group. In this manner, the DDPM can reduce the number of diffusion steps required to denoise each image in the group by performing a single series of T1 initial diffusion steps using the representative image as a conditioned image to generate a reconstructed representative image. The DDPM can then denoise the reconstructed representative image over a series of n T2 diffusion steps to generate n output images. This method eliminates the need to repeat the initial diffusion steps for each of the n images in the group of images. The remaining diffusion steps can be used to identify and denoise features that are unique to each image in the group of images or that are distinct within the image.

[0029] FIG. 1D is a schematic diagram of a bulk diffusion method according to one embodiment of the '170 application. The trained DDPM is generated from a sequence of N images (Z0, Z1, Z2, Z3, Z4, Z5, ... Z N ) can be used to denoise the image. The series of N images can be collected over a period of time or along a spatial direction. In one embodiment, the series of N images is divided into one or more groups (G0, G1...G N ), with each group containing a subset of N images. For example, as shown in FIG. 1D, group G0 can contain images Z0 through Z5, and group G1 can contain images Z6 through Z7. 11It can include etc. Taking group G0 as an example, the representative image Z (bar symbol) can be input into the DDPM as a conditional image. The representative image Z (bar symbol) can be, for example, the average value of images Z0 to Z5. The pure noise image x (bar symbol) can be input into the DDPM. The DDPM can be trained to denoise images in a series of T diffusion steps. The DDPM denoises the pure noise image x (bar symbol) based on the conditional image Z (bar symbol) in a series of T1 initial diffusion steps, where T1 < T, and the restored representative image x T-1 (bar symbol), x T-2 (bar symbol) and the like can be generated. At each diffusion step, the representative image restored from the previous diffusion step can be input into the DDPM together with the conditional image Z (bar symbol) and the time step (T - 1, T = 2, etc.). For example, the DDPM denoises the pure noise image x (bar symbol) based on the conditional image Z (bar symbol), and the representative image x T-1 (bar symbol) can be generated. Then, the DDPM denoises the image x T-1 (bar symbol) based on the conditional image Z (bar symbol), and the further restored (denoised) representative image x T-2 (bar symbol) can be generated.

[0030] After T1 diffusion steps, the DDPM can output the representative restored image x T (bar symbol). The representative restored image x T (bar symbol) can be the last image in the sequence of representative images generated by the DDPM in T1 diffusion steps. As shown in Figure 1D, the DDPM can then use each of the images G0 of the group as a conditional image to denoise the representative restored image x T (bar symbol). For example, the DDPM uses image Z0 as a conditional image in a series of subsequent diffusion steps to denoise the representative restored image x T(bar symbol) can be denoised to output a reconstructed image x0 corresponding to the original image Z0. DDPM then uses image Z1 as a conditioned image in a series of subsequent diffusion steps to produce a representative reconstructed image x T (bar symbol) can be denoised to output a reconstructed image x1 corresponding to the original image Z1. DDPM then uses image Z2 as a conditioned image in a series of subsequent diffusion steps to produce a representative reconstructed image x T (bar symbol) and output a reconstructed image x2 corresponding to the original image Z2. In one embodiment, DDPM generates a representative reconstructed image x2 in t subsequent diffusion steps, as shown in FIG. 1D. T (bar symbol), where t+T1=T total diffusion steps. The t subsequent diffusion steps may be referred to herein as quantity T2. In this manner, DDPM can denoise each image in a group of N images using T1+T2*N diffusion steps, where T1+T2=T. This method reduces the number of diffusion steps required compared to the method shown in FIG. 1B, which uses N*T steps to diffuse a series of N images.

[0031] The process of generating a representative reconstructed image for a group of images and denoising the representative reconstructed image conditioned on each image in the group of images can be repeated for each group in the series of image groups. Each group can include the same or a different number of images. The number of initial diffusion steps T1 and the number of subsequent diffusion steps T2 can vary from group to group or can be the same for each group. In one embodiment, the number of initial diffusion steps T1 can be referred to as the length of the initial diffusion process using the representative image. The length of the initial diffusion process can be adjusted based on the expected content of the images in the group. For example, the length of the initial diffusion process can be adjusted based on the expected size of one or more features. In one embodiment, the length of the initial diffusion process and / or the total number of diffusion steps ( ) can depend on the training of the DDPM.

[0032] The '170 application utilizes a one-dimensional sharing technique (grouping) along z to reduce the processing time of the diffusion-based method. The present application provides an adaptive grouping strategy along t that can further accelerate the processing flow without compromising image quality. Figure 2 shows a three-tiered sampling step sharing strategy along t to accelerate the diffusion-based restoration workflow according to an embodiment of the present application. In the example shown in Figure 2, the first tier is one group (G 0,0 ), and the second level contains two groups (G 1,0 and G 1,1 ), and the third level includes four groups (G 2,0 , G 2,1 , G 2,2 , and G 2,3 ) A person skilled in the art will recognize that other grouping configurations can be used.

[0033] 3 is a schematic diagram of a bulk diffusion-based image restoration workflow according to one embodiment of the present application. Layer 0 contains one image x G(0.0) Starting from (bar symbol), it is gradually restored from T to t0'. In the second layer, layer 1, two groups are G(1,0) (bar symbol) and x G(1,1) (bar symbol), and are gradually restored to sampling step t'1. The third stratum, stratum 2, contains four groups, x G(2.0) (bar symbol)~x G(2.3) (Bar symbol) Starting separately from ~, they are gradually restored to sampling step t'2. Thus, the final layer, layer M, contains N groups, x G(M.0) (bar symbol)~x G(M.N) (Bar symbol) Starting separately from ~, gradually increasing the sampling step t M After the last layer, the I images in each of the N groups are restored to t M '-1 to t=0 separately. Optionally, the conditional image can be G(M.N) (bar symbol) according to the hierarchy and number of groups, or y n·I+ican be applied individually as follows:

[0034] In Figures 3 and 4, G represents the image group used for early inverse sampling sharing. The groups are indexed by (m, n), where m is the hierarchical index with M being the maximum hierarchical number, and n is the grouped image index with N being the maximum number of groups in the hierarchical level. i represents the image index within the group, and I represents the total number of images in the group, which may be the same or different between groups. x represents the processed (noisy) image, x (bar symbol) represents a representative (e.g., averaged) image in the group, and y and y (bar symbol) represent the condition inputs. t is the sampling step index, and T is its maximum number. t m ' is the minimum index of the sampling step in stratum m.

[0035] Additionally, three parameters can be adjusted to balance processing time and image quality at different levels.

[0036] (1) Number of layers: This parameter determines the number of levels (layers) in a hierarchical processing workflow.

[0037] (2) Number of groups in each layer. This parameter determines the number of groups that share steps in a layer.

[0038] (3) Sampling length at each stratum: This parameter determines the minimum sampling step point within a stratum.

[0039] By taking advantage of the continuous nature of medical imaging stacks, we can assume that the noisy (early sampling steps) low-detail intermediate images of neighboring images are very similar. Therefore, by adjusting the number of layers, the number of groups in each layer, and the sampling length of each layer, we can significantly reduce processing time without significantly compromising the data integrity.

[0040] In one example, the number of layers is 4, the number of groups in the layers is 1, 2, 4, and 8, and the sampling lengths within the layers are 170, 10, 10, and 5. Considering an image stack of 1022 slices processed using a DDPM model with a sampling step of T = 200, this layered sampling step sharing approach results in 1 × 170 + 2 × 10 + 4 × 10 + 8 × 5 + 1022 × 5 = 5.38k inference steps.

[0041] In contrast, typical 2D slice-by-slice processing requires 1022 × 200 = 204.4k inference steps. When the sampling step sharing provided in the '170 application is performed with n = 8 groups and t' = 5, there are 8 × 195 + 1022 × 5 = 6.67k inference steps. The proposed layered sharing mechanism provides an additional acceleration of nearly 20% compared to the '170 application without significantly sacrificing reconstruction quality. Because the total processing steps are only five or six times longer than traditional 2D neural network approaches, this layered early information sharing strategy can substantially increase the practicality of diffusion-based approaches in routine imaging routines.

[0042] Further details and advantages of the hierarchical sampling step sharing strategy provided in this application can be seen in the examples shown in Figures 5-9. Figure 5 shows a high-level view of the proposed hierarchical sampling step sharing strategy. The sharing sampling step can be performed using a phase-representative slice as a guide, which can be the average image of the selected phase or the individual phase containing the most advanced anatomical information. Because the intermediate output of DDPM at earlier time steps (larger t) reveals coarser features, such as the rough contours of the volume (an example is shown on the left side of Figure 5), these time steps can be shared among slices throughout the volume. As more detailed structures, such as organ boundaries, are recovered, a new sharing layer with more sharing groups begins, each containing fewer steps. In the final stage of processing, the intermediate results from the latest sharing layer are distributed to individual slices from all phases within each sharing group, and the remaining sampling steps are completed for each slice.

[0043] In the example of the tiered sampling step sharing restoration workflow shown in Figure 6, the input is a four-phase scan (non-contrast, arterial, venous, and delayed), with each phase containing 1200 slices. In addition to the four phases (N, A, V, and D), an additional phase, all, represents the average image for each slice. The sharing configuration includes four sharing layers. Layer 1 shares across the entire volume (slices 1–1200) and covers sampling steps t = 200 to t = 100. Layer 2 divides slices into two groups (slices 1–600 and slices 601–1200) and covers sampling steps t = 99 to t = 50. Layer 3 divides slices into four groups (slices 1–300, 301–600, 601–900, and 901–1200) and covers sampling steps t = 49 to t = 10. Layer 4 further divided the slices into eight groups (slices 1–150, 151–300, …, 1051–1200) and covered sampling steps t = 9 to t = 5. Finally, sampling steps t = 4 to t = 1 were performed on individual slices without sharing.

[0044] According to the multi-phase stratified sampling step-sharing strategy shown in Figure 6, the inference uses a total of 200 time steps. m、g t (Bar symbol) is the intermediate result of the shared group g of layer m at sampling step t, which is shared across all phases. For simplicity, in this example, each shared layer divides the slices equally into groups. The first shared layer consists of only one group, and the guide image is obtained by averaging all slices from all four phases. t1 ‘ =100 is the sharing stopping point of sharing layer 1, and the shared intermediate result x 1,1 100 (bar symbol). Then, a second sharing hierarchy is started with two sharing groups. The first group consists of slices 1 to 600, and the second group consists of slices 601 to 1200. Both groups have x 1,1 100 (bar symbol) as the initial input. This step generates two shared intermediate results x 1,1 50 (bar symbol) and x 2,2 50 The next two shared layers operate similarly. Layer 3 covers sampling steps 49 through 10, with four shared groups of 300 slices each, and produces four intermediate results x 3,1 10 (bar symbol) to x 3,4 10 The final sharing stage involves sampling steps 9 to 5, where the phase representative images are divided into eight groups to generate eight shared intermediate results x 8,1 5 (bar symbol) to x 8,8 5A final sampling step can be performed on individual slices from each phase, with slices from all four phases in each of the eight groups sharing the intermediate results from sampling step 5 as input to sampling step 4. This sharing configuration requires 100 + 2 × 50 + 4 × 40 + 8 × 5 + 5 × 4 × 1200 = 24,400 forward processing passes of the model for a four-phase scan of n slices each, which is significantly faster than the 960,000 time steps required by a traditional DDPM processing pipeline for the same data.

[0045] Figure 7 shows a qualitative comparison of the restoration results between conventional DDPM restoration, the sampling step sharing described in the '170 application, and the layered sampling step sharing provided in this application. 872 slices are processed for each phase. The total number of inference sampling steps is 200. Conventional processing requires 4 × 200 × 872 = 697,600. In the conventional sampling step sharing process, each of the four phases is divided into eight groups. Sampling steps 200 to 5 are performed on the average image of each phase in each group. Conventional sharing requires 4 × 8 × 195 + 4 × 872 × 5 = 23,680 time steps, which is approximately 30 times faster than conventional processing. In the layered processing, a total of four sharing layers are used, with g1, g2, g3, g4 = 1, 2, 4, 8, and t1', t2', t3', t4' = 30, 20, 10, 5. The layered processing required a total of 170 + 2 × 10 + 4 × 10 + 5 × 8 + 4 × 872 × 5 = 17,710 processing steps, which is approximately 40 times faster than conventional processing and approximately 25% faster than normal sharing. Despite the significant reduction in processing time, the two sharing processing mechanisms maintained the quality of image restoration, with appropriate improvements in noise levels observed in all phases. The restoration process improved the clarity of intravascular contrast visualization, particularly in the arterial and venous phases.

[0046] Figure 8 shows a quantitative comparison of the restoration results between the conventional sharing mechanism and the proposed layered sharing approach. The restored image quality is measured by the structural similarity index (SSIM) between each slice generated by the sharing approach and the conventional DDPM process without sharing. The restoration quality tends to degrade most at the boundaries of the sharing group (compared to no sharing) because the boundary slices have low similarity to the shared image. The layered sharing mechanism further exacerbated the degradation in restoration fidelity. However, for most non-border slices, layered sharing achieved comparable performance compared to conventional sharing. Note that due to the random noise sampling behavior of DDPM, each iteration of the same DDPM process has slight inherent differences, resulting in an SSIM of ∼0.995 between restored images. The SSIM value of 0.995 is indicated using a dashed line to indicate a high visual similarity threshold, which is exceeded by the layered sharing approach in all phases for all slices.

[0047] Figure 9 shows a comparison of slice 164 in phase 1 (circled in Figure 8) reconstructed using regular DDPM, DDPM with regular sharing within each phase, and DDPM with layered multiphase sharing. While layered multiphase sharing resulted in a lower SSIM for this slice, a visual comparison in Figure 9 does not reveal any significant differences in the recovered features. Overall, the proposed layered sampling step sharing mechanism achieved high-fidelity reconstruction of multiphase CT scans with significant acceleration.

[0048] In the examples shown in Figures 8 and 9, the average image of all phases is used as the phase-representative image used in the shared phase. However, it is also possible to use a single phase that contains the most anatomical information or has the best image quality. For different multi-phase acquisition protocols, the method for obtaining phase-representative images can be adjusted accordingly for optimal performance.

[0049] In the results presented above, the shared parameters were empirically selected for a specific test dataset. Alternatively, for clinical applications, an automated approach for shared parameter selection can be developed by considering the impact of each parameter on the amount of acceleration and the quality of the final reconstructed image.

[0050] The layered sharing method and system described herein is compatible with other methods for reducing the processing time of diffusion-based image denoising models, including, but not limited to, using a denoising diffusion implicit model, reducing the number of diffusion steps, implementing early stopping of the diffusion process, using a fast ordinary differential equation solver, pre-segmentation diffusion sampling, using latent space diffusion for larger acceleration factors, and using high-frequency spatial diffusion models.

[0051] A hardware description of device 601 according to an exemplary embodiment will now be described with reference to FIG. 10 . In FIG. 10 , device 601 includes processing circuitry. Device 601 can be used to perform any of the methods described herein related to acquiring a DDPM, training a DDPM, receiving acquired images, and / or denoising images using a DDPM. In one embodiment, device 601 can be a server, a computer, or the like. In one embodiment, device 601 can be in communication with or integrated into an image acquisition device, such as the CT device illustrated in FIG. 11 . In one embodiment, the methods described herein can be distributed across one or more devices, where one or more devices include at least some of the elements of device 601. The processing circuitry includes one or more of the elements described below with reference to FIG. 10 . Process data and instructions can be stored in memory 602. These processes and instructions can also be stored on a storage medium disk 604, such as a hard drive (HDD) or a portable storage medium, or can be stored remotely. Furthermore, the claimed advancement is not limited by the form of computer-readable medium on which the instructions for the inventive process are stored. For example, the instructions may be stored on a CD, DVD, flash memory, RAM, ROM, PROM, EPROM, EEPROM, hard disk, or other information processing device with which device 601 communicates, such as a server or computer.

[0052] Additionally, the claimed advancements may be provided as utility applications, background daemons, or operating system components, or combinations thereof, and execute in cooperation with CPU 600 and an operating system such as Microsoft Windows, UNIX, Solaris, LINUX, Apple MAC-OS, and other systems known to those skilled in the art.

[0053] The hardware elements for implementing device 601 may be implemented using a variety of circuit elements known to those skilled in the art. For example, CPU 600 may be an Intel Xenon or Core processor, or an AMD Opteron processor, or other processor types recognized by those skilled in the art. Alternatively, CPU 600 may be implemented using FPGA, ASIC, PLD, or discrete logic circuitry, as recognized by those skilled in the art. Furthermore, CPU 600 may be implemented as multiple processors cooperating in parallel to execute instructions of the processes described above.

[0054] 10 also includes a network controller 606, such as an Intel Ethernet PRO network interface card from Intel Corporation of America, for interfacing with a network 650 and communicating with other devices. As can be appreciated, the network 650 can be a public network, such as the Internet, a private network, such as a LAN or WAN network, or any combination thereof, and can also include a PSTN or ISDN subnetwork. The network 650 can be wired, such as an Ethernet network, or wireless, such as a cellular network, including EDGE, 3G, 4G, and 5G wireless cellular systems. The wireless network can also be WiFi, Bluetooth, or any other known form of wireless communication.

[0055] The device 601 further includes a display controller 608, such as an NVIDIA GeForce GTX or Quadro graphics adapter from NVIDIA Corporation of the United States, for interfacing with a display 610, such as an LCD monitor. A general-purpose I / O interface 612 interfaces with a keyboard and / or mouse 614, and a touchscreen panel 616 on or separate from the display 610. The general-purpose I / O interface also connects to various peripherals 618, including printers and scanners.

[0056] A sound controller 620 is also provided in the device 601 and interfaces with a speaker / microphone 622 to provide sound and / or music.

[0057] A generic storage controller 624 connects the storage media disks 604 to a communication bus 626, which may be ISA, EISA, VESA, PCI, or the like, for interconnecting all components of the device 601. A description of the general features and functionality of the display 610, keyboard and / or mouse 614, as well as the display controller 608, storage controller 624, network controller 606, sound controller 620, and generic I / O interface 612 will be omitted herein for the sake of brevity, as these features are known.

[0058] In one embodiment, the image processed using the bulk diffusion method described herein may be a CT image acquired by a CT device or scanner. FIG. 11 illustrates an implementation of a radiation gantry included in a CT device or scanner. As shown in FIG. 11, the X-ray imaging gantry 9900 is shown from a side view and further includes an X-ray tube 9901, an annular frame 9902, and a multi-row or two-dimensional array X-ray detector 9903. The X-ray tube 9901 and the X-ray detector 9903 are mounted radially on the annular frame 9902, which is rotatably supported about a rotation axis RA, across a subject, such as a patient. A rotation unit 9907 rotates the annular frame 9902 at a high speed, such as 0.4 seconds per rotation, while the subject moves along the axis RA toward or away from the illustrated page.

[0059] Embodiments of an X-ray computed tomography (CT) apparatus according to the present application will be described below with reference to the accompanying drawings. X-ray CT apparatuses include various types of apparatuses, such as a rotating / rotating type apparatus in which an X-ray tube and an X-ray detector rotate together around an object to be examined, and a fixed / rotating type apparatus in which many detector elements are arranged in a ring or plane and only the X-ray tube rotates around an object to be examined. The present application can be applied to either type. Here, the rotating / rotating type, which is currently mainstream, will be exemplified.

[0060] The multi-slice X-ray CT apparatus further includes a high-voltage generator 9909, which generates a tube voltage applied to the X-ray tube 9901 through a slip ring 9908 so that the X-ray tube 9901 generates X-rays. The X-ray detector 9903 is located on the opposite side of the subject from the X-ray tube 9901 to detect the irradiated X-rays that have passed through and propagated through the subject. The X-ray detector 9903 is, for example, a photon-counting detector. The X-ray detector or the photon-counting detector 9903 further includes individual detector elements or units, such as a processing circuit.

[0061] The CT apparatus further includes other devices for processing detection signals from the X-ray detector 9903. A data acquisition circuit or data acquisition system (DAS) 9904 converts the signal output from the X-ray detector 9903 of each channel into a voltage signal, amplifies the signal, and further converts the signal into a digital signal. The X-ray detector 9903 and the DAS 9904 are configured to manage a predetermined total number of projections per rotation (TPPR).

[0062] The above-mentioned data is sent via a non-contact data transmitter 9905 to a pre-processing device 9906 housed in a console external to the X-ray gantry 9900. The pre-processing device 9906 performs certain corrections. A memory 9912 stores the resulting data, also called projection data, just before the reconstruction process. The memory 9912, along with a reconstruction device 9914, an input device 9915, and a display 9916, are connected to a system controller 9910 via a data / control bus 9911. The system controller 9910 controls a current regulator 9913, which limits the current to a level sufficient to drive the CT system.

[0063] In various generations of CT scanner systems, the detectors are rotated and / or fixed relative to the object being scanned, such as a patient. In one embodiment, the CT system described above may be an example of a combination of third-generation and fourth-generation geometry systems. In a third-generation system, the X-ray tube 9901 and X-ray detector 9903 are radially mounted on an annular frame 9902, which rotates around the object by rotating about an axis of rotation RA. In a fourth-generation geometry system, the detectors are fixedly positioned around the patient, and the X-ray tube 9901 rotates around the patient. In an alternative embodiment, the X-ray imaging gantry 9900 has multiple detectors arranged on an annular frame 9902 supported by a C-arm and a stand.

[0064] Post-reconstruction processing performed by the reconstruction device 9914 may include image filtering and smoothing, volume rendering, and image subtraction, as needed. The image reconstruction process may implement various CT image reconstruction methods. The reconstruction device 9914 may use memory to store, for example, projection data, reconstructed images, calibration data and parameters, and computer programs.

[0065] While this specification contains many specific implementation details, these should not be construed as limitations on the scope of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments.

[0066] Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, while features may be described above as working in a particular combination and even initially claimed as such, in some instances one or more features of a claimed combination can be deleted from that combination. Claimed combinations can also be directed to subcombinations or variations of subcombinations.

[0067] Similarly, while operations are shown in the figures in a particular order, this should not be understood as requiring that such operations be performed in the particular order or sequential order shown, or that all of the operations shown be performed, to achieve desirable results. In some situations, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in every embodiment, and it should be understood that the described program components and systems may generally be integrated together into a single software product or packaged into multiple software products.

[0068] Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. As an example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous.

[0069] Moreover, the embodiments of the present application may be as described in the following appendices.

[0070] (1) A method for performing diffusion-based image restoration on a plurality of input images, the method comprising: obtaining a diffusion-based probabilistic model trained to perform image restoration over T steps, where T is an integer greater than or equal to 3; dividing the initial T1 steps into a plurality of layers, where T1 is an integer greater than or equal to 2; for each layer of the plurality of layers, grouping the plurality of input images into a number of groups; and for each group of the groups, determining an initial representative image for the group; and performing, for each group, a sequence of sampling steps starting from the initial representative image for the group using the obtained model to generate a corresponding sequence of representative images for the group; and for each group in a last layer of the plurality of layers, for each input image in the group, where T2 is an integer greater than or equal to 1, where T=T1+T2, using the obtained model to generate a final image, which is a restored image corresponding to the input image.

[0071] (2) The method according to (1), wherein the dividing step further includes dividing the initial T1 steps of the T steps into multiple layers based on common features identified among the multiple input images.

[0072] (3) The method of (2), wherein the dividing step further includes dividing the initial T1 steps of the T steps into a preceding layer and a succeeding layer, and the grouping step further includes grouping the multiple input images into a first number of groups for the preceding layer and grouping the multiple input images into a second number of groups for the succeeding layer, the second number being greater than the first number.

[0073] (4) The method according to (1), wherein the determining step further includes determining the average of the images input to the group as the initial representative image of the group.

[0074] (5) The method according to (1), wherein the determining step further includes determining one image having the highest image quality among the images input to the group as an initial representative image of the group.

[0075] (6) The method of (1), further comprising acquiring a plurality of input images as a time sequence of reconstructed medical images.

[0076] (7) The method of (1), further comprising acquiring the plurality of input images as a spatial sequence of reconstructed medical images.

[0077] (8) An apparatus for performing diffusion-based image restoration on a plurality of input images, the apparatus including a processing circuit configured to: obtain a diffusion-based probabilistic model trained to perform image restoration over T steps, where T is an integer greater than or equal to 3; divide the initial T1 steps into a plurality of layers, where T1 is an integer greater than or equal to 2; group the plurality of input images into a number of groups, and for each group of the groups, determine an initial representative image for the group; and perform, for each group, a sequence of sampling steps starting from the initial representative image of the group using the obtained model to generate a corresponding sequence of representative images for the group; and for each group in a last layer of the plurality of groups in the plurality of layers, perform, for each input image in the group, a second sequence of T2 sampling steps starting from the last one of the sequence of representative images of the group using the obtained model, where T2 is an integer greater than or equal to 1, where T=T1+T2, to generate a final image, which is a restored image corresponding to the input image.

[0078] (9) The apparatus of (8), wherein the processing circuitry is further configured to perform the segmentation step by dividing an initial T1 number of steps of T steps into multiple layers based on common features identified among the multiple input images.

[0079] (10) The apparatus described in (9), wherein the processing circuit is further configured to perform a dividing step by dividing an initial T1 steps of the T steps into a preceding layer and a succeeding layer, and the processing circuit is further configured to perform a grouping step by grouping the plurality of input images into a first number of groups for the preceding layer and grouping the plurality of input images into a second number of groups for the succeeding layer, the second number being greater than the first number.

[0080] (11) The apparatus of (8), wherein the processing circuitry is further configured to determine an average of the images input to the group as an initial representative image for the group.

[0081] (12) The device according to (8), wherein the processing circuit is further configured to determine one image having the highest image quality among the images input to the group as an initial representative image of the group.

[0082] (13) The apparatus of (8), wherein the processing circuitry is further configured to acquire the plurality of input images as a time sequence of reconstructed medical images.

[0083] (14) The apparatus of (8), wherein the processing circuitry is further configured to acquire the plurality of input images as a spatial sequence of reconstructed medical images.

[0084] (15) A non-transitory computer-readable storage medium for storing computer-readable instructions that, when executed by a computer, cause the computer to perform a method for performing diffusion-based image restoration on a plurality of input images, the method including: obtaining a diffusion-based probabilistic model trained to perform image restoration over T steps, where T is an integer greater than or equal to 3; dividing the initial T1 steps into a plurality of layers, where T1 is an integer greater than or equal to 2; grouping the plurality of input images into a number of groups, and for each of the groups, determining an initial representative image for the group; and performing, for each group, a sequence of sampling steps starting from the initial representative image for the group using the obtained model to generate a corresponding sequence of representative images for the group; and for each of the groups in a last layer of the plurality of layers, for each input image in the group, where T2 is an integer greater than or equal to 1, where T=T1+T2, using the obtained model to generate a final image, which is a restored image corresponding to the input image.

[0085] (16) The non-transitory computer-readable storage medium of (15), wherein the dividing step further includes dividing the initial T1 steps of the T steps into multiple layers based on common features identified among the multiple input images.

[0086] (17) The non-transitory computer-readable storage medium of (16), wherein the dividing step further includes dividing an initial T1 steps of the T steps into a preceding layer and a subsequent layer, and the grouping step further includes grouping the plurality of input images into a first number of groups for the preceding layer and grouping the plurality of input images into a second number of groups for the subsequent layer, the second number being greater than the first number.

[0087] (18) The non-transitory computer-readable storage medium of (15), wherein the determining step further includes determining an average of the images input to the group as an initial representative image for the group.

[0088] (19) The non-transitory computer-readable storage medium according to (15), wherein the determining step further includes determining one image having the highest image quality among the images input to the group as an initial representative image of the group.

[0089] (20) The non-transitory computer-readable storage medium of (15), wherein the acquiring step further includes acquiring the plurality of input images as a time sequence of reconstructed medical images.

[0090] Obviously, many modifications and variations are possible in light of the above teachings. It is therefore to be understood that, within the scope of the claims, the embodiments of the present application may be practiced otherwise than as specifically described herein.

Claims

1. 1. A method for performing diffusion-based image restoration on a plurality of input images, comprising: Obtaining a diffusion-based probabilistic model trained to perform image restoration over T steps, where T is an integer greater than or equal to 3; T1 is an integer equal to or greater than 2, and the initial T1 steps of the T steps are divided into a plurality of layers; For each of the plurality of hierarchies, grouping the plurality of input images into groups; determining, for each of said groups, an initial representative image of said group; performing, for each group, a sequence of sampling steps starting from the initial representative image of the group using the obtained model to generate a corresponding sequence of representative images of the group; For each of the several groups in the last hierarchical level of the plurality of hierarchical levels, For each input image in the group, performing a second sequence of T2 sampling steps starting from the last one of the sequence of representative images of the group using the obtained model, where T=T1+T2, where T2 is an integer greater than or equal to 1, to generate a final image, which is a reconstructed image corresponding to the input image; A method comprising:

2. 2. The method of claim 1, wherein the dividing step further comprises dividing the initial T1 steps of the T steps into the plurality of layers based on common features identified among the plurality of input images.

3. the dividing step further includes dividing the initial T1 steps of the T steps into a preceding tier and a succeeding tier; The grouping step includes: grouping the plurality of input images into a first number of groups for the previous layer; The method of claim 2 , further comprising: for the subsequent layer, grouping the plurality of input images into a second number of groups, the second number being greater than the first number.

4. The method of claim 1 , wherein the determining step further comprises determining an average of the images input to the group as the initial representative image for the group.

5. The method of claim 1 , wherein the determining step further comprises determining one image having the highest image quality from among the images input to the group as the initial representative image of the group.

6. The method of claim 1 , further comprising acquiring the plurality of input images as a time sequence of reconstructed medical images.

7. The method of claim 1 , further comprising acquiring the plurality of input images as a spatial sequence of reconstructed medical images.

8. 1. An apparatus for performing diffusion-based image restoration on a plurality of input images, comprising: Obtain a diffusion-based probabilistic model trained to perform image restoration over T steps, where T is an integer greater than or equal to 3; T1 is an integer equal to or greater than 2, and the initial T1 steps of the T steps are divided into a plurality of layers; For each of the plurality of hierarchies, grouping the plurality of input images into several groups; determining, for each of said several groups, an initial representative image for said group; performing, for each group, a sequence of sampling steps starting from the initial representative image of the group using the obtained model to generate a corresponding sequence of representative images of the group; For each of the several groups in the last hierarchical level of the plurality of hierarchical levels, For each input image in the group, performing a second sequence of T2 sampling steps starting from the last one of the sequence of representative images of the group using the obtained model, where T=T1+T2, where T2 is an integer greater than or equal to 1, to generate a final image, which is a reconstructed image corresponding to the input image; a processing circuit configured to 1. An apparatus comprising:

9. 9. The apparatus of claim 8, wherein the processing circuitry is further configured to perform the segmenting step by segmenting the initial T1 steps of the T steps into the plurality of layers based on common features identified among the plurality of input images.

10. the processing circuitry is further configured to perform the dividing step by dividing the initial T1 steps of the T steps into a preceding tier and a succeeding tier; The processing circuitry grouping the plurality of input images into a first number of groups for the previous layer; grouping the plurality of input images into a second number of groups for the subsequent layer; 10. The apparatus of claim 9, further configured to perform the grouping step by: wherein the second number is greater than the first number.

11. The apparatus of claim 8 , wherein the processing circuitry is further configured to determine an average of the images input to the group as the initial representative image for the group.

12. The apparatus of claim 8 , wherein the processing circuitry is further configured to determine an image of the highest quality among the images input to the group as the initial representative image of the group.

13. The apparatus of claim 8 , wherein the processing circuitry is further configured to acquire the plurality of input images as a time sequence of reconstructed medical images.

14. The apparatus of claim 8 , wherein the processing circuitry is further configured to acquire the plurality of input images as a spatial sequence of reconstructed medical images.

15. 1. A non-transitory computer-readable storage medium for storing computer-readable instructions that, when executed by a computer, cause the computer to perform a method for performing diffusion-based image restoration on a plurality of input images, the method comprising: Obtaining a diffusion-based probabilistic model trained to perform image restoration over T steps, where T is an integer greater than or equal to 3; T1 is an integer equal to or greater than 2, and the initial T1 steps of the T steps are divided into a plurality of layers; For each of the plurality of hierarchies, grouping the plurality of input images into groups; determining, for each of said groups, an initial representative image of said group; performing, for each group, a sequence of sampling steps starting from the initial representative image of the group using the obtained model to generate a corresponding sequence of representative images of the group; For each of the several groups in the last hierarchical level of the plurality of hierarchical levels, For each input image in the group, performing a second sequence of T2 sampling steps starting from the last one of the sequence of representative images of the group using the obtained model, where T=T1+T2, where T2 is an integer greater than or equal to 1, to generate a final image, which is a reconstructed image corresponding to the input image; 1. A non-transitory computer-readable storage medium comprising:

16. 16. The non-transitory computer-readable storage medium of claim 15, wherein the dividing step further comprises dividing the initial T1 steps of the T steps into the plurality of layers based on common features identified among the plurality of input images.

17. the dividing step further includes dividing the initial T1 steps of the T steps into a preceding tier and a succeeding tier; The grouping step includes: grouping the plurality of input images into a first number of groups for the previous layer; and grouping the plurality of input images into a second number of groups for the subsequent layer, the second number being greater than the first number.

18. The non-transitory computer-readable storage medium of claim 15 , wherein the determining step further comprises determining an average of the images input to the group as the initial representative image for the group.

19. 16. The non-transitory computer-readable storage medium of claim 15, wherein the determining step further comprises determining one image having the highest image quality among the images input to the group as the initial representative image of the group.

20. The non-transitory computer-readable storage medium of claim 15 , wherein the acquiring step further comprises acquiring the plurality of input images as a time sequence of reconstructed medical images.

Images