Generalized image restoration with synthetic prior

Abstract

Systems and methods for image restoration can make use of a synthetic prior. An imaging process, such as X-ray microscopy can generate imaging data from an original subject. The imaging data can be subject to a first imaging restoration process to generate the synthetic prior, which acts as an approximation for the original subject. This synthetic prior can be subjected to a forward imaging operator to generate synthetic imaging data. A second imaging process can then be trained using the synthetic imaging data as an input and the synthetic prior as a desired output. The trained second imaging process can then be applied to the original received imaging data to generate restored imaging data. The restored imaging data would thus provide a more accurate approximation for the original subject than the synthetic prior.

Description

Docket No. 069608-000328WOPT 2023P01186WO GENERALIZED IMAGE RESTORATION WITH SYNTHETIC PRIORCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of and priority to U.S. Provisional Application No.63 / 549,818, filed February 5, 2024, which is hereby incorporated by reference herein in its entirety.FIELD OF THE INVENTION

[0002] The present disclosure relates to image restoration generally and more specifically to restoration of radiological imaging data, such as X-ray microscopy imaging data.BACKGROUND

[0003] X-ray Microscopy Imaging is a field of imaging that is used to acquire imaging data for many different types of samples across many different use cases. X-ray Microscopy Imaging has found uses in biology (e.g., imaging biomaterials, soft tissues, and the like), material science (e.g., imaging the internal microstructure of a material), manufacturing (e.g., non-destructively imaging internal components), and many other fields. Individual images (e.g., projections) can be acquired by directing radiation from an X-ray source, through a sample, towards a detector. Multiple projections can be acquired for a single sample by rotating the direction of travel of the X-ray radiation with respect to the sample (e.g., rotating the sample with respect to the X-ray source and detector). Often, the acquired imaging data (e.g., containing multiple projections) is used to generate a three-dimensional reconstructed volumes of the sample that was imaged, such as through the use of computed tomography (CT).

[0004] While X-ray imaging provides many benefits, it often suffers from a range of artifacts and imaging issues, including noise, sparse sampling artifacts, beam hardening, missing angle artifacts, and other issues. Many of these issues have historically been dealt with using a combination of specialized 3D reconstruction technology and post processing filters. These techniques, however, give suboptimal imaging results and can introduce new artifacts or biases in the results that should be avoided if possible.

[0005] There is a need for improved image processing techniques that cure these and other deficiencies.Docket No. 069608-000328WOPT 2023P01186WOBRIEF SUMMARY

[0006] The term embodiment and like terms are intended to refer broadly to all of the subject matter of this disclosure and the claims below. Statements containing these terms should be understood not to limit the subject matter described herein or to limit the meaning or scope of the claims below. Embodiments of the present disclosure covered herein are defined by the claims below, supplemented by this summary. This summary is a high-level overview of various aspects of the disclosure and introduces some of the concepts that are further described in the Detailed Description section below. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used in isolation to determine the scope of the claimed subject matter. The subject matter should be understood by reference to appropriate portions of the entire specification of this disclosure, any or all drawings and each claim.

[0007] Embodiments of the present disclosure include a computer-implemented image restoration method, which includes receiving imaging data associated with a subject, generating a synthetic prior by applying a first imaging restoration process to the received imaging data, determining a forward imaging operator, generating synthetic imaging data by applying the forward imaging operator to the synthetic prior, training a second imaging restoration process based at least in part on the synthetic imaging data and the synthetic prior, and applying the second imaging restoration process to the received imaging data to generate restored imaging data. The image restoration method may also include where applying the first imaging restoration process includes applying a trained machine learning algorithm to the received imaging data. The image restoration method may also include where the second imaging restoration process includes a machine learning algorithm, and where training the second imaging restoration process includes optimizing a hyperparameter of the machine learning algorithm. The image restoration method may also include where determining the forward imaging operator includes receiving metadata associated with the imaging data, and selecting the forward imaging operator from a set of preset forward imaging operators based at least in part on the received metadata. The image restoration method may also include where the received imaging data is in the image space and the synthetic prior is in the image space. The image restoration method may also include where determining the forward imaging operator includes calculating an initial noise distribution based at least in part on the received imaging data and the synthetic prior, calculating one or more parameters associated with the initial noise distribution, and generating a synthetic noise field based at least in part on the calculated one or more parameters, where generating the syntheticDocket No. 069608-000328WOPT 2023P01186WOimaging data includes applying the synthetic noise field to the synthetic prior. The image restoration method may also include where training the second imaging restoration process includes determining one or more regions of high gradient associated with the synthetic prior, and training a machine learning algorithm using a loss function based at least in part on the synthetic prior and the synthetic imaging data, where the loss function weights a loss in the determined one or more regions of high gradient higher than a loss not in the determined one or more regions of high gradient. The image restoration method may also include where the received imaging data is in the projection space and the synthetic prior is in the image space. The image restoration method may also include where generating the synthetic prior includes generating an initial reconstruction based at least in part on the received imaging data, determining an estimated noise field based at least in part on the initial reconstruction, determining an available angle range associated with the received imaging data, accessing a training volume, forward-projecting the training volume in the available angle range to generate constrained training volume imaging data, forward-projecting the training volume in full range to generate full training volume imaging data, generating a constrained training reconstruction based at least in part on the constrained training volume imaging data, generating a full training reconstruction based at least in part on the full training volume imaging data, updating the constrained training reconstruction by adding the estimated noise field to the constrained training reconstruction, training the first imaging restoration process based at least in part on the constrained training reconstruction and the full training reconstruction, and applying the trained first imaging restoration process to the initial reconstruction to obtain the synthetic prior. The image restoration method may also further include outputting the restored imaging data using a display device.

[0008] A system includes a control system including one or more processors, and a memory having stored thereon machine readable instructions, where the control system is coupled to the memory, and the method of any one of claims 1 to 18 is implemented when the machine executable instructions in the memory are executed by at least one of the one or more processors of the control system.

[0009] A computer program product tangibly embodied in a non-transitory machine-readable storage medium, the computer program product includes instructions which, when executed by a computer, cause the computer to carry out the method of any one of claims 1 to 18.

[0010] The image restoration method may also include where optimizing the hyperparameter includes generating interim restored imaging data by applying the second imaging restorationDocket No. 069608-000328WOPT 2023P01186WOprocess to the synthetic imaging data, and determining a difference between the synthetic prior and the interim restored imaging data, where the hyperparameter is optimized when the difference between the synthetic prior and the interim restored imaging data is minimized. The image restoration method may also include where calculating the initial noise distribution includes including applying the equation where Ni is the initial noise distribution, D is the received imaging data, and Sa is the synthetic prior. The image restoration method may also include where calculating the one or more parameters associated with the initial noise distribution includes generating a frequency domain representation of the initial noise distribution, and where generating the synthetic noise field includes generating a magnitude field from the frequency domain representation of the initial noise distribution, and creating the synthetic noise field based at least in part on the magnitude field. The image restoration method may also include where generating the synthetic noise field further includes generating a random phase field using a uniform random number generator, and where creating the synthetic noise field is further based at least in part on the random phase field. The image restoration method may also include where calculating the one or more parameters associated with the initial noise distribution further includes calculating a local spatially variant standard deviation of the initial noise distribution, and where generating the synthetic noise field further includes generating a random noise field based at least in part on the local spatially variant standard deviation, and generating a phase field based at least in part on a frequency domain representation of the random noise field, and where creating the synthetic noise field is further based at least in part on the generated phase field. The image restoration method may also include where generating the synthetic noise field includes applying the equation , where N is the synthetic noise field, Nima is the magnitude field of the frequency domain representation of the initial noise distribution, GPhase is a phase field derived from a frequency domain representation of a random noise field generated based at least in part on a calculated local spatially variant standard deviation of the initial noise distribution, i is , real is a real component function, and ifft is an inverse Fourier transform. The image restoration method may also include where the synthetic noise field has equivalent local Fourier magnitudes to the initial noise distribution, and where local phases of the synthetic noise field are fully decorrelated to the initial noise distribution. The image restoration method may also include where generating the synthetic imaging data includes generating a new projection dataset by forward-projecting the syntheticDocket No. 069608-000328WOPT 2023P01186WOprior through a projective geometry corresponding to the received imaging data, determining an outside-field-of-view (OFOV) attenuation representation based at least in part on the new projection dataset and the received imaging data, the OFOV attenuation representation corresponding to material outside a field of view of the received imaging data, denoising the OFOV attenuation representation, generating an updated new projection dataset by combining the denoised OFOV attenuation representation and the new projection dataset, and generating the synthetic imaging data by adding the estimated noise field to back-projection of the updated new projection dataset.

[0011] The image restoration method may also further include outputting the restored imaging data using a display device. The image restoration method may also include where the received imaging data is in a projection domain. The image restoration method may also include where the synthetic prior is in an image domain, the first imaging restoration process usable to generate reconstructed datasets (e g., reconstructed volume datasets) from projection datasets. The image restoration method may also include where the received imaging data includes a projection dataset acquired via an X-ray imaging device. The image restoration method may also include where the received imaging data is in an image domain and the synthetic prior is in the image domain. The image restoration method may also include where the received imaging data is representative of a plurality of projections acquired at a plurality of different angles with respect to the subject. The image restoration method may also include where the received imaging data is associated with a transmission imaging study of the subject. The image restoration method may also include where the transmission imaging study is an X-ray imaging study.

[0012] Other technical features may be readily apparent to one skilled in the art from the following figures, descriptions, and claims.BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

[0013] To easily identify the discussion of any particular element or act, the most significant digit or digits in a reference number refer to the figure number in which that element is first introduced.

[0014] FIG. 1 is a schematic diagram depicting an imaging data processing system, according to certain aspects of the present disclosure.

[0015] FIG. 2 is a schematic diagram depicting an image restoration workflow, according to certain aspects of the present disclosure.Docket No. 069608-000328WOPT 2023P01186WO

[0016] FIG. 3 is a flowchart depicting a process for performing image restoration using a synthetic prior, according to certain aspects of the present disclosure.

[0017] FIG. 4 is a flowchart depicting a process for generating synthetic imaging data, according to certain aspects of the present disclosure.

[0018] FIG. 5 is a flowchart depicting a process for generating synthetic imaging data using frequency domain transforms, according to certain aspects of the present disclosure.

[0019] FIG. 6 is a flowchart depicting a process for improving restoration of limited-angle imaging data, according to certain aspects of the present disclosure.

[0020] FIG. 7 is a set of images depicting various sets of imaging data of sandstone rock, according to certain aspects of the present disclosure.

[0021] FIG. 8 is a set of images depicting various sets of imaging data of sandstone rock, according to certain aspects of the present disclosure.

[0022] FIG. 9 is a chart depicting the performance of various imaging restoration processes, according to certain aspects of the present disclosure.

[0023] FIG. 10 is a set of image reconstructions for a sample, according to certain aspects of the present disclosure.

[0024] FIG. 11 is a set of image reconstructions for a training volume, according to certain aspects of the present disclosure.

[0025] FIG. 12 is a set of image reconstructions for a sample, according to certain aspects of the present disclosure.

[0026] FIG. 13 is a set of image reconstructions for a sample (e.g., the sample of FIG. 12), according to certain aspects of the present disclosure.

[0027] FIG. 14 is a block diagram of an example system architecture for implementing features and processes of the present disclosure.DETAILED DESCRIPTION

[0028] Certain aspects and features of the present disclosure relate to image restoration techniques making use of a synthetic prior. An imaging process, such as X-ray microscopy can generate imaging data from an original subject. The imaging data can be subject to a first imaging restoration process to generate the synthetic prior, which acts as an approximation for the original subject. This synthetic prior can be subjected to a forward imaging operator to generate syntheticDocket No. 069608-000328WOPT 2023P01186WOimaging data. A second imaging restoration process can then be trained using the synthetic imaging data as an input and the synthetic prior as a desired output. The trained second imaging restoration process can then be applied to the original received imaging data to generate restored imaging data. The restored imaging data would thus provide a more accurate approximation for the original subject than the synthetic prior.

[0029] While described herein with reference to imaging in general, certain aspects and features of the present disclosure are especially suitable for X-ray imaging and X-ray reconstruction, and especially with respect to X-ray microscopy. X-ray reconstruction, among other imaging techniques, can be prone to noise and sampling artifacts, missing wedge artifacts, beam hardening, interior tomography artifacts, cone angle / laminography artifacts, and the like. Certain aspects and features of the present disclosure are especially suitable for transmission imaging studies (e.g., X-ray imaging studies), in which electromagnetic radiation is directed at a subject being imaged and imaging is based on electromagnetic radiation detected after passing through the subject. For example, in X-ray imaging, X-rays are directed to a sample and are absorbed or scattered by the sample as the X-rays travel through the sample. The X-rays not absorbed or scattered away are transmitted through and modulated by the sample before being received by a detector. Certain aspects and features of the present disclosure are especially suitable for tomography imaging, in which multiple acquisitions are acquired of the subject at multiple different angles to permit reconstruction of one or more slices of the subject.

[0030] An original subject can be represented by a structure function, S. The original subject can be any suitable subject for which imaging is desired. When the original subject is imaged, the resulting output is imaging data, D. Thus, imaging of the subject to obtain the imaging data can be described as performing some imaging operator, G on the structure function to create the imaging data, such as shown in equation,

[0031] Imaging data itself can be stored and / or manipulated in any suitable form or domain. In some cases, imaging data is raw data acquired in the projection domain, although that need not always be the case. In some cases, imaging data can refer to images created from reconstruction of raw imaging data, such as volumetric reconstructions. As used herein, the term projection domain refers to data representative of the individual projections acquired of the sample themselves. Such data is generally in a 2-dimensional format (e.g., an individual radiograph), although that need not always be the case. As used herein, the term image domain refers to data that has been reconstructed from individual projections to generate an image or volumeDocket No. 069608-000328WOPT 2023P01186WO representative of the sample. Such data is generally in a 2-dimensional format (e.g., a reconstruction of a tomographic slice of the sample) or 3-dimensional format (e.g., a volumetric reconstruction of the sample), although that need not always be the case.

[0032] Imaging restoration processes can be applied to imaging data to improve the imaging data and make it a more accurate representation of the original structure function (e.g., the original structure). The improvements can include reconstruction of raw imaging data into two-dimensional or volumetric representations of a subject, although that need not always be the case. In some cases, an imaging restoration process includes taking a volumetric representation of a subject as input and outputting an improved volumetric representation of the subject, such as with reduced noise or other artefacts. According to certain aspects and features of the present disclosure, an imaging restoration process includes leveraging a trained machine learning model to perform some imaging task. Some imaging restoration processes, especially those based on machine learning models, when used alone, can introduce undesired effects, such as oversmoothing.

[0033] An ideal restoration process would involve applying a reverse imaging operator, G1to the imaging data to recover the original structure function from the imaging data, such as shown in equation, . However, finding the ideal inverse operator is extremely difficult.

[0034] Various technologies exist to try and approximate this image restoration process for various specific image restoration tasks. For example, accounting for the sparse angle problem or noise has been attempted using Huber regularized iterative reconstruction or deep learning reconstruction. As another example, accounting for beam hardening artefacts has been attempted using polynomial beam hardening correction. These techniques, however, suffer from their own significant challenges. They can often be computationally complex, require hyper parameter tuning, and can often introduce their own incremental artifacts above what may be desired.

[0035] Certain aspects and features of the present disclosure leverage a first imaging restoration process, Ga-1, to create an image (e.g., a 2D, 3D, or other image) that is used as an intermediate step to generating a final, restored image. This intermediate image, being already processed using a first imaging restoration process, has a feature distribution that approximates that of the original structure function. This intermediate image can be known as a synthetic prior, Sa, since it is used as a synthetic “sample” that contains a feature distribution close to the real sample, constructedDocket No. 069608-000328WOPT 2023P01186WOusing the prior information available in the sample, i.e., derived from the imaging data. The synthetic prior can be determined according to the equation,

[0036] The synthetic prior can then be degraded using some forward imaging operator, Gb. The forward imaging operator should approximate the overall impact of the degradation process (e.g., the degradation from imaging the original subject to output the received imaging data) as faithfully as possible. In some cases, an accurate forward system model may be used. In some cases, a robust heuristic model can be used. Other techniques may be used. The result of applying the forward imaging operator to the synthetic prior is synthetic imaging data, Db, which can be determined according to the equation,

[0037] This synthetic imaging data can then be used as the input for a second imaging restoration process, Gc-1. As for many problems a forward problem is easier to solve to a high fidelity than an equivalent inverse problem with no apriori prior information. Since both the synthetic imaging data is known and the synthetic prior is known, the second imaging restoration process can be optimized more effectively. Knowing both the input and output for the second imaging restoration process allows optimization of any hyperparameters (a) in the second imaging restoration process to maximize the fidelity of recovery, as seen in the equation, , where argmin is indicative of optimizing the hyperparameter(s), a, until the difference between the synthetic prior and the output of the second imaging restoration process applied to the synthetic imaging data is at a minimum.

[0038] In the aforementioned example, the hyperparameters a can be optimized using a L2 norm (mean square error), but in some cases, other loss functions are used.

[0039] The inverse imaging operator Gc-1"can be any reconstruction / restoration operation with tunable hyperparameters. Notably, convolutional neural networks (CNNs) provide a generally applicable tool that can be adapted to any image restoration task, given the presence of an input dataset (e.g., Db) and a target / label dataset (e.g., Sa).

[0040] Training the second imaging restoration process on the synthetic imaging data derived from the synthetic prior is especially beneficial in obtaining high-quality restored imaging data. Any image restoration process trained based on the received imaging data instead of the synthetic imaging data would be optimized to remove any features and introduce any artifact introduced by the imaging restoration process, and thus would not offer any performance benefit relative toDocket No. 069608-000328WOPT 2023P01186WOthe first imaging restoration process, other than potentially a speed benefit if a more computationally efficient approach was used for the trained imaging restoration process than is available for the first imaging restoration process.

[0041] The degradation process (e.g., the forward imaging operator) should not introduce any features that would not be acceptable to remove using the trained imaging restoration process, Ge1. Removed features should be absent in both the synthetic imaging data and the synthetic prior, so the hyperparameters will not be optimized to remove them. Further, any artifacts that are introduced when using the first imaging restoration process, Ga-1, should be present in both the synthetic prior and the synthetic imaging data. This correlation should lead the trained imaging restoration process, Gc1, to not be trained to introduce these artifacts as they are correlated between both the input and the labels (e.g., the desired output, which is the synthetic prior).

[0042] Once the second imaging restoration process has been trained (e.g., optimized), it can be applied to the original imaging data to generate the restored imaging data (Sb), such as seen in the equation,

[0043] While described herein with respect to one iteration of received imaging data, synthetic prior, and synthetic imaging data, in some cases additional iterations can be used. In such cases, an additional iteration can use the restored imaging data from a prior iteration as the synthetic prior for the additional iteration. For example, an original structure function, S, can be used to generate imaging data that is then used to generate a synthetic prior, Sa, which can be used to generate synthetic imaging data Db. When a single iteration is used, a second imaging restoration process can be trained using Db and Sa, and that second imaging restoration process can be used to generate the restored imaging data, Sb from the original imaging data. However, if a second iteration is used, the restored imaging data, Sb can function as the synthetic prior used to generate additional synthetic imaging data, Dc, which can be used with Sb to train a third imaging restoration process, which can then be used to generate further restored imaging data (e.g., by applying the third imaging restoration process to the original imaging data). Any number of iterations could be used.

[0044] This overall workflow can be applied across a wide range of image restoration tasks, by using a first imaging restoration process, , to initially construct the synthetic prior and a forward imaging operator, , to construct the synthetic imaging data. Apart from theseDocket No. 069608-000328WOPT 2023P01186WOoperators, the second imaging restoration process, , can be re-used, retraining the network to the next imaging restoration task.

[0045] In a first example use case, certain aspects and features of the present disclosure can be used to remove undesired noise and artifacts from initial imaging data. An example first imaging restoration process can be used to try and improve initial imaging data by subsampling projection data into two subsets (A and B), such as those representing even and odd projections, respectively. These two subsets of data are reconstructed into two volumes and a symmetrical noise-to-noise model (with loss computed from the mapping of A to B and B to A together) is trained to map from one to the other having the effect of both denoising and removing sparse sampling artifacts at the same time. (See e.g., U.S. Patent Publication No. US2003 / 0009951A1, the disclosure of which is incorporated by reference herein). In such a case, a final estimate of the structure function is given by passing a reconstruction created using all projections or a subset of projections through the trained network. The result of this first imaging restoration process is normally intended to be a final output (e.g., to be displayed to a user and / or used for further analysis).

[0046] Such a first imaging restoration process works well, but has some drawbacks. First, the network is trained with an input noise distribution found from half the total projection number, which is different from the noise distribution of the data used for final inference. Using just a projection subset for inference was found experimentally to yield less accurate results than when using the full projection number. This difference in noise distribution means that the network will tend to over-smooth the data relative to some theoretical optimal denoiser. Second, because a noise-to-noise approach is used, any imaging artifact that is correlated between the subsets will be retained in the final image. Spurious image signal in X-ray reconstruction is a combination of sparse sampling artifact and true noise. The noise will be uncorrelated but sparse sampling artifacts often have a degree of correlation between the projection subsets. The angular offset between the projection subsets and the fact that the network is trained in a symmetric fashion minimizes the impact of this suboptimal denoising, however it may still be noticeable in the final reconstructed volume.

[0047] According to certain aspects and features of the present disclosure, this first imaging restoration process can be used to generate the synthetic prior. Generally, this synthetic prior will be used as an intermediate and not provided as any sort of final output (e.g., not displayed to aDocket No. 069608-000328WOPT 2023P01186WOuser or used for substantive analysis), although that need not always be the case. Since the synthetic prior need not be displayed to a user or otherwise relied upon for substantive analysis, the first imaging restoration process used to generate the synthetic prior can be adjusted to produce intermediate imaging data that is more useful for the generation of synthetic imaging data and training of a second imaging restoration process, at the cost of visual or other degradations that may make the synthetic prior otherwise unsuitable for a final output.

[0048] Next, a forward imaging operator (e.g., forward model) can be used to re-introduce the noise and sampling artefacts to the level of the input dataset. While one could use an accurate system forward model (e g., forward projecting the synthetic prior, simulating noise, adding it to the projection, and then re-back projecting), such an approach is problematic for several reasons.

[0049] First, the problem is only well determined for samples that are wholly contained inside the Field of View. If the same extends beyond the field of view (e.g., an interior tomography), then the reconstructed volume does not constitute full information about the sample and its forward projection will not properly correspond to the data. Compensating for this effect is imperfect and corresponds to an inaccurate estimation of noise. Second, the forward and backward projection operation involves multiple interpolation steps that introduce incremental image blur. Finally, such a forward model may induce correlations between the spurious image signal and the structure function, leading to suboptimal trained removal.

[0050] However, according to certain aspects and features of the present disclosure, a heuristic model can be used. First, the imaging operator is approximated as a simple addition of noise, such as according to the equation, , where Db is the synthetic imaging data, Gb is the forward imaging operator, Sais the synthetic prior, and N is a noise function, here represented in the volumetric domain.

[0051] Noise in X-ray imaging is heteroscedastic, spatially variable, and spatially anisotropic. This is theoretically tractable in the projection (e g., transmission) domain as approximately Poissonian, however its representation in the volume domain depends on structures elsewhere in the object and is extremely hard to predict based on the values of the synthetic prior alone. However, an approximation for the initial noise distribution (Nt) with a full description of the second order complexities can be calculated as the residual from the data of the synthetic prior, such as according to equation . This spurious signal field cannot be used to recoverDocket No. 069608-000328WOPT 2023P01186WOan image Db, however it can be analyzed in detail to give insight into the specific complexities of the noise field for this dataset.

[0052] First, the noise field can be analyzed to estimate a local spatially variant standard deviation, <ry, such as according to the equation:where the local average of the square error (the object , taken here over a small 3 -dimensional 3x3x3 window, although other suitable window sizes can be used) is a representation of the local variance, as Sa can be viewed as an estimate of local measurement mean. This spatially variant standard deviation can then be used to seed a heteroscedastic normal random generator to generate a new, random noise field, G (e.g., random gaussian field).

[0053] This new noise field, however, remains an incomplete model of volumetric X-ray noise, as while it accounts for the spatial variability and heteroscedasticity of the noise field, it remains spatially isotropic and white. In other words, it has no local spatial auto-correlation in any dimension. To account for this incompleteness, the frequency domain (e.g., Fourier spectrum) of Ni can be considered. A Fourier spectrum of an image is a complex representation of the spatial frequencies in the image, and can be represented in many ways, however a particularly informative one is to split up the complex transform into two components: magnitude and phase. Given a complex number, C, where , the magnitude can be represented by and the phase can be represented by

[0054] The magnitude is a representation of the spatial frequency distribution in the image and the phase is a representation of the specific position of that frequency distribution. To say that X-ray noise is non-white is to say it has a non-uniform distribution of Fourier magnitudes, and to say it is anisotropic is to say that it is different in the 3 dimensions of the multidimensional Fourier transform.

[0055] Matching a noise field to the initial noise distribution can be achieved by matching the Fourier magnitudes of the noise field and the initial noise distribution, but not the phases. As the volumetric expression of spurious signal in X-ray imaging is spatially variable, only matching of Fourier magnitudes should be enforced between the initial noise distribution and the randomDocket No. 069608-000328WOPT 2023P01186WOgaussian field. This matching can be done by locally chunking the volume (e.g., into small cubes of geometry 64 x 64 x 64 voxels). The complex Fourier domain representation of the new noise field can be generated by taking the magnitudes from the initial noise distribution (e.g., ) and combining it with the phase from the random gaussian field G (e.g., ), such as seen in the following equation:

[0056] The final synthetic noise field (N) is then just the real component of the inverse Fourier transform of this object, as seen in the following equation:where fft(x) and ifft(x) represent the n-dimensional Fourier transform and n-dimensional inverse Fourier transform, respectively, real(x) is a real component function (e.g., function to extract the real component), and i is . This approach can be applied, chunk by chunk, across the entire image Ni and G to create a new noise field N (e.g., the synthetic noise field) having equivalent local Fourier magnitudes to Ni but fully decorrelated phase.

[0057] In some alternate implementations, the phase information could be created with a simply uniform random number generator rather than by taking the Fourier transform of the random field G

[0058] Once the synthetic noise field is obtained, it can be applied to the synthetic prior to generate synthetic imaging data. A neural network can then be trained to map the synthetic imaging data to the synthetic prior, with this network being the optimized inverse operator Gc-1.This network can then be applied to the original imaging data to output the restored imaging data, Sb, (e.g., a close approximation of the original structure function, S).

[0059] In a second example use case, certain aspects and features of the present disclosure can be used to improve reconstructions where the projection dataset is missing certain angles. Typically X-ray tomographic projections are taken through a sequence of angles evenly spaced through either 180 + fan angle degrees (e.g., for cone beam CT imaging) or 360 degrees, yieldingDocket No. 069608-000328WOPT 2023P01186WOa good reconstruction. Sometimes experimental constraints limit the number of angles available to reconstruction, particularly when using in situ equipment that requires highly attenuating / X-ray opaque equipment oriented such that it is not possible to acquire certain views. Traditional reconstruction yields artifacts in the form of streaks and anomalous reconstructed attenuations, causing ambiguity in analysis and issues with interpretation.

[0060] Unlike the noise and sparse sampling artifacts described in the first example use case, there is no existing process of record to correct for these artifacts (Ga-1). Certain aspects and features of the present disclosure relate to a new process for correcting for these artifacts.

[0061] First the dataset was reconstructed using the missing wedge projection data, both using all projections and offset projection subsets. A noise-to-noise model was trained and inferred to create a denoised volume which was then subtracted from the full projection reconstruction to give an estimate of the noise field (N).

[0062] Then a volume of arbitrary objects was created and forward projected into both the angle range available for missing wedge reconstruction and a full 360-degree series. Forward projecting the volume of arbitrary objects can include determining projective geometry associated with the received imaging data and forward projecting using that same projective geometry (e.g., source distance, detector distance, dither corrections, etc.). The two projection datasets were then re-back projected into the volume domain to create a pair of images - one corresponding to the missing wedge dataset, one for the full projection series.

[0063] The datasets were then scaled such that the mean and standard deviation of the synthetic data reconstructions matched that of the of the real data reconstructions, and the noise field was added to the missing wedge reconstruction. A residual network was trained to map from the missing wedge (+ noise) synthetic data reconstruction to the full angle synthetic data reconstruction, creating a network that has the effect of both removing the missing angle artefacts and the data noise, and this network was then applied to the real data missing wedge reconstruction (e.g., the received imaging data).

[0064] The resultant dataset constitutes the synthetic prior (Sa). However, this synthetic prior may still exhibit significant residual artifacts due to the imperfect correlation between the feature distribution in the synthetic object image and the feature distribution in the imaged structure.

[0065] A forward model (Gb) can be used to construct a synthetic data representation (Db) from the synthetic prior. In this example case, as the restoration is related to missing projection domainDocket No. 069608-000328WOPT 2023P01186WO information, first the synthetic prior is forward projected, creating a new projection dataset through a projective geometry corresponding to the original data. For a full field of view (FFOV) tomography the full information is known about the sample, but for interior tomography, correction is needed for the attenuation coming from outside the field of view to obtain an accurate forward projection model. This can be achieved by subtracting the forward projection (FP) from the data (D) to create a representation of the attenuation corresponding to materials outside the field of view (O) according to the equation,

[0066] This representation, however, will also represent all the spurious signal in D, as the forward projection does not contain this information, so this representation can be further denoised (e.g., denoised around a local kernel). This denoising could be a local mean, or gaussian filter, or similar simple denoising function, to create an outside of field of view representation Of according to the equation,

[0067] As this representation is only used to correct the attenuation values used for back projection, and the features in it are poorly correlated across views, the application of a spatial blur would not modify the reconstructed volume. It is therefore re-added to the forward projection set to create a final simulated projection set. This is then back projected (BP), creating a new dataset that has simulated missing wedge artifacts in it. Finally, the pre-estimated noise field (N) is added back to the missing wedge, creating a high fidelity synthetic data representation (Db).The construction of Db is therefore represented by the equation:

[0068] The noise field (here described as the original noise field) can be re-synthesized with the spectral matched noise method described above. A final restoration network can then be trained regressing this Db representation against the synthetic prior Sadescribed above. This can then be applied to the original data D to create the restored imaging data (Sb).

[0069] Certain aspects and features of the present disclosure improve a computing device's ability to process imaging data, especially with respect to imaging data from imagers like X-ray microscopes and the like. Certain aspects and features of the present disclosure improve the technical field of image processing by enabling new techniques for further improving image restoration. Certain aspects and features of the present disclosure improve the technical field of X-ray microscopy (and similar fields) by improving imaging restoration abilities, which canDocket No. 069608-000328WOPT 2023P01186WOresult in the need to collect fewer projections to achieve a desired image quality, and thus reduce imaging time and required data storage. Various other improvements are also achieved.

[0070] These illustrative examples are given to introduce the reader to the general subject matter discussed here and are not intended to limit the scope of the disclosed concepts. The following sections describe various additional features and examples with reference to the drawings in which like numerals indicate like elements, and directional descriptions are used to describe the illustrative embodiments but, like the illustrative embodiments, should not be used to limit the present disclosure. The elements included in the illustrations herein may not be drawn to scale.

[0071] FIG. 1 is a schematic diagram depicting an imaging data processing system 100, according to certain aspects of the present disclosure. The imaging data processing system 100 (e.g., control system) can include an imaging data source 102 that provides imaging data to a processing module 104. The imaging data source 102 can be any suitable source of imaging data, such as an imager (e.g., an imaging machine, such as an X-ray microscope or a CT scanner), a database of imaging data, a local memory storing imaging data, a removable memory storing imaging data, or the like. Certain aspects and features of the present disclosure are especially useful when the imaging data source 102 is an imager, such as an X-ray microscope.

[0072] The processing module 104 can process imaging data from the imaging data source 102. In some cases, the processing module 104 can control the imaging data source 102.

[0073] Processing module 104 can perform various operations on received imaging data, such as those described here (e.g., process 300 of FIG. 3). Memory 106 can store machine-readable instructions that enable the processing module 104 to perform its various operations. Memory 106 can also store untrained or trained models (e.g., machine learning algorithms or neural networks) that can be used in the restoration of imaging data. Memory 106 can store received imaging data, any intermediate data (e.g., synthetic prior data), and any output data (e.g., restored imaging data). Memory 106 can be implemented as locally accessible memory and / or remotely accessible memory (e.g., accessible via a network such as a local area network, a wide area network, a cloud network, or the Internet).

[0074] The processing module 104 can use imaging data (e.g., received imaging data and / or synthetic imaging data) to train one or more neural networks, such as an artificial neural network (ANN) (e.g., a deep neural network (DNN), a convolutional neural network (CNN), or the like), and / or use such a trained neural network to process imaging data or a reconstructed volume intoDocket No. 069608-000328WOPT 2023P01186WOimproved imaging data or an improved reconstructed volume. For example, received imaging data can be processed into a synthetic prior using a first imaging restoration process (e.g., a first, trained neural network), synthetic imaging data can be processed into a recreation of the synthetic prior to train a second imaging restoration process (e.g., a second neural network), and the received imaging data can be processed into restored imaging data using the trained, second imaging restoration process (e.g., the trained second neural network). In some cases, training of a first neural network can be used to facilitate training of a second neural network, such as through transfer learning.

[0075] The processing module 104 can also carry out reconstruction of imaging data (e.g., raw imaging data or improved imaging data) to generate reconstructed volumes (e.g., converting a set of acquired projections into a three-dimensional reconstructed volume). In some cases, the processing module 104 can access a pre-trained neural network from a memory 106, which can be applied as-is or can be further trained. In some cases, the pre-trained neural network can be a neural network that is generated using a federated learning technique, in which multiple trained neural networks can be collected and combined to generate a collaborative neural network that is distributed as the pre-trained neural network. In such cases, each pre-trained neural network can be associated with the same category of sample and / or the same or similar acquisition parameters, and the pre-trained neural network accessed by the processing module 104 can be accessed based on a provided category and / or set of acquisition parameters.

[0076] An input / output module 108 can be coupled to the processing module 104 to receive user input and provide output to a user. Any suitable input / output devices can be implemented in the input / output module 108, such as a keyboard, a mouse, a display (e.g., computer monitor), a touchscreen, light emitting diodes (LEDs) or other light sources, buttons, and the like. The processing module 104 can present reconstructions (e.g., reconstructed volumes) and / or further information derived therefrom to a user via the input / output module 108. In some cases, the input / output module 108 can store imaging data (e.g., received imaging data and / or restored imaging data), a reconstructed volume, and / or a trained neural network (e g., on a local memory, removable memory, or network-accessible memory). In some cases, a neural network trained using a processing module 104 can be stored in association with the imaging data and / or reconstructed volume.

[0077] In some cases, any of the imaging data source 102, the processing module 104, the memory 106, and the input / output module 108 can be incorporated into one or more housings inDocket No. 069608-000328WOPT 2023P01186WOany suitable combination. Any combination of one or more of the imaging data source 102, the processing module 104, the memory 106, and the input / output module 108 can be implemented locally (e.g., on the same device as one another or on devices coupled by a bus or local area network) or remotely (e.g., via a wide area network, the Internet, or a cloud network). In an example, a processing module 104 can be implemented on a user’s laptop computer, the imaging data source 102 can be implemented on a cloud-based health record database (e.g., one or more servers accessible via the Internet), and at least a portion of the memory 106 can be implemented on a separate cloud-based analysis database (e.g., one or more servers accessible via the Internet).

[0078] In another example, the processing module 104 can be incorporated into an imaging data source 102, such as a computer for processing imaging data that is also used to control an X-ray microscope. In another example, the processing module 104 can be incorporated into an individual computer that accesses, via a network-accessible database, imaging data supplied from a separate CT scanner or X-ray microscope.

[0079] FIG. 2 is a schematic diagram depicting an image restoration workflow 200, according to certain aspects of the present disclosure. The workflow 200 can be implemented using any suitable system, such as imaging data processing system 100 of FIG. 1. The workflow 200 depicts various imaging operators that connect various elements and imaging data. The workflow 200 is arranged on an image quality scale from left to right, with the leftmost end of the scale representing perfect quality, which by definition is the original subject original subject 202 being imaged.

[0080] The original subject 202 can be represented by a structure function S. When the original subject 202 is imaged (e.g., using an X-ray microscope), the result is received imaging data 206 (D). Various aspects of the imaging process (e.g., settings, mechanical limitations, etc.) result in the received imaging data 206 being degraded by some amount. This degradation can be represented by imaging device function 204, G, such that

[0081] The received imaging data 206 can be used to generate a synthetic prior 210 (Sa) through a first imaging restoration process 208 (GA-1), such that

[0082] The synthetic prior 210 can then be used to generate synthetic imaging data 214 (Db) by applying a forward imaging operator 212 (Gb), such that

[0083] The synthetic imaging data 214 can then be supplied as input to a second imaging restoration process 216 (Gc1) to generate recreations of the synthetic prior 210. By using theDocket No. 069608-000328WOPT 2023P01186WOsynthetic prior 210 as a desired output (e.g., as “labeled data”), the second imaging restoration process 216 can be trained / optimized until the second imaging restoration process 216 is able to generate a sufficiently suitable recreation of the synthetic prior 210 from the synthetic imaging data 214.

[0084] Then, the same second imaging restoration process 218 can be applied to the received imaging data 206 to generate the restored imaging data 220 (Sb), such that . The restored imaging data 220 is the improved imaging data that is improved over the received imaging data 206 and the synthetic prior 210.

[0085] The depiction of the received imaging data 206, synthetic prior 210, synthetic imaging data 214, and restored imaging data 220 at certain locations along an image quality scale is for illustrative purposes. In some cases, one or more of these elements may be located at different locations along the image quality scale. For example, the synthetic imaging data 214 may be located to the left or right of the received imaging data 206, rather than at the same point along the scale.

[0086] FIG. 3 is a flowchart depicting a process 300 for performing image restoration using a synthetic prior, according to certain aspects of the present disclosure. Process 300 can be performed using any suitable hardware, such as imaging data processing system 100 of FIG. 1.

[0087] At block 302, imaging data is received. Imaging data can be received from any suitable source, such as directly from an imager (e.g., an X-ray microscope), from a database of previously acquired imaging data, from a removable storage device, or the like. The imaging data can be in any suitable form. In some cases, the imaging data is raw imaging data representative of a set of projections acquired of a subject. Such raw imaging data may have undergone some degree of preprocessing prior to being received at block 302, although that need not always be the case. In some cases, the imaging data is an image reconstruction based on a set of projections acquired of a subject. In some cases, receiving imaging data at block 302 includes receiving metadata associated with the imaging data. In some cases, receiving imaging data at block 302 includes operating (e.g., sending signals to operate) an imager to acquire the imaging data.

[0088] At block 304, a first imaging restoration process can be determined. The first imaging restoration process can be determined, optionally based at least in part on the received imaging data. In some cases, a pre-determined first imaging restoration process can be used. In some cases, a pre-determined first imaging restoration process can be selected from a set of availableDocket No. 069608-000328WOPT 2023P01186WOpre-determined first imaging restoration processes, such as based on the received imaging data and / or metadata associated with the received imaging data (e.g., a particular first imaging restoration process can be selected for a particular imager and / or imaging settings used during acquisition of the received imaging data).

[0089] At block 306, a synthetic prior is generated using a first imaging restoration process. The first imaging restoration process can be any suitable image restoration process (e.g., an inverse imaging operator) that attempts to approximate a representation of the original subject that was imaged. Generally, the first imaging restoration process can be a state-of-the-art imaging restoration process, such as a state-of-the-art deep-learning-based image restoration / reconstruction tool (e.g., ZEISS® DeepRecon Pro™). In some cases, the first imaging restoration process can include other techniques, such as those described with reference to process 600 of FIG. 6.

[0090] The synthetic prior can be a form of imaging data. Often, the received imaging data will be in the form of a dataset of projections (e.g., in the projection domain) while the synthetic prior is in the form of a reconstruction (e.g., in the image domain or volume domain) based on that dataset of projections, although that need not always be the case.

[0091] At block 308, a forward imaging operator is determined. The forward imaging operator can be determined, optionally based at least in part on the synthetic prior and the received imaging data. In some cases, a pre-determined forward imaging operator can be used. In some cases, a pre-determined forward imaging operator can be selected from a set of available predetermined forward imaging operators, such as based on the synthetic prior, the received imaging data, and / or metadata associated with the received imaging data (e.g., a particular forward imaging operator can be selected for a particular imager and / or imaging settings used during acquisition of the received imaging data).

[0092] In some cases, however, determining the forward imaging operator can include extracting certain information from the synthetic prior and / or the received imaging data. For example, in some cases, determining the forward imaging operator at block 308 can include extracting information about the noise distribution of the received imaging data, such as described with reference to process 400 of FIG. 4.

[0093] At block 310, the forward imaging operator is applied to the synthetic prior to generate synthetic imaging data. The synthetic imaging data can be a form of imaging data. The syntheticDocket No. 069608-000328WOPT 2023P01186WOimaging data can be in the same form or domain as the received imaging data. For example, in some cases, the received imaging data from block 302 is in the projection domain, the synthetic prior of block 306 is in the image domain, and the synthetic imaging data of block 310 is in the projection domain. The synthetic imaging data can be a set of projections that represent the data an imager would output if it had acquired imaging data from the synthetic prior. The synthetic imaging data is referred to as synthetic because it is generated from the synthetic prior. Applying the forward imaging operator at block 310 can include applying the forward imaging operator such that the synthetic imaging data has the same level of noise and / or sampling artifacts as the received imaging data.

[0094] At block 312, a second imaging restoration process is trained using the synthetic prior and the synthetic imaging data. Training the second imaging restoration process can include using the synthetic imaging data as an input and using the synthetic prior as a desired output. Thus, as adjustments are made to the second imaging restoration process (e.g., adjustments to weights in nodes of a neural network) during training, the output of the second imaging restoration process supplied with the synthetic imaging data can be compared with the synthetic prior. Training the second imaging restoration process can include minimizing a loss function that is based on a difference between the synthetic prior and the output of the second imaging restoration process supplied with the synthetic imaging data. Any suitable machine learning techniques can be used to train the second imaging restoration process. The second imaging restoration process can take any suitable form, although often includes a neural network.

[0095] At block 314, the trained second imaging restoration process can be applied to the received imaging data to output the restored imaging data. The restored imaging data is imaging data that has been improved through the synthetic prior restoration process. The restored imaging data can be in any suitable form. The restored imaging data can be in the same form or domain as the synthetic prior.

[0096] While described with certain blocks in a certain order, the process 300, the sequence may be altered without departing from the scope of the present disclosure. For example, some blocks may be performed in parallel or in a different sequence that does not materially affect the function of the process 300. In other examples, different components of an example device or system that implements the process 300 may perform functions at substantially the same time or in a specific sequence. In some cases, process 300 may include fewer blocks or additional blocks. For example, in some cases process 300 may include an additional block after block 314 relatedDocket No. 069608-000328WOPT 2023P01186WOto outputting the restored imaging data, such as via displaying the restored imaging data using a display device.

[0097] FIG. 4 is a flowchart depicting a process 400 for generating synthetic imaging data, according to certain aspects of the present disclosure. Process 400 can be performed using any suitable hardware, such as imaging data processing system 100 of FIG. 1.

[0098] At block 402, imaging data is received. Receiving imaging data at block 402 can be the same as or similar to block 302 of FIG. 3.

[0099] At block 404, a synthetic prior is received. Receiving the synthetic prior at block 404 can include generating the synthetic prior (e.g., as described herein and with reference to block 306 of FIG. 3). In some cases, receiving the synthetic prior can include accessing a previously generated synthetic prior from a memory.

[0100] At block 406, an initial noise distribution is calculated based on the received imaging data from block 402 and the synthetic prior from block 404. The initial noise distribution is a noise field that is imparted on the received imaging data from the imaging process. The initial noise distribution can be calculated as the residual remaining after subtracting the synthetic prior from the received imaging data (e.g., subtracting an image domain representation of the received imaging data from an image domain representation of the synthetic prior). This residual, which includes the noise removed during the first imaging restoration process, can be used as an estimation or representation of the initial noise distribution.

[0101] At block 408, one or more parameters of the initial noise distribution are calculated. Calculating the one or more parameters of the initial noise distribution can include determining any suitable parameter of the initial noise distribution from the estimation of block 406. These parameters can include any measurable metric of the initial noise distribution, such as a local spatially variant standard deviation (e.g., as described herein and with respect to block 504 of FIG. 5). In some cases, calculating the one or more parameters at block 408 includes calculating a magnitude distribution of a Fourier transform of the initial noise distribution (e.g., as described herein and with respect to block 10 of FIG. 5).

[0102] At block 410, a synthetic noise field is generated based at least in part on the one or more parameters from block 408. The synthetic noise field can be generated to have one or more parameters that match or substantially match (e.g., within 1%, 2%, 3%, 4%, 5%, 6%, 7%, 8%,Docket No. 069608-000328WOPT 2023P01186WO9%, or 10% of) those of the initial noise distribution. In some cases, generating the synthetic noise field can occur as described herein or with reference to process 500 of FIG. 5.

[0103] At block 412, the synthetic noise field can be applied to the synthetic prior to obtain synthetic imaging data. Applying the synthetic noise field to the synthetic prior can include adding the synthetic noise field to the synthetic prior as at least part of the forward imaging operator.

[0104] While described with certain blocks in a certain order, the process 400, the sequence may be altered without departing from the scope of the present disclosure. For example, some blocks may be performed in parallel or in a different sequence that does not materially affect the function of the process 400. In other examples, different components of an example device or system that implements the process 400 may perform functions at substantially the same time or in a specific sequence. In some cases, process 400 may include fewer blocks or additional blocks.

[0105] FIG. 5 is a flowchart depicting a process 500 for generating synthetic imaging data using frequency domain transforms, according to certain aspects of the present disclosure. Process 500 can be performed using any suitable hardware, such as imaging data processing system 100 of FIG. 1.

[0106] At block 502, an initial noise distribution can be received. The initial noise distribution can be received from memory or can be calculated (e.g., as described herein and with reference to block 406 of FIG. 4).

[0107] At block 504, a local spatially variant standard deviation can be calculated from the initial noise distribution of block 502. Calculating the local spatially variant standard deviation can include determining a working window size defining the size of a local area (e.g., a 3 -voxel x 3 voxel x 3 voxel volumetric window), then calculating the spatially variant standard deviation within that window. In some cases, block 504 includes calculating the local spatially variant standard deviation for one, some, or all voxels in the initial noise distribution. For example, a local spatially variant standard deviation field can be calculated for the entire initial noise distribution by calculating the local spatially variant standard deviation at each voxel in the initial noise distribution.

[0108] At block 506, a random noise field can be generated using the local spatially variant standard deviation calculation(s). Generating the random noise field can include generating a random noise field that has local spatially variant standard deviation(s) that match orDocket No. 069608-000328WOPT 2023P01186WO substantially match (e.g., within 1%, 2%, 3%, 4%, 5%, 6%, 7%, 8%, 9%, or 10% of) the local spatially variant standard deviation(s) of block 504. In some cases, the random noise field can be generated to have its own local spatially variant standard deviation field that matches or substantially matches the local spatially variant standard deviation field calculated at block 504. As used herein, the term random can include truly random or pseudorandom.

[0109] At block 508, a phase field is generated from the frequency domain transformation of the random noise field from block 506. The random noise field can be passed through a frequency domain transformation (e.g., a Fourier transform) to create a frequency domain representation of the random noise field containing both phase and magnitude information at each voxel. The phase field can be generated by taking only the phase information from this frequency domain representation of the random noise field.

[0110] At block 510, a magnitude field can be generated from the frequency domain representation of the initial noise distribution. The initial noise distribution can be passed through a frequency domain transformation (e.g., a Fourier transform) to create a frequency domain representation of the initial noise distribution containing both phase and magnitude information at each voxel. The magnitude field can be generated by taking only the magnitude information from this frequency domain representation of the initial noise distribution.[OHl] At block 512, a synthetic noise field can be generated from the phase field of block 508 and the magnitude field of block 512. A complex frequency domain representation of the synthetic noise field can be generated by the equation , where N is the synthetic noise field, Ni mag is the magnitude field, Gphase is the phase field, and i is . An inverse frequency domain transformation (e.g., inverse Fourier transform) can be applied to the frequency domain representation of the synthetic noise field to generate a complex representation of the synthetic noise field. The actual synthetic noise field to be used can be extracted as the real portion of the resultant complex representation of the synthetic noise field.

[0112] At block 514, synthetic imaging data is generated using the synthetic prior and the synthetic noise field. Generating the synthetic imaging data can include applying the synthetic noise field to the synthetic prior, such as described herein and with reference to block 412 of FIG. 4.Docket No. 069608-000328WOPT 2023P01186WO

[0113] While described with certain blocks in a certain order, the process 500, the sequence may be altered without departing from the scope of the present disclosure. For example, some blocks may be performed in parallel or in a different sequence that does not materially affect the function of the process 500. In other examples, different components of an example device or system that implements the process 500 may perform functions at substantially the same time or in a specific sequence. In some cases, process 500 may include fewer blocks or additional blocks. For example, in some cases blocks 504, 506, 508 can be replaced by a block that includes generating a phase field using a uniform random number generator, in which case generating the synthetic noise field at block 512 includes using that phase field instead of one generated based on the local spatially variant standard deviation calculations.

[0114] FIG. 6 is a flowchart depicting a process 600 for improving restoration of limited-angle imaging data, according to certain aspects of the present disclosure. Process 600 can be performed using any suitable hardware, such as imaging data processing system 100 of FIG. 1.

[0115] At block 602, imaging data is received. Receiving imaging data at block 602 can be the same as or similar to block 302 of FIG. 3. The imaging data can include raw imaging data representing projections acquired through a limited set of angles, rather than a full set of angles. This limited set of angles can be referred to as the available angle range. A full set of angles may include projections from 0 degrees up through 180 degrees, 360 degrees, and / or some value between 180 degrees and 360 degrees. A limited set of angles would have one or more “wedges” of angles missing from the projection dataset. As an example, received imaging data may include projections only from 0 degrees up to 150 degrees, or from 0 degrees up to 120 degrees and 150 degrees up to 180 degrees (missing a wedge of projections between 120 degrees to 150 degrees). If the imaging data received at block 602 is not in the image domain, a reconstruction of the imaging data can be generated. Thus, the imaging data output from block 602 can be in the image domain.

[0116] At block 604, an estimated noise field is determined from the received imaging data. This estimated noise field can be determined through any suitable technique. In some cases, determining the estimated noise field can include first processing the received imaging data using a first imaging restoration process that denoises the received imaging data. The resultant denoised imaging data can be subtracted from the received imaging data to generate the estimated noise field.Docket No. 069608-000328WOPT 2023P01186WO

[0117] At block 606, a training volume is obtained. Obtaining a training volume can include using a pre-determined training volume, selecting a pre-determined training volume from a set of available training volumes, or generating a training volume. In some cases, selecting the predetermined training volume can include selecting the pre-determined training volume based on the received imaging data or metadata associated with the received imaging data (e.g., an indication of the type of material being examined by an X-ray microscope). In some cases, generating the pre-determined training volume can include generating a volume of arbitrary objects to simulate the scale of features present in the sample imaged to generate the received imaging data.

[0118] At block 608, the training volume from block 606 can be forward projected in the available angle range to generate a dataset of projections, for the training volume, associated with the available angle range. Any suitable forward imaging operator can be used, such as those described herein. The resultant dataset of projections can be referred to as the constrained training volume imaging data.

[0119] At block 610, a constrained training reconstruction can be generated from the constrained training volume imaging data. This constrained training reconstruction will be a reconstruction of the training volume as if the training volume were imaged only through the available angle range.

[0120] At block 612, the estimated noise field from block 604 is added to the constrained training reconstruction. As a result, the constrained training reconstruction will now exhibit a noise field that is equivalent to that of the received imaging data.

[0121] At block 614, the training volume from block 606 can be forward projected in the full range to generate a dataset of projections, for the training volume, associated with a full range of angles (e.g., 360 degrees). Any suitable forward imaging operator can be used, such as those described herein. The resultant dataset of projections can be referred to as the full training volume imaging data.

[0122] At block 616, a full training reconstruction can be generated from the full training volume imaging data. This full training reconstruction will be a reconstruction of the training volume as if the training volume were imaged through the full range of angles.

[0123] At block 618, an imaging restoration process is trained using the constrained training reconstruction and the full training reconstruction. The image imaging restoration process canDocket No. 069608-000328WOPT 2023P01186WOtake the constrained training reconstruction as input and can use the full training reconstruction as a desired output. Thus, the imaging restoration process will be trained specifically to take as input a reconstruction that is constrained to the specific limited set of angles available in the received imaging data (e.g., the available angle range) and that contains the same noise field as the received imaging data, and to output a reconstruction that is based on the full range of angles and without noise.

[0124] At block 620, the trained imaging restoration process from block 618 is applied to the imaging data from block 602 (e.g., a reconstruction of raw imaging data) to generate an improved reconstruction. In some cases, this improved reconstruction can be used as a final output (e.g., for display to a user). However, in some cases, this improved reconstruction is used as a synthetic prior.

[0125] Blocks 622, 624, 626, 628 can be used to correct for attenuation corresponding to materials outside of the field of view.

[0126] At block 622, a synthetic prior (e g., the synthetic prior from block 620) is forward projected to obtain a new projection dataset. Any suitable forward imaging operator can be used, such as those described herein.

[0127] At block 624, an attenuation representation is determined using the received imaging data from block 602 and the new projection dataset from block 622. Determining the attenuation representation can include subtracting the new projection dataset (e.g., a reconstruction thereof) from the received imaging data (e.g., a reconstruction). The residual from this subtraction includes data that is present in the new projection dataset but not in the received imaging data, which is a representation of the attenuation occurring due to materials outside the field of view. This representation can be referred to as the outside-field-of-view (OFOV) attenuation representation.

[0128] At block 626, this OFOV attenuation representation can be denoised to obtain a denoised OFOV attenuation representation, which more accurately represents the attenuation from outside the of field of view as it does not include other spurious signal from the received imaging data.

[0129] At block 628, synthetic imaging data based on the synthetic prior can be output. The denoised OFOV attenuation representation from block 626 can be added to the new projection dataset to achieve a new projection dataset with attenuation. Then, the new projection dataset with attenuation outside the FOV can be back-projected into the image domain. Then, theDocket No. 069608-000328WOPT 2023P01186WOestimated noise field from block 604 can be added to this back-projection. The resultant imaging data can be used as the synthetic imaging data.

[0130] The synthetic imaging data from block 628 can then be used for further purposes, such as to train a second imaging restoration process using this synthetic imaging data as input and the synthetic prior as the desired output. The trained second imaging restoration process can then be applied to the received imaging data to generate restored imaging data.

[0131] While described with certain blocks in a certain order, the process 600, the sequence may be altered without departing from the scope of the present disclosure. For example, some blocks may be performed in parallel or in a different sequence that does not materially affect the function of the process 600. In other examples, different components of an example device or system that implements the process 600may perform functions at substantially the same time or in a specific sequence. In some cases, process 600 may include fewer blocks or additional blocks. For example, in some cases, especially when the imaging is a full field of view imaging, blocks 622, 624, 626, and 628 may be removed and the reconstruction resulting from block 620 can be used as a final output (e.g., for display to a user or for further analysis) or as a synthetic prior to which a different second imaging restoration process is applied (e g., as described with reference to process 300 of FIG. 3).

[0132] FIG. 7 is a set of images depicting various sets of imaging data of sandstone rock, according to certain aspects of the present disclosure. The imaging data depicted in FIG. 7 was acquired from X-ray microscopy of sandstone rock. Each column depicts a full field of view image at the top and a zoomed-in portion below. The imaging data depicted in FIG. 7 represents different imaging data at different times in a synthetic prior image restoration method, such as process 300 of FIG. 3. Specifically, the imaging data depicted in FIG. 7 was generated through processes the same as or similar to processes 400, 500 of FIG. 4 and FIG. 5, respectively.

[0133] The full field received imaging data 710 depicts an image of the sandstone rock as acquired from the imager, prior to any restoration processes. As seen in the zoomed received imaging data 712, the image quality is somewhat low, with much noise and various artifacts.

[0134] Applying a first imaging restoration process to the received imaging data results in the full field synthetic prior 714. As seen in the full field synthetic prior 714 and the zoomed synthetic prior 716, the image quality is much improved from the full field received imaging data 710 and zoomed received imaging data 712. The noise is greatly reduced, and some artifacts are reducedDocket No. 069608-000328WOPT 2023P01186WOor eliminated. However, certain artifacts remain (e.g., residual artifacts 730) and certain structures are not clearly depicted (e.g., faint structure 728).

[0135] As described herein, a forward imaging operator is applied to the synthetic prior to generate the full field synthetic imaging data 718. The full field synthetic imaging data 718 and zoomed synthetic imaging data 720 appears somewhat similar to the full field received imaging data 710 and zoomed received imaging data 712, respectively, which is to be expected, as the synthetic imaging data is intended to be used as an input to train a second imaging restoration process.

[0136] A second imaging restoration process can be trained using the full field synthetic prior 714 and full field synthetic imaging data 718 (or at least using portions thereof, such as the zoomed synthetic prior 716 and the zoomed synthetic imaging data 720). Once trained, the second imaging restoration process can be applied to the full field received imaging data 710 to generate the full field restored imaging data 722.

[0137] As seen in FIG. 7, the full field restored imaging data 722 and the zoomed restored imaging data 724 is not only much improved over the full field received imaging data 710 and zoomed received imaging data 712, but also substantially improved over the full field synthetic prior 714 and zoomed synthetic prior 716. The full field restored imaging data 722 has further reduced noise and artifacts, while retaining better clarity of small structures. For example, improved structure 732 is much more visible in the zoomed restored imaging data 724 than the faint structure 728 in the zoomed synthetic prior 716. Likewise, the residual artifacts 730 from the full field synthetic prior 714 have been much improved in the zoomed restored imaging data 724, resulting in improved residual artifacts 734 (e.g., no artifacts or reduced artifacts).

[0138] Overall, the amount of recovery achieved by the full field restored imaging data 722 is substantially improved over that of the full field synthetic prior 714.

[0139] FIG. 8 is a set of images depicting various sets of imaging data of sandstone rock, according to certain aspects of the present disclosure. The imaging data depicted in FIG. 8 was acquired from X-ray microscopy of sandstone rock. Each column depicts a full field of view image at the top and a zoomed-in portion below.

[0140] The full field received imaging data 804 depicts an image of the sandstone rock as acquired from the imager, prior to any restoration processes. As seen in the zoomed received imaging data 806, the image quality is somewhat low, with much noise and various artifacts.Docket No. 069608-000328WOPT 2023P01186WO

[0141] The full field synthetic prior using standard first imaging restoration process 808 is a synthetic prior obtained using a standard first imaging restoration process, such as that used to generate the full field synthetic prior 714 of FIG. 7. The standard first imaging restoration process is one that is intended to output an image suitable for final output, such as to be displayed to a user or used for substantive analysis. As seen in the zoomed synthetic prior using standard first imaging restoration process 810, the first imaging restoration process has been trained to achieve good looking results with reduced noise and certain artifacts (e.g., in the center of grains), such as at region 820, however is subject to over-smoothing of small features (e.g., small feature 822).

[0142] If the full field synthetic prior using standard first imaging restoration process 808 were used as the synthetic prior for generating synthetic imaging data and training a second imaging restoration process, the final restored imaging data that is output can be the same as or similar to zoomed restored imaging data 724 of FIG. 7.

[0143] However, since the synthetic prior is only used as intermediate imaging data according to certain aspects and features of the present disclosure, it is not otherwise be displayed to user or used for substantive analysis. Thus, in some cases, the first imaging restoration process used to generate the synthetic prior can be adjusted to produce intermediate imaging data that is more useful for the generation of synthetic imaging data and training of a second imaging restoration process, at the cost of visual or other degradations that may make the synthetic prior otherwise unsuitable for a final output.

[0144] The full field synthetic prior using adjusted first imaging restoration process 812 is a synthetic prior that has been obtained using an adjusted first imaging restoration process. In the example used with FIG. 8, the adjusted first imaging restoration process has an adjusted network structure and uses an asymmetric loss function rather than a symmetric loss function. As seen in the zoomed synthetic prior using adjusted first imaging restoration process 814, this synthetic prior gives results with worse residual artifacts (e g., residual artifact 824) in the center of the grains (e.g., due to an increase in correlation between spurious signals between the subsets), but with the benefit of reducing the over-smoothing of small features (e.g., small feature 826).

[0145] Since the residual artifacts (e.g., residual artifact 824) of the synthetic prior are correlated with the synthetic imaging data, the second imaging restoration process does not learn to recover these residual artifacts. Thus, the residual artifacts are not present in the full fieldDocket No. 069608-000328WOPT 2023P01186WOrestored imaging data 816 and zoomed restored imaging data 818, as seen from the artifact-free region 828.

[0146] Further, the presence of not-over-smoothed, distinct small features (e.g., small feature 830) in the zoomed restored imaging data 818 means that the feature distribution present in the full field synthetic prior using adjusted first imaging restoration process 812 more closely correlates to the true structure feature distribution, resulting in better feature recover than if the standard first imaging restoration process were used in the creation of the synthetic prior.

[0147] Many other such adjustments can be made to the workflow (e g., process 300 of FIG. 3) to achieve a synthetic prior that closely correlates to the structure function, such as loss weighting in the final recovery network, weighting the loss computed in regions of high gradient, or corresponding to small features, higher than the loss in the center of large uniform objects, potentially giving better recovery of small features.

[0148] FIG. 9 is a chart 902 depicting the performance of various imaging restoration processes, according to certain aspects of the present disclosure. Chart 902 depicts the performance of various imaging restoration processes at various projection sampling factors. The projection sampling factors are based on a full 1601 projection dataset, such that a projection sampling factor of 1 corresponds to a reconstruction that is provided all 1601 projections to use, whereas a projection sampling factor of 2 corresponds to a reconstruction that is provided 800 projections to use. The performance of each imaging restoration process is depicted as the root mean square error (RMSE) of the reconstructed dataset (e.g., restored imaging data), computed relative to a best structure function (S) estimate (e.g., best restored imaging data) found using synthetic prior reconstruction using the full projection set of 1601 projections (e.g., a projection sampling of 1).

[0149] Line 904 represents the Feldkamp, Davis and. Kress (FDK) reconstruction algorithm, which always shows a high RMSE due to its extensive noise and artifacts. Dotted line 916 is a fitted polynomial regression curve for line 904.

[0150] Line 908 is an existing imaging restoration process that relies on deep learning based X-ray reconstruction. Line 908 shows significantly better fidelity of reconstruction than line 904. Dotted line 918 is a fitted polynomial regression curve for line 908.

[0151] Line 910 is a synthetic prior reconstruction (e.g., as described with reference to process 300 of FIG. 3) that uses the imaging restoration process from line 908 as its first imaging restoration process, to generate its synthetic prior, which is then used to generate syntheticDocket No. 069608-000328WOPT 2023P01186WOimaging data and train a second imaging restoration process, which is then used to output the restored imaging data from the received imaging data. Dotted line 920 is a fitted polynomial regression curve for line 910. It can be seen that line 910 provides significantly better fidelity of reconstruction than line 904 and line 908.

[0152] Line 912 is a synthetic prior reconstruction that uses an adjusted imaging restoration process, such as that described with reference to FIG. 8, to generate its synthetic prior. Specifically, the imaging restoration process used for line 912 is the imaging restoration process from line 908 with a modified loss and network structure. Dotted line 922 is a fitted polynomial regression curve for line 912. It can be seen that line 912 provides significantly better fidelity of reconstruction than line 904 and line 908, and slightly better fidelity of reconstruction than line 910.

[0153] The dashed line 924 is an indication of the improvement achieved. Dashed line 924 is set at the RMSE for line 912 at a projection sampling factor of 4 (e.g., 400 projections). The position on the X axis of the intersection of this dashed line 924 with the other lines corresponds to the number of projections that these other techniques require to achieve the same RMSE as synthetic prior with an adjusted network and loss achieves at 400 projections. For example, line 908 shows that the existing imaging restoration process would require use of approximately 670 projections to achieve the same results as the synthetic prior (with adjusted network and loss) can achieve with 400 projections, an effective speed increase of 67%.

[0154] FIG. 10 is a set of image reconstructions for a sample, according to certain aspects of the present disclosure. The all-angle reconstruction 1002 is a reconstruction of a sample generated by using all projection angles (e.g., 0-180 degrees). The limited-angle reconstruction 1004 is a reconstruction of the same sample generated by using a limited set of projection angles (e.g., 30-150 degrees). As seen in limited-angle reconstruction 1004, streaks and anomalous reconstructed attenuations are present where none appear in the all-angle reconstruction 1002.

[0155] FIG. 11 is a set of image reconstructions for a training volume, according to certain aspects of the present disclosure. The training volume can be any suitable volume containing various structures that can somewhat approximate the intended sample for which imaging restoration is intended. For example, for an imaging of sandstone, a volume of arbitrary objects (e.g., spheres, cubes, other shapes) can be generated. This training volume can then be forward projected into both an all-angle dataset and a limited-angle dataset. Specifically, the limited setDocket No. 069608-000328WOPT 2023P01186WOof projection angles used for the limited-angle dataset the same limited set of projection angles available in the received imaging data (e.g., the projection angles associated with limited-angle reconstruction 1004 of FIG. 10).

[0156] The all-angle reconstruction 1102 is a reconstruction of the training volume generated by using the all-angle dataset. The limited-angle reconstruction 1104 is a reconstruction of the same training volume generated by using the limited-angle dataset.

[0157] FIG. 12 is a set of image reconstructions for a sample, according to certain aspects of the present disclosure. The initial reconstruction 1204 is a reconstruction of received imaging data corresponding to imaging of a sample. Here, the received imaging data contains projections from only a limited set of angles (e.g., 30-150 degrees). As such, the initial reconstruction 1204 contains various artifacts seen in other limited-angle reconstructions (e.g., limited-angle reconstruction 1004 of FIG. 10).

[0158] The synthetic prior 1206 is a corrected version of the initial reconstruction 1204 after application of an imaging restoration process that has been trained using a synthetic training volume (e.g., the training volume described with reference to FIG. 11). The synthetic prior 1206 exhibits a significant reduction in the missing wedge artifact.

[0159] However, some further artifacts remain in the synthetic prior 1206, such as anomalously non-uniform reconstructed absorption values (e.g., region 1210) and anomalously high absorption values (ghosting) around the edge of features (e.g., region 1212) and around high-intensity grains (e.g., region 1210). This synthetic prior 1206 can be further used as described with reference to process 600 of FIG. 6.

[0160] FIG. 13 is a set of image reconstructions for a sample (e.g., the sample of FIG. 12), according to certain aspects of the present disclosure. The synthetic prior 1304 can be the same as or similar to the synthetic prior 1206 of FIG. 12. As seen in the synthetic prior 1304, various artifacts remain, such as anomalously non-uniform reconstructed absorption values (e.g., region 1310) and anomalously high absorption values (ghosting) around the edge of features (e.g., region 1308) and around high-intensity grains (e.g., region 1312).

[0161] The restored imaging data 1306 is imaging data that has been processed by a second imaging restoration process that was trained based on the synthetic prior 1206 and synthetic imaging data generated from the synthetic prior (e.g., generated according to process 600 of FIG.6).Docket No. 069608-000328WOPT 2023P01186WO

[0162] As seen in the restored imaging data 1306, the anomalously low reconstructed values are reduced, and the ghosting near the edge of features and around high-intensity grains is removed.

[0163] FIG. 14 is a block diagram of an example system architecture 1400 for implementing features and processes of the present disclosure, such as those presented with reference to processes 300, 400, 500, 600, of FIGs. 3, 4, 5, 6, respectively. The features and processes disclosed herein can be implemented using one or multiple instances of 1400. The system architecture 1400 can be used to implement a server (e.g., a cloud-accessible server), a user device (e.g., a smartphone or personal computer), or any other suitable device for performing some or all of the aspects of the present disclosure. The system architecture 1400 can be implemented on any electronic device that runs software applications derived from compiled instructions, including without limitation personal computers, servers, imagers (e.g., X-ray microscopes), smart phones, electronic tablets, game consoles, email devices, and the like. In some implementations, the system architecture 1400 can include one or more processors 1404, one or more input devices 1412, one or more display devices 1410, one or more network interfaces 1408, and one or more computer-readable media 1420. Each of these components can be coupled by bus 1418.

[0164] Display device 1410 can be any known display technology, including but not limited to display devices using Liquid Crystal Display (LCD) or Light Emitting Diode (LED) technology. Processor(s) 802 can use any known processor technology, including but not limited to graphics processors and multi-core processors. Input device 1412 can be any known input device technology, including but not limited to a keyboard (including a virtual keyboard), mouse, track ball, and touch-sensitive pad or display. In some cases, audio inputs can be used to provide audio signals, such as audio signals of an individual speaking. Bus 1418 can be any known internal or external bus technology, including but not limited to ISA, EISA, PCI, PCI Express, NuBus, USB, Serial ATA or FireWire.

[0165] Computer-readable medium 1420 can be any medium that participates in providing instructions to processor 1404 for execution, including without limitation, non-volatile storage media (e.g., optical disks, magnetic disks, flash drives, etc.) or volatile media (e.g., SDRAM, ROM, etc.). The computer-readable medium (e.g., storage devices, mediums, and memories) can include, for example, a cable or wireless signal containing a bit stream and the like. However,Docket No. 069608-000328WOPT 2023P01186WOwhen mentioned, non-transitory computer-readable storage media expressly exclude media such as energy, carrier signals, electromagnetic waves, and signals per se.

[0166] Computer-readable medium 1420 can include various instructions for implementing operating system 1414 and applications 1416 such as computer programs. The operating system 1414 can be multi-user, multiprocessing, multitasking, multithreading, real-time and the like. The operating system 1414 performs basic tasks, including but not limited to: recognizing input from input device 1412; sending output to display device 1410; keeping track of fdes and directories on computer-readable medium 1420; controlling peripheral devices (e.g., storage drives, interface devices, etc.) which can be controlled directly or through an I / O controller; and managing traffic on bus 1418. Computer-readable medium 1420 can include various instructions for implementing firmware processes, such as a BIOS. Computer-readable medium 1420 can include various instructions for implementing any of the processes described herein, including at least processes 300, 400, 500, 600 of FIGs. 3, 4, 5, 6, respectively.

[0167] Memory 1406 can include high-speed random access memory and / or non-volatile memory, such as one or more magnetic disk storage devices, one or more optical storage devices, and / or flash memory (e g., NAND, NOR). The memory 1406 (e g., computer-readable storage devices, mediums, and memories) can include a cable or wireless signal containing a bit stream and the like. However, when mentioned, non-transitory computer-readable storage media expressly exclude media such as energy, carrier signals, electromagnetic waves, and signals per se. The memory 1406 can store an operating system, such as Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embedded operating system such as VxWorks.

[0168] System controller 1402 can be a service processor that operates independently of processor 1404. In some implementations, system controller 1402 can be a baseboard management controller (BMC). For example, a BMC is a specialized service processor that monitors the physical state of a computer, network server, or other hardware device using sensors and communicating with the system administrator through an independent connection. The BMC is configured on the motherboard or main circuit board of the device to be monitored. The sensors of a BMC can measure internal physical variables such as temperature, humidity, powersupply voltage, fan speeds, communications parameters and operating system (OS) functions.

[0169] The described features can be implemented advantageously in one or more computer programs that are executable on a programmable system including at least one programmableDocket No. 069608-000328WOPT 2023P01186WOprocessor coupled to receive data and instructions from, and to transmit data and instructions to, a data storage system, at least one input device, and at least one output device. A computer program is a set of instructions that can be used, directly or indirectly, in a computer to perform a certain activity or bring about a certain result. A computer program can be written in any form of programming language (e.g., Objective-C, Java), including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0170] Suitable processors for the execution of a program of instructions include, by way of example, both general and special purpose microprocessors, and the sole processor or one of multiple processors or cores, of any kind of computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memories for storing instructions and data. Generally, a computer will also include, or be operatively coupled to communicate with, one or more mass storage devices for storing data files; such devices include magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and optical disks. Storage devices suitable for tangibly embodying computer program instructions and data include all forms of non-volatile memory, including by way of example semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, ASICs (application-specific integrated circuits).

[0171] To provide for interaction with a user, the features can be implemented on a computer having a display device such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor for displaying information to the user and a keyboard and a pointing device such as a mouse or a trackball by which the user can provide input to the computer.

[0172] The features can be implemented in a computing system that includes a back-end component, such as a data server, or that includes a middleware component, such as an application server or an Internet server, or that includes a front-end component, such as a client computer having a graphical user interface or an Internet browser, or any combination thereof. The components of the system can be connected by any form or medium of digital data communication such as a communication network. Examples of communication networks include, e.g., a LAN, a WAN, and the computers and networks forming the Internet.Docket No. 069608-000328WOPT 2023P01186WO

[0173] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0174] One or more features or steps of the disclosed embodiments can be implemented using an application programming interface (API). An API can define one or more parameters that are passed between a calling application and other software code (e.g., an operating system, library routine, function) that provides a service, that provides data, or that performs an operation or a computation.

[0175] The API can be implemented as one or more calls in program code that send or receive one or more parameters through a parameter list or other structure based on a call convention defined in an API specification document. A parameter can be a constant, a key, a data structure, an object, an object class, a variable, a data type, a pointer, an array, a list, or another call. API calls and parameters can be implemented in any programming language. The programming language can define the vocabulary and calling convention that a programmer will employ to access functions supporting the API.

[0176] In some implementations, an API call can report to an application the capabilities of a device running the application, such as input capability, output capability, processing capability, power capability, communications capability, and the like.

[0177] The foregoing description of the embodiments, including illustrated embodiments, has been presented only for the purpose of illustration and description and is not intended to be exhaustive or limiting to the precise forms disclosed. Numerous modifications, adaptations, and uses thereof will be apparent to those skilled in the art. Numerous changes to the disclosed embodiments can be made in accordance with the disclosure herein, without departing from the spirit or scope of the disclosure. Thus, the breadth and scope of the present disclosure should not be limited by any of the above described embodiments.

[0178] Although certain aspects and features of the present disclosure have been illustrated and described with respect to one or more implementations, equivalent alterations and modifications will occur or be known to others skilled in the art upon the reading and understanding of this specification and the annexed drawings. In addition, while a particular feature may have been disclosed with respect to only one of several implementations, such feature may be combinedDocket No. 069608-000328WOPT 2023P01186WOwith one or more other features of the other implementations as may be desired and advantageous for any given or particular application.

[0179] The terminology used herein is for the purpose of describing particular embodiments only, and is not intended to be limiting of the disclosure. As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. Furthermore, to the extent that the terms “including,” “includes,” “having,” “has,” “with,” or variants thereof, are used in either the detailed description and / or the claims, such terms are intended to be inclusive in a manner similar to the term “comprising.”

[0180] One or more elements or aspects or steps, or any portion(s) thereof, from one or more of any of the claims below can be combined with one or more elements or aspects or steps, or any portion(s) thereof, from one or more of any of the other claims below or combinations thereof, to form one or more additional implementations and / or claims of the present disclosure.

Claims

Docket No. 069608-000328WOPT 2023P01186WOCLAIMSWhat is claimed is:

1. A computer-implemented image restoration method, comprising:receiving imaging data associated with a subject;generating a synthetic prior by applying a first imaging restoration process to the received imaging data;determining a forward imaging operator;generating synthetic imaging data by applying the forward imaging operator to the synthetic prior;training a second imaging restoration process based at least in part on the synthetic imaging data and the synthetic prior; andapplying the second imaging restoration process to the received imaging data to generate restored imaging data.

2. The image restoration method of claim 1, wherein applying the first imaging restoration process includes applying a trained machine learning algorithm to the received imaging data.

3. The image restoration method of claim 1 or 2, wherein the second imaging restoration process includes a machine learning algorithm, and wherein training the second imaging restoration process includes optimizing a hyperparameter of the machine learning algorithm.

4. The image restoration method of claim 3, wherein optimizing the hyperparameter includes:generating interim restored imaging data by applying the second imaging restoration process to the synthetic imaging data; anddetermining a difference between the synthetic prior and the interim restored imaging data, wherein the hyperparameter is optimized when the difference between the synthetic prior and the interim restored imaging data is minimized.

5. The image restoration method of any one of claims 1 to 4, wherein determining the forward imaging operator includes:receiving metadata associated with the imaging data; andselecting the forward imaging operator from a set of preset forward imaging operators based at least in part on the received metadata.Docket No. 069608-000328WOPT 2023P01186WO6. The image restoration method of any one of claims 1 to 5, wherein the received imaging data is in the image space and the synthetic prior is in the image space.

7. The image restoration method of any one of claims 1 to 6, wherein determining the forward imaging operator includes:calculating an initial noise distribution based at least in part on the received imaging data and the synthetic prior;calculating one or more parameters associated with the initial noise distribution; and generating a synthetic noise field based at least in part on the calculated one or more parameters, wherein generating the synthetic imaging data includes applying the synthetic noise field to the synthetic prior.

8. The image restoration method of claim 7, wherein calculating the initial noise distribution includes including applying the equationwhere Ni is the initial noise distribution, D is the received imaging data, and Sa is the synthetic prior.

9. The image restoration method of claim 7 or 8, wherein calculating the one or more parameters associated with the initial noise distribution includes generating a frequency domain representation of the initial noise distribution, and wherein generating the synthetic noise field includes:generating a magnitude field from the frequency domain representation of the initial noise distribution; andcreating the synthetic noise field based at least in part on the magnitude field.

10. The image restoration method of claim 9, wherein generating the synthetic noise field further includes generating a random phase field using a uniform random number generator, and wherein creating the synthetic noise field is further based at least in part on the random phase field.

11. The image restoration method of claim 9, wherein calculating the one or more parameters associated with the initial noise distribution further includes calculating a local spatially variant standard deviation of the initial noise distribution, and wherein generating the synthetic noise field further includes:Docket No. 069608-000328WOPT 2023P01186WOgenerating a random noise field based at least in part on the local spatially variant standard deviation; andgenerating a phase field based at least in part on a frequency domain representation of the random noise field; andwherein creating the synthetic noise field is further based at least in part on the generated phase field.

12. The image restoration method of claim 9, wherein generating the synthetic noise field includes applying the equationwhere N is the synthetic noise field, Nimag is the magnitude field of the frequency domain representation of the initial noise distribution, Gphase is a phase field derived from a frequency domain representation of a random noise field generated based at least in part on a calculated local spatially variant standard deviation of the initial noise distribution, i is , real is a real component function, and ifft is an inverse Fourier transform.

13. The image restoration method of any one of claims 7 to 12, wherein the synthetic noise field has equivalent local Fourier magnitudes to the initial noise distribution, and wherein local phases of the synthetic noise field are fully decorrelated to the initial noise distribution.

14. The image restoration method of any one of claims 1 to 13, wherein training the second imaging restoration process includes:determining one or more regions of high gradient associated with the synthetic prior; and training a machine learning algorithm using a loss function based at least in part on the synthetic prior and the synthetic imaging data, wherein the loss function weights a loss in the determined one or more regions of high gradient higher than a loss not in the determined one or more regions of high gradient.

15. The image restoration method of any one of claims 1 to 14, wherein generating the synthetic prior includes:generating an initial reconstruction based at least in part on the received imaging data; determining an estimated noise field based at least in part on the initial reconstruction; determining an available angle range associated with the received imaging data;Docket No. 069608-000328WOPT 2023P01186WOaccessing a training volume;forward-projecting the training volume in the available angle range to generate constrained training volume imaging data;forward-projecting the training volume in full range to generate full training volume imaging data;generating a constrained training reconstruction based at least in part on the constrained training volume imaging data;generating a full training reconstruction based at least in part on the full training volume imaging data;updating the constrained training reconstruction by adding the estimated noise field to the constrained training reconstruction;training the first imaging restoration process based at least in part on the constrained training reconstruction and the full training reconstruction; andapplying the trained first imaging restoration process to the initial reconstruction to obtain the synthetic prior.

16. The image restoration method of claim 15, wherein generating the synthetic imaging data includes:generating a new projection dataset by forward-projecting the synthetic prior through a projective geometry corresponding to the received imaging data;determining an outside-field-of-view OFOV attenuation representation based at least in part on the new projection dataset and the received imaging data, the OFOV attenuation representation corresponding to material outside a field of view of the received imaging data;denoising the OFOV attenuation representation;generating an updated new projection dataset by combining the denoised OFOV attenuation representation and the new projection dataset; andgenerating the synthetic imaging data by adding the estimated noise field to a back-projection of the updated new projection dataset.

17. The image restoration method of any one of claims 1 to 16, further comprising outputting the restored imaging data using a display device.Docket No. 069608-000328WOPT 2023P01186WO18. The image restoration method of any one of claims 1 to 17, wherein the received imaging data is in a projection domain.

19. The image restoration method of claim 18, wherein the synthetic prior is in an image domain, the first imaging restoration process usable to generate reconstructed datasets from projection datasets.

20. The image restoration method of claim 18 or 19, wherein the received imaging data includes a projection dataset acquired via an X-ray imaging device.

21. The image restoration method of any one of claims 1 to 17, wherein the received imaging data is in an image domain and the synthetic prior is in the image domain.

22. The image restoration method of any one of claims 18 to 21, wherein the received imaging data is representative of a plurality of projections acquired at a plurality of different angles with respect to the subject.

23. The image restoration method of any one of claims 1 to 22, wherein the received imaging data is associated with a transmission imaging study of the subject.

24. The image restoration method of claim 23, wherein the transmission imaging study is an X-ray imaging study.

25. A system comprising:a control system including one or more processors; anda memory having stored thereon machine readable instructions;wherein the control system is coupled to the memory, and the method of any one of claims 1 to 24 is implemented when the machine executable instructions in the memory are executed by at least one of the one or more processors of the control system.

26. A computer program product tangibly embodied in a non-transitory machine-readable storage medium, the computer program product comprising instructions which, when executed by a computer, cause the computer to carry out the method of any one of claims 1 to 25.

Images