Cyclic measurement consistent training for physics-driven deep learning reconstruction and uncertainty guidance

Abstract

Described here are systems and methods for training a neural network, or other machine learning model, using a cyclic measurement of consistency in a self-supervised learning framework. In general, the disclosed systems and methods implement an uncertainty estimation process that focuses on the data fidelity component of a physics-driven deep learning (PD-DL) model by characterizing the cyclic consistency between different forward models. Subsequently, this uncertainty estimate is used to guide the training of the PD-DL model. It is an advantage of the disclosed systems and methods that this uncertainty-guided PD-DL strategy improves reconstruction quality.

Description

UMN 2024-020 920171.00622 CYCLIC MEASUREMENT CONSISTENT TRAINING FOR PHYSICS-DRIVEN DEEP LEARNING RECONSTRUCTION AND UNCERTAINTY GUIDANCE CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Provisional Patent Application Serial No. 63 / 622,229, filed on January 18, 2024, and entitled “Uncertainty-Guided Physics-Driven Deep Learning Reconstruction via Cyclic Measurement Consistency,” which is herein incorporated by reference in its entirety. STATEMENT OF FEDERALLY SPONSORED RESEARCH

[0002] This invention was made with government support under HL153146, EB027061, and EB032830, awarded by the National Institutes of Health. The government has certain rights in the invention. BACKGROUND

[0003] Physics-driven deep learning (PD-DL) techniques have recently emerged as a powerful means for improved computational imaging, including in magnetic resonance imaging (MRI) applications. These methods use the physics information by incorporating the known forward model for data fidelity, while performing regularization using neural networks. There has been progress in the training of PD-DL reconstruction methods, ranging from simple supervised learning to more practical self-supervised learning and generative models that allow training without reference data. Similarly, efforts have been made to characterize the errors associated with PD-DL methods via uncertainty quantification, mostly focusing on generative models. SUMMARY OF THE DISCLOSURE

[0004] The present disclosure provides a method for training a deep learning model to solve an inverse problem.

[0005] In some aspects, a method for training a physics-driven deep learning model to reconstruct an image from k-space data acquired with a magnetic resonance imaging (MRI) system is provided. The method includes accessing undersampled k-space data with a computer system, where the undersampled k-space data have been acquired from a subject using an MRI system; accessing a physics-driven deep learning (PD-DL) model with the computer system; 1 QB\920171.00622\94070024.2UMN 2024-020 920171.00622 inputting the undersampled k-space data to the PD-DL model, generating synthesized k-space data corresponding to k-space locations not sampled in the undersampled k-space data; generating uncertainty measurement data by computing a cyclic measurement may include between the synthesized k-space data and the undersampled k-space data; and updating the PD- DL model by incorporating an additional regularizer based on the uncertainty measurement data. Other embodiments of this aspect include corresponding systems (e.g., computer systems), programs, algorithms, and / or modules, each configured to perform the steps of the methods.

[0006] It is another aspect to provide a method for training a deep learning model that includes accessing a subsampled dataset with a computer system; accessing a deep learning model with the computer system; generating synthesized data by inputting the subsampled dataset to the deep learning model, where the synthesized data contain data samples corresponding to sample locations not sampled in the subsampled dataset; and training the deep learning model to solve an inverse problem using a self-supervised learning guided by an uncertainty measurement estimated by computing a cyclic measurement may include between the subsampled dataset and the synthesized data. Other embodiments of this aspect include corresponding systems (e.g., computer systems), programs, algorithms, and / or modules, each configured to perform the steps of the methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] FIG. 1 illustrates an example of undersampling patterns Ω and ^Δ^^ at acceleration rate R = 3 for equispaced acquisitions.

[0008] FIG. 2 illustrates an example of uncertainty estimation using measurement consistency.

[0009] FIG.3 illustrates another example implementation of a cyclic consistency self- supervised learning via data undersampling (SSDU) using equispaced undersampling at R = 3as an example. ^^n ^ show simulated k-space undersampling patterns with the samedistribution as. x ˆ^ is obtained and measurements ^ are simulatedfrom them. These simulated measurements are then reconstructed,ran through theforward operator E ^ , which are then compared to the original acquired data y ^ for cyclicconsistency ^^^ n ^^ to be used in PD-DL training.2 QB\920171.00622\94070024.2UMN 2024-020 920171.00622

[0010] FIG. 4 is a flowchart setting forth the steps of an example method for uncertainty-guided deep learning model reconstruction using a cyclic measurement consistency.

[0011] FIG. 5 shows representative examples of the proposed uncertainty estimation భ based on measurement consistency. The uncertainty maps, ^^మvisibly captures the artifacts in the error image corresponding to the network output.

[0012] FIG. 6 shows representative reconstructions from ^^ = 6 equispaced undersampling using supervised, self-supervised and the proposed uncertainty-guided PD-DL. The proposed approach reduces aliasing artifacts associated with the standard PD-DL methods. The corresponding error maps are scaled by a factor of 5 for display purposes.

[0013] FIG. 7 is a block diagram of an example system for uncertainty-guided deep learning reconstruction in accordance with some aspects of the present disclosure.

[0014] FIG. 8 is a block diagram of example components that can implement the system of FIG.7. DETAILED DESCRIPTION

[0015] Described here are systems and methods for training a neural network, or other machine learning model, using a cyclic measurement of consistency in a self-supervised learning framework. In general, the disclosed systems and methods implement an uncertainty estimation process that focuses on the data fidelity component of a physics-driven deep learning (PD-DL) model by characterizing the cyclic consistency between different forward models. Subsequently, this uncertainty estimate is used to guide the training of the PD-DL model. It is an advantage of the disclosed systems and methods that this uncertainty-guided PD-DL strategy improves reconstruction quality.

[0016] In some embodiments described in the present disclosure, cyclic consistency is used it to perform uncertainty estimation through an unrolled PD-DL network. The estimated uncertainty is subsequently used to train an improved PD-DL network via an uncertainty- guided reconstruction approach that incorporates knowledge about the uncertainties to the network.

[0017] Advantageously, these disclosed systems and methods also provide an improved approach for the self-supervised (e.g., reference-less) training of PD-DL reconstruction in the scarce data regime, such as when using high sub-sampling and / or high acceleration rates. For instance, in some embodiments, the cyclic consistency-based techniques 3 QB\920171.00622\94070024.2UMN 2024-020 920171.00622 described in the present disclosure simulate new measurements based on inference results with a different known (e.g., forward) model, whose inference results should be consistent with the original data (e.g., the acquired data). As a non-limiting example, cyclic consistency can be used, along with the aim that PD-DL reconstruction should be generalizable to undersampling patterns with similar distributions as the sampling pattern used for data acquisition, to improve multi-mask self-supervised learning, in a method that can be referred to as CC-SSDU. More generally, the disclosed systems and methods are applicable to inverse problems. For illustrative purposes, an example is provided below with respect to MRI reconstruction.

[0018] As mentioned above, using the disclosed systems and methods, a deep learning model (e.g., a neural network, a PD-DL model) is run on acquired data and to synthesize unacquired data with similar characteristics as the acquired data (e.g., shifted k-space trajectory). The deep learning model is then run only on this synthesized data to synthesize the acquired data positions, which are then compared to the true acquired data. This cyclic consistency between acquired- to-synthesized-to-acquired data is used to guide the training, as described below in more detail. Additionally or alternatively, consistency can be enforced among different simulated trajectories, and simulated data can be incorporated into the deep learning model directly instead of only through training.

[0019] Within the context of MRI, self-supervised learning can be performed by masking part of the k-space data and hiding this from the PD-DL network, which learns to predict this hidden part from the remainder of the k-space data. This allows training with only undersampled data without requiring a reference. In some implementations, this can be achieved by using a multi-mask approach, where pairs of disjoint sets ^Θ^,Λ^^ are generatedfor ^^ ∈ ^1, … ,^^^ such that Θ^ ∪ Λ^ ൌ Ω which is the index set of acquired k-space points.Then trainingperformed using: argmin ∑^^^^^^ ^ஃ,^^ஃ^^^^^^^^,^^ ^^, ^^ୀ^ ೖ ೖ ೖ ^ೖ

[0020] wherenetwork (parametrized by ^^) output for input k-space ^^^ೖand corresponding encoding operator ^^^ೖ. In effect, the network only sees the points in ^^^ೖand learns to predict the points in ^^ஃೖ, which are disjoint. By cycling through ^^ of these, we cover the full k-space.

[0021] At high acceleration rates, the further masking of the sub-sampling data can lead to data scarcity. In these instances, a self-supervised PD-DL network may degrade faster than 4 QB\920171.00622\94070024.2UMN 2024-020 920171.00622 a supervised PD-DL network at high acceleration rates. Advantageously, the systems and methods described in the present disclosure overcome this problem.

[0022] In general, the disclosed systems and method can implement the followingframework. As one non-limiting example, assume Ω ൌ ^1,^^ ^ 1,2^^ ^ 1, … ^ is an equispacedundersampling pattern with acceleration rate ^^. Additionally or alternatively, the starting pattern, Ω, could also be a random sampling pattern, a non-Cartesian sampling pattern, and so on. A network can then be trained to interpolate the missing lines in a different spot, say Δ^ൌ^2,^^ ^ 2,2^^ ^ 2, … ^ from Ω. Then, this same network can be used to interpolate Δଶ ൌ ^3,^^ ^3,2^^ ^ 3, … ^ from Δ^. This process can be repeated ^^-1 times to interpolate the lines at ^^^ ^1,2^^ ^ 1, … ^, which is a subset of Ω. The iteratively interpolated lines can be compared withthe acquired lines to guide the training process, without requiring any calibration data.

[0023] This framework can be formulated in the context of physics-driven neural networks, or other suitable deep learning models, by posing it as using a reconstruction from Ω to estimate a new set of lines on a trajectory Δ^and then using these new lines to re-estimate Ω, to ensure cyclic consistency between trajectories. This leads to the following loss function argmin ^^∑^^ୀ^ ^^^^^ஃೖ ,^^ஃೖ^^^^^^^^ೖ ,^^^ೖ^^ ^. Further details of thisframework are below, including an alternative loss function in Eqn. (20) that mayalso be implemented to incorporate cyclic consistency into a PD-DL reconstruction model.

[0025] As noted above, the disclosed systems and methods are applicable to a range of inverse problems. For illustrative purposes, an image reconstruction problem in magnetic resonance imaging is presented as an example.

[0026] Given a linear multi-coil encoding operator ^^ஐassociated with sampling pattern Ω, and a regularization term ℛ^⋅^, PD-DL aims to solve a regularized least squares problem: argmin||^ ଶ௫^ஐ െ ^^ஐ^^||ଶ ^ ℛ^^^^, (1)

[0027] an unrolled network approach, an optimization technique for solving Eqn. (1) is unrolled for a fixed number, ^^, of iterations, and then trained end-to-end. Such algorithms include variable splitting that alternates between a proximal operator: ^^^^^ ൌ argm ^^||^^^^ି^^ െ ^^||ଶଶ ^ ℛ^^^^, (2)

[0028] which is5 QB\920171.00622\94070024.2UMN 2024-020 920171.00622 ^^^^^ ൌ argmin ||^^ െ ^^ ^^ ଶ ^^^ଶ௫ஐ ஐ ||ଶ ^ ^^||^^ െ ^^ ||ଶ, (3)

[0029] ^^ ^^

[0030] a trained PD-DL reconstruction can generalize well to undersampling patterns drawn from similar distributions. Thus, by performing the reconstruction across different undersampling patterns drawn from a given distribution that matches the true pattern, Ω, the mean and variance of the outputs can be computed to characterize the uncertainty. Because only one true undersampling pattern Ω may be available at test time, a cyclic measurement consistency can be used to simulate additional patterns. To this end, let the output of the unrolled network parametrized by ^^ be denoted as: ^^ஐ ൌ ^^^^^ஐ,^^ஐ;^^^, (4)

[0031] ^^

[0032] can patterns ^Δ^^ from a similar distribution as Ω, as follows: ^^^^^ ൌ ^^^^^^ஐ, (5)

[0033] Δ^and coil sensitivity profiles matching ^^ஐ. In some implementations, a noise term can also be added to Eqn. (5). Note, similar distribution of the undersampling patterns here assumes matched acceleration rate with same number of central lines and the same underlying distribution (e.g., equispaced or variable density random with same underlying distribution). Asimple example of equispaced sampling at ^^ ൌ 3 is depicted in FIG. 1. Once these rate ^^accelerated measurements are simulated, then the PD-DL reconstruction can be performed again with the corresponding inputs to the unrolled network as: ^^^^^ ൌ ^^^^^^^^ ,^^^^;^^^. (6)

[0034] ^^^ஐ ൌ ^^^^^ஐ^^ஐ,^^ஐ; ^^^. (7)

[0035] incur more error than the first step alone, which may be imperfect to begin with. As one non- limiting example, Lipschitz bounds on the proximal operator neural network and a minimum eigenvalue of the forward operator can be used to show that the overall ℓଶerror is within a constant of the error from the first step. 6 QB\920171.00622\94070024.2UMN 2024-020 920171.00622

[0036] For illustrative purposes, suppose the unrolled network has an estimation error as follows. ||^^^^^^,^^^;^^^ െ ^^||ଶ ^ ^^||^^||ଶ

[0037] for undersampling pattern Υ from the set of undersampling patterns of interest,^Ω,Δ^, … ,Δ^^ and for the corresponding measurements ^^^ of the true image of interest. Thenthe estimation error incurred by the process of subsequently using Eqn. (6) satisfies: ||^^^^^ െ ^^||ଶ ^ ^^ ⋅ ^^||^^||ଶ,

[0038] where ^^ is a eigenvalue of the multi-coilencoding operator, the learned Eqn. (3), the Lipschitz constant of the neural network that implements the proximal operation in Eqn. (2), and the number of unrolls ^^.

[0039] Using these set of reconstructions ^^^^ஐ, ^^^^భ , … , ^^^^ಿ^, the element-wise mean ^̅^,and the element-wise variance can be latter gives a diagonalapproximation to the covariance matrix ^^. of ^^ is given as: ^^ ^ ^^ൌ ேା^ ൫|^^்^ ^^^^ஐ െ ^^^|ଶ ^ ∑ே^ୀ^ |^^்^ ^^^^^ଶ^െ ^̅^^|൯, (8)

[0040] estimation is summarized in FIG.2.

[0041] The uncertainty information obtained from this measurement consistency approach can be incorporated into guiding the training of a PD-DL algorithm or other suitable deep learning model. In the example physics-driven model, the uncertainty estimation is used to modify the objective function solved in Eqn. (1) rather than the training loss. In particular, an additional regularizer that centers the estimation around the mean with weights inversely proportional to the uncertainty is added. As a non-limiting example, this additional regularizer may be added via a score function of a Gaussian distribution, as follows: argmin||^^ െ ^^ ^^||ଶ ^ ℛ^ ^ ு ି^௫ஐ ஐ ଶ ^^ ^ ^^^^^ െ ^̅^^ ^^ ^^^ െ ^̅^^, (9)

[0042] amplification from the two-step estimation described above. Using a variable splitting approach similar to the one described above leads to ^^^^^ ൌ argm ^^௭in ^^||^^ ି^^ െ ^^||ଶଶ ^ ℛ^^^^, (10)^^^^^ ^^ ^ ^ ^^^

[0043] the context of MRI and other medical imaging applications, the uncertainty estimation is scan- 7 QB\920171.00622\94070024.2UMN 2024-020 920171.00622 specific. Thus, for MRI applications, a zero-shot self-supervised learning approach can be implemented. Thus, the overall training process first involves training of the basic PD-DL network to solve Eqn. (1). Subsequently, uncertainty estimation is performed for a given test input using this trained network, leading to the generation of the element-wise mean and covariance ^^̅^,^^^. These are then used to formulate a new uncertainty-guided PD-DL network for solving the objective function in Eqn. (9), which can be trained end-to-end.

[0044] A natural choice for Δ^is to keep the characteristics original sampling trajectory Ω but just transform it. For instance in the previous example, when Ω is equispaced sampling,Δ^ are just shifted versions of equispaced trajectories (and ^^ ൌ ^^ െ 1). In another example, Ωcould be a spiral trajectory, and Δ^may be different rotations of that trajectory. Another example (where SSDU does not work) is partial Fourier imaging, where e.g. Ω is a 6 / 8 partial Fourier pattern, and Δ^is the corresponding 6 / 8 pattern on the opposing side.

[0045] The scheme described above ensures the following cyclic consistency Ω→ Δ^ → Ω

[0046] Different cycles can also be considered, such as Ω→ Δ^^^^^^^Δ^ᇱ^^^^^^ℎ^^^^^^^^^^^^^^^^^^^^^^Δ^ ↔ Δ^ᇱ,

[0047] which corresponds to the following loss: ^ ^ ^^ ^^^^

[0048] consistency can be used in higher degrees, such as ^^^ n ^^ m ^^ for n^ m .

[0049] Another approach is to bringFirst recall the structure and objective of the PD-DL network is to solve a regularized least squares problem: ||^^ െ ^^ ^^ ଶஐ ஐ ||ଶ ^ ^^^^^^, (13)

[0050] which can be solved using various iterative algorithms that can be unrolled. Variable splitting with quadratic penalty can be used as an example, leading to ^^^^^ ൌ ^^||^^ െ ^^^^ି^^||ଶଶ ^ ^^^^^^, (14)8UMN 2024-020 920171.00622 ^^^^^ ൌ argmin ||^^ െ ^^ ^^||ଶ ^ ^^||^^ െ ^^^^^|ଶ ு ି^ு௫ஐ ஐ ଶ |ଶ ൌ ^^^ஐ^^ஐ ^ ^^^^^ ^^^ஐ^^ஐ^

[0051] equationgiven to the acquired data versus the estimated image.

[0052] Based on this, the data fidelity can be extended to include the simulated k-space points. To this end, suppose training (e.g., using the loss function above) has been performed and some network weights ^^^, have been learned. This trained model can be used to generate a number of newer simulated trajectories:^^^^ ≜ ^^^^^^^^బ^^^ஐ,^^ஐ^ (16)Thus, a new objective function can be posed, such as the following: ||^^ െ ^^ ^^||ଶ ^ ∑^ ^^||^^ െ ^^ ^^||ଶஐ ஐ ଶ ^ୀ^ ^ ^^ ^^ ଶ ^ ^^^^^^, (17)

[0054] can as an leading to ^^^^^ ൌ argmin ^ ^^ି^^ ଶ௭^||^^ െ ^^ ||ଶ ^ ^^^^^^, (18)

[0055] include ^^^^^, and trained using multi-mask SSDU, as described above. In some implementations, both the first network (^^^) and this new network can be learned together end- to-end (e.g. sharing parameters for the regularization part).

[0056] For instance, Eqn. (6) suggests that these reconstructions can reliably map to y^ using the corresponding forward mappingE^ x^ ^ n . To this end, the following loss functioncan be used to incorporate cyclic consistency to multi-mask self-supervised PD-DL: ^ 1 M ^^^ ^^^ ^ ^ ^UMN 2024-020 920171.00622

[0057] where ^ is a weight term and L ^ ^ ^ denotes a loss function, such meansquared error (MSE), ^ 1^ ^ 2 loss, or the like. The first term corresponds to a multi-maskingstrategy as in MM- with multiple pairs of disjoint ^^m, ^ m subsets of ^ . The second^term incorporates cyclic consistency with respect to acquired data y ^ by applying E ^ overx^^n (e.g., as shown in FIG. 3). This consistency can be denoted by ^^^ n ^^ . By wayof example, MM-SSDU augments SSDU by training performed with the following loss: M min^ ^1L y , E f ^^^^ ^ ^ ^ m ^ m^ ^ y ^ m , E ^ m ;^^ ^ ^ ^ , (21)^

[0058]

[0059] now a as of an example method for a physics-driven deep learning model reconstruction using cyclic measurement consistency.

[0060] The method includes accessing subsampled data with a computer system, as indicated at step 302. Accessing the subsampled data may include retrieving such data from a memory or other suitable data storage device or medium. Additionally or alternatively, accessing the subsampled data may include acquiring such data with a suitable imaging or measurement system and transferring or otherwise communicating the data to the computer system, which may be a part of the computer system.

[0061] In some non-limiting examples, the subsampled data may be undersampled medical imaging data, such as undersampled k-space data acquired with an MRI system.

[0062] A deep learning model, such as a PD-DL model or other neural network model, is then accessed with the computer system, as indicated at step 304. In general, the deep learning model structured to solve an inverse problem, such as an image reconstruction problem. For example, the deep learning model can be formulated in the context of physics- driven neural networks, or other suitable deep learning models, by posing it as using a reconstruction from the sampled data points, Ω, to estimate a new set of data points Δ^and then using these new data points to re-estimate Ω, to ensure cyclic consistency between the sets of data points.

[0063] Accessing the deep learning model may include accessing model parameters (e.g., weights, biases, or both) that have been optimized or otherwise estimated by training the deep learning model on training data. In some instances, retrieving the deep learning model 10 QB\920171.00622\94070024.2UMN 2024-020 920171.00622 can also include retrieving, constructing, or otherwise accessing the particular model architecture to be implemented. For instance, data pertaining to the layers in a neural network architecture (e.g., number of layers, type of layers, ordering of layers, connections between layers, hyperparameters for layers) may be retrieved, selected, constructed, or otherwise accessed.

[0064] The subsampled data are then input to the deep learning model, generating output as synthesized data, as indicated at step 306. As described above, the synthesized data correspond to sample points not sampled in the original subsampled data. In some implementations, noise can be added to the synthesized data in order to keep signal-to-noise ratio (SNR) constant or otherwise consistent between the subsampled data and the synthesized data. As one example, random noise can be added to the synthesized data, such as random Gaussian noise. As another example, a noise profile can be estimated from the subsampled data and the estimated noise profile can be added to the synthesized data.

[0065] A measurement consistency between the original subsampled data and the synthesized data can then be computed, as indicated at step 308. For instance, the synthesized data can be used to re-estimate the subsampled data, as described above. An estimate of uncertainty between the re-estimated subsampled data and the original subsampled data can be computed to ensure cyclic consistency between the sets of data points. This process can be repeated to estimate additional sets of synthesized data, checking for cyclic consistency along the way, as indicated at decision block 310.

[0066] The uncertainty measurement data generated in this process is stored at step 312 and used to guide training of the deep learning model at step 314. For instance, the uncertainty measurement data can be used to formulate an additional regularization on the inverse problem being solved by the deep learning model. Training the deep learning model may include initializing the model, such as by computing, estimating, or otherwise selecting initial model parameters (e.g., weights, biases, or both). During training, the deep learning model receives the inputs for a training example and generates an output using the bias for each node, and the connections between each node and the corresponding weights. For instance, training data can be input to the initialized deep learning model, generating an output. The output can be passed to a loss function, such as one of the loss functions described in the present disclosure, to compute an error. The current deep learning model can then be updated based on the calculated error (e.g., using backpropagation methods based on the calculated error). For instance, the current deep learning model can be updated by updating the model parameters (e.g., weights, 11 QB\920171.00622\94070024.2UMN 2024-020 920171.00622 biases, or both) in order to minimize the loss according to the loss function. The training continues until a training condition is met. The training condition may correspond to, for example, a predetermined number of training examples being used, a minimum accuracy threshold being reached during training and validation, a predetermined number of validation iterations being completed, and the like.

[0067] When the training condition has been met (e.g., by determining whether an error threshold or other stopping criterion has been satisfied), the current deep learning model and its associated model parameters represent the trained deep learning model. When training is completed, the trained deep learning model is stored at step 316 and used to generate an output from the input subsampled data and / or the combination of the subsampled data and the synthesized data sets, as indicated at step 318. As a non-limiting example, the output may be a reconstructed image. The output from the deep learning model (e.g., a reconstructed image) can then be displayed to a user, stored for later use or further processing, or both, as indicated at step 320.

[0068] In an example implementation of the methods described in the present disclosure, imaging experiments were conducted using magnetic resonance imaging data acquired using a coronal proton density with fat suppression scanning protocol. The data were acquired using the following imaging parameters: matrix size = 320ൈ368, in-plane resolution = 0.49ൈ0.44 mm2. Fully-sampled data were retrospectively undersampled with an acceleration rate ^^ = 6 using an equispaced undersampling pattern, with 24 lines of autocalibrated signal (ACS) in the center of k-space.

[0069] The baseline PD-DL networks for solving Eqn. (1) were trained using both supervised and self-supervised learning. A total of 320 slices from 16 subjects were used for training. The self-supervised PD-DL network was used for subsequent uncertainty estimation and the training of the proposed uncertainty-guided PD-DL network for solving Eqn. (9). Thus, the uncertainty-guided PD-DL network was trained using only undersampled data, without requiring reference fully-sampled datasets. In all cases, the PD-DL networks were unrolled for ^^ = 10 iterations. The proximal operator in Eqn. (3) was implemented using a convolutional neural network (CNN) with 592,128 learnable parameters. The data fidelity units were implemented using an unrolled conjugate gradient approach with 10 conjugate gradient steps. All implementations used Pytorch 1.10, and experiments were performed on a server with NVIDIA A100 GPU. A normalized ℓ^-ℓଶloss was minimized using the ADAM optimizer with a learning rate of 0.0003 for 200 epochs. 12 QB\920171.00622\94070024.2UMN 2024-020 920171.00622

[0070] FIG. 5 depicts example uncertainty maps generated using the approach described above. The standard deviation map obtained using Eqn. (8) visibly matches the error in the reconstruction in two representative slices. FIG. 6 shows representative reconstruction results of ^^ = 6 equispaced undersampling, using supervised, self-supervised and uncertainty- guided PD-DL. Due to relatively high acceleration and equispaced undersampling pattern, which is more difficult to resolve compared to random undersampling, visible aliasing artifacts are observed in the self-supervised PD-DL, and to a lesser extent in the supervised PD-DL. These artifacts are largely alleviated by the proposed uncertainty-guided PD-DL, which successfully reduces such aliasing artifacts.

[0071] An uncertainty estimation procedure for unrolled PD-DL networks has thus been developed based on cyclic measurement consistency, which shows close match to the error maps of the reconstructions. The uncertainty estimates are used to guide the reconstruction with an additional regularization term. The proposed uncertainty-guided PD-DL reduced artifacts visible in conventionally trained PD-DL, offering promise for improving accelerated MRI.

[0072] Referring now to FIG. 6, an example of a system 600 for uncertainty-guided deep learning reconstruction in accordance with some embodiments of the systems and methods described in the present disclosure is shown. As shown in FIG.6, a computing device 650 can receive one or more types of data (e.g., undersampled k-space data, other subsampled data) from data source 602. In some embodiments, computing device 650 can execute at least a portion of a uncertainty-guided deep learning reconstruction system 604 to reconstruct images from data received from the data source 602, or to otherwise solve other inverse problems modeled by a deep learning model.

[0073] Additionally or alternatively, in some embodiments, the computing device 650 can communicate information about data received from the data source 602 to a server 652 over a communication network 654, which can execute at least a portion of the uncertainty- guided deep learning reconstruction system 604. In such embodiments, the server 652 can return information to the computing device 650 (and / or any other suitable computing device) indicative of an output of the uncertainty-guided deep learning reconstruction system 604.

[0074] In some embodiments, computing device 650 and / or server 652 can be any suitable computing device or combination of devices, such as a desktop computer, a laptop computer, a smartphone, a tablet computer, a wearable computer, a server computer, a virtual 13 QB\920171.00622\94070024.2UMN 2024-020 920171.00622 machine being executed by a physical computing device, and so on. The computing device 650 and / or server 652 can also reconstruct images from the data.

[0075] In some embodiments, data source 602 can be any suitable source of data (e.g., measurement data, images reconstructed from measurement data, processed image data), such as a medical imaging system, another computing device (e.g., a server storing measurement data, images reconstructed from measurement data, processed image data), and so on. In some embodiments, data source 602 can be local to computing device 650. For example, data source 602 can be incorporated with computing device 650 (e.g., computing device 650 can be configured as part of a device for measuring, recording, estimating, acquiring, or otherwise collecting or storing data). As another example, data source 602 can be connected to computing device 650 by a cable, a direct wireless link, and so on. Additionally or alternatively, in some embodiments, data source 602 can be located locally and / or remotely from computing device 650, and can communicate data to computing device 650 (and / or server 652) via a communication network (e.g., communication network 654).

[0076] In some embodiments, communication network 654 can be any suitable communication network or combination of communication networks. For example, communication network 654 can include a Wi-Fi network (which can include one or more wireless routers, one or more switches, etc.), a peer-to-peer network (e.g., a Bluetooth network), a cellular network (e.g., a 3G network, a 4G network, etc., complying with any suitable standard, such as CDMA, GSM, LTE, LTE Advanced, WiMAX, etc.), other types of wireless network, a wired network, and so on. In some embodiments, communication network 654 can be a local area network, a wide area network, a public network (e.g., the Internet), a private or semi-private network (e.g., a corporate or university intranet), any other suitable type of network, or any suitable combination of networks. Communications links shown in FIG. 6 can each be any suitable communications link or combination of communications links, such as wired links, fiber optic links, Wi-Fi links, Bluetooth links, cellular links, and so on.

[0077] Referring now to FIG. 7, an example of hardware 700 that can be used to implement data source 602, computing device 650, and server 652 in accordance with some embodiments of the systems and methods described in the present disclosure is shown.

[0078] As shown in FIG. 7, in some embodiments, computing device 650 can include a processor 702, a display 704, one or more inputs 706, one or more communication systems 708, and / or memory 710. In some embodiments, processor 702 can be any suitable hardware processor or combination of processors, such as a central processing unit (“CPU”), a graphics 14 QB\920171.00622\94070024.2UMN 2024-020 920171.00622 processing unit (“GPU”), and so on. In some embodiments, display 704 can include any suitable display devices, such as a liquid crystal display (“LCD”) screen, a light-emitting diode (“LED”) display, an organic LED (“OLED”) display, an electrophoretic display (e.g., an “e- ink” display), a computer monitor, a touchscreen, a television, and so on. In some embodiments, inputs 706 can include any suitable input devices and / or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, and so on.

[0079] In some embodiments, communications systems 708 can include any suitable hardware, firmware, and / or software for communicating information over communication network 654 and / or any other suitable communication networks. For example, communications systems 708 can include one or more transceivers, one or more communication chips and / or chip sets, and so on. In a more particular example, communications systems 708 can include hardware, firmware, and / or software that can be used to establish a Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on.

[0080] In some embodiments, memory 710 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor 702 to present content using display 704, to communicate with server 652 via communications system(s) 708, and so on. Memory 710 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof. For example, memory 710 can include random-access memory (“RAM”), read-only memory (“ROM”), electrically programmable ROM (“EPROM”), electrically erasable ROM (“EEPROM”), other forms of volatile memory, other forms of non-volatile memory, one or more forms of semi-volatile memory, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on. In some embodiments, memory 710 can have encoded thereon, or otherwise stored therein, a computer program for controlling operation of computing device 650. In such embodiments, processor 702 can execute at least a portion of the computer program to present content (e.g., images, user interfaces, graphics, tables), receive content from server 652, transmit information to server 652, and so on. For example, the processor 702 and the memory 710 can be configured to perform the methods described herein (e.g., the process shown in FIG.2, the method of FIG.3).

[0081] In some embodiments, server 652 can include a processor 712, a display 714, one or more inputs 716, one or more communications systems 718, and / or memory 720. In some embodiments, processor 712 can be any suitable hardware processor or combination of processors, such as a CPU, a GPU, and so on. In some embodiments, display 714 can include 15 QB\920171.00622\94070024.2UMN 2024-020 920171.00622 any suitable display devices, such as an LCD screen, LED display, OLED display, electrophoretic display, a computer monitor, a touchscreen, a television, and so on. In some embodiments, inputs 716 can include any suitable input devices and / or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, and so on.

[0082] In some embodiments, communications systems 718 can include any suitable hardware, firmware, and / or software for communicating information over communication network 654 and / or any other suitable communication networks. For example, communications systems 718 can include one or more transceivers, one or more communication chips and / or chip sets, and so on. In a more particular example, communications systems 718 can include hardware, firmware, and / or software that can be used to establish a Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on.

[0083] In some embodiments, memory 720 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor 712 to present content using display 714, to communicate with one or more computing devices 650, and so on. Memory 720 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof. For example, memory 720 can include RAM, ROM, EPROM, EEPROM, other types of volatile memory, other types of non-volatile memory, one or more types of semi-volatile memory, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on. In some embodiments, memory 720 can have encoded thereon a server program for controlling operation of server 652. In such embodiments, processor 712 can execute at least a portion of the server program to transmit information and / or content (e.g., data, images, a user interface) to one or more computing devices 650, receive information and / or content from one or more computing devices 650, receive instructions from one or more devices (e.g., a personal computer, a laptop computer, a tablet computer, a smartphone), and so on.

[0084] In some embodiments, the server 652 is configured to perform the methods described in the present disclosure. For example, the processor 712 and memory 720 can be configured to perform the methods described herein (e.g., the process shown in FIG. 2, the method of FIG.3).

[0085] In some embodiments, data source 602 can include a processor 722, one or more data acquisition systems 724, one or more communications systems 726, and / or memory 728. In some embodiments, processor 722 can be any suitable hardware processor or combination of processors, such as a CPU, a GPU, and so on. In some embodiments, the one or more data 16 QB\920171.00622\94070024.2UMN 2024-020 920171.00622 acquisition systems 724 are generally configured to acquire data, images, or both, and can include a medical imaging system, such as an MRI system. Additionally or alternatively, in some embodiments, the one or more data acquisition systems 724 can include any suitable hardware, firmware, and / or software for coupling to and / or controlling operations of a medical imaging system, such as an MRI system. In some embodiments, one or more portions of the data acquisition system(s) 724 can be removable and / or replaceable.

[0086] Note that, although not shown, data source 602 can include any suitable inputs and / or outputs. For example, data source 602 can include input devices and / or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, a trackpad, a trackball, and so on. As another example, data source 602 can include any suitable display devices, such as an LCD screen, an LED display, an OLED display, an electrophoretic display, a computer monitor, a touchscreen, a television, etc., one or more speakers, and so on.

[0087] In some embodiments, communications systems 726 can include any suitable hardware, firmware, and / or software for communicating information to computing device 650 (and, in some embodiments, over communication network 654 and / or any other suitable communication networks). For example, communications systems 726 can include one or more transceivers, one or more communication chips and / or chip sets, and so on. In a more particular example, communications systems 726 can include hardware, firmware, and / or software that can be used to establish a wired connection using any suitable port and / or communication standard (e.g., VGA, DVI video, USB, RS-232, etc.), Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on.

[0088] In some embodiments, memory 728 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor 722 to control the one or more data acquisition systems 724, and / or receive data from the one or more data acquisition systems 724; to generate images from data; present content (e.g., data, images, a user interface) using a display; communicate with one or more computing devices 650; and so on. Memory 728 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof. For example, memory 728 can include RAM, ROM, EPROM, EEPROM, other types of volatile memory, other types of non-volatile memory, one or more types of semi-volatile memory, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on. In some embodiments, memory 728 can have encoded thereon, or otherwise stored therein, a program for controlling operation of data source 602. In such embodiments, 17 QB\920171.00622\94070024.2UMN 2024-020 920171.00622 processor 722 can execute at least a portion of the program to generate images, transmit information and / or content (e.g., data, images, a user interface) to one or more computing devices 650, receive information and / or content from one or more computing devices 650, receive instructions from one or more devices (e.g., a personal computer, a laptop computer, a tablet computer, a smartphone, etc.), and so on.

[0089] In some embodiments, any suitable computer-readable media can be used for storing instructions for performing the functions and / or processes described herein. For example, in some embodiments, computer-readable media can be transitory or non-transitory. For example, non-transitory computer-readable media can include media such as magnetic media (e.g., hard disks, floppy disks), optical media (e.g., compact discs, digital video discs, Blu-ray discs), semiconductor media (e.g., RAM, flash memory, EPROM, EEPROM), any suitable media that is not fleeting or devoid of any semblance of permanence during transmission, and / or any suitable tangible media. As another example, transitory computer- readable media can include signals on networks, in wires, conductors, optical fibers, circuits, or any suitable media that is fleeting and devoid of any semblance of permanence during transmission, and / or any suitable intangible media.

[0090] As used herein in the context of computer implementation, unless otherwise specified or limited, the terms “component,” “system,” “module,” “framework,” and the like are intended to encompass part or all of computer-related systems that include hardware, software, a combination of hardware and software, or software in execution. For example, a component may be, but is not limited to being, a processor device, a process being executed (or executable) by a processor device, an object, an executable, a thread of execution, a computer program, or a computer. By way of illustration, both an application running on a computer and the computer can be a component. One or more components (or system, module, and so on) may reside within a process or thread of execution, may be localized on one computer, may be distributed between two or more computers or other processor devices, or may be included within another component (or system, module, and so on).

[0091] In some implementations, devices or systems disclosed herein can be utilized or installed using methods embodying aspects of the disclosure. Correspondingly, description herein of particular features, capabilities, or intended purposes of a device or system is generally intended to inherently include disclosure of a method of using such features for the intended purposes, a method of implementing such capabilities, and a method of installing disclosed (or otherwise known) components to support these purposes or capabilities. 18 QB\920171.00622\94070024.2UMN 2024-020 920171.00622 Similarly, unless otherwise indicated or limited, discussion herein of any method of manufacturing or using a particular device or system, including installing the device or system, is intended to inherently include disclosure, as embodiments of the disclosure, of the utilized features and implemented capabilities of such device or system.

[0092] The present disclosure has described one or more preferred embodiments, and it should be appreciated that many equivalents, alternatives, variations, and modifications, aside from those expressly stated, are possible and within the scope of the invention. 19 QB\920171.00622\94070024.2

Claims

UMN 2024-020 920171.00622 CLAIMS 1. A method for training a physics-driven deep learning model to reconstruct an image from k-space data acquired with a magnetic resonance imaging (MRI) system, the method comprising: accessing undersampled k-space data with a computer system, wherein the undersampled k-space data have been acquired from a subject using an MRI system; accessing a physics-driven deep learning (PD-DL) model with the computer system; inputting the undersampled k-space data to the PD-DL model, generating synthesized k-space data; generating uncertainty measurement data by computing a cyclic measurement consistency between the synthesized k-space data and the undersampled k- space data; and updating the PD-DL model by incorporating an additional regularizer based on the uncertainty measurement data.

2. The method of claim 1, further comprising reconstructing an image from the undersampled k-space data and the synthesized k-space data using the updated PD-DL model.

3. The method of claim 1, wherein the synthesized k-space data correspond to k- space locations not sampled in the undersampled k-space data.

4. The method of claim 1, wherein the synthesized k-space data correspond to k- space locations that at least partially overlap with the undersampled k-space data.

5. The method of claim 1, wherein the uncertainty measurement data comprise at least one of an element-wise mean or an element-wise standard deviation.

6. The method of claim 1, wherein the additional regularizer centers an estimation of a reconstructed image around a mean using weights that are inversely proportional to uncertainty estimated in the uncertainty measurement data. 20 QB\920171.00622\94070024.2UMN 2024-020 920171.00622 7. The method of claim 1, wherein the cyclic measurement consistency is used to simulate additional k-space sampling patterns.

8. The method of claim 7, wherein generating the uncertainty measurement data using the cyclic measurement consistency comprises inputting the synthesized k-space data to the PD-DL model to estimate k-space positions sampled in the undersampled k-space data and comparing the estimated k-space positions with sampled k-space positions in the undersampled k-space data.

9. The method of claim 1, wherein the uncertainty measurement data are generated by further computing a cyclic measurement consistency between different trajectories in the synthesized k-space data.

10. The method of claim 1, further comprising adding noise to the synthesized k- space data prior to generating the uncertainty measurement data in order to maintain a consistent signal-to-noise ratio between the synthesized k-space data and the undersampled k- space data.

11. A method for training a deep learning model, the method comprising: accessing a subsampled dataset with a computer system; accessing a deep learning model with the computer system; generating synthesized data by inputting the subsampled dataset to the deep learning model, wherein the synthesized data contain data samples corresponding to sample locations not sampled in the subsampled dataset; and training the deep learning model to solve an inverse problem using a self-supervised learning guided by an uncertainty measurement estimated by computing a cyclic measurement consistency between the subsampled dataset and the synthesized data.

12. The method of claim 11, wherein the deep learning model is trained to solve an inverse problem comprising an image reconstruction problem. 21 QB\920171.00622\94070024.2UMN 2024-020 920171.00622 13. The method of claim 12, wherein the subsampled data comprise undersampled k-space data acquired with a magnetic resonance imaging (MRI) system.

14. The method of claim 11, wherein the uncertainty measurement estimated by computing the cyclic measurement consistency between the subsampled dataset and the synthesized data comprises at least one of an element-wise mean or an element-wise standard deviation.

15. The method of claim 11, wherein training the deep learning model comprises adding a regularizer to the deep learning model, wherein the regularizer is based on the uncertainty measurement.

16. The method of claim 15, wherein the regularizer centers an estimation of an output of the deep learning model around a mean using weights that are inversely proportional to the uncertainty measurement.

17. The method of claim 11, wherein the deep learning model comprises a physics-driven deep learning (PD-DL) reconstruction model.

18. The method of claim 11, further comprising adding noise to the synthesized k- space data prior to training the deep learning model in order to maintain a consistent signal- to-noise ratio between the synthesized k-space data and the undersampled k-space data.

19. A method for training a physics-driven deep learning model to reconstruct an image from k-space data acquired with a magnetic resonance imaging (MRI) system, the method comprising: accessing undersampled k-space data with a computer system, wherein the undersampled k-space data have been acquired from a subject using an MRI system; accessing a physics-driven deep learning (PD-DL) model with the computer system; inputting the undersampled k-space data to the PD-DL model, generating synthesized k-space data; 22 QB\920171.00622\94070024.2UMN 2024-020 920171.00622 computing a cyclic consistency measurement between a reconstruction performed on the synthesized k-space data with the PD-DL model and the undersampled k- space data; and updating the PD-DL model by training the PD-DL model on training data using a loss function that incorporates the cyclic consistency measurement.

20. The method of claim 19, wherein the loss function includes a first term corresponding to a self-supervised learning term and a second term corresponding to a cyclic consistency term that incorporates the cyclic consistency measurement.

21. The method of claim 20, wherein the self-supervised learning term comprises a self-supervised learning via data undersampling (SSDU) term.

22. The method of claim 21, wherein the SSDU term comprises a multi-mask SSDU term that incorporates a multi-masking strategy with multiple pairs of disjoint subsets of the undersampled k-space data.

23. The method of claim 19, wherein the synthesized k-space data correspond to k-space locations not sampled in the undersampled k-space data.

24. The method of claim 19, wherein the synthesized k-space data correspond to k-space locations that at least partially overlap with the undersampled k-space data.

25. The method of claim 19, further comprising adding noise to the synthesized k- space data prior to computing the cyclic consistency measurement in order to maintain a consistent signal-to-noise ratio between the synthesized k-space data and the undersampled k- space data. 23 QB\920171.00622\94070024.2

Images