Systems and methods for dynamic image reconstruction

Abstract

An example method includes initializing a plurality of time‑varying latent codes and at least one time‑invariant latent code for a plurality of undersampled magnetic resonance data; generating, by a first neural network, a deformation field for each time point from a corresponding time‑varying latent code; generating, by a second neural network, a set of combination weights for each time point; generating, by a third neural network, a time‑invariant spatial basis from the time‑invariant latent code; forming, for each time point, an intermediate image by combining the spatial basis using the combination weights for that time point; generating a final image for each time point by warping the intermediate image using the deformation field for that time point; and jointly training the neural networks and the latent codes to reconstruct an image from the plurality of undersampled magnetic resonance data.

Description

SYSTEMS AND METHODS FOR DYNAMIC IMAGE RECONSTRUCTIONCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U. S. provisional patent application No.63 / 716,363, filed on November 5, 2024, and titled “SYSTEMS AND METHODS FOR DYNAMIC IMAGE RECONSTRUCTION,” the disclosure of which is expressly incorporated herein by reference in its entirety,STATEMENT REGARDING FEDERALLY FUNDED RESEARCH

[0002] This invention was made with government support under R01 EB029957 awarded by the National Institutes of Health. The government has certain rights in the invention.BACKGROUND

[0003] Magnetic Resonance Imaging (MRI) includes types of non-mvasive imaging technology that produce images of soft tissue structures from inside the body, such as organs and vessels. One type of MRI is Cardiovascular Magnetic Resonance (CMR). CMR data acquisition is time-consuming and can be accelerated by acquiring the undersampled data. Improvements to MRI and CMR image reconstruction can be used to improve the quality of images from the undersampled data, and can improve imaging, diagnosis, and treatment of various diseases.SUMMARY

[0004] In some aspects, implementations of the present disclosure include a computer-implemented method including: initializing a plurality of time-varying latent codes and at leastone time-invariant latent code for a plurality of undersampled magnetic resonance data; generating, by a first neural network, a deformation field for each time point from a corresponding time- varying latent code; generating, by a second neural network, a set of combination weights for each time point; generating, by a third neural network, a time-invariant spatial basis from the time-invariant latent code; forming, for each time point, an intermediate image by combining the spatial basis using the combination weights for that time point; generating a final image for each time point by warping the intermediate image using the deformation field for that time point; and jointly training the neural networks and the latent codes to reconstruct an image from the plurality of undersampled magnetic resonance data

[0005] In some aspects, implementations of the present disclosure include a method, wherein the corresponding time- varying latent code for a given time point drives both the generation of the deformation field and the generation of the combination weights for that time point.

[0006] In some aspects, implementations of the present disclosure include a method, wherein training is instance-specific and self-supervised, is performed without pre-training on external datasets, and enforces consistency with a model of a magnetic-resonance acquisition.

[0007] In some aspects, implementations of the present disclosure include a method, wherein the training further includes a constraint that promotes smoothness or continuity of the deformation fields across space and / or over time.

[0008] In some aspects, implementations of the present disclosure include a method, wherein the first neural network is configured for motion handling.

[0009] In some aspects, implementations of the present disclosure include a method, wherein the plurality of undersampled magnetic resonance data corresponds to at least one of cardiovascular magnetic resonance imaging applications.

[0010] In some aspects, implementations of the present disclosure include a method, wherein acquisition parameters and sampling patterns vary across frames.

[0011] In some aspects, implementations of the present disclosure include a method, wherein the time-invariant spatial basis includes multiple learned basis images or features that are shared across all frames.

[0012] In some aspects, implementations of the present disclosure include a method, wherein at least one of the latent codes is initialized randomly,

[0013] In some aspects, implementations of the present disclosure include a method, wherein at least one of the latent codes is initialized from information derived from a motion sensor or the imaging data itself.

[0014] In some aspects, implementations of the present disclosure include a system including: an MRI imaging device; a computing device operably coupled to the MRI imaging device, the computing device operably including a processor and a memory, the memory including computer executable instructions stored thereon that, when executed by the processor, cause the processor to: receive a plurality of undersampled magnetic resonance data from the MRI imaging device; initialize a plurality of time-varying latent codes and at least one timeinvariant latent code; generate, by a first neural network, a deformation field for each time point from a corresponding time-varying latent code; generate, by a second neural network, a set of combination weights for each time point; generate, by a third neural network, a time- invariant spatial basis from the time-invariant latent code; form, for each time point, an intermediate imageby combining the spatial basis using the combination weights for that time point; generate a final image for each time point by warping the intermediate image using the deformation field for that time point; and jointly train the first neural network, second neural network, and third neural network to reconstruct an image from the undersampled magnetic resonance data

[0015] In some aspects, implementations of the present disclosure include a system, wherein the memory’ further includes computer-executable instructions that, when executed by the processor, cause the processor to: input the plurality of undersampled magnetic resonance data into the first neural network, second neural network, and third neural network, and output, by first neural network, second neural network, and third neural network, a reconstructed image.

[0016] In some aspects, implementations of the present disclosure include a system, wherein the memory further includes computer-executable instructions that, when executed by the processor, cause the processor to: display the reconstructed image on a display.

[0017] In some aspects, implementations of the present disclosure include a system, wherein the corresponding time- varying latent code for a given time point drives both the generation of the deformation field and the generation of the combination weights for that time point.

[0018] In some aspects, implementations of the present disclosure include a system, wherein training is instance-specific and self-supervised, is performed without pre-training on external datasets, and enforces consistency with a model of a magnetic-resonance acquisition.

[0019] In some aspects, implementations of the present disclosure include a system, wherein the training further includes a constraint that promotes smoothness or continuity’ of the deformation fields across space and / or over time.

[0020] In some aspects, implementations of the present disclosure include a system, wherein the reconstructed image corresponds to at least one of cardiac cine, myocardial perfusion, or late gadolinium enhancement imaging.

[0021] In some aspects, implementations of the present disclosure include a system, wherein acquisition parameters and sampling patterns vary across frames.

[0022] In some aspects, implementations of the present disclosure include a system, wherein the time-invariant spatial basis includes multiple learned basis images or features that are shared across all frames.

[0023] In some aspects, implementations of the present disclosure include a system, wherein at least one of the latent codes is initialized randomly,

[0024] It should be understood that the above-described subject matter may also be implemented as a computer-controlled apparatus, a computer process, a computing system, or an article of manufacture, such as a computer-readable storage medium.

[0025] Other systems, methods, features and / or advantages will be or may become apparent to one with skill in the art upon examination of the following drawings and detailed description. It is intended that all such additional systems, methods, features and / or advantages be included within this description and be protected by the accompanying claims.BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The components in the drawings are not necessarily to scale relative to each other. Like reference numerals designate corresponding parts throughout the several views.

[0027] FIG. 1A illustrates a flow chart of an example method for image reconstruction, according to implementations of the present disclosure.

[0028] FIG. 1B illustrates an example method for performing image reconstruction in cardiac magnetic resonance imaging, according to implementations of the present disclosure.

[0029] FIG. 2 illustrates an example system including a magnetic resonance imaging machine, according to implementations of the present disclosure.

[0030] FIG. 3 is an example computing device.

[0031] FIGS. 4A-4C illustrate example CINE image reconstructions according to a study described herein. FIG. 4A illustrates a reconstruction using an example implementation of the present disclosure. FIG. 4B illustrates an example reconstruction using LR-DIP. FIG. 4C illustrates an example reconstruction using L+S.

[0032] FIGS, 5A-5C illustrate example CINE image reconstructions according to a study described herein. FIG. 5A illustrates a reconstruction using an example implementation of the present disclosure. FIG. 5B illustrates an example reconstruction using LR-DIP. FIG. 5C illustrates an example reconstruction using L+S.

[0033] FIGS. 6A-6C illustrate example first-pass perfusion image reconstructions according to a study described herein. FIG. 6A illustrates a reconstruction using an example implementation of the present disclosure. FIG. 6B illustrates an example reconstruction using LR-DIP. FIG. 6C illustrates an example reconstruction using L+S.

[0034] FIGS. 7A-7C illustrate example first-pass perfusion image reconstructions according to a study described herein. FIG. 7A illustrates a reconstruction using an example implementation of the present disclosure. FIG. 7B illustrates an example reconstruction using LR-DIP. FIG. 7C illustrates an example reconstruction using L+S.

[0035] FIGS. 8A-8C illustrate example LGE image reconstructions according to a study described herein. FIG. 8A illustrates a reconstruction using an example implementation ofthe present disclosure. FIG. 8B illustrates an example reconstruction using LR-DIP. FIG. 8C illustrates an example reconstruction using L+S.DETAILED DESCRIPTION

[0036] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. Methods and materials similar or equivalent to those described herein can be used in the practice or testing of the present disclosure. As used in the specification, and in the appended claims, the singular forms “a,” “an,” “the” include plural referents unless the context clearly dictates otherwise. The term “comprising” and variations thereof as used herein is used synonymously with the term “including” and variations thereof and are open, non-limiting terms. The terms “optional” or “optionally” used herein mean that the subsequently described feature, event or circumstance may or may not occur, and that the description includes instances where said feature, event or circumstance occurs and instances where it does not. Ranges may be expressed herein as from "about" one particular value, and / or to "about" another particular value. When such a range is expressed, an aspect includes from the one particular value and / or to the other particular value. Similarly, when values are expressed as approximations, by use of the antecedent "about," it will be understood that the particular value forms another aspect. It will be further understood that the endpoints of each of the ranges are significant both in relation to the other endpoint, and independently of the other endpoint. While implementations will be described for performing reconstruction of cardiovascular magnetic resonance (CMR) images, it will become evident to those skilled in the art that the implementations are not limited thereto, but are applicable for reconstructing other types of images.

[0037] Implementations of the present disclosure include improvements to image processing and analysis for cardiac magnetic resonance imaging (“CMR”), in particular accelerated and / or undersampled CMR. CMR can have significant benefits, including providing a comprehensive assessment of the cardiovascular structure, function, and morphology. Image reconstruction can be required to obtain a diagnostic-quality image from CMR images, in particular images that are obtained using techniques where the data is undersampled and / or acquisition process is accelerated.

[0038] Existing CMR reconstruction techniques do not fully use the spatial and / or temporal structure of CMR data for image reconstruction, and / or require large amounts of training data to be configured for accurate image reconstruction. Implementations of the present disclosure can improve these and other problems with existing CMR reconstruction techniques, and / or reconstruction of any other type of image data (e.g., other MRI images).

[0039] For example, implementations of the present disclosure include systems and methods through multi-dynamic Deep-image prior (M-DIP) techniques as described herein.

[0040] An example implementation of the present disclosure includes a method of reconstructing moving magnetic resonance images by combining multiple complementary methods inside one model. The example implementation can learn a time-invariant set of basis images that capture anatomy and learn for each time point both mixing weights that form an intermediate stage image and a deformation field that warps the image to match the motion. The same time-dependent code drives both the weights and the deformation, so appearance-changes in motion are coordinated. The model can be trained in a self-supervised manner directly from k-space data of the scan being reconstructed with a loss that enforces magnetic resonance, data consistency, and smooths motion. This joint design improves over prior DIP approaches thatlearn only a basis or only a deformation or that require separate motion estimation. Implementations of the present disclosure overcome technical limitations of transforming raw magnetic resonance images into diagnostic images using specific data structures and a specific training objective to improve image reconstruction for accelerated and undersampled cardiac MRI. For example, implementations of the present disclosure exploit both geometry and intensity dynamics in a unified model. The model can be performant enough to run on standard computing hardware that can be coupled to an MRI scanner,

[0041] Fig. 1 A is a flowchart of an example method for reconstructing a time-series of magnetic-resonance images, according to implementations of the present disclosure. The method of FIG. 1A can be used to reconstruct images using temporal information. It should be understood that steps 110-180 can be completed in different orders, and that the steps can be optionally completed in parallel.

[0042] The method shown in FIG. 1 A can be used to reconstruct a time series of images based on k-space imaging data collected using any MRI methodology. Optionally, the k-space data can be undersampled data and / or data that includes motion artifacts. Implementations of the present disclosure can overcome particular limitations of cardiac cine imaging, myocardial perfusion image, and late gadolinium enhancement imaging by enabling reconstruction of images with significant motion artifacts. Optionally, the k-space data can include data collected from an acquisition process where the acquisition parameters of the MRI methodology and / or the sampling pattern used vary over time.

[0043] At step 110, the method includes initializing a plurality of time-varying latent codes and at least one time-invariant latent code. Optionally, the latent codes can be randomly generated and / or based on motion sensor data or the MRI data.

[0044] At step 120, the method includes generating, by a first neural network, a deformation field for each time point from a corresponding time-varying latent code. The trainable time-varying latent code can be used to adjust or configure both the generation of the deformation field.

[0045] At step 130, the method includes generating, by a second neural network, a set of time-varying combination weights from the time-varying latent code. These weights are used to compose an intermediate image for Step 150.

[0046] At step 140, the method includes generating, by a third neural network, a timeinvariant spatial basis from the time-invariant latent code. The time-invariant spatial basis can include one or more learned features that are shared across image frames of the time-series of magnetic resonance images.

[0047] At step 150, the method includes forming, for each time point, an intermediate image by combining the spatial basis using the combination weights for that time point.

[0048] At step 160, the method includes generating a final image for each time point by warping the intermediate image using the deformation field for that time point.

[0049] At step 170, the method includes jointly training the neural networks and the static and dynamic latent code. The neural networks can be trained to enforce consistency between the outputs of the neural networks using a model of magnetic-resonance acquisition. The training performed at step 170 can be self-supervised so that no external preexisting dataset is required. Optionally, the training at step 170 can include one or more constraints configured to enforce smoothness or continuity of deformation fields across space and / or time.

[0050] A study was performed of an example implementation of the present disclosure configured for cardiac MRI image reconstruction. With reference to FIG. IB, the exampleimplementation includes comprehensive framework for self-supervised accelerated CMR reconstruction based on deep image prior (DIP), referred to herein as multi-dynamic deep image prior (M-DIP). The present disclosure includes a study of the M-DIP implementation, where the study trained three generators on an instance-specific basis to recover dynamic imaging series from undersampled k-space data.

[0051] In the example implementation, a first code-to-image generator creates a dictionary of static images; a second code-to-weight generator generates time-dependent weights to combine these dictionary elements; The weights and inputs (random codes) of the generators are jointly learned during training. The input to the first generator is a static code, while the inputs to the other two generators vary with time to capture cardiac and respiratory motions as well as contrast dynamics. The example implementation improves previous DIP frameworks and applies to a broad range of CMR applications. The study results herein demonstrate the feasibility and performance of M-DIP.

[0052] FIG. IB illustrates an example implementation of the present disclosure referred to herein as M-DIP, including example steps A-L. It should be understood that while the steps A-L can be performed in parallel and / or series with one another according to various implementations of the present disclosure. Step (A) can include initializing with trainable dynamic codes of dimensionality K. Step (B) can include initializing trainable static code of dimensionality N, Step (C) can include using a networkthat generates a different deformation field for each t. Step (D) can include using a networkthat generates a different set of weights for each t. Step (E) can include using a networkthat generates time-invariant L spatial basis. Step (F) can include using a deformation field generated byat t = T. Step (G) can include outputting weights generated byat t = T. Step (H) can include a spatial basis b1: Lgeneratedby Qe. Step (I) can include determining the sum of the basis after they have been multiplied with time-specific weights. Step (J) can include generating an intermediate composite image at t ~ T. Step (K) can include performing a warping operation that applies deformation to the intermediate image. Step (L) includes outputting the reconstructed frame at t — T.

[0053] The study included recovering an image serieswhere each of the 7’ frames represents a d-dimensional image with IV pixels or voxels.^ e tthframe, z 6 if®17represents the static code.nts the dynamic codes for the6 CLrepresents dynamicweights for the tthframe. bVL= {h f=1, where b, E CNrepresents the ithelement of the spatial basis.where <p^ EdjVrepresents the deformation field for the tthframe. The optimization problem solved in M-DIP:z^'^.z,^),^, & = argwhere,

[0054] With reference to FIG. 2, an example system is shown according to implementations of the present disclosure. The system of FIG. 2 can be used to implement any of the methods described herein, including, for example, the methods of FIGS. 1A and IB. Thesystem can include an MRI imaging device 210, a computing device 220, and a display 230. The computing device 220 can optionally be the computing device 300 described with reference to FIG. 3. As shown in FIG. 2, the MRI imaging device 210 can optionally include a motion sensor 212 that can be used to collect motion sensor data. The computing device 220 can store the first neural network 222, the second neural network 224, the third neural network 226. In some implementations, multiple computing devices are used to store the neural networks separately. Alternatively or additionally, any combination of the first neural network 222, the second neural network 224, the third neural network 226 can be stored on any number of computing devices.

[0055] The computing device 220 can be coupled to the MRI imaging device 210 by any network, wired or wireless, as well as to the display by any network, wired or wireless. The MRI imaging device 210 can be used to collect training data to train the first neural network 222, second neural network 224, and third neural network 226 according to the methods described herein with reference to FIGS. 1 A and IB. The reconstructed image can optionally be output to the display 230 for viewing by a user. This allows the networks to be trained for each individual patient and each imaging session, allowing for improvements over techniques where one or more networks are trained and re-used for multiple patients or imaging sessions.

[0056] It should be appreciated that the logical operations described herein with respect to the various figures may be implemented (1) as a sequence of computer-implemented acts or program modules (i.e., software) running on a computing device (e.g., the computing device described in FIG. 3), (2) as interconnected machine logic circuits or circuit modules (i.e., hardware) within the computing device and / or (3) a combination of software and hardware of the computing device. Thus, the logical operations discussed herein are not limited to any specific combination of hardware and software. The implementation is a matter of choice dependent onthe performance and other requirements of the computing device. Accordingly, the logical operations described herein are referred to variously as operations, structural devices, acts, or modules. These operations, structural devices, acts and modules may be implemented in software, in firmware, in special purpose digital logic, and any combination thereof. It should also be appreciated that more or fewer operations may be performed than shown in the figures and described herein. These operations may also be performed in a different order than those described herein.

[0057] Referring to FIG. 3, an example computing device 300 upon which the methods described herein may be implemented is illustrated. It should be understood that the example computing device 300 is only one example of a suitable computing environment upon which the methods described herein may be implemented. Optionally, the computing device 300 can be a well-known computing system including, but not limited to, personal computers, servers, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, network personal computers (PCs), minicomputers, mainframe computers, embedded systems, and / or distributed computing environments including a plurality of any of the above systems or devices. Distributed computing environments enable remote computing devices, which are connected to a communication network or other data transmission medium, to perform various tasks. In the distributed computing environment, the program modules, applications, and other data may be stored on local and / or remote computer storage media.

[0058] In its most basic configuration, computing device 300 typically includes at least one processing unit 306 and system memory 304. Depending on the exact configuration and type of computing device, system memory 304 may be volatile (such as random access memory (RAM)), non-volatile (such as read-only memory (ROM), flash memory, etc.), or somecombination of the two. This most basic configuration is illustrated in Fig. 3 by dashed line 302. The processing unit 306 may be a standard programmable processor that performs arithmetic and logic operations necessary for operation of the computing device 300. The computing device 300 may also include a bus or other communication mechanism for communicating information among various components of the computing device 300.

[0059] Computing device 300 may have additional features / functionality. For example, computing device 300 may include additional storage such as removable storage 308 and non-removable storage 310 including, but not limited to, magnetic or optical disks or tapes. Computing device 300 may also contain network connection(s) 16 that allow the device to communicate with other devices. Computing device 300 may also have input device(s) 314 such as a keyboard, mouse, touch screen, etc. Output device(s) 312 such as a display, speakers, printer, etc. may also be included. The additional devices may be connected to the bus in order to facilitate communication of data among the components of the computing device 300. All these devices are well known in the art and need not be discussed at length here.

[0060] The processing unit 306 may be configured to execute program code encoded in tangible, computer-readable media. Tangible, computer-readable media refers to any media that is capable of providing data that causes the computing device 300 (i.e., a machine) to operate in a particular fashion. Various computer-readable media may be utilized to provide instructions to the processing unit 306 for execution. Example tangible, computer-readable media may include, but is not limited to, volatile media, non-volatile media, removable media and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. System memory 304, removable storage 308, and non-removable storage 310 are all examples oftangible, computer storage media. Example tangible, computer-readable recording media include, but are not limited to, an integrated circuit (e.g., field-programmable gate array or application-specific IC), a hard disk, an optical disk, a magneto-optical disk, a floppy disk, a magnetic tape, a holographic storage medium, a solid-state device, RAM, ROM, electrically erasable program read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices,

[0061] In an example implementation, the processing unit 306 may execute program code stored in the system memory 304. For example, the bus may carry data to the system memory 304, from which the processing unit 306 receives and executes instructions. The data received by the system memory 304 may optionally be stored on the removable storage 308 or the non-removable storage 310 before or after execution by the processing unit 306.

[0062] Example Results:

[0063] An example implementation of the present disclosure, referred to herein as M-DIP, was tested using 10-second real-time cine one-slice data. The reconstruction time was approximately 10 minutes using a GPU sold under the trade name NVIDIA RTX 4090. The example implementation required about 5 gigabytes of GPU memory to perform the reconstructions performed in the study.

[0064] CINE results were collected to compare an example implementation of the present disclosure to LR-DIP (low-rank deep image prior) and L+S (low rank and sparse) reconstruction techniques. FIGS. 4A-4C illustrate an example result comparing the reconstructed image from M-DIP (FIG. 4A) to LR-DIP (FIG. 4B) and L+S (FIG. 4C). FIGS. 5A-5C further illustrate a comparison of reconstructed image from M-DIP (FIG. 5A) to LR-DIP (FIG. 5B) andL+S (FIG. 5C). The results of FIGS. 4A-5C illustrate that M-DIP performs well when resolving periodic motion that can be present in CINE images.

[0065] Perfusion results were collected to compare an example implementation of the present disclosure to LR-DIP and L+S. FIGS. 6A-6C illustrate an example result comparing the reconstructed image from M-DIP (FIG. 6A) to LR-DIP (FIG. 6B) and L+S (FIG. 6C). FIGS. 7A-7C further illustrate a comparison of reconstructed image from M-DIP (FIG. 7A) to LR-DIP (FIG. 7B) and L+S (FIG. 7C). The results of FIGS. 6A-7C illustrate that M-DIP performs well when resolving rapid contrast changes and motion that can be present in perfusion images,

[0066] Late Gadolinium Enhancement (LGE) results were also collected to compare an example implementation of the present disclosure to LR-DIP and L+S. FIGS. 8A-8C illustrate an example result comparing the reconstructed image from M-DIP (FIG, 8A) to LR-DIP (FIG, 8B) and L+S (FIG. 8C). The results of FIGS. 8A-8C illustrate that M-DIP performs well when resolving non-periodic respiratory motion that can be present in LGE images.

[0067] In sum, the results herein show that implementations of the present disclosure can improve over conventional methods by providing higher reconstruction quality through the use of deformation fields and low-rank representation. Additionally, the example implementation is robust against artifacts and to enable perfusion and free breathing LGE data to be recovered as demonstrated in the phantom study described herein.

[0068] It should be understood that the various techniques described herein may be implemented in connection with hardware or software or, where appropriate, with a combination thereof. Thus, the methods and apparatuses of the presently disclosed subject matter, or certain aspects or portions thereof, may take the form of program code (i.e., instructions) embodied in tangible media, such as floppy diskettes, CD-ROMs, hard drives, or any other machine-readablestorage medium wherein, when the program code is loaded into and executed by a machine, such as a computing device, the machine becomes an apparatus for practicing the presently disclosed subject matter. In the case of program code execution on programmable computers, the computing device generally includes a processor, a storage medium readable by the processor (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. One or more programs may implement or utilize the processes described in connection with the presently disclosed subject matter, e.g., through the use of an application programming interface (API), reusable controls, or the like. Such programs may be implemented in a high level procedural or object-oriented programming language to communicate with a computer system. However, the program(s) can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language and it may be combined with hardware implementations.

[0069] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

WHAT IS CLAIMED:

1. A computer-implemented method comprising:initializing a plurality of time-varying latent codes and at least one time- invariant latent code for a plurality of undersampled magnetic resonance data;generating, by a first neural network, a deformation field for each time point from a corresponding time-varying latent code;generating, by a second neural network, a set of combination weights for each time point;generating, by a third neural network, a time-invariant spatial basis from the time-invariant latent code;forming, for each time point, an intermediate image by combining the spatial basis using the combination weights for that time point;generating a final image for each time point by warping the intermediate image using the deformation field for that time point; andjointly training the neural networks and the latent codes to reconstruct an image from the plurality of undersampled magnetic resonance data.

2. The method of claim 1, wherein the corresponding time-varying latent code for a given time point drives both the generation of the deformation field and the generation of the combination weights for that time point.

3. The method of claim 1, wherein training is instance-specific and self-supervised, is performed without pre-training on external datasets, and enforces consistency with a model of a magnetic-resonance acquisition.

4. The method of claim 1, w’herein the training further includes a constraint that promotes smoothness, continuity, or diffeomorphism of the deformation fields across space and / or over time.

5. The method of claim 1, wherein the training further includes a constraint that promotes smoothness or continuity on the image basis or the intermediate image.

6. The method of claim 1, wherein the first neural network is configured for motion handling.

7. The method of claim 1, wherein the plurality of undersampled magnetic resonance data corresponds to at least one of cardiovascular magnetic resonance imaging applications.

8. The method of claim 1, wherein acquisition parameters and sampling patterns vary across frames.

9. The method of claim 1, wherein the time-invariant spatial basis comprises multiple learned basis images or features that are shared across all frames.

10. The method of claim 1, wherein at least one of the latent codes is initialized randomly.

11. The method of claim 1, wherein at least one of the latent codes is initialized from information derived from a motion sensor or the magnetic resonance data.

12. A system comprising:an MRI imaging device;a computing device operably coupled to the MRI imaging device, the computing device operably comprising a processor and a memory7, the memory comprising computer executable instructions stored thereon that, when executed by the processor, cause the processor to:receive a plurality of undersampled magnetic resonance data from the MRI imaging device;initialize a plurality of time-varying latent codes and at least one time-invariant latent code;generate, by a first neural network, a deformation field for each time point from a corresponding time-varying latent code;generate, by a second neural network, a set of combination weights for each time point;generate, by a third neural network, a time-invariant spatial basis from the time-invariant latent code;form, for each time point, an intermediate image by combining the spatial basis using the combination weights for that time point;generate a final image for each time point by warping the intermediate image using the deformation field for that time point; andjointly train the first neural network, second neural network, and third neural network to reconstruct an image from the undersampled magnetic resonance data.

13. The system of claim 12, wherein the memory further comprises computer-executable instructions that, when executed by the processor, cause the processor to: input the plurality of undersampled magnetic resonance data into the first neural network, second neural network, and third neural network, and output, by first neural network, second neural network, and third neural network, a reconstructed image.

14. The system of claim 12, wherein the memory further comprises computer-executable instructions that, when executed by the processor, cause the processor to: display the reconstructed image on a display.

15. The system of claim 12, wherein the corresponding time-varying latent code for a given time point drives both the generation of the deformation field and the generation of the combination weights for that time point.

16. The system of claim 12, wherein training is instance-specific and self-supervised, is performed without pre-training on external datasets, and enforces consistency with a model of a magnetic-resonance acquisition.

17. The system of claim 12, wherein the training further includes a constraint that promotes smoothness, continuity, or diffeomorphism of the deformation fields across space and / or over time.

18. The system of claim 12, wherein the training further includes a constraint that promotes smoothness or continuity on the image basis or the intermediate image.

19. The system of claim 12, wherein the reconstructed image corresponds to at least one of cardiac cine, myocardial perfusion, or late gadolinium enhancement imaging.

20. The system of claim 12, wherein acquisition parameters and sampling patterns vary across frames.

21. The system of claim 12, wherein the time-invariant spatial basis comprises multiple learned basis images or features that are shared across all frames.The system of claim 12, wherein at least one of the latent codes is initialized randomly.

Images