Deep learning autonavigation of free breathing magnetic resonance imaginig (MRI) data

Abstract

Systems and methods for extraction and use of respiratory signals for k-space datasets are disclosed herein. A method can include identifying a spatiotemporal dataset comprising a projections corresponding to k-space datasets acquired in accordance with MRI. The method can include providing the spatiotemporal dataset as an input to a machine learning (ML) model. The method can include generating, based on providing the spatiotemporal dataset to the ML model, a signal identifying the motion of the torso region of the subject relative to the axis over the time period.

Description

Atty. Dkt. No.: 115872-3062DEEP LEARNING AUTONAVIGATION OF FREE BREATHING MAGNETIC RESONANCE IMAGING (MRI) DATA STATEMENT OF U.S. GOVERNMENT SUPPORT|0001] This invention was made with government support under CA255661, awarded by the National Institutes of Health. The government has certain rights in the invention.TECHNICAL FIELD

[0002] The present application relates generally to the extraction of motion signals from the acquired k-space data using deep learning and its use to reconstructed motion-resolved images, as may be generated from magnetic resonance imaging (MRI) devices.BACKGROUND

[0003] A computing system may use a computer vision algorithm to process a digital image to generate outputs.SUMMARY

[0004] At least one aspect of the present disclosure is directed to a method of generating motion signals from magnetic resonance imaging (MRI) data. The method can include identifying, by one or more processors, a spatiotemporal dataset including a plurality of projections corresponding to a plurality of k-space datasets acquired in accordance with MRI. The k-space datasets can correspond to a torso region of a subject in motion relative to an axis over a time period. The method can include providing, by the one or more processors, the spatiotemporal dataset as an input to a machine learning (ML) model. The ML model can be established using a plurality of examples. Each of the plurality of examples can include a respective spatiotemporal dataset including a respective plurality of projections corresponding to a respective torso region of a respective subject in a respective motion relative to a respective axis over a respective time period. Each of the plurality of examples can include a respective signal identifying the respective motion of the respective torso region of the respective subject over the respective time period. The method can include generating, by the one or more processors, based on providing the spatiotemporal dataset to the ML model, a signal identifying the motion of the torso region of the subject relative to the axis-1- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062over the time period. The method can include storing, by the one or more processors, using one or more data structures, an association between the subject and the signal.

[0005] In some implementations, the method includes comparing, by the one or more processors, for each of the plurality of k-space datasets, a corresponding segment of the signal with a plurality of amplitude ranges corresponding to a plurality of motion states. The method can include classifying, by the one or more processors, each k-space dataset of the plurality of k-space datasets into a motion state of the plurality of motion states in accordance with comparison of the corresponding segment with the plurality of amplitude ranges.

[0006] In some implementations, the method includes applying, by the one or more processors, an image reconstruction function on the plurality of k-space datasets and the motion state for each of the plurality of k-space datasets. The method can include generating, by the one or more processors, based on applying the image reconstruction function, a plurality of MRI image frames of the torso region of the subject relative to the axis over the time period.

[0007] Classifying each k-space of the plurality of k-space datasets can include sorting the plurality of k-space datasets by the motion state for each k-space of the plurality of k-space datasets.

[0008] In some implementations, the method includes receiving, by the one or more processors, via an MRI machine, the plurality of k-space datasets, each k-space dataset of the plurality of k-space datasets including a set of signal measurements corresponding to a respective segment of the time period acquired in accordance with MRI. Identifying the spatiotemporal dataset can include generating the spatiotemporal dataset using the plurality of k-space datasets in accordance with an inverse Fourier transform.

[0009] In some implementations, the method includes receiving, by the one or more processors, an identification of a tumor of the torso region, within a plurality of MRI image frames corresponding to the torso region of the subject in motion. The method can include causing, by the one or more processors, administration of radiotherapy to the tumor of the torso region of the subject, in accordance with the motion identified by the signal.

[0010] In some implementations, the radiotherapy includes at least one of includes at least one of a stereotactic body radiation therapy (SBRT), an intensity-modulated radiation -2- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062therapy (IMRT), a volumetric modulated arc therapy (VMAT), a conformal radiation therapy (CRT), or a proton beam therapy (PBT).

[0011] In some implementations, the subject is not under anesthesia during acquisition by an MRI machine used to acquire the of k-space datasets. In some implementations, the signal identifying the motion of the torso region can be determined, by the ML model, using the k-space datasets without external sensor data. In some implementations, the respective signal of at least one of the plurality of examples is generated in accordance with principal component analysis (PCA) using the respective spatiotemporal dataset including the respective plurality of projections.

[0012] In some implementations, the ML model is established by providing the respective spatiotemporal dataset of at least one example of the plurality of examples as input to the ML model to generate an output signal identifying the respective motion of the respective torso region of the respective subject over the respective time period. The ML model can be established by determining an error metric based on a comparison between the output signal and the respective signal. The ML model can be established by updating one or more weights of the ML model in accordance with the error metric.

[0013] In some implementations, the subject is not under anesthesia during acquisition by the MRI machine. The subject can be at risk of or diagnosed with cancer associated with the torso region. The cancer can include at least one of lung cancer, esophageal cancer, stomach cancer, colorectal cancer, liver cancer, pancreatic cancer, small intestine cancer, kidney cancer, bladder cancer, prostate cancer, ovarian cancer, uterine cancer, or cervical cancer.

[0014] At least one aspect of the present disclosure is directed to a system for motion signal management for magnetic resonance imaging (MRI) data. The system includes one or more processors coupled with memory. The one or more processors can identify a spatiotemporal dataset including a plurality of projections corresponding to a plurality of k-space datasets acquired in accordance with MRI, the plurality of k-space datasets corresponding to an torso region of a subject in motion relative to an axis over a time period. The one or more processors can provide the spatiotemporal dataset as an input to a machine learning (ML) model, wherein the ML model is established using a plurality of examples, each of the plurality of examples including (i) a respective spatiotemporal dataset including -3- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062a respective plurality of projections corresponding to a respective torso region of a respective subject in a respective motion relative to a respective axis over a respective time period and (ii) a respective signal identifying the respective motion of the respective torso region of the respective subject over the respective time period. The one or more processors can generate, based on providing the spatiotemporal dataset to the ML model, a signal identifying the motion of the torso region of the subject relative to the axis over the time period. The one or more processors can store, using one or more data structures, an association between the subject and the signal.

[0015] In some implementations, the one or more processors are configured to compare, for each of the plurality of k-space datasets, a corresponding segment of the signal with a plurality of amplitude ranges corresponding to a plurality of motion states. The one or more processors can classify each k-space dataset of the plurality of k-space datasets into a motion state of the plurality of motion states in accordance with comparison of the corresponding segment with the plurality of amplitude ranges.|0016] In some implementations, the one or more processors are configured to apply an image reconstruction function on the plurality of k-space datasets and the motion state for each of the plurality of k-space datasets. The processors can generate, based on applying the image reconstruction function, a plurality of MRI image frames of the torso region of the subject relative to the axis over the time period.

[0017] In some implementations, classifying each k-space of the plurality of k-space datasets further includes sorting the plurality of k-space datasets by the motion state for each k-space of the plurality of k-space datasets.

[0018] In some implementations, the one or more processors are configured to receive, via an MRI machine, the plurality of k-space datasets, each k-space dataset of the plurality of k-space datasets including a set of signal measurements corresponding to a respective segment of the time period acquired in accordance with MRI, wherein identifying the spatiotemporal dataset further includes generating the spatiotemporal dataset using the plurality of k-space datasets in accordance with an inverse Fourier transform.

[0019] In some implementations, the one or more processors are configured to receive an identification of a tumor of the torso region, within a plurality of MRI image frames corresponding to the torso region of the subject in motion. The one or more processors can -4- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062cause administration of radiotherapy to the tumor of the torso region of the subject, in accordance with the motion identified by the signal.

[0020] In some implementations, the radiotherapy includes at least one of includes at least one of a stereotactic body radiation therapy (SBRT), an intensity-modulated radiation therapy (IMRT), a volumetric modulated arc therapy (VMAT), a conformal radiation therapy (CRT), or a proton beam therapy (PBT).

[0021] In some implementations, the respective signal of at least one of the plurality of examples is generated in accordance with principal component analysis (PCA) using the respective spatiotemporal dataset including the respective plurality of projections.

[0022] In some implementations, to establish the ML model, the one or more processors are configured to provide the respective spatiotemporal dataset of at least one example of the plurality of examples as input to the ML model to generate an output signal identifying the respective motion of the respective torso region of the respective subject over the respective time period; and determine an error metric based on a comparison between the output signal and the respective signal; and updating one or more weights of the ML model in accordance with the error metric.

[0023] In some implementations, the one or more processors are configured to receive the plurality of k-space datasets corresponding to a subject: not under anesthesia during acquisition by the MRI machine; and at risk of or diagnosed with a cancer associated with the torso region, wherein the cancer includes at least one of lung cancer, esophageal cancer, stomach cancer, colorectal cancer, liver cancer, pancreatic cancer, small intestine cancer, kidney cancer, bladder cancer, prostate cancer, ovarian cancer, uterine cancer, or cervical cancer.

[0024] At least one aspect of the present disclosure is directed to a system for dynamic image reconstruction. The system can include one or more processors coupled to a non-transitory memory. The system can receive a plurality of k-space datasets. The system can detect a motion signal from the plurality of k-space datasets. The system can classify each of the plurality of k-space data sets according to a state of the motion signal. The system can resolve the plurality of the k-space datasets to a plurality of first images, each of the plurality of first Euclidean space images corresponding to one of the plurality of k-space datasets. The system can convey the plurality of first Euclidean space images and corresponding image -5- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062acquisition data to an image reconstruction network. The system can resolve, by the image reconstruction network, a second Euclidean space image, based on the plurality of first Euclidean space images without checking data consistency in k-space.

[0025] In some implementations, the k-space data sets are radially sampled. In some implementations, the k-space data sets are generated for each of a plurality of coils of the MRI machine. In some implementations, the motion signal corresponds to a cardiac cycle or a respiratory cycle. In some implementations, the states of the motion signal are defined according to an amplitude of the motion signal. In some implementations, the k-space data sets are three dimensional. In some implementations, the image reconstruction network is a residual U-net network. In some implementations, the image reconstruction network employs patch mixing between at least Euclidean, coil, and motion signal state axes.

[0026] At least one aspect of the present disclosure relates to a method for image reconstruction. The method can be performed, for example, by a data processing system. The method includes receiving a plurality of k-space data sets. The method includes detecting a motion signal from the plurality of k-space. The method includes classifying each of the plurality of k-space data sets according to a state of the motion signal. The method includes resolving the plurality of the k-space data sets to a plurality of first images, each of the plurality of first images corresponding to one of the plurality of k-space data sets. The method includes conveying the plurality of first images and corresponding image acquisition data to an image reconstruction network of the data processing system. The method includes resolving, by an image reconstruction network, a second image, based on the plurality of first images.

[0027] In some implementations, the k-space data sets are radially sampled. In some implementations, the k-space data sets are generated for each of a plurality of coils of the MRI machine. In some implementations, the k-space data sets are three dimensional. In some implementations, the motion signal is one-dimensional. In some implementations, the motion signal corresponds to a cardiac cycle or a respiratory cycle. In some implementations, the states of the motion signal are defined according to an amplitude of the motion signal. In some implementations, image reconstruction network is a residual U-net network. In some implementations, the image reconstruction network employs patch mixing between at least a plurality of Euclidean axes, a coil axis, and a motion signal state axis.-6- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062

[0028] At least one aspect of the present disclosure relates to a computer-readable medium storing instructions that, when executed by a one or more processors, cause it to perform a process. The process can include receiving a plurality of k-space data sets. The process can include detecting a motion signal from the plurality of k-space data sets. The process can include classifying each of the plurality of k-space data sets according to a state of the motion signal. The process can include resolving the plurality of the k-space data sets to a plurality of first images, each of the plurality of first images corresponding to one of the plurality of k-space data sets. The process can include conveying the plurality of first images and corresponding image acquisition data to an image reconstruction network of the data processing system. The process can include resolving a second image, based on the plurality of first images.

[0029] In some implementations, the k-space data sets are received from each of a plurality of coils of an MRI machine. In some implementations, the k-space data sets are three dimensional. In some implementations, the motion signal is one-dimensional. In some implementations, the motion signal corresponds to a cardiac cycle or a respiratory cycle. In some implementations, the states of the motion signal are defined according to an amplitude of the motion signal. In some implementations, the image reconstruction network is a residual U-net network. In some implementations, the image reconstruction network employs patch mixing between at least a plurality of Euclidean axes, a coil axis, and a motion signal state axis.

[0030] These and other aspects and implementations are discussed in detail below. The foregoing information and the following detailed description include illustrative examples of various aspects and implementations, and provide an overview or framework for understanding the nature and character of the claimed aspects and implementations. The drawings provide illustration and a further understanding of the various aspects and implementations, and are incorporated in and constitute a part of this specification. It will be readily appreciated that features described in the context of one aspect of the invention can be combined with other aspects. Aspects can be implemented in any convenient form, for example, by appropriate computer programs, which may be carried on appropriate carrier media (computer readable media), which may be tangible carrier media (e.g., disks) or intangible carrier media (e.g., communications signals). Aspects may also be implemented-7- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062using suitable apparatus, which may take the form of programmable computers running computer programs arranged to implement the aspect.BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The foregoing and other objects, aspects, features, and advantages of the disclosure will become more apparent and better understood by referring to the following description taken in conjunction with the accompanying drawings, in which:|0032] FIG. 1 depicts an example system for performing dynamic motion-resolved reconstruction from continuously acquired radial MRI data, in accordance with one or more implementations.

[0033] FIGs. 2A and 2B show an example of a feature and the feature having a plurality of radial samples thereof, in accordance with one or more implementations.

[0034] FIG. 3A, 3B, and 3C show classified images of the features; FIG. 3D shows an average of the images of 3 A, 3B, and 3C, in accordance with one or more implementations.

[0035] FIG. 4 depicts a motion signal map 400, according to some embodiments.

[0036] FIG. 5 depicts a flow diagram for constructing an image from a plurality of source images, in accordance with one or more implementations.

[0037] FIG. 6 depicts a system for performing the flow of FIG. 5, in accordance with one or more implementations.

[0038] FIG. 7 depicts another system for performing the flow of FIG. 5, in accordance with one or more implementations.

[0039] FIG. 8 shows a flow diagram for generating an image from, in accordance with an illustrative embodiment.

[0040] FIG. 9 depicts (A) a RANGR motion estimation: After free-breathing acquisition of 3D MRI data using a golden-angle stack-of-stars k-space trajectory, ID projections for each angle / time point were obtained by applying inverse Fourier transform (TFT) along the kz line defined at kx=ky=0. Projections were coil-combined using sum-of-squares and concatenated as a z-t matrix (input to RANGR). The filtering+PCA results served -8- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062as a reference to train the network and (B) a motion-resolved MRI using RANGR: The motion signal from RANGR was used to sort the acquired k-space data into motion states and perform motion-resolved reconstruction using for example Movienet.

[0041] FIG. 10 depicts RANGR using a modified VGG-16 architecture. Tensor dimensions are [batch size, number of channels, height, width], where height corresponds to the z-dimension (the original number of slices (50-54) were interpolated to 400 points), and width to the temporal dimension (900 angles in this example). The final block mimics the dense layer block of a standard VGG-16, where 2-D convolutional layers of up to 4096 channels are used instead.

[0042] FIG 11 shows a motion-resolved MRI using Movienet reconstruction in combination with PCA navigation (left) and RANGR navigation (right). PCA navigation failed to detect respiratory motion, while RANGR was able to detect respiratory motion changes, leading to improvement in image quality and clear visualization of motion.

[0043] FIG. 12 is a block diagram of a server system and a client computer system in accordance with an illustrative embodiment.

[0044] FIG. 13 depicts a flow diagram for training a model to extract a motion signal from a k-space dataset, in accordance with one or more implementations.

[0045] FIG. 14 depicts a flow diagram for extracting a motion signal from a k-space dataset, in accordance with one or more implementations.

[0046] FIG. 15 depicts a flow diagram for generating an output based on a motion signal, in accordance with one or more implementations.

[0047] FIG. 16 depicts a flow diagram for a method of motion signal management for magnetic resonance imaging (MRI) data, in accordance with one or more implementations.

[0048] FIG. 17 is a block diagram of a computing environment according to an example implementation of the present disclosure.DETAILED DESCRIPTION

[0049] The present techniques can reconstruct motion-resolved 4D MR images using time-space-coil convolutional networks without k-space data consistency to remove aliasing -9- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062artifacts from acceleration and significantly reduce computation time relative to iterative reconstruction algorithms. The images can be resolved at a quality or time to inform diagnosis, treatment, planning, treatment evaluation or other care. It should be appreciated that various concepts introduced above and discussed in greater detail below may be implemented in any of numerous ways. The disclosed concepts are not limited to any particular manner of implementation. Examples of specific implementations and applications are provided primarily for illustrative purposes. For the purpose of better understanding the present disclosure, a brief overview of the sections of the detailed description may be helpful:

[0050] Section A describes systems and methods for performing dynamic motion-resolved image reconstruction.

[0051] Section B describes an example embodiment, using the systems and methods herein to generate a motion signal which can be used to generate an output from k-space datasets.

[0052] Section C describes systems and methods for the extraction and use of motion signals from k-space datasets.

[0053] Section D describes an example embodiment, using the systems and methods herein to generate a series of image space representations of an anatomical feature displaced by a motion signal during a data acquisition time.

[0054] Section E describes a network environment and computing environment which may be useful for practicing various embodiments described herein.A. Fast Motion-Resolved MRI Reconstruction Using Space-Time-Coil Convolutional Networks Without k-space Data Consistency (Movienet)

[0055] Medical images can include a plurality of captured data which is combined to form high resolution images. For example, a magnetic resonance imaging (MRI) machine can capture data in a frequency transform domain, such as k-space. The MRI machine can include a plurality of coils such that n k-space datasets can be generated for each Euclidean space image. Further, some motion can be of interest for some patients (e.g., swallowing, or a heartbeat), or can interfere with an area of interest (e.g., a heartbeat can interfere with a liver scan by displacing the liver). Thus, it may be advantageous to capture motion data and-10- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062combine data from the plurality of coils to resolve an image, containing information from a plurality of datasets. However, transforms between the k-space and the Euclidean domain can be computationally expensive, and particularly, so for resolving images from a plurality of k-space images that results from a plurality of coils or a plurality of motion states, wherein coherency between the images is maintained.

[0056] The computational expense can increase an energy use to resolve an image, as well as increase a time for resolving the image. For example, a patient may need to wait a substantial period of time for an image to be resolved so that a provider can offer a diagnosis, treatment evaluation or other care following an MRI, or may need to schedule a later appointment. For some issues requiring immediate response (such as when a medical procedure is delayed until results are available), the processing time may delay care or result in a suboptimal decision, based on unavailable data or relatively low-information data.

[0057] As noted above, the time to resolve certain k-space data to Euclidean space images can be greater than desired. Thus, an acceleration may be desirable. Detecting certain features of an image based on the k-space data, and individually resolving each of the k-space data sets can be less computationally involved. However, the various Euclidean space images may not, individually, contain adequate information or clarity, in at least some implementations. Thus, it may be advantageous for a system to receive the individually resolved images in the Euclidean domain, along with the data extracted from the k-space data. Such a system may resolve the plurality of images to at least one higher quality image, based on Euclidean space information. For example, the Euclidean space information can be computationally less expensive or can be more admitting of parallel processing. Thus, the systems and methods herein disclosed may reduce an energy usage, mean time to diagnosis, improve image quality, and otherwise advance the state of the art of therapeutic and / or diagnostic imaging, such as MRI imaging.

[0058] Referring now to FIG. 1, depicted is block diagram of an example data processing system 100, in accordance with one or more implementations. The data processing system 100 can include at least one data capture device 102. The data processing system 100 can include at least one motion signal extractor 104. The data processing system 100 can include at least one k-space image resolver 106. The data processing system 100 can include at least one classification component 108. The data processing system 100 can include at least one residual U-net network 110. The data processing system 100 can include at least one -11- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062spatiotemporal patch embedding network 112. The data processing system 100 can include at least one data repository 120.

[0059] The data capture device 102, motion signal extractor 104, k-space image resolver 106, classification component 108, residual U-net network 110, or spatiotemporal patch embedding network 112 can each include a processing unit or other logic device such as programmable logic array engine, or module configured to communicate with the data repository 120 or database. The data capture device 102, motion signal extractor 104, k-space image resolver 106, classification component 108, residual U-net network 110, or spatiotemporal patch embedding network 112 can be separate components, a single component, or part of the data processing system 100. The data processing system 100 can include hardware elements, such as one or more processors, logic devices, or circuits. For example, the data processing system 100 can include one or more components, structures or functionality of a computing device depicted in FIG. 9.

[0060] The data repository 120 can include one or more local or distributed databases, and can include a database management system. The data repository 120 can include computer data storage or memory and can store a plurality of medical images 122. The images 122 can include an MRI image (e.g., slice thereof) such as an MRI image of an anatomical feature of interest such as a pathological feature (e.g., a lesion). The images 122 can comprise k-space data sets or Euclidean space images (e.g., derived from the k-space data sets). The data repository 120 can store a motion signal 124 or associated data. For example, the motion signal 124 can be a displacement associated with a cardiac or respiratory cycle which may be associated with one or a plurality of the medical images 122. The data repository 120 can store image acquisition data 126. For example, the image acquisition data 126 can include an acquisition time, acquisition coil, or other information associated with the acquisition of the data (e.g., the k-space data).

[0061] Still referring to FIG. 1, and in further detail, the data processing system 100 can include at least one data capture device 102 designed, constructed or operational to acquire data of an anatomical feature. For example, the data capture device 102 can capture k-space data for a three-dimensional anatomical feature. The data capture device can include a plurality of coils disposed radially around a feature of interest (e.g., an anatomical feature of interest). An electric field (e.g., an RF signal) can be passed through the feature of interest which can alternatively align the poles of portions thereof, such that upon a relaxation of the -12- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062electric field (e.g., according to an electric field aligned with a spatial dimension) detectable energy can be emitted from the re-alignment (e.g., un-alignment) of the poles. For example, the energy can be measured in response to a gradient pulse and an RF (radio frequency) emission, and such a transfer function (e.g., measured response) can be mapped to the gradient or RF pulse (e.g., diagnostic or therapeutic information can be received in a k-space domain).

[0062] The data capture device 102 can capture a k-space data set, such as 2D data sets (also referred to as “slices,” herein) or 3D data sets. In some embodiments, the data capture device 102 can sample k-space data in alignment with a plurality of Euclidean space coordinates (e.g., Cartesian coordinates). For example, the data can be sampled according to an orthogonal frequency and gradient phase direction. The data capture device 102 cam pass such k-space data sets to the k-space image resolver 106. The data capture device 102 can maintain a Cartesian coordinate system. Such a coordinate system can maintain a constant phase and frequency directions which can localize movement. Such a coordinate system can sample a low frequency region of the k-space located near the origin thereof less than other methods disclosed herein, and peripheral regions with relatively high frequencies, which may emphasize high frequency data (e.g., detail) over low frequency data (e.g., contrast of major features).

[0063] In some embodiments, the data capture device 102 can employ a nonCartesian coordinate system. For example, the data capture device 102 can fill one or more radial portions of the k-space by various methods, such as filling the k-space associated with one or more rotating parallel lines. Such a radial system can sample the relatively information-dense center of the k-space more frequently that the Cartesian grids and the relatively information-sparse periphery of the k-space less frequently. Moreover, the changing angle of the data collection can disperse motion across the image. FIG. 2 discusses one radial method in further detail. The method is not intended to be limiting, indeed, the methods and systems disclosed herein can be performed according to data collected by various techniques.

[0064] The data processing system 100 can include at least one motion signal extractor 104 designed, constructed, or operational to define or identify a motion signal 124 from a plurality of the datasets captured by the data capture device 102. For example, motion signal extractor 104 can define or identify the motion signals 124 in k-space. In some embodiments, the motion signal extractor 104 can a define a period of a motion, such as a respiratory cycle which is controlled (e.g., coached) during data acquisition to maintain a -13- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062regular and defined cadence. For example, the breathing can be responsive to a human machine interface communicatively coupled to the motion signal extractor 104 for provision to a patient (e.g., by audial or visual indication). In some embodiments, a patient can free-breathe during the course of an examination, and thus a rate of respiration may be unknown. Further, a heartrate of a patient may not be readily controlled, and thus may require detection. Examples of respiratory and cardiac cycles are further discussed with regard to FIG. 4. In some embodiments, the motion signal extractor 104 can detect the motion signal 124 in Euclidean space.

[0065] In some embodiments, the motion signal extractor 104 can detect the motion signal in k-space. For example, the motion signal extractor 104 can detect a displacement of a center of a k-space line (e.g., relative to other k-space lines) and fit the displacements to predefined frequencies (e.g., according to a single dimensional Fourier transform). For example, a predefined frequency range can be defined, having a range of 0.1 Hz to 0.5 Hz, and another predefined frequency range can be defined having a range of 0.5 to 2.5 Hz. A peak detected frequency within the predefined ranges can be selected as the motion signal 124 of interest. The frequency ranges can be selected according to one or more patient attributes (e.g., to a cardiac or respiratory cycle), though such an embodiment is non-limiting. For example, the predefined frequency range can be associated with a nearby generator. In some embodiments, a plurality of frequencies can be associated with a feature. For example, a cardiac cycle can be disaggregated into a plurality of constituent cycles.

[0066] The data processing system 100 can include at least one k-space image resolver 106 designed, constructed, or operational to resolve a Euclidean space image from the k-space data. In some embodiments, the k-space image resolver 106 can resolve images by performing an inverse Fourier transform on k-space data sorted according to a Cartesian scheme. In some embodiments, the k-space image resolver 106 can associate a plurality of points disposed along a non-Cartesian scheme, such as a radial sampling of the k-space data to Cartesian coordinates and thereafter transforming the Cartesian coordinates to Euclidean space.

[0067] In some embodiments, the k-space image resolver 106 can manipulate a plurality of individual slices of k-space data to generate a plurality of two-dimensional images which can then be combined (e.g., stitched) to form a three-dimensional image. For example, the k-space image resolver 106 can combine two dimensional images captured by the data -14- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062capture device 102. In some embodiments, the k-space image resolver 106 can receive three-dimensional k-space data from the data capture device 102. The k-space image resolver 106 can resolve images according to the three-dimensional data. For example, the k-space image resolver 106 can receive data sampled by the data capture device 102, according to a three-dimensional method (e.g., a three-dimensional radial method, sometimes referred to as a “kooshball” method).

[0068] The k-space image resolver 106 can generate a three-dimensional image for each of a plurality of coils of the data capture device 102. In some embodiments, the k-space image resolver 106 can combine a plurality of three-dimensional images taken from a plurality of coils. For example, the k-space image resolver 106 can perform temporal compressed sensing reconstruction of a plurality of images to resolve a high-resolution image. For example, the k-space image resolver 106 can generate training data for either of the residual U-net network 110 or the spatio-temporal patch embedding network 112. The k-space image resolver 106 can receive a classification of a plurality of images from the classification component 108, which may increase a sparsity of device features (relative to a complete set of images comprising images in each of a plurality of the states of the motion signal 124). For example, the k-space image resolver 106 can resolve the function:

[0069] Where F defines the non-uniform FFT (fast Fourier transform) or another Fourier transform, C represents a matrix sensitivity array, d is the 2D image series with a cardiac dimension and a respiratory dimension. Si is the sparsifying transform applied in the cardiac motion dimension with regularization parameter i and S2 is the sparsifying transform applied along the extra respiratory-state dimension with regularization parameter2. R is a reordering operator along the tR dimension. In some embodiments, various terms may be substituted, replaced, omitted, or the like. For example, the R term can be omitted in embodiments having limited cardiac motion or which is otherwise accounted for.|0070] The data processing system 100 can include at least one classification component 108 designed, constructed, or operational to classify the plurality of images to one or more portions of the motion signals 124. For example, the classification component 108 can classify one or more images (or portions thereof) according to a position. As will be further discussed with regard to FIG. 4, the classification component 108 can classify the -15- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062images according to a position or a period. For example, an inhalation state which results in a similar displacement as an exhalation state can be classified as a same or different state. The classification component 108 can detect the presence of one or more outlier images (e.g., data which does not fit into a pre-defined or dynamically scaled class or state). The classification component 108 can pre-define the classes or can align the classes to the image data or k-space data. In some embodiments, the classification component 108 can dynamically define the states to maintain an equal number of data sets in each class, a minimum number of data sets per state, a maximum number of data sets per state, a minimum variance between states, or otherwise establish or adjust thresholds associated with one or more states to disperse the available data between the states.

[0071] In some embodiments, the classification component 108 can classify Euclidean space images or k-space data sets according to a plurality of motion signals. For example, the cardiac rhythm and the respiratory rhythm can be classified along a two-dimensional space (e.g., defined by the displacement of the cardiac and respiratory cycle). In some embodiments, the classification component 108 can include user controlled or automatic detection of a desired state or defining cycle. For example, an MRI of a meniscus may have an associated displacement associated with a patient breathing, but little or no discernable relationship with a cardiac cycle. The classification component 108 can thus exclude the cardiac cycle, which may be responsive to a failure to detect a cardiac cycle exceeding a threshold, or an operator entry (e.g., manually deselecting a cardiac motion signal or selecting a knee MRI).

[0072] The data processing system 100 can include at least one residual U-net network 110 designed, constructed or operational to reconstruct a high-quality image from a plurality of three-dimensional k-space images. The residual U-net network 110 can receive a training set of images from the k-space image resolver 106 which combine the 3D (or 2D) images from the plurality of coils and the plurality of motion signals states to a single processed image or a series of sorted processed images for each of the respective motion states. Subsequent to the training, the residual U-net network 110 can further receive a plurality of 3D images from the image resolver. For example, the residual U-net network 110 can receive a plurality of 3D images associated with image acquisition data 126 such as a coil associated with the image collection (e.g., RF coils to transmit or receive the data). The image acquisition data 126 can further include processed information such as a state of the motion-16- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062signal 124 of an image, as classified by the classification component 108. In some embodiments, the U-net network 110 can operate on a plurality of two-dimensional planes of the three-dimensional images. For example, the residual U-net network 110 can generate 256 two dimensional images having as resolution of 256x256 from a 3D image having a resolution of 256x256x256.

[0073] The U-net network 110 can concatenate the state of the motion signal 124 and the coils. For example, the U-net network 110 can receive the image data from a data capture device having eight coils, and a classification component classifying the images in ten motion signal states. The concatenated coils and motion signal 124 states can be disposed in the color dimension. Continuing the above example, each two-dimensional image can thus have a resolution of 256x256x80 (where 8 coils x 10 motion signal states yields the resolution of 80). The U-net network 110 can process the images as is further described with respect to FIG. 6. For example, the U-net network 110 can down sample / encode the data, and thereafter up-sample / decode the image. In some embodiments, further networks such as a Mask-R CNN network, Feature Pyramid Network, and the like can be employed to perform, supplement, or validate the operation of the U-net network 110. In some embodiments, the residual U-net network 110 can be omitted, such as an embodiment exclusively employing the spatiotemporal patch embedding network 112.

[0074] The data processing system 100 can include at least one spatiotemporal patch embedding network 112 designed, constructed, or operational to reconstruct a high-quality image from the plurality of three-dimensional images. The spatiotemporal patch embedding network 112 can receive the training data discussed with regard to the U-net network 110 and train the spatiotemporal patch embedding network 112. The loss function to train the spatiotemporal patch embedding network 112 can include a perceptual loss. The training of the model of the spatiotemporal patch embedding network 112 can be defined by or include a perceptual loss function. For example, the perceptual loss function can be another CNN, such as VGG (visual geometry group) 16 or VGG 19. The perceptual component of the loss function can be defined according to a formula:Lperceptual = Sj =|| j(y)- j(y)||2 + kp|| j(y)- j(y)||l

[0075] The perceptual loss function can be combined with a pixelwise Li or L2, which may mitigate or remove artifacts generated or tolerated by the perceptual loss function, such-17- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062as high frequency artifacts. In some embodiments, the loss function can be applied to train a model on an image basis or slice basis. In some embodiments, the loss function can be applied to a plurality of models, such as based on patches of the image or images slices. The patches can be overlapping or non-overlapping. The spatiotemporal patch embedding network 112 can mix adjoining patches. For example, the spatiotemporal patch embedding network 112 can receive training from adjacent Euclidean space images (e.g., between the x, y, and z planes, or adjacent portions thereof), or image acquisition data dimensions (e.g., adjacent motion signal 124 states or coils). The adjacency of motion signal states or coils can be based on various criteria of adjacency including temporal, spatial, or associational. For example, a respiratory cycle can include spatially adjacent but temporally non-adjacent states (e.g., the middle of an exhale or inhale); non-adjacent coils can have a close association which are not spatially adjacent (e.g., offset 180°).

[0076] The spatiotemporal patch embedding network 112 can receive a plurality of images (e.g., images having associated image acquisition data 126 such as coil and motion states) to generate one or more processed images therefrom (e.g., can generate a plurality of high resolution or high contrast images therefrom). The spatiotemporal patch embedding network 112 can define patches across coordinates in the Euclidean space associated with a three dimensional image, as well as across the image acquisition data 126, including the motion signal 124 state or the coils associated with the image from the data capture device, which may be considered as five dimensions, or concatenated to four dimensions (as for the U-net residual network 110). The spatiotemporal patch embedding network 112 can thereafter mix the various dimensions, to train (or generate) images, as is further discussed with regards to FIG. 7. In some embodiments, the spatiotemporal patch embedding network 112 can be omitted, such as an embodiment exclusively employing the residual U-net network 110. In some embodiments, both networks can be employed such as to direct particular images to one or the other network based on operator selection, data content, or parallel resolution which can be compared (e.g., by the data processing system or an operator) for sharpness, contrast, information density, or the like.

[0077] Referring to FIG. 2A, an example of k-space data 205 is depicted. The k-space data 205 is depicted with relation to a coordinate system including an x-axis 210 separating a positive and negative component. The positive and negative components can be symmetrical. For example, the positive and negative components can be used to correct for various-18- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062measurement or calculation errors by interpolation therebetween. The x-axis 210 can define a frequency of x-component frequencies of the k-space or an image corresponding thereto. The y-axis 215 defines a magnitude of the frequency of y-component frequencies of the k-space or an image corresponding thereto. As depicted, FIG. 2A includes a relatively high concentration of low frequencies data (e.g., smooth shapes having high contrast therebetween) and a relatively low concentration of high frequency data (e.g., fine detail or rough / spiked patterns).

[0078] FIG. 2B depicts a series of radial lines disposed over the k-space, and can describe information sampled from the k-space. For example, a first radial line 220 samples a portion of the k space. A second radial line 225 samples a second portion of the k-space. The second radial line 225 can be offset from the first radial line a predefined distance, such as about 111.246°. A third radial line 230 and fourth radial line 235 can be offset a same or different distance. The angle can be selected to capture diverse components of the k-space data 205. For example, a golden angle can be selected to avoid duplicate or near-duplicate (e.g., with 1 degree, 0.1 degrees, or so on) readings when a total number of samples is undefined. In some embodiments, a number of radial lines can be predefined, and the relative angles therebetween can be selected to equally divide the k-space data 205. For example, for 180 pre-defined radial lines, the respective lines can be offset by 1°. The disclosure of one or more golden angle radial lines is not intended to be limiting. In some embodiments, one or more other methods (e.g., radial, spiral, linear, stack of stars, etc.) can be employed. The sampling can relate to points along the line selected for conversion to Euclidean space.

[0079] Referring to FIGs. 3A-3D, Euclidean space images of a feature 305 is depicted, according to a plurality of motion states. The feature is depicted as circular, merely for ease of depiction. One skilled in the art will understand that the feature can represent various anatomical organs, components, thereof, pathological features, and so on. For example, the feature can represent a liver or a lesion thereof. The motion can represent one or more motion signals 124 such as a cyclic motion signal (e.g., a cardiac cycle, or a respiratory cycle) or other time-variant behavior, such as swallowing, coughing, or the like.

[0080] At FIG. 3 A, the feature 305 is depicted as aligned with an arbitrary axis 310. At FIG. 3B, the feature 305 is depicted as having an offset relative to the same arbitrary axis 310. At FIG. 3C, the feature 305 is depicted as having an opposite offset relative to the same arbitrary axis 310 as FIG. 3B. FIG. 3D represents a combinatorial depiction of the feature 305-19- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062relative to the arbitrary axis 310. FIGs. 3A-3C can depict single images, or combinations of a plurality of images of the feature 305 defined by a classification component 108 of the data processing system 100. For example, FIG. 3A can be a high quality image (e.g., high contrast image or high resolution image) generated by a combination of a plurality of images, such as by the residual u-net network 110, the spatiotemporal patch embedding network 112, or the k-space image resolver 106 (e.g., to train the residual u-net network 110 or the spatiotemporal patch embedding network 112). In some embodiments, the feature 305 may deform between states, which may further differentiate the variance between the images, and cause further information loss in FIG. 3D, or an analogue thereof. For example, the feature 305 can be a pericardial sack, and can deform according to a cardiac cycle.

[0081] Referring to FIG. 4, a motion signal map 400 is depicted, according to some embodiments. The motion signal map 400 includes a respiratory signal 405 and a cardiac signal 410. The respiratory signal 405 can include an inhalation peak 415, an exhalation peak 420, an inhalation midpoint 425 and an exhalation midpoint 430. The respiratory signal 405 can be generally cyclic, such that various measurements can be aligned along an axis 406 from 0° to 360°. For example, a plurality of measurements can be disposed in a frequency domain. For example, the center points of the radial lines of FIG. 2 can be transformed to generate a sinusoidal wave, such as the sinusoid of the respiratory signal 405. The respiratory signal can be subdivided into a plurality of states. For example, a first state 435 can include the inhalation peak 415. A second state 440 can be predefined to include additional data disposed there-within. A third state 445 can include an inhalation midpoint 425 and the exhalation midpoint 430. A fourth state 450 can include the exhalation peak 420. In some embodiments, the states can be disposed vertically over the sinusoid or other signal describing the respiratory motion (e.g., by angle, such that the inhalation midpoint 425 and exhalation midpoint 430 can be different states).

[0082] The cardiac signal 410 can be disposed along another axis (not depicted). For example, the cardiac signal 410 can be one or more higher frequency signals generated from the k-space 205. For example, the cardiac signal 410 can be resolved from k-space data 205 in combination with the respiratory signal 405 or additional signals. In some embodiments, the motion signal may not be non-regularly periodic. For example, a function (e.g., swallowing) can be repeated and aligned to an axis. As depicted, the cardiac signal 410 can include variance to the sinusoidal signal, such as frequency or amplitude variances. The-20- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062variances can be included in the states, be defined as outliers, or can have additional states defined to capture the data thereof.

[0083] FIG. 5 depicts a flow diagram 500 for constructing a Euclidean space image from a plurality of Euclidean space input images 510, in accordance with one or more implementations. The flow diagram 500 can include a delivery of a plurality of input images 510 to a machine learning engine 505. The plurality of input images 510 can be disposed along a motion axis 515 (e.g., states of a motion signal 124). For example, each image can be associated with image acquisition data defining the position along an axis. The plurality of input images 510 can be associated with (e.g., can be generated responsive to k-state data 205 received by, or in response to a signal sent from the coil) various coils, along a coil axis 520. The output 525 of the machine learning engine 505 can contain a time-series of images, which can be generated from the input images 510. In some embodiments, each of the output images 525 can be derived from a single array of the input images 510. In some embodiments, the each of the output images 525 can be derived from multiple arrays of the input images 510 (e.g., based on the mixing between adjacent motion state images of the spatiotemporal patch embedding network 112).

[0084] FIG. 6 depicts an example representation 600 of a residual U-net network 110, in accordance with one or more implementations. The residual U-net network 110 receives an input image 605. For example, the input image 605 can be a 256-by-256 pixel image of a slice of a 3D image. A number of coils and time-series images can define a third dimension (sometimes referred to as a color dimension) of the input image 605 or set. In some embodiments, any of the dimensions of the images can be aligned to the residual U-net network 110 (e.g., by an alignment function performed by the residual U-net network 110 or another component of the data processing system 100). For example, the images can be adjusted centrally into the voxels of the residual U-net network 110 or aligned to a pre-defined edge thereof. In some embodiments, one or more padding datasets can be introduced, or the input image 605 can be otherwise resized to conform to a dimension of the U-net network 110.

[0085] The U-net network 110 includes a series of encoder blocks 610 and decoder blocks 615, and a series of lateral connections 625 therebetween. The encoder blocks 610 can pass pixel -wise data to the sequential encoder 610 or decoder 615 blocks; the lateral connections can pass information derived at various stages of the U-net network 110. For -21- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062example, edge information can be derived from the input images having a higher resolution than the filtered images at other operations. Put differently, the image-wide information can be gathered in upper stages (e.g., disposed relatively near the input 605 or the output 620). The encoder blocks 610 and decoder blocks 615 can be symmetrical. For example, the encoder blocks 610 can perform a maximum pooling function (e.g., having a 2x2 pool and a stride of 2); the decoder block can perform a symmetrical up-sampling function.

[0086] FIG. 7 depicts a spatiotemporal patch embedding network 112, in accordance with one or more implementations. The spatiotemporal patch embedding network 112 can receive an input image 605. For example, the input image can be a same resolution, padding, and content as the input image of FIG. 6. In some embodiments, additional dimensions can be defined relative to the input image. For example, the input image 605 can be received as a 5-dimensional image including Cartesian dimensions, a dimension of the movement signal 124, and a dimension for the coils. Although X-Y-Z coordinates are described with respect to FIG. 7, the method can be performed with various coordinates for Euclidean space (e.g., polar coordinates or 2D coordinates).

[0087] At a first stage 705 of the spatiotemporal patch embedding network 112, pixels, or other pools of at least the X and Y coordinate are combined. For example, a convolutional layer can include a pixel or pool (e.g., the 2x2 pool of the U-net network 110) that can pass between the x and y dimensions, such that local information or context of a volumetric feature of the input image (e.g., a tumor or other lesion) having features disposed across the x and y dimensions can train or provide information for the spatiotemporal patch embedding network 112, or a portion thereof (e.g., a portion for a local patch). In some embodiments, other dimensions can be selected, such as the x and z dimensions or polar dimensions. Indeed, for the various stages disclosed herein, any two-dimensional projection of the 5-dimensional data can be selected. For example, the methods herein can include a two-dimensional projection including the coil dimensions.

[0088] At a second stage 710 of the spatiotemporal patch embedding network 112, the x dimension is mixed between adjacent dimensions of the motion signal 124. In some embodiments, the motion signal can include a linear set of images. In some embodiments, the motion signal 124 can “wrap around” such that all motion signals are adjacent to two other periodic signals, which can increase a number of relationships between periodic or repeating motion signals. At a third stage 715 of the spatiotemporal patch embedding network 112, y-t -22- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062mixing is performed similarly to the x-y mixing of the second stage 710. The sequence of the stages disclosed herein is not intended to be limiting. Indeed, the sequences can be alternated or performed in various sequences (e.g., alternatively for separate embodiments, or sequentially in an embodiment). Advantageously, the mixing of the two-dimensional images can increase specificity of information to a patch of the 5D image or reduce memory bandwidth required to convolve across the data sets.

[0089] FIG. 8 depicts a method of generating images, according to some embodiments. The method 800 can be performed by one or more components or systems described in FIGs. 1, 9, or throughout this disclosure. In brief summary, at operation 805, k-space data is collected. At operation 810, a motion signal 124 is derived from the k-space data. The data is classified according to the motion signal 124 at operation 815. The k-space data is resolved into image-domain datasets at operation 820. Resolved images are upscaled at operation 825.

[0090] At operation 805, the data capture device 102 collects the k-space data. For example, the data capture device can include the RF and gradient coils of an MRI machine transmitting or receiving information in a k-space domain to measure a magnitude of energy released from a slice or volume of tissue upon demagnetization. The k-space data can be collected according to a Cartesian coordinate system, or another coordinate system such as a radial coordinate system which can be resolved to generate a corresponding Cartesian coordinate. For example, a series of radial images can be taken, wherein each radial image reduces a maximum radially defined un-sampled area of the k-space.

[0091] At operation 810, the motion signal extractor 104 extracts one or more motion signals 124 from the k-space data. For example, the motion signal extractor 104 can receive or calculate a pre-defined bound of a motion signal 124 such as a cardiac signal. The motion signal extractor can evaluate a center-point of the radial arm of operation 805 and resolve the information therein to a corresponding signal in the image space (e.g., may perform an inverse Fourier transform).

[0092] At operation 815, the classification component 108 classifies the k-space data according to the motion signal 124. For example, a time-series of images can be classified according to states of the motion signal 124. In some embodiments, the states can be dynamically generated to sort a number of k-space data sets for each state. In some -23- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062embodiments, state bounds can be predefined, such a where a signal is well defined, and a number or approximate number of k-space datasets can be predicted (e.g., a heart rate of a sedated patient).

[0093] At operation 820, the k-space data is resolved into Euclidean space data (e.g., an image). The k-space data can be resolved into images for each of a plurality of states of the motion signal 124 and coils. The images can be organized (e.g., sorted, tagged, or associated with metadata) according to associated coils or states of the motion signal 124. For example, the three-dimensional images can be disposed across a coil axis and a motion signal axis. The motion signal axis can also be referred to as a temporal axis or a motion axis, though the images thereof are not necessarily of a sequential temporal order. For example, for at least some motion signals 124, the beginning and ending of the motion signal 124 can be defined arbitrarily.

[0094] At operation 825, the resolved images are upscaled. For example, the images can be received by either or both of the Residual U-net network 110 or the Spatio-temporal patch embedding network 112. In some embodiments, the resolved images can be compared (e.g., pixel-wise, patch-wise, or image-wise) to select an image, or images associated with both systems can be presented. In some embodiments, one or both networks can be selected according to a nature of the image, image acquisition data 126 such as a reason for the image, a region of a body imaged, or other criteria, such as a manual selection. Upscaling can improve one or more components of image quality and does not require an increase in resolution, per se. Indeed, the methods and systems disclosed herein may, in at least some embodiments, improve an image quality based on a sharpness, contract, or any other characteristic that can be associated with a loss function.B. Example Embodiment I

[0095] Respiratory motion remains a challenge for torso MRIs, as can capture data related to an abdomen or chest of a subject. Among different solutions, motion-resolved imaging represents a powerful alternative to breath-hold or gating. Instead of avoiding or correcting for motion, motion-resolved imaging can capture motion using a continuous acquisition and sort the acquired data into motion frames (e.g., where each frame is free of motion artifacts). However, the long scan times have prevented clinical adoption in some implementations. The combination of radial k-space acquisition and compressed sensing to -24- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062reduce scan time can be demonstrated with motion-resolved imaging with clinically feasible scan times of 4-5 minutes. However, reconstruction times can be in the order of tens of minutes, due to the iterative nature of the reconstruction algorithm, as may inhibit adoption. A deep learning reconstruction technique called Movienet was developed to further increase acquisition speed and reduce image reconstruction times with respect to compressed sensing. Movienet can reduce scan times (e.g., to 1 minute) and reconstruction times (e.g., to 1 second). The autonavigation technique of Movienet to sort acquired data into motion states can include a combination of adaptive spatial and temporal filtering with principal component analysis (PCA). This method can use manual tuning of numerous parameters and can fail to detect respiratory motion in cases with limited diaphragm coverage or high-intensity regions in the image. According to the present disclosure, robust autonavigation techniques can extract a motion signal from the data for motion sorting purposes, as useful for clinical MRI of organs affected by motion, particularly in the abdomen and chest.|0096] A deep learning network can use auto-navigation golden-angle radial k-space acquisitions and extract a respiratory motion signal from the data. The network can be referred to as RANGR (Respiratory AutoNavigation for Golden-angle Radial MRI). RANGR can be used together with Movienet to provide a complete deep learning solution for motion-resolved MRI with fast acquisition, autonavigation and fast reconstruction.

[0097] A RANGR workflow can include an input to RANGR of the matrix of z-proj ections over time computed from the acquired golden-angle stack-of-stars data after taking an inverse Fourier transform along the central position (kx= ky= 0) and sum-of-squares coil combination (as depicted in Figure 9). The output of RANGR can include a ID respiratory motion signal of one motion amplitude per time point. Respiratory waveforms obtained via filtering and PCA can be used as a target signal. For testing cases, the respiratory motion signal estimated by RANGR can be used to sort the acquired k-space data into motion states and perform motion-resolved reconstruction with compressed sensing such as XD-GRASP or Movienet.

[0098] RANGR can employ a modified VGG-16 architecture, which is described with regard to Figure 10. The fully connected classification head of VGG-16 can be replaced by three convolutional layers (e.g., up to 4096 channels), preserving the final feature dimensionality of VGG-16. Max-pooling can be employed on the spatial dimension after each of the first five convolutional blocks, thus preserving the temporal dimension. Ablation tests -25- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062can be performed to simplify RANGR’s architecture to its present iteration by reducing the convolutional blocks of VGG-16 (e.g., to one layer per block, with RANGR ultimately containing six convolutional layers in total). The depicted example of RANGR takes as input a z-t matrix and outputs a ID time signal of the full temporal dimension (912 in Figures 9 and 10).

[0099] RANGR can be trained based on free-breathing torso MRI data previously acquired from subjects used for training. The respiratory motion signal detected using a conventional PCA technique can be used for training.

[0100] In vivo validation of RANGR was performed by acquiring new data using the same stack-of-stars pulse sequence on a 3T scanner (GE Healthcare). Acquisition parameters were TR = 4ms, TE = 2ms, flip angle = 12°, in-plane resolution = 1.5 mm, slice thickness = 5 mm, number of radial spokes = 900, and scan time = 2.25 min. Respiratory signals detected with RANGR and PCA (for comparison purposes) was employed to sort the acquired data into 10 undersampled motion states and Movienet reconstruction was employed to obtain final motion-resolved images. RANGR processing time was about 0.7 seconds and Movienet reconstruction time was about 1 second, which would aid implementation in a clinical setting.

[0101] RANGR can work in combination with Movienet to provide a deep learning solution for motion-resolved MRI with accelerated acquisition time and processing time in the order of seconds for various application including free-breathing autonavigated MRI of the abdomen and chest, MRI-guided adaptive radiotherapy of tumors affected by respiratory motion using an MR-linac system, pediatric MRI with reduced or without anesthesia, or MRI-guided adaptive radiotherapy of tumors affected by respiratory motion using an MR-linac system.C. Deep Learning Autonavigation of Free-Breathing MRI

[0102] FIG. 9 depicts flow diagrams for motion estimation. For example, the motion estimates can refer to Respiratory AutoNavigation for Golden Angle Radial MRI (RANGR) motion estimates. References to RANGR motion estimation are provided as illustrative examples and should not be construed as limiting. For example, aspects of the present disclosure referred to herein as RANGR examples can be substituted for further examples using different stack of star approaches for data acquisition (e.g., golden-angle radial or-26- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062pseudo-golden-angle radial sequences), or further data acquisition approaches, including propeller or spiral trajectories.

[0103] Referring more particularly to FIG. 9A, a flow diagram 900 for training a machine learning (ML) model 910 to generate motions signals 912 is provided. Data is provided according to a first pathway 902 for model-processed data (sometimes referred to as sample or input data) and a second pathway 904 for further data (sometimes referred to as target, annotated, expected, or ground truth data). Each of the first pathway 902 and the second pathway 904 can receive k-space data 906 as may be acquired according to, for example, the golden-angle stack of stars acquisition technique as described above. Such acquired k-space data 906 is sometimes referred to as “raw” k-space data 906 to distinguish from subsequently processed data, without limiting effect. For example, the “raw” k-space data 906 can incorporate certain processing incident to its acquisition. In the depicted flow diagram 900, an acquisition of k-space data 906 can refer to live acquisition of the k-space data 906 from a subject (e.g., in an MRI machine), an acquisition of previously captured k-space data 906 of a subject from a storage device (e.g., storage device 1716 or 1718 of FIG.17), or synthetic data approximating k-space data 906 for a subject.

[0104] In some examples, the subject may be not under anesthesia during acquisition by the MRI machine, such that respiratory cycles can reflect physiological patterns unaffected by sedative drugs. The subject may be of any age, such as a child (e.g., below the age of 18) or an adult (e.g., above the age of 18). In some examples, the subject can include a child; such a pediatric subject can exhibit varied physiological patterns relative to adult patients (e.g., faster or more irregular breathing patterns, even when anesthetized). Implementations of the present disclosure can identify motion signals for pediatric and adult subjects, either of which can be under anesthesia or un-anesthetized during acquisition by the MRI machine. The subject can be at risk of or diagnosed with cancer associated with the torso region (e.g., the chest and abdominal region). For example, the cancer can include one or more of lung cancer, esophageal cancer, stomach cancer, colorectal cancer, liver cancer, pancreatic cancer, small intestine cancer, kidney cancer, bladder cancer, prostate cancer, ovarian cancer, uterine cancer, or cervical cancer, such that the cancer can move along with respiratory excursions.

[0105] With continued reference to the first pathway 902, various data transforms 907 can be applied to the acquired k-space data 906. The data transforms 907 can include central k-space projections to extract a central portion of the k-space data 906 to exclude aberrant or -27- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062noisy high frequency data. Application of such a transform prior to at least some further transforms can aid the further transforms 907, by excluding noise or outlier data from the subsequent transform 907.

[0106] The data transforms 907 can include an inverse Fourier transform 907 to generate spatial data from the k-space data 906. The spatial data can be generated in any number of spatial dimensions, and a time dimension. Such data is sometimes referred to as a spatiotemporal dataset 908. Aspects of the present disclosure related to the extraction of motion signals related to respiratory function sometimes use two-dimensional data (e.g., a two-dimensional matrix) having a first spatial direction in a z-direction, along with a temporal direction. Such an approach can capture vertical respiratory motion of a recumbent subject (e.g., a subject laying in an MRI machine). The exclusion of lateral movements can reduce complexity and computational load for practicing the various techniques of the present disclosure.

[0107] Spatiotemporal datasets 908 including information from further dimensions are contemplated. For example, in some embodiments, the spatiotemporal dataset 908 can include two or three spatial dimensions, as may provide a motion signal better approximating ground truth of the subject. However, various illustrative examples of the present disclosure adopt a convention of a two-dimensional spatiotemporal datasets 908 to aid the clarity and brevity of the disclosure. The generation of the spatiotemporal datasets 908 can include exclusion of one or more spatial dimensions prior to, or subsequent to, performing the inverse Fourier transform. For example, cropping, filtering, or dimensionality reduction may be applied in k-space or image space.

[0108] For acquisition devices including multiple sensors, such as multi-coil MRI devices, the various sensors can be weighted for combination according to, for example, a sum-of-squares coil combination. The coils of the MRI machine can be used to the exclusion of other sensors (sometimes referred to as external sensors or breath sensors) to capture the motion signal 912, in some embodiments. Such an implementation can obviate temporal alignment between sensors, and can capture internal motion. For example, the motion signal 912 can be based on a location of feature of interest (e.g., a tumor or other pathological feature) which may not be detected by an external sensor, the external sensor referring to a sensor external to the MRI device. The acquired data can further be normalized, such as according to 0-1 normalization. Such acquisition can include normalization to a -28- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062predetermined or dynamic range, as may be greater than, equal to, or less than a received data range. In some embodiments, normalization can include quantization, according to a selected model (e.g., a model not configured to operate on floating point inputs). For example, the normalization can include 8- or 16-bit quantization for a corresponding model 910.|0109] The ML model 910 can be provided as any ML model 910 configured to generate an output signal. For example, some implementations of the present disclosure can use a convolutional neural network (CNN) to generate a signal output (e.g., according to a one-dimensional or hybrid convolution). For example, an CNN ML model 910 can include the modified VGG-16 architecture provided, in further detail, at FIG. 10, among others. Moreover, the ML model 910 can include further models, such as general adversarial networks or attention-based transformer networks. Some ML models 910 can include recurrent processing to generate the output signal, such as recurrent neural networks (RNN) or long short-term memory networks (LSTM). The output of the ML model 910 can be referred to as a motion signal 912. For example, the motion signal 912 can represent a vertical dimension represented in the spatiotemporal dataset 908 (e.g., the rising and falling excursion of an abdomen or diaphragm corresponding to respiratory function of a subject) across a time period (e.g., ranging between 30-90 minutes).

[0110] Referring now to the second pathway 904, further data is generated from the acquired k-space data 906 to operate as target data relative to the output of the first pathway 902 (e.g., the output of the ML model 910). A principal component analysis (PCA) technique 914 is used to generate the target data of the second pathway 904. Such a technique can use a combination of adaptive spatial and temporal filters, as may be tuned according to a manual tuning process to generate an output of an expected motion signal 916. In some cases, the various spatial or temporal filters can be applied or tuned on a per-subject basis, as may improve a fidelity of the expected motion signal 916 to a ground truth. Moreover, further techniques or input data can be used to improve a fidelity of the expected motion signal 916 to a subject ground truth.

[0111] A loss function metric 918 can estimate a loss function between the motion signal 912 and the subject ground truth according to a determination of a loss function between the motion signal 912 generated by the ML model 910 and an expected motion signal 916 generated according to the second pathway 904. For example, the loss function metric-29- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062918 can include mean squared error or other loss function to adjust the weightings of the ML model 910.

[0112] Referring more particularly to FIG. 9B, a flow diagram 950 for using a machine learning (ML) model 910 to generate motions signals 912 is provided. As described with regard to the first path 902 of FIG. 9A, a spatiotemporal dataset 908 is generated from k-space data 906 and ingested by an ML model 910. The ML model 910 generates a motion signal 912 used to classify the k-space data 906 according to a respiratory position, to generated sorted k-space data 920. The sorted k-space data 920 can be used to generate fast motion-resolved MRI reconstructed images 924 using space-time-coil convolutional networks 922, as described above with regard to section A of the present disclosure.

[0113] The respiratory positions for the sorted k-space data 920, used to generate the reconstructed images 924 can include one or more positions related to peak inhalation, peak exhalation, or mid phase respiration, as depicted in FIG. 4. Peak inhalation or exhalation can refer to a local minimum or local maximum of a respiratory cycle. For example, for a maximal breath respiratory cycle, the peak inhalation or peak exhalation can correspond to an inspiratory capacity of a subject. For tidal breathing, corresponding to reduced z-magnitude of the spatiotemporal dataset 908, peak inhalation or peak exhalation can correspond to a lower magnitude than the maximal breath. A quantity of the respiratory positions can be predetermined or dynamically adjusted. For example, five positions can be predetermined. Two of the positions can correspond to peak inhalation and peak exhalation. A remaining three of the positions can be allocated to mid phase respiration. For example, the respiratory cycle can be subdivided along a temporal or spatial range (e.g., z-axis magnitude). In some embodiments, a quantity of positions is determined based on a temporal period or spatial magnitude of the respiratory cycle. For example, each position can correspond to a centimeter of chest or abdominal excursion or a second of the breathing cycle.

[0114] FIG. 10 depicts an illustrative example of a modified VGG-16 architecture used to implement an ML model 910. The ML model 910 is configured to generate an output signal 912 from an input of time-series spatial data (e.g., the spatiotemporal datasets 908 of FIG. 9).

[0115] A first convolutional layer 1002 of the ML model 910 includes a two-dimensional convolution to generate feature maps. For example, the convolution can be -30- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062provided via a 3x3 kernel implementing a Rectified Linear Unit (ReLU) activation function. The first layer 1002 of the ML model 910 includes an input dimensionality of [ 1 , 1 , 400, 900], with an output dimensionality of [1, 64, 400, 900] according to the application of 64 filters, though various embodiments can include various resolutions or filters. For example, further color dimensionality can be included to provide further information of the k-space data 906. The second layer 1004 of the ML model 910 is provided as a max-pooling layer to downsample feature maps of the first layer. For example, according to the depicted example, the second layer 1004 flattens the dataset to a dimensionality of [1, 64, 133, 900],

[0011] Alternating convolutional and max-pooling layers include a third 1006, fourth 1008, fifth 1010, sixth 1012, seventh 1014, eighth 1016, ninth 1018, tenth 1020, and eleventh layer 1022 to continue to flatten the output. According to various embodiments, the quantity and dimensionality of the layers can vary according to a selected kernel and input dimensionality, among other parameters. A squeeze operation 1024 can generate a time-series signal of a motion signal 912 from the final convolutional layer (e.g., the eleventh layer 1022). For example, the motion signal 912 can depict a vertical torso excursion of the spatiotemporal datasets 908. In some embodiments, the output can include multiple spatial dimensions, to include lateral or radial dimensions, or separate ML models 910 can be used to generate output signals corresponding to various spatial dimensions over time.

[0117] FIG. 11 depicts spatial images 1102 generated according to the flow diagram 950 of FIG. 9B. Reference images 1104 are depicted as generated according to the PCA technique 914. For example, the motion signal 912 of FIG. 9B can be substituted for the expected motion signal 916 of FIG. 9A to generate the reference signals.

[0118] The spatial images 1102 include a first spatial image 1102A, second spatial image 1102B, and a third spatial image 1102C, as may be referred to collectively as the spatial images 1102. The reference images 1104 include a first reference image 1104 A, second reference image 1104B, and a third reference image 1104C, as may be referred to collectively as the reference images 1104. The spatial images 1102 can exhibit improved contrast, relative to the reference images 1104. Moreover, according to the techniques of the present disclosure, one or more computing systems of FIG. 17 can exhibit performant advantages for the generation of the spatial images 1102, relative to the reference images 1104. For example, the computing systems can generate the spatial images 1102 according to a near-real time image-31- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062(e.g., video frame), as may aid in providing an output of an low-latency image or a radiotherapy beam based on the output image.

[0119] Referring now to FIG. 12, a block diagram 1200 is depicted as including a data processing system 1205, in accordance with one or more implementations. The data processing system 1205 can include at least one data acquirer 1230. The data processing system 1205 can include at least one domain transformer 1235. The data processing system 1205 can include at least one model trainer 1240. The data processing system 1205 can include at least one model applier 1245. The data processing system 1205 can include at least one signal analyzer 1250. The data processing system 1205 can include at least one image reconstructor 1255. The data processing system 1205 can include at least one output handler 1260. The data processing system 1205 can include at least one ML model 910. The data processing system 1205 can include or interface with at least one database 1270.

[0120] The data acquirer 1230, domain transformer 1235, model trainer 1240, model applier 1245, signal analyzer 1250, image reconstructor 1255, output handler 1260, or ML model 910 can each include a processing unit or other logic device such as programmable logic array engine, or module configured to communicate with the database 1270. The data acquirer 1230, domain transformer 1235, model trainer 1240, model applier 1245, signal analyzer 1250, image reconstructor 1255, output handler 1260, or ML model 910 can be separate components, a single component, or part of the data processing system 1205. Moreover, the depicted data processing system 1205 can be separate from, coupled with, include, or be hosted by the data processing system 100 of FIG. 1, according to various embodiments of the present disclosure. The data processing system 1205 can include hardware elements, such as one or more processors, logic devices, or circuits. For example, the data processing system 1205 can include one or more components, structures or functionality of a computing device depicted in FIG. 17.

[0121] The various components of the data processing system 1205 are in network communication with one-another using a local or other network 1225. For example, such a network 1225 can include Ethernet, PCIe, or other data links. The network 1225 can communicatively couple with an MRI scanner 1210 to receive k-space data 906 or an administrative device 1220 to send or receive commands to control the operation of the data processing system, or to view generated images. The network 1225 can couple with further components, such as a device configured to generate or direct a radiation beam.-32- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062

[0122] The MRI scanner 1210 can be any device to acquire an MRI image of a volume of a subject. The MRI scanner 1210 may include one or more components to carry out MRI imaging of a volume within the subject. The components may include a magnet to generate a uniform static magnetic field, gradient coils for spatial encoding, and radiofrequency (RF) coils to transmit excitation pulses and receive emitted signals from nuclei within the volume of the subject. The anatomic MRI may be acquired in accordance with T1 -weight, T2-weight, proton density (PD) weight, fluid-attenuated inversion recovery (FLAIR), brain volume imaging (BRAVO), magnetization prepared rapid gradient echo (MPRAGE), and susceptibility weighted imaging (SWI), among others.

[0123] The database 1270 (e.g., the data repository 120 of FIG. 1) can include one or more local or distributed databases, and can include a database management system. The database 1270 can include computer data storage or memory and can store a plurality of medical images. The images can include any of the k-space data 906, spatial images, or spatiotemporal datasets 908 discussed herein, or information derived from, used for, or otherwise related to such images. The data processing system 100 can host, access, maintain, or otherwise provide the model 910 to other of the components (e.g., the model trainer 1240 and the model applier 1245).

[0124] FIG. 13 depicts a flow diagram 1300 for training a model to extract a motion signal from a k-space dataset 1310, in accordance with one or more implementations. Some aspects of the present flow diagram can parallel or otherwise relate to the flow diagram of FIG. 9 A. For example, the expected motion signal 916 be generated according to the second flow path 904 of FIG. 9A, and various components of the present figure can correspond to various of the operations or data depicted according to the first flow path 902 of the flow diagram 900 of FIG. 9 A.

[0125] The data acquirer 1230 is designed, constructed or operational to obtain k-space data 906 and an expected motion signal 916. The k-space data 906 and an expected motion signal 916 can correspond to a torso region of a subject in motion relative to an axis over a time period (e.g., at least one respiratory cycle). For example, the k-space data 906 and an expected motion signal 916 can be received paired in a training dataset 1305.

[0126] A first k-space dataset 1310A can refer to information generated based on an MRI of a subject at a first time, a second k-space dataset 1310B can refer to information -33- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062generated based on an MRI of the subject at a second time, and an / / th k-space datasets 1310 / / can refer to information generated based on an MRI of the subject at a third time. For example, the various datasets 1310 can be received according to a time-sequence of captures, such as according to a Cartesian-radial trajectory (e.g., stack of stars) or other acquisition approach. The training datasets 1305 can include motion signals 916 and k-space data 906 of k-space datasets 1310 corresponding to one or more MRI scans. While described herein primarily in context of stack of stars acquisition, the k-space datasets 906 or 1310 can be in accordance with other coordinate systems for MRI imaging, such as Cartesian sampling, radial sampling (e.g., golden-angle radial sampling), spiral sampling (E.g., Cartesian spiral (CASPR)), stack-of-stars (e.g., variable density stack-of-stars), or echo-planar imaging (EPI), among others.

[0127] The one or more MRI scans can include information for any quantity of scanned regions, patients or other subjects. For example, the various k-space datasets 1310 can include one or more sequences for any quantity of subjects, as may enhance a diversity of a training set. In some implementations, training datasets 1305 can be selected to provide a broad range of data, across various pathological or non-pathological subjects having various respiratory patterns. In some implementations, training datasets 1305 can be selected based on a subject characteristic to match one or more subjects of interest. For example, a training dataset 1305 of sedated or non-sedated patients can be selected to train an ML model 910 for one or more sedated or non-sedated patients; a training set of pediatric subjects can train an ML model 910 for a pediatric subject, and so forth.

[0128] The data acquirer 1230 can receive the k-space data 906 from the database 1270, as is depicted in the present figure. The data acquirer 1230 can receive one or more expected motion signals 916 generated in accordance with principal component analysis (PC A) using the respective spatiotemporal dataset 908 comprising respective projections of k-space data 906. For example, one or more of the example motion signals 916 can be generated according to the PC A technique 914, as described above with regard to FIG. 9 A. The expected motion signal 916 may identify a motion of a torso region of a respective subject relative to an axis (e.g., along plane formed by surface of table on which subject is situated) over a time period (e.g., ranging between 15-90 minutes). In some embodiments, precomputed or measured motion signals 916 can be retrieved from the database 1270. For example, various example datasets of the database can include pre-computed or measured motion signals 916, as may be determined according to PCA or other approaches. In some-34- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062embodiments, the data acquirer 1230 receives the expected motion signal 916 from a further component. For example, a further component can determine the expected motion signals 916 based on k-space data 906 retrieved from the database 1270 and convey such an expected motion signals 916 to the data acquirer 1230.|0129] In some embodiments, the k-space data 906 received by the data acquirer 1230 includes contemporary (e.g., near real-time) data. For example, the data acquirer 1230 can receive training sets of k-space data 906 from the MRI scanner 1210. However, for at least some computing systems, the generation of the expected motion signal 916 can incur substantial delay (e.g., the PCA technique 914 may be less performant than the generation of the motion signal 912 according to the depicted flow 1300). Accordingly, the data acquirer 1230 can be configured to receive contemporary data for a subject in the MRI scanner 1210, as depicted hereinafter with regard to FIG. 14, but receive pre-computed or measured training data from the database 1270.

[0130] The domain transformer 1235 may be designed, constructed or operational to generate or otherwise identify a spatiotemporal dataset 908. For example, the domain transformer 1235 can apply various data transforms 907 to generate a spatiotemporal dataset comprising various projections corresponding to a plurality of k-space datasets 1310 acquired in accordance with MRI. The various transforms can include an inverse Fourier transform 907 to generate a spatiotemporal dataset 908 (e.g., z-t matrix) based on the k-space data 1310. The domain transformer 1235 can perform various other spatial or spectral transforms 907 (e.g., prior or subsequent to the Fourier transform 907). Some illustrative examples of such transforms 907 are discussed with regard to FIG. 9.

[0131] The model trainer 1240 may be designed, constructed or operational to train (e.g., update) or otherwise establish the ML model 910 using the spatiotemporal dataset 908. For example, the model trainer 1240 can provide at least one example of a spatiotemporal dataset 908 as input to the ML model 910 to generate an output signal of a motion signal 912 identifying the respective motion of the respective torso region 1215 of the respective subject 1410 over the respective time period of the k-space data 906. A loss estimator 1242 of the model trainer 1240 can determine an error metric 918 based on a comparison between the motion signal 912 output by the ML model 910 and the expected motion signal 916. For example, the loss estimator 1242 can use an autoencoder to estimate reconstruction loss, or-35- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062apply regression to determine the loss metric 918 according to a mean squared error, mean absolute error, or other technique.

[0132] The model trainer 1240 can update one or more weights of the ML model 910 in accordance with the loss metric 918 (sometimes referred to as an error metric, without limiting effect). The updating the of the ML model 910 may in accordance with objective function. The objective function may define one or more rates at which the weights of the ML model 910 are to be updated. The objective function may be in accordance with stochastic gradient descent, and may include, for example, an adaptive moment estimation (Adam), implicit update (ISGD), and adaptive gradient algorithm (AdaGrad), among others. The updating of the parameters of the ML model 910 may be repeated until convergence. For example, the model trainer 1240 can iteratively update the weights using various spatiotemporal datasets 908. Further, the model trainer 1240 can implement regularization or induce dropout layers to prevent overfitting.

[0133] FIG. 14 depicts a flow diagram 1400 for a method of extracting a motion signal 912 from a k-space dataset 1420, in accordance with one or more implementations. The present figure includes a subject 1410 having a torso region 1415 operatively coupled with an MRI scanner 1210, such that the MRI scanner 1210 can generate k-space data 906 of at least one k-space dataset 1420. The data acquirer 1230 is communicatively coupled with the MRI scanner 1210 to receive k-space data 906 from the MRI scanner 1210. For example, the k-space data 906 can include information related to a motion signal 912 for a respiratory cycle of the subject 1410, embedded according to a vertical excursion of the torso region 1415 of the subject 1410. The torso region 1415 can correspond to at least a portion of a lung, esophagus, stomach, colorectal, liver, pancreas, small intestine, kidney, bladder, prostate, ovarian, uterine, or cervix, among others, in the subject 1410. The torso region 1415 subject 1410 may be scanned to evaluate for cancer and administration of therapies thereto.

[0134] As discussed above with regard to FIGS. 9 and 13, the data acquirer 1230 can receive k-space data 906 from the MRI scanner 1210. The k-space data 906 may be arrayed into one or more k-space datasets 1420, as may maintain a temporal sequence relative to each other. The data acquirer 1230 can receive the k-space data 906 including the sequence and any other information (e.g., a coil used to sense the k-space data 906, calibration parameters, or so forth). The data acquirer 1230 can provide any acquired information, including the k-space data sets 1420, to the domain transformer 1235. In turn, the domain transformer 1235-36- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062can identify a spatiotemporal dataset 908 comprising various projections corresponding to a plurality of k-space datasets 1310 acquired in accordance with MRI as discussed above with regard to, for example, FIG. 13 (e.g., the k-space datasets 1420 can correspond to a torso region 1415 of the subject 1410 in motion relative to an axis over a time period).|0135] To identify the spatiotemporal dataset 908, the domain transformer 1235 can, upon receipt of the various k-space datasets 1420 acquired in accordance with MRI, generate the spatiotemporal dataset 908 using the various k-space datasets 1420 in accordance with an inverse Fourier transform 907. For instance, the spatiotemporal dataset 908 may include signal intensity across slices forming the k-space datasets 1410 and time period over which the MRI scanning is performed to form the k-space datasets 1420. Each of the k-space datasets 1420 can include a set of signal measurements corresponding to a respective segment of the time period. For each signal measurement corresponding to the segment, the spatiotemporal dataset 908 can include a z-t matrix including an indication of torso excursions over time relating to respiratory cycle.|0136] The model applier 1245 may be designed, constructed or operational to generate a motion signal 912. For example, the model applier 1245 can provide the spatiotemporal dataset 908 as an input to a machine learning (ML) model established by the model trainer 1240 (e.g., as discussed with regard to FIG. 13). The model applier 1245 can generate the motion signal 912 based on providing the spatiotemporal dataset 908 to the ML model 910. The motion signal 912 can, in turn, identify the motion of the torso region 1415 of the subject 1410 relative to the axis (e.g., along a surface on which the subject 1410 situated while scanning) over the time period (e.g., 15-90 minutes). For example, the model applier 1245 can execute, access, or otherwise communicatively couple with the ML model 910 to generate the motion signal 912 using the spatiotemporal dataset 908 as ingested by the ML model 910. The model applier 1245 or other components of the data processing system can store an association between the subject 1410 and the motion signal 912 using one or more using one or more data structures. For example, the data structures can include a temporal association alone, or can be embedded in an image, such as a reconstructed image generated according to the operation of the data processing system 100 of FIG. 1. The data structures may be accessible to further components, as may retrieve such an output. For example, a signal analyzer 1250 can receive the stored association, or the image reconstructor 1255 can generate the stored association according to the execution of the flow of FIG. 15.-37- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062

[0137] FIG. 15 depicts a flow diagram 1500 for generating an output 1530 based on a motion signal, in accordance with one or more implementations. For example, the output 1530 can include a generation of an output image, or a position, engagement, power, or other aspect of a radiation beam 1540 for radiotherapy.|0138] The signal analyzer 1250 may be designed, constructed or operational to classify motion states. For example, the signal analyzer 1250 can generate the sorted k-space data 920 of FIG. 9B according to various techniques discussed with regard to the classification component 108 of the data processing system 100 of FIG. 1. To generate the sorted k-space data 920, the signal analyzer 1250 can compare a corresponding segment of the motion signal 912 with various amplitude ranges corresponding to various corresponding motion states (e.g., a vertical excursion associated with a respiratory cycle). The signal analyzer 1250 can compare the motion signal 912 for each of the various k-space datasets 1420 to classify each k-space dataset 1420 into a motion state in accordance with a comparison of the corresponding segment with the plurality of amplitude ranges.]0139] In some embodiments, the signal analyzer 1250 can sort each of the various k-space datasets 1420 by the motion state. For example, referring back to the illustrative example of a vertical excursion, the signal analyzer 1250 can sort the k-space dataset 1420 from a lowest vertical excursion (e.g., maximum exhalation) to a highest vertical excursion (e.g., maximum inhalation). Upon sorting the various k-space datasets 1420, the signal analyzer 1250 can group or otherwise classify the k-space datasets 1420 according to various criteria. For example, the signal analyzer 1250 can divide the k-space datasets 1420 according to a predetermined number of classes (e.g., five classes), or according to a predetermined displacement (e.g., a separate class for each centimeter of excursion). The signal analyzer 1250 can annotate the various k-space datasets 1420 with a class, or otherwise communication the classification to the image reconstructor 1255.

[0140] The image reconstructor 1255 may be designed, constructed or operational to generate spatial images. The image reconstructor 1255, like the other components of the data processing system 1205, can be or include various components of the data processing system 100 of FIG. 1. For example, the image reconstructor 1255 can execute a reconstruction model 1510 to apply an image reconstruction function on the various k-space datasets 1420 and the motion states thereof. The reconstruction model 1510 can include one or more of a residual u-net 110 or spatiotemporal patch embedding network 112. According to the application of -38- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062the image reconstruction technique, the image reconstructor 1255 can generate one or more images (e.g., image frames) of the torso region 1415 of the subject 1410 relative to the axis (e.g., the vertical axis, z) over the time period. For example, a performant image reconstructor 1255 can generate time-series images to provide a near-real time indication of a position of various features of a subject. For example, the near-real time indication can include a position of a tumor or other pathological feature. That is, each of the time-series images can comprise frames 1520 of a near-real time video, having a latency of less than two seconds (e.g., less than one second) from the acquisition of k-space datasets 1420 from the data acquirer 1230.

[0141] An output handler 1260 may be designed, constructed, or operational to generate outputs 1530 based on the one or more images generated by the image reconstructor 1255. For example, the output 1530 can include an output of the MRI images via an administrative device 1220. Such an image may be used to display the MRI image contemporaneously to its capture. Such a display may aid a further component (or a medical practitioner) to diagnose, locate, or indicate a presence of pathological tissue, such as a tumor. For example, by providing the output image contemporaneous to its capture, tissue can be manipulated, a position of the subject 1410 can be adjusted, or further diagnostic procedures conducted that may otherwise be challenging based on information latency of several seconds, minutes, or hours later.

[0142] The output handler 1260 can receive, from the administrative device 1220 or another component, an identification of pathological tissue (e.g., a tumor of the torso region) within various of the MRI image frames 1520 corresponding to the torso region 1415 of the subject 1410 in motion (e.g., undergoing respiratory motion). Such an identification can aid the output handler 1260 to cause administration of the radiotherapy 1540 to the tumor of the torso region of the subject, in accordance with the motion identified by the signal. The radiation beam can include a bean configured to effect stereotactic body radiation therapy (SBRT), intensity-modulated radiation therapy (IMRT), volumetric modulated arc therapy (VMAT), conformal radiation therapy (CRT), or proton beam therapy (PBT).

[0143] In some embodiments, the output handler 1260 can generate an output 1530 to indicate or actuate a position, engagement, power, or other aspect of the radiation beam for radiotherapy. For instance, the output 1530 may include delivery parameters for an radiotherapy administration device (e.g., linear accelerator (LINAC)) to emit or provide the radiation beam for the radiotherapy 1540. The delivery parameters may be generated by the -39- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062output handler 1260 based on an identification of the target (e.g., tumor associated with cancer) in the MRI image frames 1520 or based on the motion of the torso region 1415 of the subject 1410 relative to the axis over the time period, as identified by the motion signal 912. The delivery parameters may include, for example, beam energy, gantry angle, subject orientation, collimator angle, field size, and beam type, among others. The output handler 1260 can generate an output 1530 to control the radiation beam to induce a therapeutic effect in a torso region 1415 of the subject 1410. For example, the radiation beam can induce tumor cell apoptosis (sometimes referred to as a therapeutic beam). Using the output 1530, the operator of the radiotherapy administration device may carry out the delivery of the radiotherapy 1540 to the subject 1410.

[0144] In some embodiments, the output handler 1260 can interface with further components to provide further functionality. For example, in some embodiments, the output handler 1260 can determine a capture or processing latency, as may be useful to predict a current state of a respiratory cycle, as may correspond to a position of a subject (e.g., a position of a tumor or other pathological feature). The output handler 1260 can present such information using the administrative device, or use the information to control further outputs, including radiotherapy devices. For example, the output handler 1260 can determine a period of a respiratory cycle, the present state of the respiratory cycle represented in an output image, and a total latency. The output handler 1260 can predict a present position of a subject based on the determined information. For example, for a total latency of two seconds, the output handler can predict that a respiratory cycle will reach a local maximum or local maximum by summing a current position in the period with the latency. Such a prediction may be useful for any interaction (e.g., radiotherapy) or further observation of the subject.|0145] In this manner, the data processing system 1205 can use the ML model 910 extract motion signals 912 defining movement of a torso region 1415 of a subject 1410 through the duration of the scanning. From a computing perspective, the use of the ML model 910 may generate motion signals 912 from the raw k-space datasets 906, without the reliance on computationally complex and slow techniques, such as PCA. This can allow the data processing system 1205 to generate MRI image frames 1520 with little to no artefacts, in a smaller amount of time compared to such techniques. The use of the ML model 910 can thus reduce the consumption of computing resources such as processor and memory on the data processing system 1205, and free up these resources to execute other processes.-40- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062

[0146] In addition, from a clinical perspective, the use of the ML model 910 can eliminate having to instruct the subject to hold their breath while the MRI scanning is being performed. The ML model 910 can also reduce or eliminate the provision of anesthesia to the subject to place the subject under sedation. The fast and accurate generation of MRI image frames 1520 with little to no artefacts using the motion signal 912 can allow for adjustments of the provision of radiotherapy 1540 to more precisely and accurate target tumors associated with cancers within the subject 1410.

[0147] FIG. 16 depicts a flow diagram for a method of motion signal management for magnetic resonance imaging (MRI) data, in accordance with one or more implementations. The method 1600 may be implemented or performed by any of the components of the computing systems detailed herein, such as the data processing system 100 or the data processing system 1205. Under the method 1600, a computing system may identify a spatiotemporal dataset (1605). The computing system may provide the spatiotemporal model to a machine learning (ML) model (1610). Based on applying the ML architecture, the computing system may generate a signal identifying subject motion (1615). The computing system may generate an output, such as an image or a radiation beam (1620).D. Computing and Network Environment

[0148] Various operations described herein can be implemented on computer systems. FIG. 17 shows a simplified block diagram of a representative server system 1700, client computer system 1714, and network 1726 usable to implement certain embodiments of the present disclosure. In various embodiments, server system 1700 or similar systems can implement services or servers described herein or portions thereof. Client computer system 1714 or similar systems can implement clients described herein. The system 100 described herein can be similar to the server system 1700. Server system 1700 can have a modular design that incorporates a number of modules 1702 (e.g., blades in a blade server embodiment); while two modules 1702 are shown, any number can be provided. Each module 1702 can include processing unit(s) 1704 and local storage 1706.

[0149] Processing unit(s) 1704 can include a single processor, which can have one or more cores, or multiple processors. In some embodiments, processing unit(s) 1704 can include a general-purpose primary processor as well as one or more special-purpose co- -41- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062processors such as graphics processors, digital signal processors, or the like. In some embodiments, some or all processing units 1704 can be implemented using customized circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions that are stored on the circuit itself. In other embodiments, processing unit(s) 1704 can execute instructions stored in local storage 1706. Any type of processors in any combination can be included in processing unit(s) 1704.

[0150] Local storage 1706 can include volatile storage media (e.g., DRAM, SRAM, SDRAM, or the like) and / or non-volatile storage media (e.g., magnetic or optical disk, flash memory, or the like). Storage media incorporated in local storage 1706 can be fixed, removable or upgradeable as desired. Local storage 1706 can be physically or logically divided into various subunits such as a system memory, a read-only memory (ROM), and a permanent storage device. The system memory can be a read-and-write memory device or a volatile read-and-write memory, such as dynamic random-access memory. The system memory can store some or all of the instructions and data that processing unit(s) 1704 need at runtime. The ROM can store static data and instructions that are needed by processing unit(s) 1704. The permanent storage device can be a non-volatile read-and-write memory device that can store instructions and data even when module 1702 is powered down. The term “storage medium” as used herein includes any medium in which data can be stored indefinitely (subject to overwriting, electrical disturbance, power loss, or the like) and does not include carrier waves and transitory electronic signals propagating wirelessly or over wired connections.

[0151] In some embodiments, local storage 1706 can store one or more software programs to be executed by processing unit(s) 1704, such as an operating system and / or programs implementing various server functions such as functions of the system 100 of FIG.1 or any other system described herein, or any other server(s) associated with system 100 or any other system described herein.

[0152] Software” refers generally to sequences of instructions that, when executed by processing unit(s) 1704 cause server system 1700 (or portions thereof) to perform various operations, thus defining one or more specific machine embodiments that execute and perform the operations of the software programs. The instructions can be stored as firmware residing in read-only memory and / or program code stored in non-volatile storage media that -42- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062can be read into volatile working memory for execution by processing unit(s) 1704. Software can be implemented as a single program or a collection of separate programs or program modules that interact as desired. From local storage 1706 (or non-local storage described below), processing unit(s) 1704 can retrieve program instructions to execute and data to process in order to execute various operations described above.

[0153] In some server systems 1700, multiple modules 1702 can be interconnected via a bus or other interconnect 1708, forming a local area network that supports communication between modules 1702 and other components of server system 1700. Interconnect 1708 can be implemented using various technologies including server racks, hubs, routers, etc.

[0154] A wide area network (WAN) interface 1710 can provide data communication capability between the local area network (interconnect 1708) and the network 1726, such as the Internet. Technologies can be used, including wired (e.g., Ethernet, IEEE 802.3 standards) and / or wireless technologies (e.g., Wi-Fi, IEEE 802.11 standards).

[0155] In some embodiments, local storage 1706 is intended to provide working memory for processing unit(s) 1704, providing fast access to programs and / or data to be processed while reducing traffic on interconnect 1708. Storage for larger quantities of data can be provided on the local area network by one or more mass storage subsystems 1712 that can be connected to interconnect 1708. Mass storage subsystem 1712 can be based on magnetic, optical, semiconductor, or other data storage media. Direct attached storage, storage area networks, network-attached storage, and the like can be used. Any data stores or other collections of data described herein as being produced, consumed, or maintained by a service or server can be stored in mass storage subsystem 1712. In some embodiments, additional data storage resources may be accessible via WAN interface 1710 (potentially with increased latency).

[0156] Server system 1700 can operate in response to requests received via WAN interface 1710. For example, one of modules 1702 can implement a supervisory function and assign discrete tasks to other modules 1702 in response to received requests. Work allocation techniques can be used. As requests are processed, results can be returned to the requester via WAN interface 1710. Such operation can generally be automated. Further, in some embodiments, WAN interface 1710 can connect multiple server systems 1700 to each other,-43- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062providing scalable systems capable of managing high volumes of activity. Other techniques for managing server systems and server farms (collections of server systems that cooperate) can be used, including dynamic resource allocation and reallocation.

[0157] Server system 1700 can interact with various user-owned or user-operated devices via a wide-area network such as the Internet. An example of a user-operated device is shown in FIG. 17 as client computing system 1714. Client computing system 1714 can be implemented, for example, as a consumer device such as a smartphone, other mobile phone, tablet computer, wearable computing device (e.g., smart watch, eyeglasses), desktop computer, laptop computer, and so on.

[0158] For example, client computing system 1714 can communicate via WAN interface 1710. Client computing system 1714 can include computer components such as processing unit(s) 1716, storage device 1718, network interface 1720, user input device 1722, and user output device 1724. Client computing system 1714 can be a computing device implemented in a variety of form factors, such as a desktop computer, laptop computer, tablet computer, smartphone, other mobile computing device, wearable computing device, or the like.

[0159] Processor 1716 and storage device 1718 can be similar to processing unit(s) 1704 and local storage 1706 described above. Suitable devices can be selected based on the demands to be placed on client computing system 1714; for example, client computing system 1714 can be implemented as a “thin” client with limited processing capability or as a high-powered computing device. Client computing system 1714 can be provisioned with program code executable by processing unit(s) 1716 to enable various interactions with server system 1700.

[0160] Network interface 1720 can provide a connection to the network 1726, such as a wide area network (e.g., the Internet) to which WAN interface 1710 of server system 1700 is also connected. In various embodiments, network interface 1720 can include a wired interface (e.g., Ethernet) and / or a wireless interface implementing various RF data communication standards such as Wi-Fi, Bluetooth, or cellular data network standards (e.g., 3G, 4G, LTE, etc ).

[0161] User input device 1722 can include any device (or devices) via which a user can provide signals to client computing system 1714; client computing system 1714 can -44- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062interpret the signals as indicative of particular user requests or information. In various embodiments, user input device 1722 can include any or all of a keyboard, touch pad, touch screen, mouse or other pointing device, scroll wheel, click wheel, dial, button, switch, keypad, microphone, and so on.|0162] User output device 1724 can include any device via which client computing system 1714 can provide information to a user. For example, user output device 1724 can include a display-to-display images generated by or delivered to client computing system 1714. The display can incorporate various image generation technologies, e.g., a liquid crystal display (LCD), light-emitting diode (LED) including organic light-emitting diodes (OLED), projection system, cathode ray tube (CRT), or the like, together with supporting electronics (e.g., digital-to-analog or analog-to-digital converters, signal processors, or the like). Some embodiments can include a device such as a touchscreen that functions as both input and output device. In some embodiments, other user output devices 1724 can be provided in addition to or instead of a display. Examples include indicator lights, speakers, tactile “display” devices, printers, and so on.

[0163] Some embodiments include electronic components, such as microprocessors, storage and memory that store computer program instructions in a computer readable storage medium. Many of the features described in this specification can be implemented as processes that are specified as a set of program instructions encoded on a computer readable storage medium. When these program instructions are executed by one or more processing units, they cause the processing unit(s) to perform various operations indicated in the program instructions. Examples of program instructions or computer code include machine code, such as is produced by a compiler, and files including higher-level code that are executed by a computer, an electronic component, or a microprocessor using an interpreter. Through suitable programming, processing unit(s) 1704 and 1716 can provide various functionality for server system 1700 and client computing system 1714, including any of the functionality described herein as being performed by a server or client, or other functionality.

[0164] It will be appreciated that server system 1700 and client computing system 1714 are illustrative and that variations and modifications are possible. Computer systems used in connection with embodiments of the present disclosure can have other capabilities not specifically described here. Further, while server system 1700 and client computing system 1714 are described with reference to particular blocks, it is to be understood that these -45- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062blocks are defined for convenience of description and are not intended to imply a particular physical arrangement of component parts. For instance, different blocks can be but need not be located in the same facility, in the same server rack, or on the same motherboard. Further, the blocks need not correspond to physically distinct components. Blocks can be configured to perform various operations, e.g., by programming a processor or providing appropriate control circuitry, and various blocks might or might not be reconfigurable depending on how the initial configuration is obtained. Embodiments of the present disclosure can be realized in a variety of apparatus including electronic devices implemented using any combination of circuitry and software.E. Example Embodiment II

[0165] Motion-resolved 4D imaging (motion as the fourth dimension) is an important tool for radiation therapy of tumors affected by physiological motion, including respiratory motion, cardiac motion, peristaltic motion, swallowing motion and combinations. 4D imaging can be used to personalize treatment planning according to the motion range of the tumor and the surrounding organs with the goal to maximize radiation dose in the tumor and minimize radiation dose in the organs at risk. MRI presents several advantages over CT for 4D imaging, including superior soft tissue contrast to image organs at risk next to the tumor and the absence of ionizing radiation, which enables to scan for a longer time and obtain an improved characterization of motion. However, despite the extensive research on 4D MRI, there is no 4D MRI method currently available in clinical practice. One of the main reasons is the high data acquisition and computational burden to generate 4D MRI with sufficient spatial and temporal resolution.

[0166] eXtra-Dimensional Golden-angle RAdial Sparse Parallel MRI (XDGRASP) performs a continuous 3D golden-angle radial acquisition during motion and reconstructs a motion-resolved 4D image by sorting the acquired data into motion states and performing a temporal compressed sensing reconstruction. XD-GRASP enables 4D MRI with high spatial resolution (1-1.5 mm) and respiratory motion resolution (e.g., 10-20 states) within clinical acquisition times (5-6 min). However, the image reconstruction time can be long (10-30 minutes), which has prevented clinical implementation. There are several factors that increase reconstruction time, including (1) compressed sensing reconstruction can be iterative and requires several iterations to achieve high reconstruction performance, (2) data consistency in k-space can be performed in each iteration, which can involve computing two non-uniform -46- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062fast Fourier transform (NUFFT). Moreover, the NUFFT can be applied in each coil and motion state, which can increase computation time.

[0167] Deep learning using convolutional neural networks can be applied to reconstruct dynamic MRI data, such as for cardiac motion, which can include obtaining a fully-sampled reference for training. These networks can unroll the iterations of compressed sensing as layers of the network and since they are trained based on several datasets, the number of layers can be reduced with respect to the number of iterations. However, they still perform data consistency in each layer, in at least some embodiments, and thus reconstruction time is still significant, particularly for high-dimensional data sets such as 4D MRI.

[0168] The present example can include a convolutional neural network that exploits correlations in 5D image space (x, y, z, time, coil), without enforcing data consistency in k-space, to remove aliasing artifacts from accelerated data acquisition and reconstruct an unaliased motion-resolved 4D image (e.g., FIG. 5). The dimension “time” can refer to motion states. The ability to remove k-space data consistency can accelerate the image reconstruction process, offering shorter reconstruction times than unrolled-loop convolutional neural networks, which can use data consistency in each layer.

[0169] The present example can operate entirely in the image domain without enforcing data consistency in k-space to remove aliasing artifacts in the 5D input image and reconstruct a motion-resolved unaliased 4D image. Time in this example refers to the motion dimension.

[0170] The input 5D image (x-y-z -time-coil) can be obtained according to at least of methods and techniques of the XD-GRASP technique, such as before temporal compressed reconstruction to convert 3D k-space data into motion resolved 4D k-space data. The 3D k-space data can be, for example, stack-of-stars (radial ky-kx, Cartesian kz) or 3D radial (kooshball). The following steps can be performed: 1) the center of k-space in each radial line can be sorted as a matrix and principal component analysis can be used to estimate a respiratory motion signal, 2) amplitude binning can be used to divide the amplitude of the respiratory motion signal into motion states, and 3) 3D k-space data in each coil can be sorted into motion states, such as according to the amplitude of each radial line in the respiratory motion signal.-47- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062

[0171] The present example can be employed with at least a U-net with residual learning units, or convolutional space-time-coil patch embedding (e.g., the residual U-net network 110 or the spatiotemporal patch embedding network 112). Both versions can use iterative temporal compressed sensing results as a reference to train the network.|0172] The residual U-net network 110 can concatenate the coil and time dimensions to exploit spatial correlations along both the coil and time dimensions to separate image features from image artifacts. Skip connections (dashed lines of FIG. 6) can be used to transfer features at the different levels of down-sampling and up-sampling. K-space data consistency can be obviated which can reduce a number of computationally expensive NUFFT transforms in each time point and coil.

[0173] The network can use a smooth Li-loss function given by X / «, for In =with N representing the batch size, and In. defined as 0.5(x„ - „)2, if abs(x„ - jA) < 1, and abs(x„ - y ) - 0.5 otherwise, for a given output xnand target yn. This loss function can combine some advantages of the Li-loss function that may be less impacted by outliers of the L2-I0SS function according to some implementations.

[0174] Figure 7 shows space-time-coil patch embedding and mixing, where spatial features localized in small patches are mixed along the time and coil dimensions. Conceptually, the idea is related to applying compressed sensing or low-rank matrix completion on small patches rather than on the whole image. Some image features are localized, and they are shared among temporal points and coils. By operating on patches, fewer datasets may be required to train the network, since multiple patches can be extracted from a single dataset, according to some embodiments. Mixing can separate learning across or within patches, creating highly specialized, efficient networks that may employ fewer trainable parameters, according to some implementations. Combining 3D patch embedding with 2D cross-sectional mixing layers enables exploration of correlations in various planes. Additionally, mixing may reduce computational complexity of convolutional blocks, which may further minimize reconstruction time, according to some implementations.

[0175] This network can optimize a multi-space loss function that can span at least two landscapes: (1) a perceptual loss that reduces the semantic differences between the reconstruction and target volumes in abstract feature spaces, which can be implemented via forward propagation through and probing of a separate, pre-trained vision network, such as -48- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062VGG-16, and (2) a low-level quantitative pixelwise Li loss in raw image space that aims to correct any high-frequency artifacts produced by perceptual loss.

[0176] While the disclosure has been described with respect to specific embodiments, one skilled in the art will recognize that numerous modifications are possible. Embodiments of the disclosure can be realized using a variety of computer systems and communication technologies including but not limited to specific examples described herein. Embodiments of the present disclosure can be realized using any combination of dedicated components and / or programmable processors and / or other programmable devices. The various processes described herein can be implemented on the same processor or different processors in any combination. Where components are described as being configured to perform certain operations, such configuration can be accomplished, e.g., by designing electronic circuits to perform the operation, by programming programmable electronic circuits (such as microprocessors) to perform the operation, or any combination thereof. Further, while the embodiments described above may make reference to specific hardware and software components, those skilled in the art will appreciate that different combinations of hardware and / or software components may also be used and that particular operations described as being implemented in hardware might also be implemented in software or vice versa.

[0177] Computer programs incorporating various features of the present disclosure may be encoded and stored on various computer readable storage media; suitable media include magnetic disk or tape, optical storage media such as compact disk (CD) or DVD (digital versatile disk), flash memory, and other non-transitory media. Computer readable media encoded with the program code may be packaged with a compatible electronic device, or the program code may be provided separately from electronic devices (e.g., via Internet download or as a separately packaged computer-readable storage medium).

[0178] Thus, although the disclosure has been described with respect to specific embodiments, it will be appreciated that the disclosure is intended to cover all modifications and equivalents within the scope of the following claims.

[0179] Aspects can be combined, and it will be readily appreciated that features described in the context of one aspect can be combined with other aspects. Aspects can be implemented in any convenient form. For example, by appropriate computer programs, which may be carried on appropriate carrier media (computer readable media), which may be -49- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062tangible carrier media (e.g., disks) or intangible carrier media (e.g., communications signals). Aspects may also be implemented using a suitable apparatus, which can take the form of one or more programmable computers running computer programs arranged to implement the aspect. As used in the specification and in the claims, the singular form of 'a', 'an', and 'the' include plural referents unless the context clearly dictates otherwise. References to “or” may be construed as inclusive so that any terms described using “or” may indicate any of a single, more than one, and all of the described terms. In some instances, this disclosure recites “and / or” which can also refer to all of the described terms as well as a single instance of the described terms or another subset of the described terms, without limiting effect to other instances of “or.”4939-1144-9655.1

Claims

Atty. Dkt. No.: 115872-3062WHAT IS CLAIMED IS:

1. A method of generating motion signals from magnetic resonance imaging (MRI) data, comprising:identifying, by one or more processors, a spatiotemporal dataset comprising a plurality of projections corresponding to a plurality of k-space datasets acquired in accordance with MRI, the plurality of k-space datasets corresponding to a torso region of a subject in motion relative to an axis over a time period;providing, by the one or more processors, the spatiotemporal dataset as an input to a machine learning (ML) model, wherein the ML model is established using a plurality of examples, each of the plurality of examples comprising (i) a respective spatiotemporal dataset comprising a respective plurality of projections corresponding to a respective torso region of a respective subject in a respective motion relative to a respective axis over a respective time period and (ii) a respective signal identifying the respective motion of the respective torso region of the respective subject over the respective time period;generating, by the one or more processors, based on providing the spatiotemporal dataset to the ML model, a signal identifying the motion of the torso region of the subject relative to the axis over the time period; andstoring, by the one or more processors, using one or more data structures, an association between the subject and the signal.

2. The method of claim 1, further comprising:comparing, by the one or more processors, for each of the plurality of k-space datasets, a corresponding segment of the signal with a plurality of amplitude ranges corresponding to a plurality of motion states; andclassifying, by the one or more processors, each k-space dataset of the plurality of k-space datasets into a motion state of the plurality of motion states in accordance with comparison of the corresponding segment with the plurality of amplitude ranges.

3. The method of claim 2, further comprising:applying, by the one or more processors, an image reconstruction function on the plurality of k-space datasets and the motion state for each of the plurality of k-space datasets; and-51- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062generating, by the one or more processors, based on applying the image reconstruction function, a plurality of MRI image frames of the torso region of the subject relative to the axis over the time period.

4. The method of claim 2, wherein classifying each k-space of the plurality of k-space datasets further comprises sorting the plurality of k-space datasets by the motion state for each k-space of the plurality of k-space datasets.

5. The method of claim 1, further comprising:receiving, by the one or more processors, via an MRI machine, the plurality of k-space datasets, each k-space dataset of the plurality of k-space datasets comprising a set of signal measurements corresponding to a respective segment of the time period acquired in accordance with MRI,wherein identifying the spatiotemporal dataset further comprises generating the spatiotemporal dataset using the plurality of k-space datasets in accordance with an inverse Fourier transform.

6. The method of claim 1, further comprising:receiving, by the one or more processors, an identification of a tumor of the torso region, within a plurality of MRI image frames corresponding to the torso region of the subject in motion; andcausing, by the one or more processors, administration of radiotherapy to the tumor of the torso region of the subject, in accordance with the motion identified by the signal.

7. The method of claim 6, wherein the radiotherapy comprises at least one of comprises at least one of a stereotactic body radiation therapy (SBRT), an intensity-modulated radiation therapy (IMRT), a volumetric modulated arc therapy (VMAT), a conformal radiation therapy (CRT), or a proton beam therapy (PBT).

8. The method of claim 1, wherein the subject is not under anesthesia during acquisition by an MRI machine used to acquire the plurality of k-space datasets.-52- 4939-1144-9655.1Atty. Dkt. No.: 115872-30629. The method of claim 1, wherein the signal identifying the motion of the torso region is determined, by the ML model, using the plurality of k-space datasets without external sensor data.

10. The method of claim 1, wherein the respective signal of at least one of the plurality of examples is generated in accordance with principal component analysis (PCA) using the respective spatiotemporal dataset comprising the respective plurality of projections.

11. The method of claim 1, wherein the ML model is established by:providing the respective spatiotemporal dataset of at least one example of the plurality of examples as input to the ML model to generate an output signal identifying the respective motion of the respective torso region of the respective subject over the respective time period;determining an error metric based on a comparison between the output signal and the respective signal; andupdating one or more weights of the ML model in accordance with the error metric.

12. The method of claim 1, wherein the subject is not under anesthesia during acquisition by the MRI machine, wherein the subject is at risk of or diagnosed with cancer associated with the torso region, and wherein the cancer comprises at least one of lung cancer, esophageal cancer, stomach cancer, colorectal cancer, liver cancer, pancreatic cancer, small intestine cancer, kidney cancer, bladder cancer, prostate cancer, ovarian cancer, uterine cancer, or cervical cancer.

13. A system for motion signal management for magnetic resonance imaging (MRI) data, the system comprising one or more processors coupled with memory, configured to:identify a spatiotemporal dataset comprising a plurality of projections corresponding to a plurality of k-space datasets acquired in accordance with MRI, the plurality of k-space datasets corresponding to a torso region of a subject in motion relative to an axis over a time period;provide the spatiotemporal dataset as an input to a machine learning (ML) model, wherein the ML model is established using a plurality of examples, each of the plurality of examples comprising (i) a respective spatiotemporal dataset comprising a respective plurality of projections corresponding to a respective torso region of a respective subject in a respective-53- 4939-1144-9655.1Atty. Dkt. No.: 115872-3062motion relative to a respective axis over a respective time period and (ii) a respective signal identifying the respective motion of the respective torso region of the respective subject over the respective time period;generate, based on providing the spatiotemporal dataset to the ML model, a signal identifying the motion of the torso region of the subject relative to the axis over the time period; andstore, using one or more data structures, an association between the subject and the signal.

14. The system of claim 13, wherein the one or more processors are further configured to:compare, for each of the plurality of k-space datasets, a corresponding segment of the signal with a plurality of amplitude ranges corresponding to a plurality of motion states; and classify each k-space dataset of the plurality of k-space datasets into a motion state of the plurality of motion states in accordance with comparison of the corresponding segment with the plurality of amplitude ranges.

15. The system of claim 14, wherein the one or more processors are further configured to:apply an image reconstruction function on the plurality of k-space datasets and the motion state for each of the plurality of k-space datasets; andgenerate, based on applying the image reconstruction function, a plurality of MRI image frames of the torso region of the subject relative to the axis over the time period.

16. The system of claim 14, wherein classifying each k-space of the plurality of k-space datasets further comprises sorting the plurality of k-space datasets by the motion state for each k-space of the plurality of k-space datasets.

17. The system of claim 13, wherein the one or more processors are further configured to:receive, via an MRI machine, the plurality of k-space datasets, each k-space dataset of the plurality of k-space datasets comprising a set of signal measurements corresponding to a respective segment of the time period acquired in accordance with MRI,wherein identifying the spatiotemporal dataset further comprises generating the spatiotemporal dataset using the plurality of k-space datasets in accordance with an inverse Fourier transform.-54- 4939-1144-9655.1Atty. Dkt. No.: 115872-306218. The system of claim 13, wherein the one or more processors are further configured to:receive an identification of a tumor of the torso region, within a plurality of MRI image frames corresponding to the torso region of the subject in motion; andcause administration of radiotherapy to the tumor of the torso region of the subject, in accordance with the motion identified by the signal.

19. The system of claim 18, wherein the radiotherapy comprises at least one of comprises at least one of a stereotactic body radiation therapy (SBRT), an intensity-modulated radiation therapy (IMRT), a volumetric modulated arc therapy (VMAT), a conformal radiation therapy (CRT), or a proton beam therapy (PBT).

20. The system of claim 13, wherein the subject is not under anesthesia during acquisition by an MRI machine used to acquire the plurality of k-space datasets.

21. The method of claim 13, wherein the signal identifying the motion of the torso region is determined, by the ML model, using the plurality of k-space datasets without external sensor data.

22. The system of claim 13, wherein the respective signal of at least one of the plurality of examples is generated in accordance with principal component analysis (PCA) using the respective spatiotemporal dataset comprising the respective plurality of projections.

23. The system of claim 13, wherein, to establish the ML model, the one or more processors are further configured to:provide the respective spatiotemporal dataset of at least one example of the plurality of examples as input to the ML model to generate an output signal identifying the respective motion of the respective torso region of the respective subject over the respective time period;determine an error metric based on a comparison between the output signal and the respective signal; andupdating one or more weights of the ML model in accordance with the error metric.-55- 4939-1144-9655.1Atty. Dkt. No.: 115872-306224. The system of claim 13, wherein the subject is not under anesthesia during acquisition by the MRI machine, wherein the subject is at risk of or diagnosed with cancer associated with the torso region, and wherein the cancer comprises at least one of lung cancer, esophageal cancer, stomach cancer, colorectal cancer, liver cancer, pancreatic cancer, small intestine cancer, kidney cancer, bladder cancer, prostate cancer, ovarian cancer, uterine cancer, or cervical cancer.4939-1144-9655.1

Images