Reconstruction network and method for reconstructing movie MRI images

Abstract

A reconstruction network and method for reconstructing a movie MRI image are disclosed. A reconstruction network (12) for reconstructing a movie MRI image is described, which is a variational network for reconstructing an image from movie MRI data and comprising a cascaded architecture with cascaded modules (C1, C2, C3). The input of the first cascade module (C1, C2, C3) is an input stack (S) of a plurality of N frames (K1, K2, K3, K4), and the input of each subsequent cascade module (C1, C2, C3) is the input stack (S) and the output stack (P, P ') of the preceding cascade module (C1, C2). The reconstruction network (12) comprises a selection unit (U) designed to select, as a basis for the output data set, a single frame (F) corresponding to the i-th frame of the input stack (S) processed by the cascade modules (C1, C2, C3). Reconstruction methods, training methods, and related magnetic resonance imaging systems are also described.

Description

Technical Field

[0001] The present invention describes a reconstruction network and a reconstruction method for reconstructing movie MRI images, as well as a training method and an MRI system for training the reconstruction network. In particular, the present invention relates to low-latency deep learning-based reconstruction of interactive real-time (preferably cardiac) movie MRI. Background Art

[0002] Cine MRI (also called "cine sequence" or "cine imaging") is a type of MRI recording used to capture motion. Cine images are acquired by repeatedly imaging a region of interest over a certain period of time. Typically, a single slice is recorded, but 3D images can also be recorded.

[0003] Cine MRI is commonly used for cardiac imaging. Interactive cardiac MRI is often used for real-time diagnostic and interventional applications and places unique demands on reconstruction latency (ie, the time between acquisition and image display).

[0004] For example, for device navigation, a maximum delay of 200ms is considered tolerable. Achieving the desired spatial and temporal resolution usually requires high acceleration rates, which in turn require complex and time-consuming reconstruction techniques. Summary of the invention

[0005] The object of the present invention is to improve known systems and methods to facilitate the improvement of reconstruction of cine MRI images using trained (machine learning) reconstruction networks. In particular, the object of the present invention is to provide a solution for low-latency deep learning-based reconstruction of interactive real-time (preferably cardiac) cine MRI.

[0006] This object is achieved by a reconstruction network, a reconstruction method, a training method and an MRI system according to the present invention.

[0007] The (machine learning) reconstruction network for reconstructing movie MRI images according to the present invention is a variational network designed for reconstructing images from movie MRI data and comprises an architecture having a cascade of cascaded modules (particularly arranged in series). Each cascaded module is similar to an update step in a reconstruction based on iterative gradient descent. The input of the first cascaded module is an input stack of multiple N frames and the input of each subsequent cascaded module is the input stack and the output stack of the previous cascaded module. The reconstruction network (additionally) comprises a selection unit designed to select a single frame corresponding to the i-th frame of the input stack processed by the cascaded module as the basis for the output data set (frame or reconstructed image in k-space).

[0008] The general architecture of variational networks and suitable training processes are known in the art. The reconstruction network according to the present invention must be able to reconstruct movie MRI images, i.e. images from undersampled MRI data sets (k-space frames). The internal architecture of the reconstruction network is a cascade of cascade modules arranged in series (the input of each subsequent cascade module is the output stack of the previous cascade module). Therefore, each cascade module is similar to an update step in the reconstruction based on iterative gradient descent. The input of the first cascade module is an input stack of multiple N (k-space) frames.

[0009] Preferably, the number of cascaded modules arranged in series is between 2 and 20, in particular between 5 and 10. More cascaded modules may improve reconstruction quality, but may also increase reconstruction time. The inventors have found that increasing the model complexity beyond 6 to 8 cascaded modules does not significantly improve image quality.

[0010] The input to each cascaded module is preferably also a coil sensitivity map.The cascaded modules may have the same internal architecture, but with different weighting factors and / or calculation parameters.

[0011] A very important feature of the reconstruction network according to the invention is the selection unit, which is designed to select a single frame corresponding to the i-th frame of the input stack (as the output of the last cascade). This means that the selected frame is different from the input frame, as it has been processed by the cascade, but always has the same position with respect to the input stack (e.g. always the last frame).

[0012] This frame is then the basis for the output data set. The output data set may include the frame and / or the reconstructed image in k-space. Since the variational network can use the image information of the entire frame stack, showing a scene, e.g., a beating heart, the quality of a single frame is enhanced by the reconstruction network, and therefore the quality of the image reconstructed from the single frame is also enhanced.

[0013] The reconstruction method according to the present invention for reconstructing a cine MRI image using a reconstruction network according to the present invention comprises the following steps:

[0014] a) recording multiple MRI data frames from a region of interest (particularly from the heart) at different time points,

[0015] b) forming an input stack from a plurality of N MRI data frames recorded last in time,

[0016] c) inputting the input stack into the reconstruction network and reconstructing the image from one single frame selected by the selection unit of the reconstruction network (i.e., processed by the cascade modules of the reconstruction network and corresponding to the i-th frame of the input stack),

[0017] d) displaying the image,

[0018] e) In the presence of another recorded MRI data frame, steps b) to e) are repeated until a termination condition is reached.

[0019] It is known to record multiple MRI data frames from a region of interest at different time points. Typically (at least in cine MRI), these data sets are undersampled to record the data set fast enough to show a moving ROI (video). The reconstruction method is particularly advantageous for cardiac MRI.

[0020] The input stack is formed in a similar manner as described above for the training frames. In order to have a small delay, the input stack should be selected from the N MRI data frames that were last recorded in time. The number of frames of the input stack should be the same as the number of training frames, because the reconstruction network has been trained for this number.

[0021] This input stack can simply be fed into a (trained) reconstruction network, which will automatically reconstruct an image from the individual frames of the input stack, and this image can then be displayed.

[0022] To generate a video, a new stack can be generated for each repetition whenever there is another recorded MRI data frame, or the oldest frame can be deleted at one end and the newest frame added to the other end of the stack. With the new input frames, a new image can be reconstructed. Displaying the reconstructed image will produce a video. Since the reconstruction time is very short, the video shows the recorded ROI in almost real time.

[0023] The training method according to the invention is capable of training a (machine learning) reconstruction network according to the invention for reconstructing movie MRI images. The reconstruction network is preferably a deep machine learning network, and the training method is then a deep learning-based method for real-time reconstruction of (particularly cardiac) movie MRI. Its goal is to reduce reconstruction delays and enhance image quality, in particular to assess its feasibility for interactive real-time (IRT) MRI applications. The training method comprises the following steps:

[0024] a) recording a plurality of MRI data frames within a time period, wherein the MRI data frames show a region of interest at different time points,

[0025] b) selecting a subset of N temporally consecutive training frames from the MRI data frames as an input stack,

[0026] c) inputting a subset of the training frames into the reconstruction network and generating an output dataset based on one single frame selected by the selection unit of the reconstruction network (i.e., the frame processed by the cascade modules of the reconstruction network and corresponding to the i-th frame of the input stack),

[0027] d) calculating a loss based on the output dataset and updating the parameters of the reconstruction network based on the loss,

[0028] e) selecting another subset of N temporally consecutive training frames from the MRI data frames,

[0029] f) Repeat steps c) to f) until the termination condition is reached.

[0030] Variational networks are known and are used in practice to reconstruct images from MRI data frames. Since the technical field is cine MRI, these MRI data frames are data sets of MRI recordings acquired at certain points in time. Although a frame may include data from a 3D region, it is usually only data from a slice of a region of interest (ROI).

[0031] Thus, each MRI data frame can be imagined as k-space data that (when an image can be reconstructed from it) shows at least a slice of the ROI. Now, many frames are recorded over a period of time, wherein each frame can be imagined as representing a slice. Since the region of interest can change over time, not all frames necessarily represent the same position of a slice in the patient. However, they always represent a portion of the recorded region. For example, frames can "show" (they are in k-space) a beating heart, wherein each frame "shows" another state of the heartbeat. Frames can also follow the path of an instrument, which "shows" different slices of the patient, depending on where the tip of the instrument is located at a point in time. Thus, an MRI data frame can be imagined (when reconstructed) as a movie of a region of interest (e.g., a certain slice of the heart).

[0032] After recording the MRI data frames, a subset of N training frames is selected from these MRI data frames. The training frames must be temporally consecutive (i.e., a series of temporally adjacent data frames), but they do not necessarily include the latest or oldest MRI data frames. Typically, not all MRI data frames are extracted, but only a (small) part (N is preferably smaller than the total number of MRI data frames). For this step, it is not necessary to stop the recording of the MRI data frames.

[0033] These training frames are then input into the reconstruction network and an output dataset is generated. The output dataset may be an image reconstructed from a single frame selected by the selection unit (the loss is then calculated based on this image). The output dataset may also be the selected single training frame (the loss can then be calculated directly in k-space).

[0034] It should be noted that there is no need to reconstruct other images or select other single frames, as the goal of the training method is to obtain the best data for one single frame for each input stack.

[0035] Preferably, the relative position of the selected single frame remains the same for all sets of training frames during training. The selected single frame may correspond to the last of the N training frames, but may also correspond to the second to last or third to last. The position of the selected single frame may be selected based on the tolerated delay. When there are some frames following the selected single frame, the "future" state may be considered to enhance the accuracy of reconstruction (there are some frames in the past that "show" the selected training frame, and some frames in the future that "show" the selected training frame). On the other hand, when the last frame is selected as a single frame, the delay will be minimal. Since in typical applications, the delay may be about 200ms, it may be advantageous to select a single frame that is temporally located in the last 200ms of the N training frames. In short: by delaying the reconstruction by one or two frames, the network can incorporate future information, which further improves image quality at the expense of increased reconstruction delay.

[0036] Based on the output data set (the single frame or the image reconstructed from the single frame), the loss of the image is calculated, and the parameters of the reconstruction network are updated based on the loss (i.e., the loss is propagated back to the network). The construction of such losses in image space and k-space and the updating of parameters are known in the art and belong to the typical training process. However, in this training method, the loss is calculated (exclusively) based on a single frame (e.g., from the frame or the image reconstructed from the frame) rather than based on multiple frames.

[0037] As described above, a typical variational network includes several cascades of machine networks, for example, a convolutional network combined with a consistency module. The goal of training is to find the best set of parameters for these cascades, which can have the same architecture but different parameter values. Typically, the loss function must be minimized on a single image with respect to the parameters. The loss function typically defines the similarity between the reconstructed image and the reference image that should have good quality. For example, the data used to generate the training data can be MRI data of good quality recorded normally (recording time is not critical for training). The reference image can be reconstructed from this "good" data. The training data set can now be an undersampled set of this "good" data. Then, there is a clean, artifact-free reference image. The loss function can be a mean square error (MSE loss), SSIM loss (SSIM-Loss) (SSIM refers to "structural similarity index metric") or the so-called "⊥ loss", which is a symmetric loss function for magnetic resonance imaging reconstruction and image registration using deep learning.

[0038] Since the complete training data set is input into the variational network, the information of all frames is present in a single frame output by the variational network. It can be said that the frame is improved by the information of the previous frame and possible subsequent frames. Therefore, the quality of the selected single frame is better than the quality of the corresponding training frame of the input training data set. Therefore, the output data set is based on the features of a single frame selected from a subset of training frames, which means selecting a frame at a defined position of the stack of frames of the training data set. The corresponding single frame of the output stack is then selected as the output data set. Although the single frame corresponds to the input training frame (i.e., the same object is shown at the same time, such as the same stage of the heartbeat), it includes information of the entire stack of the input training data set.

[0039] Then, another subset of the N training frames is selected and training is continued. In particular, the MRI data set may be recorded over a (long) period of time and the oldest training frame deleted at the "past time end" of the N training frames is always replaced by the latest MRI data frame copied at the "future time end" of the N training frames.

[0040] It should be noted that in the case where the loss is calculated directly in k-space, the reconstruction network does not need to directly reconstruct images during training. There is no need to train the reconstruction module of the reconstruction network, since the known reconstruction module of the last image can be used. However, when the loss is calculated from the reconstructed image, the reconstruction module already exists.

[0041] In short, the reconstruction network is trained to output a video stream from an acquired MRI dataset. When a subset of N frames is selected from the recorded MRI data using a temporal sliding window, the reconstructed (single) image will show the moving scene. Due to the training, the reconstruction can be optimized even if the MRI dataset is undersampled (see further description).

[0042] In particular, this trained reconstruction network facilitates the reconstruction of interactive real-time (IRT) cardiac cine MRI. Its performance in terms of reconstruction time and quality is evaluated in a "retrospective simulation" of the described IRT scenario, which would require low reconstruction latency due to the trained network that can provide output in a very short time.

[0043] It should be noted that this "sliding window" is not absolutely necessary for training, since the reconstruction works on a single image.

[0044] The MRI system according to the invention comprises a reconstruction network according to the invention, which is trained in particular by means of a training method according to the invention.

[0045] It should be noted that the reconstruction network is preferably trained with a selection unit that is adapted to select a single frame that always has the same position (i-th) corresponding to the frames of the input stack. However, when video with more or less delay is desired, the position of the selected frame must be changed. Looking at these specially trained reconstruction networks, it is preferred that there are multiple reconstruction networks, all trained for different positions of the selected single frame. For this embodiment, the MRI system preferably comprises means for controlling the selection unit or for selecting the reconstruction network.

[0046] The control of the selection unit can be a control at the start of training. The position of a single frame can be selected. After the selection, the selected position is preferably no longer changeable for the reconstruction network (e.g. written to an EPROM).

[0047] A preferred embodiment of the MRI system comprises a plurality of reconstruction networks according to the present invention, the plurality of reconstruction networks having selection units designed to select single frames at different positions of a processed stack, wherein each reconstruction network comprises a selection unit for selecting a single frame at a separate constant position (unchangeable, always remaining the same) of the processed stack.

[0048] Preferably, the MRI system additionally comprises a user interface designed to enable a user to select between positions of selected individual frames, in particular by selecting between given positions or between given delay times.

[0049] Some of the above-mentioned units or modules of the present invention can be implemented in whole or in part as software modules running on a processor of a computing system. The implementation mainly in the form of software modules can have the following advantages: applications installed on existing computing systems can be updated with relatively little effort to install and run these units of the present application. The purpose of the present invention is also achieved by a computer program product, which has a computer program that can be directly loaded into the memory of a computing system and includes a program unit for executing the steps of the method when the program is executed by the computing system - at least these steps that can be executed by a computer. In addition to the computer program, this computer program product may also include other parts, such as documents and / or additional components, as well as hardware components such as hardware keys (dongles, etc.) to facilitate access to the software.

[0050] Computer readable media (e.g., memory sticks, hard disks, or other removable or permanently installed carriers) can be used to transmit and / or store executable portions of a computer program product so that they can be read from a processor unit of a computing system. The processor unit may include one or more microprocessors or their equivalents.

[0051] Particularly advantageous embodiments and features of the invention are given by the dependent claims, as revealed in the following description. Features of different claim categories may be combined as appropriate to give further embodiments not described herein.

[0052] According to a preferred reconstruction network, (in particular each) cascade module comprises a convolutional neural network (CNN) designed to create an output image from an input image, in particular a U-net or a V-net (in an arrangement where the image is reconstructed from a processed frame as an input image to the CNN). The images are processed by the convolutional neural network, in particular by using temporal and / or spatial convolutions, and are then converted into k-space frames. Temporal convolutions or combined temporal / spatial convolutions are advantageous because they enable the use of data redundancy in the frames. Since the input and output are in k-space, the architecture of the cascaded modules is such that the reconstruction module (k-space to image) is followed by a convolutional neural network, which in turn is followed by an inverse reconstruction module (image to k-space).

[0053] Preferably, (in particular each) cascade module additionally comprises a data consistency module operating in parallel with the convolutional neural network, wherein outputs of the consistency module and the convolutional neural network are combined, in particular together with the input.

[0054] The preferred reconstruction network comprises an image reconstruction module arranged to receive the selected single frame of the selection unit and to reconstruct an image based on the selected single frame.It should be noted that the selected single frame is a frame processed by the cascade modules of the reconstruction network.

[0055] The preferred reconstruction network comprises a loss module arranged to receive the selected single frame of the selection unit and a reference frame (e.g. a high quality frame of the scene shown by the selected single frame) and calculate a loss. The loss is based on a comparison between data according to the selected single frame and data according to the reference frame.

[0056] The preferred reconstruction network is specifically trained by the method described above. This means that the reconstruction network is trained according to the following steps:

[0057] a) recording a plurality of MRI data frames within a time period, wherein the MRI data frames show a region of interest at different time points,

[0058] b) selecting a subset of N temporally consecutive training frames from the MRI data frames as an input stack,

[0059] c) inputting a subset of the training frames into the reconstruction network and generating an output dataset based on one single frame (i.e., the one processed by the cascade and corresponding to the i-th frame of the input stack) selected by the selection unit of the reconstruction network,

[0060] d) calculating a loss based on the output dataset and updating the parameters of the reconstruction network based on the loss,

[0061] e) selecting another subset of N temporally consecutive training frames from the MRI data frames,

[0062] f) Repeat steps c) to f) until the termination condition is reached.

[0063] According to a preferred training method, MRI data frames are undersampled by a predefined factor F (e.g., 8 times), and a subset of training frames are selected from the undersampled MRI data frames. Since fast data acquisition is not critical for training, "complete" data can be acquired for the MRI data frames, and in order to generate training data, these "complete" data frames can be undersampled to simulate movie MRI data. This has the following advantages: the original non-undersampled MRI data frames (which show accurate images when reconstructed) can be used for supervised training. Preferably, a temporally varying pattern of undersampling is used for each frame. This is advantageous for simulating fluctuations in the acquired data.

[0064] According to a preferred training method, for a subset of N training frames, N is greater than 4 and / or less than 30, preferably, wherein N is selected between 7 and 14. This has shown a good trade-off between reconstruction accuracy and the time for the reconstruction.

[0065] According to a preferred training method, the time difference between consecutive MRI data frames is constant over the MRI data (i.e. the temporal resolution is constant) and is preferably less than 1 / 10 second, in particular less than 1 / 20 second. This allows for good temporal resolution. Alternatively or additionally, a predefined data sampling pattern allows combining data with different temporal resolutions (in particular radial or spiral golden angles).

[0066] According to a preferred training method, each selected single frame has the same relative position in the corresponding subset of the N training frames. This means that always, for example, the last frame is selected, or in case some "future" frame should optimize the quality of the selected frame, the i-th last frame is selected. Preferably, frames are selected from the temporally last third of the subset of N training frames, wherein preferably, there are no more than 5 frames temporally after the selected single frame, in particular, wherein the selected single frame temporally corresponds to the latest (last) frame in the subsample. As described above, by delaying the reconstruction by one or two frames, the network can incorporate future information, which further improves the image quality at the expense of an increased reconstruction delay.

[0067] According to a preferred training method, a first selected subset of training frames starts from a frame of the MRI data frame, and a subsequent subset of N training frames starts from the second frame of the corresponding previous subset of training frames. Thus, preferably, a subset of N training frames is selected by a temporal sliding window.

[0068] According to a preferred training method, the reconstruction unit comprises a reconstruction module designed to reconstruct an image from a selected single frame, wherein the loss is calculated based on a comparison between a reconstructed image of the MRI data frame and an image reconstructed from the corresponding selected single frame. As described above, the loss can be calculated in k-space, however, it may be advantageous to calculate the loss in image space. It should be noted here that the information of the training frame changes during propagation through the reconstruction network. Therefore, the data used to reconstruct the image is not the same as the training frame originally input, however, the data still represents the corresponding selected single frame and is based on the frame.

[0069] Preferably, the MSE loss, the SSIM loss or the so-called "⊥ loss" is calculated, wherein a combination of the SSIM loss and the ⊥ loss is preferred. The MSE loss (referring to mean squared error) and the SSIM loss (SSIM refers to "structural similarity index measure") are known in the art. In addition, a symmetric loss function ("⊥ loss") for magnetic resonance imaging reconstruction and image registration using deep learning is known.

[0070] In a preferred embodiment of the invention, the components of the reconstruction network are part of a data network, wherein preferably the data network and the MRI system providing the image data are in data communication with each other, wherein preferably the data network comprises parts of the Internet and / or a cloud-based computing system, wherein preferably the reconstruction network is implemented in the cloud-based computing system. For example, the components of the system are part of a data network, wherein preferably the data network and the medical imaging system providing the image data are in data communication with each other. Such a networking solution can be implemented via an Internet platform and / or a cloud-based computing system

[0071] The method may also include elements of "cloud computing". In the field of "cloud computing" technology, IT infrastructure, such as storage space or processing power and / or application software, is provided via a data network. Communication between the user and the "cloud" is achieved with the help of data interfaces and / or data transmission protocols.

[0072] In the context of "cloud computing", in a preferred embodiment of the method according to the invention, data are provided to a "cloud" via a data channel (e.g. a data network). The "cloud" comprises a (remote) computing system, e.g. a computer cluster which usually does not include a local machine of the user. The cloud can be provided in particular by a medical facility which also provides a medical imaging system. In particular, the image acquisition data are sent to a (remote) computer system ("cloud") via a RIS (Radiology Information System) or a PACS (Picture Archiving and Communication System).

[0073] The training method according to the invention enables training a neural network to reconstruct interactive real-time (especially cardiac) cine MRI, in particular optimizing in terms of latency and image quality. The trained reconstruction model according to the invention achieves good reconstruction results with little temporal ambiguity. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Other objects and features of the present invention will become apparent from the following detailed description considered in conjunction with the accompanying drawings. It should be understood, however, that the drawings are designed for purposes of illustration only and not as a definition of the limits of the present invention.

[0075] Figure 1 A simplified MRI system according to an embodiment of the present invention is shown.

[0076] Figure 2 The training process of the prior art is shown.

[0077] Figure 3 The reconstruction process of the prior art is shown.

[0078] Figure 4 The reconstruction process of the prior art is shown.

[0079] Figure 5 The training method and the reconstruction method according to the present invention are shown.

[0080] Figure 6 A reconstruction network according to the invention is shown.

[0081] In the drawings, like numbers refer to like objects throughout. Objects in the drawings are not necessarily drawn to scale. DETAILED DESCRIPTION

[0082] Figure 1 A schematic representation of a magnetic resonance imaging system 1 ("MRI system") is shown. The MRI system 1 comprises an actual magnetic resonance scanner (data acquisition unit) 2 with an examination space 3 or patient channel in which a patient or a test person is positioned on a drive couch 8 and in which the actual examination object is located within the body of the patient or the test person.

[0083] The magnetic resonance scanner 2 is usually equipped with a basic field magnet system 4, a gradient system 6 and an RF transmission antenna system 5 and an RF reception antenna system 7. In the exemplary embodiment shown, the RF transmission antenna system 5 is a whole-body coil permanently installed in the magnetic resonance scanner 2, whereas the RF reception antenna system 7 is formed as a local coil (here only symbolized by a single local coil) for arrangement on a patient or test subject. In principle, however, a whole-body coil can also be used as an RF reception antenna system and the local coils can each be switched into different operating modes.

[0084] The basic field magnet system 4 is designed in a conventional manner so that it generates a basic magnetic field in the longitudinal direction of the patient, i.e. along the longitudinal axis of the magnetic resonance scanner 2 running in the z direction. The gradient system 6 generally comprises individually controllable gradient coils in order to be able to switch (activate) the gradients in the x-direction, y-direction or z-direction independently of one another.

[0085] The MRI system 1 shown here is a full-body system with a patient channel into which the patient can be completely introduced. However, in principle, the invention can also be used in other MRI systems, such as C-shaped housings with lateral openings, and in smaller magnetic resonance scanners in which only one body part can be positioned.

[0086] Furthermore, the MRI system 1 has a central control device 13 for controlling the MRI system 1. The central control device 13 comprises a sequence control unit 14 for measuring sequence control. With the sequence control unit 14, a series of radio frequency pulses (RF pulses) and gradient pulses can be controlled according to a selected pulse sequence or respectively according to a series of multiple pulse sequences to acquire magnetic resonance images within a measuring session. For example, such a series of pulse sequences can be predetermined within a measuring or control protocol. Different control protocols for different measurements or measuring sessions are usually stored in a memory 19 and can be selected by an operator (and can be modified as required) and then used to realize the measurement.

[0087] In order to output the individual RF pulses of the pulse sequence, the central control device 13 has a radio frequency transmission device 15, which generates and amplifies the RF pulses and feeds them via a suitable interface (not shown in detail) to the RF transmission antenna system 5. In order to control the gradient coils of the gradient system 6, the control device 13 has a gradient system interface 16. The sequence control unit 14 communicates with the radio frequency transmission device 15 and the gradient system interface 16 in a suitable manner to transmit the pulse sequence.

[0088] Furthermore, the control device 13 has a radio frequency receiving device 17 (likewise communicating in a suitable manner with the sequence control unit 14 ) for acquiring magnetic resonance signals for the individual measurements (ie raw data) which are received in a coordinated manner from the RF receiving antenna system 7 within the scope of a pulse sequence.

[0089] The reconstruction unit 18 receives the acquired raw data and reconstructs magnetic resonance image data therefrom for the measurement. The reconstruction is usually performed based on parameters that can be specified in a corresponding measurement or control protocol. For example, the image data can then be stored in a memory 19 .

[0090] The reconstruction unit 18 comprises a trained reconstruction network 12, e.g. Figure 6 shown.

[0091] The central control device 13 can be operated via a terminal 10 having an input unit and a display unit 9, so that the operator can also operate the entire MRI system 1 via this terminal 10. MR images can also be displayed at the display unit 9, and measurements can be planned and started with the help of the input unit (which can be combined with the display unit 9), and in particular, suitable control protocols can be selected (and can be modified) using the series of suitable pulse sequences PS as explained above.

[0092] The MRI system 1 according to the invention and in particular the control device 13 can have a plurality of additional components which are not shown in detail but are usually present in such systems, such as a network interface in order to connect the entire system to a network and to be able to exchange raw data and / or image data or respectively parameter maps, and additional data (e.g. patient-related data or control protocols).

[0093] The manner in which suitable raw data are acquired by irradiation of RF pulses and generation of gradient fields and MR images are reconstructed from the raw data is known to a person skilled in the art and therefore does not need to be explained in detail herein.

[0094] Figure 2 The training process of the prior art is shown. First, a plurality of N MRI data frames K1, K2, K3, K4 are recorded from the region of interest at different time points. Then, the stack is input into the neural network N to be trained. The neural network N reconstructs N images B1, B2, B3, B4. For training, the loss is calculated for all reconstructed images B1, B2, B3, B4, and the neural network N is updated based on the loss L.

[0095] Figure 3 The reconstruction process of the prior art is shown. Here, Figure 2As shown, MRI data sets K1 , K2 , K3 , K4 are recorded as k-space data K1 , K2 , K3 , K4 , a stack is generated, and all images B1 , B2 , B3 , B4 are reconstructed from the stack.

[0096] Figure 4 The reconstruction process of the prior art is shown. Figure 3 In contrast, the MRI data sets K1 , K2 , K3 , K4 are recorded as k-space data K1 , K2 , K3 , K4 and an image B1 , B2 , B3 , B4 of each MRI data set K1 , K2 , K3 , K4 is reconstructed immediately after recording.

[0097] Figure 5 A training method and a reconstruction method according to the invention are shown. The difference between training and reconstruction is that during training a loss L is generated and the parameters of the reconstruction network are updated (dashed arrows).

[0098] First, a plurality of MRI data frames K1, K2, K3, K4 are recorded from the region of interest (particularly from the heart) at different time points. For training, stored data sets can be used, for clinical applications, at least when aiming to show "real-time" video, actual recorded frames should be used.

[0099] From these MRI data sets K1, K2, K3, K4, a stack of frames is formed. For training, this is the training stack T (the frames can be artificially undersampled), for application, this is the input stack S. The training stack T and the input stack S should include the same number of frames. With respect to "real-time" video, the input stack S should include the N MRI data frames K1, K2, K3, K4 that were recorded last in time. With respect to training, only temporally consecutive frames need to exist.

[0100] The stack is then input into a reconstruction network 12.

[0101] For application there is a reconstruction step where the image B3 is reconstructed from a single frame of the input stack S. For training this image B3 can also be reconstructed from a single frame of the training stack T (as shown in this example), however it is also possible to use pure k-space data without any reconstruction.

[0102] The process is then repeated. For the application, the reconstructed image B3 is displayed, wherein the user can see a "real time" video of the displayed image due to the multiple repetitions of the actual image.

[0103] For training (along the dashed branch), the loss is calculated and the parameters of the reconstruction network are updated.

[0104] Figure 6A reconstruction network 12 for reconstructing a movie MRI image according to the present invention is shown. The reconstruction network 12 is designed to reconstruct an image from movie MRI data and uses Figure 5 The variational network trained by the training method.

[0105] The reconstruction network comprises an architecture having (in this example) a cascade of three cascade modules C1, C2, C3 arranged in series, each cascade module being similar to an update step in a reconstruction based on iterative gradient descent. The three cascade modules C1, C2, C3 each produce outputs P, P', P'' from inputs T, P, P', and are followed by a selection unit U receiving the output P'' of the last cascade module C3 and selecting a single frame F and an image reconstruction module R, reconstructing an image B3 based on the selected single frame F (here always the third frame).

[0106] The inputs of the first cascade modules C1, C2, C3 are a subset of training frames T selected from the MRI datasets K1, K2, K3, K4 for training, and input data S from the MRI datasets K1, K2, K3, K4 for actual reconstruction work.

[0107] In this example, the inputs of each subsequent cascade module C1, C2, C3 are a subset of the training frame T (upper arrows), the outputs P, P' (middle arrows) of the previous cascade modules C1, C2, and the coil sensitivity maps (lower arrows). In this example, the cascade modules have the same internal architecture as shown at the bottom, but with different weighting factors and / or calculation parameters.

[0108] As shown, each cascade module C1, C2, C3 includes a convolutional neural network CNN, in particular a U-net or a V-net, designed to create an output image from an input image. The convolutional neural network CNN obtains its input data from a module (which may be a reconstruction module R), which generates an image from k-space. Its output is transmitted back to k-space again by another module. In addition, each cascade module C1, C2, C3 includes a data consistency module DC working in parallel with the convolutional neural network CNN, wherein the outputs of the data consistency module DC and the convolutional neural network CNN are combined, in particular combined together with the input.

[0109] have Figure 6 Network and Figure 5The actual approach of the method in may be as follows: a fully sampled segmented cine acquisition from the OCMR dataset (multi-coil k-space data from a fully sampled cardiac cine series) was retrospectively undersampled by a factor of eight to construct the training dataset. To simulate interactive real-time MRI, a "sliding window" approach of the method was chosen, where a subset of eleven consecutive frames from one slice are reconstructed at a time. The loss is then calculated on the i-th frame (e.g. the last frame or the third to last frame) and propagated back into the network for optimization, where N is the size of the sliding window (here 11) and i∈[0,N[. Not selecting the last frame increases latency but allows the network to not only rely on past information for reconstruction.

[0110] Although the present invention has been disclosed in the form of preferred embodiments and variations thereof, it will be appreciated that many additional modifications and variations may be made thereto without departing from the scope of the present invention. For the sake of clarity, it will be appreciated that the use of "a" or "an" in this application does not exclude a plurality, and "comprising" does not exclude other steps or elements. The expression "plurality" means "at least one". Reference to a "unit" or "device" does not exclude the use of more than one unit or device. The expression "plurality" must be understood as "at least one". Independent of the usage of grammatical terms, individuals with various gender identities, male and female, are included in the term.

Claims

1. A reconstruction network (12) for reconstructing cine MRI images, the reconstruction network (12) being a variational network designed for reconstructing images from cine MRI data and comprising an architecture having a cascade of cascaded modules (C1, C2, C3), wherein: The input of a first cascade module (C1, C2, C3) is an input stack (S) of a plurality of N frames (K1, K2, K3, K4) and the input of each subsequent cascade module (C1, C2, C3) is the input stack (S) and an output stack (P, P') of the preceding cascade module (C1, C2), wherein the reconstruction network (12) comprises a selection unit (U) designed to select a single frame (F) corresponding to the i-th frame of the input stack (S) processed by the cascade module (C1, C2, C3) as the basis for an output data set.

2. The reconstruction network according to claim 1, wherein: Each cascaded module (C1, C2, C3) comprises a convolutional neural network (CNN), in particular a U-net or a V-net, designed to create output images from input images, these images being processed by said convolutional neural network (CNN), in particular by using temporal and / or spatial convolutions, and then converted into k-space frames, Preferably, each cascade module (C1, C2, C3) additionally comprises a data consistency module (DC) working in parallel with the convolutional neural network (CNN), wherein the outputs of the data consistency module (DC) and the convolutional neural network (CNN) are combined, in particular together with the input.

3. The reconstruction network according to claim 1 or 2, comprising: An image reconstruction module (R), which is arranged to receive the selected single frame (F) of the selection unit (U) and reconstruct the image (B3) based on the selected single frame (F); and / or a loss module, which is arranged to receive the selected single frame (F) of the selection unit (U) and a reference frame and calculate the loss (L) based on the comparison between the data according to the selected single frame (F) and the data according to the reference frame.

4. The reconstruction network according to one of the preceding claims, trained according to the following steps: a) recording a plurality of MRI data frames (K1, K2, K3, K4) within a time period, wherein: The MRI data frames (K1, K2, K3, K4) show regions of interest at different time points. b) selecting a subset of N temporally consecutive training frames (T) from said MRI data frames (K1, K2, K3, K4) as an input stack (S), c) inputting a subset of the training frames (T) into the reconstruction network (12) and generating an output data set (B3) based on one single frame (F) selected by a selection unit (U) of the reconstruction network, d) calculating the loss (L) based on the output data set (B3) and updating the parameters of the reconstruction network (12) based on the loss (L), e) selecting another subset of N temporally consecutive training frames (T) from said MRI data frames (K1, K2, K3, K4), f) Repeat steps c) to f) until a termination condition is reached.

5. A reconstruction method for reconstructing cine MRI images using a reconstruction network (12) according to one of the preceding claims, comprising the following steps: a) recording a plurality of MRI data frames (K1, K2, K3, K4) from a region of interest, in particular from the heart, at different time points, b) forming an input stack (S) from a plurality of N MRI data frames (K1, K2, K3, K4) recorded last in time, c) inputting the input stack (S) into the reconstruction network (12) and reconstructing an image (B3) from one single frame (F) selected by a selection unit (U) of the reconstruction network (12), d) displaying said image (B3), e) If there is another recorded MRI data frame (K1, K2, K3, K4), repeating steps b) to e) until a termination condition is reached.

6. A training method for training a reconstruction network according to one of claims 1 to 4, comprising the following steps: a) recording a plurality of MRI data frames (K1, K2, K3, K4) within a time period, wherein the MRI data frames (K1, K2, K3, K4) show regions of interest at different time points, b) selecting a subset of N temporally consecutive training frames (T) from said MRI data frames (K1, K2, K3, K4) as an input stack (S), c) inputting a subset of the training frames (T) into the reconstruction network (12) and generating an output data set (B3) based on one single frame (F) selected by a selection unit (U) of the reconstruction network (12), d) calculating the loss (L) based on the output data set (B3) and updating the parameters of the reconstruction network (12) based on the loss (L), e) selecting another subset of N temporally consecutive training frames (T) from said MRI data frames (K1, K2, K3, K4), f) Repeat steps c) to f) until a termination condition is reached.

7. The training method according to claim 6, wherein: The MRI data frames (K1, K2, K3, K4) are processed to be undersampled by a predefined factor F, and a subset of the training frames (T) are selected from the undersampled MRI data frames (K1, K2, K3, K4), preferably wherein a temporally varying undersampling pattern is used for each frame.

8. The training method according to claim 6 or 7, wherein: For the subset of N training frames, N is greater than 4 and / or less than 30, preferably, wherein N is selected between 7 and 14.

9. The training method according to one of claims 6 to 8, wherein: Each selected single frame has the same relative position in the corresponding subset of training frames (T), preferably, wherein each selected single frame is a training frame from the temporally last third of the subset of training frames (T), preferably, wherein there are no more than 5 training frames temporally after the training frame corresponding to the selected single frame (F), in particular, wherein the selected single frame (F) corresponds to the latest training frame in the subsample.

10. The training method according to one of claims 6 to 9, wherein: A first selected subset of training frames (T) starts from a frame of said MRI data frames (K1, K2, K3, K4) and subsequent subsets of training frames (T) start from the second frame of the respective previous subset of training frames (T).

11. The training method according to one of claims 6 to 10, wherein: The reconstruction unit (12) comprises a reconstruction module (R) which is designed to reconstruct an image (B3) from a selected single frame (F), wherein the loss (L) is calculated based on a comparison between the reconstructed image of the MRI data frame (K1, K2, K3, K4) and the image (B3) reconstructed from the corresponding single frame (F).

12. An MRI system (1) comprising a reconstruction network (12) according to one of claims 1 to 4, the reconstruction network (12) being trained in particular by means of a training method according to one of claims 6 to 11.

13. An MRI system according to claim 12, comprising a plurality of reconstruction networks (12) according to one of claims 1 to 4, having a selection unit (U) designed to select individual frames (F) at different positions of the processed stack (P"), wherein Each reconstruction network (12) comprises a selection unit (U) for selecting a single frame (F) at a single constant position of the processed stack (P"), Preferably, the MRI system (1) additionally comprises a user interface (11) designed to enable a user to select between positions of selected single frames (F), in particular by selecting between given positions or between given delay times.

14. A computer program product, comprising instructions, which, when a computer executes the program, cause the computer to perform the steps of the training method according to any one of claims 6 to 11 or the reconstruction method according to claim 5.

15. A computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to perform the steps of the training method of any one of claims 6 to 11 or the reconstruction method of claim 5.

Images