Method and system for multi-plane recombination with machine learning-based image enhancement

Abstract

The invention relates to multi-plane recombination of three-dimensional medical images. In particular, the present invention provides a method for recombining a sequence of images, comprising: determining a landmark plane intersecting a volume; acquiring an image sequence; recombining the image sequence along the landmark plane to generate a first recombined image sequence; perturbing the landmark plane to produce a perturbed landmark plane; recombining the first recombined image sequence along the disturbed landmark plane to generate a second recombined image sequence; mapping the second recombined image sequence, the image sequence and the landmark plane to a resolution-enhanced image sequence by using a trained image enhancement network; and displaying, via a display device, the sequence of resolution enhanced images. The present disclosure provides a method that can reduce image artifacts in retrospectively recombined image sequences, particularly in the case of retrospectively recombining medium-resolution image sequences or low-resolution image sequences, without relying on acquiring high-resolution 3D images.

Description

Technical Field

[0001] Embodiments of the subject matter disclosed herein relate to medical imaging, and more particularly, to multi-planar reformatting of three-dimensional medical images. Background Art

[0002] Non-invasive imaging techniques allow images of the internal structures and / or features of a subject to be acquired without the need to perform invasive surgery on the subject. In particular, non-invasive imaging techniques rely on various physical principles to acquire data and construct images or otherwise represent the observed internal structures and / or features of the subject. Such physical principles include differential transmission of x-rays through a target volume, reflection of sound waves within a volume, paramagnetic properties of different tissues and materials within a volume, decomposition of targeted radionuclides within the subject, etc.

[0003] Data acquired via non-invasive imaging at an anatomical region of interest (ROI) can be reconstructed into an image or image sequence. For example, multi-planar reconstruction or reconstructing (MPR) can be used to convert a first 2D image sequence parallel to a first plane into a second 2D image sequence parallel to a second plane, wherein the first plane is not parallel to the second plane. MPR can be used to reconstruct a previously acquired image sequence to capture a view of an anatomical region of interest at a standard / predetermined orientation that was not originally captured in the first image sequence. This process of reconstructing a previously acquired image sequence is called retrospective MPR. One limitation of retrospective MPR is that the reconstructing of an image sequence that is not of high enough resolution may result in artifacts, such as image blurring or step edges.

[0004] A conventional solution to this problem is to impose constraints on the data resolution requirements (e.g., 0.25 mm isotropic) before using the reconstruction process. However, this may limit the practicality of such applications in clinical practice because it increases the scan time and may be disturbed by patient motion or contrast changes in the data. Even if high-resolution image sequences are utilized, the reconstruction process may introduce blurring due to interpolation, which may affect the quality of the reconstructed image. Therefore, it is generally desirable to explore methods for reducing image artifacts in retrospectively reconstructed image sequences, particularly in the case of retrospective reconstruction of moderate-resolution or low-resolution image sequences. Summary of the invention

[0005] In one embodiment, a method for recombining an image sequence includes: determining a landmark plane that intersects a volume; acquiring an image sequence; recombining the image sequence along the landmark plane to produce a first recombined image sequence; perturbing the landmark plane to produce a perturbed landmark plane; recombining the first recombined image sequence along the perturbed landmark plane to produce a second recombined image sequence; mapping the second recombined image sequence, the image sequence, and the landmark plane to a resolution enhanced image sequence using a trained image enhancement network; and displaying the resolution enhanced image sequence via a display device. In this manner, the image quality of the retrospectively recombined images can be improved without relying on acquiring high-resolution 3D images. In addition, the inventors herein have determined that there is a relationship between the orientation of the retrospectively recombined image sequence relative to the original acquired image sequence (e.g., the orientation of the landmark plane relative to the orientation of an image including the original acquired image sequence) and the presence of artifacts in the recombined image sequence. Therefore, by informing the image enhancement network of the relative orientation of the recombined image sequence relative to the original acquired image sequence (e.g., by feeding a landscape plane to the image enhancement network), the degree of artifact attenuation in the resolution enhanced image sequence can be increased.

[0006] It should be understood that the above brief description is provided to introduce in a simplified form selected concepts that are further described in the detailed description. It is not meant to identify key or essential features of the claimed subject matter, the scope of which is solely defined by the claims that follow the detailed description. Furthermore, the claimed subject matter is not limited to implementations that solve any disadvantages noted above or in any part of this disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The present disclosure will be better understood by reading the following description of non-limiting embodiments with reference to the accompanying drawings, in which:

[0008] Figure 1 is a block diagram of an MRI system according to an embodiment of the present disclosure.

[0009] Figure 2 is a block diagram of an imaging system according to an embodiment of the present disclosure.

[0010] Figure 3 is a flowchart of a first method for generating training data for an image enhancement network according to an embodiment of the present disclosure.

[0011] Figure 4 is a diagram illustrating a first method for generating training data for an image enhancement network according to an embodiment of the present disclosure.

[0012] Figure 5is a flowchart of a second method for generating training data for an image enhancement network according to an embodiment of the present disclosure.

[0013] Figure 6 is a diagram illustrating a second method for generating training data for an image enhancement network according to an embodiment of the present disclosure.

[0014] Figure 7 is a flow chart of one embodiment of a method for training an image enhancement network using training data generated according to the methods disclosed herein.

[0015] Figure 8 is a diagram illustrating a method of training an image enhancement network according to an embodiment of the present disclosure.

[0016] Fig. 9 is a flow chart of a first embodiment of a method for performing retrospective reconstruction of an image sequence using a trained image enhancement network according to an embodiment of the present disclosure.

[0017] Fig.10 is a diagram illustrating a first embodiment of a method for performing retrospective reconstruction of an image sequence using a trained image enhancement network according to an embodiment of the present disclosure.

[0018] Fig.11 is a flow chart of a second embodiment of a method for performing retrospective reconstruction of an image sequence using a trained image enhancement network according to an embodiment of the present disclosure.

[0019] Fig.12 is a diagram illustrating a second embodiment of a method for performing retrospective reconstruction of an image sequence using a trained image enhancement network according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0020] The following description relates to various methods and systems for recombining an image sequence using multi-planar reconstruction / recombination (MPR). In some imaging workflows, one or more planes of interest (also referred to herein as scan plans) that capture a predetermined anatomical region at a predetermined orientation may be automatically identified prior to acquisition of a diagnostic image sequence (e.g., by identifying the planes of interest using a three-plane localizer image) or after acquisition of the diagnostic image sequence (e.g., by identifying the planes of interest within the diagnostic image sequence itself). In either case, it is unlikely that the planes of interest are parallel to the images comprising the diagnostic image sequence, and thus MPR may be performed to generate a "synthetic" image sequence from the diagnostic image sequence, wherein the "synthetic" (i.e., recombined) image sequence includes multiple images that are parallel to the planes of interest, thereby enabling the anatomical features of interest to be observed in the predetermined orientation and plane.

[0021] One limitation of the above methods is that the reconstruction of image sequences with insufficient resolution may result in artifacts, such as image blur or step edges. A traditional solution to this problem is to impose constraints on the data resolution requirements before using the reconstruction process (e.g., by acquiring a diagnostic scan with a voxel size of 0.25 mm × 0.25 mm × 0.25 mm). However, this may limit the practicality of such applications in clinical practice because it increases scan time and may be interfered by patient motion or contrast changes in the data. Even if high-resolution image sequences are utilized, the reconstruction process may introduce blurring due to interpolation, which may affect the quality of the reconstructed image. Therefore, it is generally desirable to explore methods for reducing image artifacts in retrospectively reconstructed image sequences, particularly in the case of retrospective reconstruction of medium-resolution or low-resolution image sequences.

[0022] The inventors herein have determined that there is a relationship between the orientation of a reconstructed image sequence relative to an acquired diagnostic image sequence (e.g., relative to the orientation of a plane of interest defined by an image comprising an originally acquired "native" image sequence) and the presence of artifacts in the reconstructed image sequence. Conventional deep learning methods for reducing blur or other image artifacts in medical images do not take into account the reconstructed orientation dependence of such artifacts, and therefore perform with uneven efficacy at different planes of interest. However, the inventors herein have developed a method that at least partially addresses the above-mentioned problem. In one embodiment, by informing an image enhancement network of the relative orientation of a reconstructed image sequence relative to the native image sequence (e.g., by feeding a plane of interest / landmark plane to the image enhancement network along with the native image sequence and the reconstructed image sequence), a resolution-enhanced image sequence can be produced in which artifact attenuation and consistency at various planes of interest are superior to conventional methods. Furthermore, by training an image enhancement network using input data comprising a reconstructed image sequence and landmark planes used to generate the reconstructed image sequence and using ground truth data comprising high-resolution equivalents of the reconstructed image sequence, the image enhancement network can implicitly learn the relationship between the reconstructed plane orientation (relative to the diagnostic scan plane orientation / native scan plane orientation) and the reconstruction / interpolation artifacts present in the reconstructed image sequence.

[0023] In one example, you can use Figure 1 The MRI apparatus 10 shown is used to acquire a sequence of diagnostic images of a volume (eg, an imaging subject). The MRI apparatus 10 may also acquire a three-plane localizer image of the volume before acquiring the sequence of diagnostic images. Figure 2The imaging system 200 shown may receive a diagnostic image sequence and a volumetric three-plane localizer image, and may determine a landmark plane or other plane of interest based on the three-plane localizer image, and may reassemble the diagnostic / native image sequence along the landmark plane to generate one or more images showing a predetermined anatomical region at a predetermined orientation. The imaging system 200 may perform Fig. 9 One or more operations of the method 900 are shown to generate a resolution enhanced image sequence from a sequence of images reconstructed along landmark planes using a trained image enhancement network, wherein the method 900 is exemplified in Fig.10 Alternatively, the imaging system 200 may perform Fig.11 The second method 1100 is shown to generate a resolution enhanced image sequence from the image sequence reconstructed along the perturbed landmark plane, wherein the method 1100 is exemplified in Fig.12 Shown in.

[0024] The image enhancement network of method 900 or 1100 can be based on Figure 7 As shown and Figure 8 In a first embodiment, the training data used by method 700 can be trained by performing one or more operations of the illustrated method 700. Figure 3 As shown and Figure 4 In a second embodiment, the training data used by method 700 can be generated by performing one or more operations of the illustrated method 300. Figure 5 As shown and Figure 6 The image enhancement network is configured and trained to utilize the reconstructed plane and the reconstructed image sequence to intelligently attenuate artifacts introduced into the reconstructed image sequence by the MPR process.

[0025] As used herein, the term native image sequence may be used synonymously with the term unreconstructed image sequence or originally acquired image sequence, and refers to an image sequence that has not been subjected to MPR and therefore retains an original or "native" orientation. In some cases, an image sequence that has been modified from its originally acquired state, such as by downsampling or adding noise, may also be referred to as a native image sequence to indicate that the orientation of the 2D images comprising the "native" image sequence retains the same orientation as the originally acquired image sequence from which it was generated. However, regardless of the orientation of the 2D images comprising the image sequence, an image sequence that has been subjected to MPR (i.e., a reconstructed image sequence) is not referred to herein as a native image or native image sequence, even if the orientation of the 2D images is the same as the orientation of the originally acquired image sequence from which the reconstructed image sequence was generated.

[0026] Reference now Figure 1, a magnetic resonance imaging (MRI) apparatus 10 is shown, wherein the MRI apparatus 10 includes a static magnetic field magnet unit 12, a gradient coil unit 13, an RF coil unit 14, an RF body or volume coil unit 15, a transmission / reception (T / R) switch 20, an RF driver unit 22, a gradient coil driver unit 23, a data acquisition unit 24, a controller unit 25, a patient examination table or examination bed 26, a data processing unit 31, an operation console unit 32, and a display unit 33. In some embodiments, the RF coil unit 14 is a surface coil, which is a local coil that is usually placed near an anatomical structure of interest of a subject 16. Here, the RF body coil unit 15 is a transmission coil that transmits RF signals, and the local surface RF coil unit 14 receives MR signals. Therefore, the transmission body coil (e.g., the RF body coil unit 15) and the surface receiving coil (e.g., the RF coil unit 14) are independent but electromagnetically coupled components. The MRI apparatus 10 transmits electromagnetic pulse signals to a subject 16 placed in an imaging space 18 forming a static magnetic field to perform scanning to obtain magnetic resonance signals from the subject 16. One or more images of the subject 16 can be reconstructed based on the magnetic resonance signals thus obtained by the scanning.

[0027] The static magnetic field magnet unit 12 includes, for example, an annular superconducting magnet installed in an annular vacuum container. The magnet defines a cylindrical space around the subject 16 and generates a constant main static magnetic field B 0 .

[0028] The MRI apparatus 10 further includes a gradient coil unit 13, which forms a gradient magnetic field in an imaging space 18 so as to provide three-dimensional position information for a magnetic resonance signal received by an RF coil array. The gradient coil unit 13 includes three gradient coil systems, each of which generates a gradient magnetic field along one of three spatial axes perpendicular to each other, and generates a gradient field in each of a frequency encoding direction, a phase encoding direction, and a slice selection direction according to an imaging condition. More specifically, the gradient coil unit 13 applies a gradient field in a slice selection direction (or a scanning direction) of the subject 16 to select a slice; and the RF body coil unit 15 or the local RF coil array can transmit an RF pulse to the selected slice of the subject 16. The gradient coil unit 13 also applies a gradient field in a phase encoding direction of the subject 16 to phase encode a magnetic resonance signal from a slice excited by an RF pulse. Then the gradient coil unit 13 applies a gradient field in a frequency encoding direction of the subject 16 to frequency encode a magnetic resonance signal from a slice excited by an RF pulse.

[0029] The RF coil unit 14 is provided, for example, to surround a region to be imaged of the subject 16. In some examples, the RF coil unit 14 may be referred to as a surface coil or a receiving coil. 0 In the static magnetic field space or imaging space 18, the RF coil unit 15 transmits RF pulses as electromagnetic waves to the subject 16 based on a control signal from the controller unit 25, and thereby generates a high-frequency magnetic field B 1 This excites the proton spins in the slice to be imaged of the subject 16. The RF coil unit 14 receives the electromagnetic waves generated when the proton spins thus excited in the slice to be imaged of the subject 16 return to alignment with the initial magnetization vector as magnetic resonance signals. In some embodiments, the RF coil unit 14 may transmit RF pulses and receive MR signals. In other embodiments, the RF coil unit 14 may be used only to receive MR signals, and not to transmit RF pulses.

[0030] The RF body coil unit 15 is provided, for example, to surround the imaging space 18, and generates a main magnetic field B generated by the static magnetic field magnet unit 12 in the imaging space 18. 0 Orthogonal RF magnetic field pulses are used to excite the nucleus. Compared with the RF coil unit 14, which can be disconnected from the MRI device 10 and replaced with another RF coil unit, the RF body coil unit 15 is fixedly attached and connected to the MRI device 10. In addition, although the local coil (such as the RF coil unit 14) can transmit or receive signals only from a local area of ​​the subject 16, the RF body coil unit 15 generally has a larger coverage area. For example, the RF body coil unit 15 can be used to transmit or receive signals to the whole body of the subject 16. The use of a receive-only local coil and a transmission body coil provides uniform RF excitation and good image uniformity, at the expense of higher RF power deposited in the subject. For a transmit-receive local coil, the local coil provides RF excitation to the region of interest and receives MR signals, thereby reducing the RF power deposited in the subject. It should be understood that the specific use of the RF coil unit 14 and / or the RF body coil unit 15 depends on the imaging application.

[0031] The T / R switch 20 may selectively electrically connect the RF body coil unit 15 to the data acquisition unit 24 when operating in the receive mode, and may selectively electrically connect the RF body coil unit to the RF driver unit 22 when operating in the transmit mode. Similarly, the T / R switch 20 may selectively electrically connect the RF coil unit 14 to the data acquisition unit 24 when the RF coil unit 14 operates in the receive mode, and may selectively electrically connect the RF coil unit to the RF driver unit 22 when operating in the transmit mode. When both the RF coil unit 14 and the RF body coil unit 15 are used for a single scan, for example, if the RF coil unit 14 is configured to receive MR signals and the RF body coil unit 15 is configured to transmit RF signals, the T / R switch 20 may direct the control signal from the RF driver unit 22 to the RF body coil unit 15 while directing the received MR signal from the RF coil unit 14 to the data acquisition unit 24. The coil of the RF body coil unit 15 may be configured to operate in a transmit-only mode or a transmit-receive mode. The coils of the local RF coil unit 14 may be configured to operate in a transmit-receive mode or a receive-only mode.

[0032] The RF driver unit 22 includes a gate modulator (not shown), an RF power amplifier (not shown), and an RF oscillator (not shown), which are used to drive an RF coil (e.g., the RF coil unit 15) and form a high-frequency magnetic field in the imaging space 18. The RF driver unit 22 modulates an RF signal received from the RF oscillator into a signal of a predetermined timing having a predetermined envelope based on a control signal from the controller unit 25 and using the gate modulator. The RF signal modulated by the gate modulator is amplified by the RF power amplifier and then output to the RF coil unit 15.

[0033] The gradient coil driver unit 23 drives the gradient coil unit 13 based on a control signal from the controller unit 25, thereby generating a gradient magnetic field in the imaging space 18. The gradient coil driver unit 23 includes three driver circuit systems (not shown) corresponding to the three gradient coil systems included in the gradient coil unit 13.

[0034] The data acquisition unit 24 includes a preamplifier (not shown), a phase detector (not shown), and an analog / digital converter (not shown) for acquiring a magnetic resonance signal received by the RF coil unit 14. In the data acquisition unit 24, the phase detector uses the output of the RF oscillator from the RF driver unit 22 as a reference signal, performs phase detection on the magnetic resonance signal received from the RF coil unit 14 and amplified by the preamplifier, and outputs the analog magnetic resonance signal after the phase detection to the analog / digital converter to be converted into a digital signal. The digital signal thus obtained is output to the data processing unit 31.

[0035] The MRI apparatus 10 includes a couch 26 for placing the subject 16 thereon. By moving the couch 26 based on a control signal from the controller unit 25, the subject 16 can be moved inside and outside the imaging space 18.

[0036] The controller unit 25 includes a computer and a recording medium on which a program to be executed by the computer is recorded. The program, when executed by the computer, causes the various parts of the device to perform operations corresponding to a predetermined scan. The recording medium may include, for example, a ROM, a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, or a non-volatile memory card. The controller unit 25 is connected to the operation console unit 32 and processes an operation signal input to the operation console unit 32, and also controls the examination bed 26, the RF driver unit 22, the gradient coil driver unit 23, and the data acquisition unit 24 by outputting control signals to them. The controller unit 25 also controls the data processing unit 31 and the display unit 33 based on the operation signal received from the operation console unit 32 to obtain a desired image.

[0037] The operation console unit 32 includes a user input device such as a touch screen, a keyboard, and a mouse. The operator uses the operation console unit 32 to, for example, input such data as an imaging protocol and set an area where an imaging sequence will be executed. The data on the imaging protocol and the imaging sequence execution area are output to the controller unit 25.

[0038] The data processing unit 31 includes a computer and a recording medium on which a program to be executed by the computer to perform predetermined data processing is recorded. The data processing unit 31 is connected to the controller unit 25, and performs data processing based on a control signal received from the controller unit 25. The data processing unit 31 is also connected to the data acquisition unit 24, and generates spectrum data by applying various image processing operations to the magnetic resonance signals output from the data acquisition unit 24.

[0039] In one example, the recording medium of the data processing unit 31 may include instructions for applying machine learning-assisted MPR to magnetic resonance signals to retrospectively reconstruct images. For example, landmark planes determined via an automatic scan plane scheme may be used to retrospectively reconstruct a diagnostic image sequence to produce a reconstructed image sequence, and the trained image enhancement network may be configured to remove blur and artifacts from the reconstructed image reconstructed from the diagnostic image sequence based on the landmark planes. Thus, reliance on high-resolution diagnostic images may be reduced because the image enhancement network may learn anatomical and orientation specific filters for attenuating blur and reconstructing specific artifacts from the reconstructed image. Further details of machine learning-assisted MPR are provided below.

[0040] The display unit 33 includes a display device, and displays an image on a display screen of the display device based on a control signal received from the controller unit 25. The display unit 33 displays, for example, images regarding input items for which an operator inputs operation data from the operation console unit 32. The display unit 33 also displays a two-dimensional (2D) slice image or a three-dimensional (3D) image of the subject 16 generated by the data processing unit 31.

[0041] Although an MRI system is described by way of example, it should be understood that the present techniques may also be useful when applied to images acquired using other imaging modalities, such as CT, tomosynthesis, PET, C-arm angiography, etc. The present discussion of the MRI imaging modality is provided merely as an example of one suitable imaging modality.

[0042] When a sequence of diagnostic images acquired by an imaging system is reconstructed along a plane different from the plane used to acquire the original diagnostic images and displayed on a display device, artifacts and blurring may be observed in the reconstructed image sequence. Artifacts may be attributed to step edges in the sequence of diagnostic images, while blurring may be caused by motion of the imaged subject. In some examples, efforts to mitigate the presence of image artifacts may include acquiring the sequence of diagnostic images at high resolution, which may require extended scan times and exacerbate contrast variations.

[0043] In one example, as described herein, resolution-enhanced reconstructed images (e.g., images with attenuated artifacts or blur and having higher spatial resolution than conventional reconstructed image sequences) can be obtained without increasing scan time via a workflow that utilizes deep learning to intelligently remove and enhance the resolution of the reconstructed image sequence based on the plane used to perform the reconstruction. The workflow can include reconstruction of a diagnostic image sequence according to specific landmark planes and target specifications (such as output resolution and thickness). The reconstructed images can be processed using a trained image enhancement network to remove blur and artifacts. For example, the image enhancement network can be trained to recognize blur based on different plane orientations predicted by an automatic scan plane scheme.

[0044] refer to Figure 2, an imaging system 200 is shown. The imaging system 200 may be configured to acquire a 3D medical image of an imaged subject using an imaging device 226, and to process the acquired 3D image using an image processing system 202, for example by recombining the 3D image along a plane of interest. The processed image may be displayed to a user via a display device 224. The imaging system may receive user input via a user input device 222, wherein one or more of image acquisition and image processing may be adjusted based on the received user input. As an example, after the 3D image acquisition, the user may specify a predetermined anatomical region of interest that the user wishes to view one or more standard views. The image processing system 202 may receive the user input from the user input device 222, and recombines the 3D image along one or more standard orientations / planes according to machine executable instructions stored in a multi-planar reformation (MPR) module 212 to generate a standard view of the requested anatomical region of interest. In addition, the image processing system may post-process the standard view of the anatomical region of interest using a trained image enhancement network stored in an image enhancement module 208 to generate a standard view with enhanced resolution. The post-processed standard view of the anatomical region of interest may have a greater resolution than the originally acquired 3D image and may display reduced number and severity of blurring and reconstruction artifacts compared to conventional MPR methods, thereby enabling a user to view the anatomical region of interest in greater detail than using conventional methods. In some embodiments, the image processing system 202 may also be configured to train the image enhancement network by executing instructions stored in the training module 210, such as referring to Figures 3 to 8 Described in more detail.

[0045] In some embodiments, at least a portion of the image processing system 202 is disposed at a device (e.g., an edge device, a server, etc.) that is communicatively coupled to the MRI system 10 via a wired connection and / or a wireless connection. In some embodiments, at least a portion of the image processing system 202 is disposed at a separate device (e.g., a workstation) that can receive images from the MRI system or from a storage device storing images generated by the MRI system. The image processing system 202 can be operatively / communicatively coupled to a user input device 222 and a display device 224. The user input device 222 can be integrated into the MRI system, such as at the operating console unit 32 of the MRI system 10. Similarly, the display device 224 can be integrated into the MRI system, such as at the display unit 33 of the MRI system 10.

[0046] The image processing system 202 includes a processor 204 configured to execute machine-readable instructions stored in a non-transitory memory 206. The processor 204 may be a single-core or multi-core processor, and the program executed thereon may be configured for parallel processing or distributed processing. In some embodiments, the processor 204 may optionally include separate components distributed on two or more devices, which may be located at a distance and / or configured for coordinated processing. In some embodiments, one or more aspects of the processor 204 may be virtualized and performed by a remotely accessible networked computing device configured in a cloud computing configuration.

[0047] The non-volatile memory 206 may store an image enhancement module 208, a training module 210, and an MPR module 212. The image enhancement module 208 may include parameters of one or more image enhancement networks and instructions for processing data via the image enhancement network. In one example, the image enhancement module 208 may include an ML model that is trained to map a recombined image sequence, a landmark plane / plane of interest used to generate the recombined image sequence, and an originally acquired image sequence (i.e., an image sequence before MPR processing) to a resolution-enhanced image sequence corresponding to the recombined image sequence. The image enhancement network stored in the image enhancement module 208 may include trained and / or untrained machine learning (ML) models, and may also include various data or metadata related to one or more ML models stored therein.

[0048] The non-transitory memory 206 may also store a training module 210, which may include instructions for training one or more image enhancement networks stored in the image enhancement module 208. The training module 210 may include instructions that, when executed by the processor 204, cause the image processing system 202 to perform one or more steps of the method 700, for training the image enhancement network to map the reconstructed image sequence to a resolution-enhanced image sequence, wherein the resolution-enhanced image sequence may have a higher resolution than the input reconstructed image sequence and may exhibit attenuation of blur and / or image artifacts present in the input reconstructed image sequence. In some embodiments, the training module 210 may include instructions for implementing one or more gradient descent algorithms, applying one or more loss functions and / or training routines for adjusting parameters of one or more neural networks of the image enhancement module 208. The training module 210 may include a training data set for one or more image enhancement networks of the image enhancement module 208, and may also include instructions for generating the training data from a 3D image / image sequence according to one or more of the methods disclosed herein.

[0049] The non-transitory memory 206 also stores an MPR module 212, which may include instructions for executing one or more MPR algorithms known in the art of medical imaging, and may include resampling image data from an image sequence along a landmark plane or other plane of interest. In some embodiments, the MPR module 212 may stack image data from an image sequence acquired via an imaging device 226 or received via a communication coupling with a remotely located imaging device and / or image storage system, and reslice the image data along a landmark plane or plane of interest. In some embodiments, the MPR module 212 may be configured to receive an automatically determined landmark plane (e.g., a landmark plane generated by a landmark plane segmentation model based on a three-plane locator image), and automatically retrospectively reassemble the image sequence along the landmark plane, thereby accelerating the generation of a standard view of the anatomical region of interest.

[0050] In some embodiments, non-transitory memory 206 may include components disposed on two or more devices that may be remotely located and / or configured for coordinated processing. In some embodiments, one or more aspects of non-transitory memory 206 may include a remotely accessible networked storage device configured in a cloud computing configuration.

[0051] The user input device 222 may include one or more of a touch screen, keyboard, mouse, trackpad, motion sensing camera, or other device configured to enable a user to interact with and manipulate data within the image processing system 202. In one example, the user input device 222 may enable a user to make selections of landmark planes / planes of interest relative to an image sequence and / or provide MPR settings such as number of slices, slice thickness, slice interval, etc.

[0052] The display device 224 may include one or more display devices utilizing almost any type of technology. In some embodiments, the display device 224 may include a computer monitor and may display medical images, such as resolution-enhanced images generated according to one or more of the methods disclosed herein. The display device 224 may be combined with the processor 204, the non-transitory memory 206, and / or the user input device 222 in a common housing, or may be a peripheral display device and may include a monitor, a touch screen, a projector, or other display devices known in the art that may enable a user to view the MRI generated by the MRI system and / or may interact with various data stored in the non-transitory memory 206.

[0053] It should be understood that Figure 2 The imaging system 200 shown is for the purpose of illustration and not limitation. Another suitable imaging system may include more, fewer, or different components.

[0054] refer to Figure 3 , shows a first embodiment of a method 300 for generating training data for training an image enhancement network. The training data generated by the method 300 can be used to train the image enhancement network to learn to intelligently remove blurring artifacts and enhance the resolution of the reconstructed image sequence based on the plane used to perform the reconstruction. In some embodiments, the method 300 can be performed by an imaging system (such as Figure 2 The imaging system 200 shown is used to perform.

[0055] The method 300 begins at operation 302, where an imaging system acquires a native image sequence of a volume, where the native image sequence includes a first plurality of 2D images parallel to a first plane. In some embodiments, the first plane may be one of a predetermined number of planes (e.g., a coronal plane, a sagittal plane, or an axial plane) relative to an imaged subject. In other embodiments, the native image sequence may be substantially parallel to a landmark plane, but may deviate from the landmark plane due to motion of the imaged subject or inaccuracies in segmentation of the landmark plane. In some embodiments, the native image sequence may have an isotropic resolution of 0.25 mm.

[0056] At operation 304, the imaging system selects a predetermined landmark plane that intersects the volume. In some embodiments, a landmark plane segmentation model may be used to determine a landmark plane based on a three-plane locator image of the volume, wherein the landmark plane intersects a predetermined anatomical region. The landmark plane segmentation model may include a deep learning model configured to receive the three-plane locator image as an input feature and output one or more landmark planes (also referred to herein as planes of interest). In some embodiments, the landmark plane segmentation model includes one or more learned convolutional filters. In some embodiments, multiple landmark planes may be determined, and thus multiple landmark planes may be selected at operation 304. In some embodiments, an image enhancement network trained using training data generated by method 300 may be used in conjunction with the same or substantially similar landmark plane segmentation model used at operation 304, thereby ensuring that the landmark planes fed to the image enhancement network at inference time are substantially similar to those used during training of the image enhancement network.

[0057] At operation 306, the imaging system reassembles the sequence of native images along the landmark plane to produce a first reassembled sequence of images including a second plurality of 2D images parallel to the landmark plane. Image reassembly may be performed using one or more MPR algorithms known in the art of medical imaging and may include resampling image data from the sequence of images acquired at operation 302 along the landmark plane. In some embodiments, MPR may be performed by "stacking" the image data from the sequence of native images acquired at operation 302 and reslicing the image data along the landmark plane. In some embodiments, additional parameters such as slice thickness and slice interval may be provided, thereby enabling the sequence of native images to be resliced ​​using a specified thickness and slice separation.

[0058] At operation 308, the imaging system determines an inverse plane of the landmark plane. In some embodiments, the imaging system may determine the inverse plane by determining a set of transformations that map from the landmark plane to the first plane. In some embodiments, the inverse plane may be determined by determining an inverse affine transformation of the first MPR transformation performed at operation 306.

[0059] At operation 310, the imaging system reassembles the first reassembled image sequence along the inverse plane to produce a second reassembled image sequence including a third plurality of 2D images parallel to the inverse plane (and also parallel to the first plane). Image reconstruction may be performed using one or more MPR algorithms known in the art of medical imaging and may include resampling image data from the first reassembled image sequence produced at operation 306 along the inverse plane. In some embodiments, MPR may be performed by "stacking" image data from the first reassembled image sequence produced at operation 306 and reslicing the image data along the inverse plane. In one embodiment, the image reconstruction process may involve the use of interpolation techniques. For example, nearest neighbor interpolation, linear interpolation, or cubic convolution interpolation may be employed to estimate pixel values ​​in the second reassembled image sequence.

[0060] At operation 312, the imaging system determines an overlap of a foreground region of the native image sequence with a foreground region of the second recombined image sequence based on an intersection of the foreground of the second recombined image sequence and the foreground of the native image sequence. Because the 2D images comprising the second recombined image sequence are in the same orientation as the 2D images from the native image sequence, determining the overlap region between the anatomical regions / foreground is computationally efficient.

[0061] At operation 314, the imaging system stores the native image sequence, the second recombined image sequence, the landmark plane, and the overlap region as training data pairs. In particular, the native image sequence, the second recombined image sequence, and the landmark plane are stored as input data / input features, and the overlap region is stored as a ground truth image sequence. After operation 314, method 300 may end.

[0062] Reference now Figure 4 , shows an example 400 of the training data generation method 300. The example 400 includes a native image sequence 402 covered by a landmark plane 404. Figure 4 As can be seen in , native image sequence 402 includes a plurality of 2D images having a first orientation (ie, parallel to a first plane). Landmark plane 404 is not parallel to the plurality of 2D images comprising native image sequence 402 because landmark plane 404 intersects the 2D images of native image sequence 402.

[0063] The reorganization of the native image sequence 402 along the landmark plane 404 (as described above at operation 306) produces a first reorganized image sequence 406, which includes a plurality of 2D images that are parallel to the landmark plane 404 and not parallel to the plurality of 2D images that comprise the native image sequence 402. The reorganization of the first reorganized image sequence 406 along the inverse plane (as described above at operation 310) produces a second reorganized image sequence 408, wherein the second reorganized image sequence 408 includes a plurality of 2D images that are parallel to the plurality of 2D images of the native image sequence 402. By reorganizing the first reorganized image sequence 406 along the inverse plane of the landmark plane 404, the 2D images of the second reorganized image sequence will be more easily registered to the native image sequence 402.

[0064] An overlap region 410 may be determined, wherein the overlap region 410 includes light portions of the intersecting mask 412. The overlap region 410 includes an area of ​​the native image sequence 402 that is equivalent to an area captured by the second recombined image sequence 408, and thus may be used as a ground truth image sequence to train an image enhancement network to map from an image space with recombined artifacts and blur (e.g., the image space embodied by the second recombined image sequence) to a high-resolution image space without blur and recombined artifacts.

[0065] refer to Figure 5 , shows a second embodiment of a method 500 for generating training data for an image enhancement network. The training data generated by the method 500 can be used to train the image enhancement network to learn to intelligently remove and enhance the resolution of the reconstructed image sequence based on the plane used to perform the reconstruction. In some embodiments, the method 500 can be performed by an imaging system (such as Figure 2The imaging system 200 shown is used to perform.

[0066] Method 500 begins at operation 502, where an imaging system acquires a high-resolution native image sequence of a volume. The high-resolution native image sequence includes a first plurality of 2D images parallel to a first plane. In some embodiments, the first plane may be one of a predetermined number of planes (e.g., a coronal plane, a sagittal plane, or an axial plane) relative to an imaged subject. In other embodiments, the high-resolution native image sequence may be substantially parallel to a landmark plane, but may deviate from the landmark plane due to motion of the imaged subject or inaccuracies in segmentation of the landmark plane. In some embodiments, the high-resolution native image sequence may have an isotropic resolution of 0.25 mm.

[0067] At operation 504, the imaging system downsamples the high-resolution native image sequence and / or adds noise to the high-resolution native image sequence to generate a low-resolution native image sequence (hereinafter referred to as the native image sequence). In some embodiments, the imaging system may simultaneously downsample the high-resolution native image sequence along all three dimensions of the imaging data by downsampling within the 2D image including the high-resolution native image sequence and by discarding or combining the 2D images / slices of the high-resolution native image sequence. In some embodiments, the downsampling may include one or more of decimation: wherein every other row, every other column, and every other slice of the high-resolution native image sequence is discarded; maximum pooling: wherein a single maximum voxel intensity value in a voxel block of a predetermined size is used to replace the voxel block; bilinear interpolation: wherein a weighted average of eight nearest voxels is used to generate a new voxel value; cubic interpolation: wherein a weighted average of 64 nearest voxels is used to generate a new voxel value; and nearest neighbor interpolation: wherein the nearest voxel is selected to generate a new voxel value. The native image sequence has a lower spatial resolution than the high-resolution native image sequence, and thus approximates an image sequence that may be acquired at an extrapolated time when image acquisition time may be limited.

[0068] Additionally, at operation 504, the imaging system may add noise to the downsampled native image sequence, thereby providing another mechanism for image quality adjustment, which may both increase the variety of training data generated (e.g., by adjusting the amount of noise added), and may also more closely approximate image sequences that may be encountered at inference time (e.g., when the image enhancement network is deployed at an imaging center or hospital). In some embodiments, the imaging system may add Gaussian noise to the low-resolution native image sequence by adding a random value from a Gaussian distribution to each voxel in the low-resolution native image sequence. In some embodiments, the imaging system may add Gaussian white noise to the low-resolution native image sequence by adding a random value from a normal distribution to each voxel in the low-resolution native image sequence. In another embodiment, the imaging system may add Rice noise to the low-resolution native image sequence by combining both Gaussian noise and noise due to the random phase of the MRI signal to the voxels of the low-resolution native image sequence.

[0069] At operation 506, the imaging system reconstructs the native image sequence along the landmark plane to produce a first reconstructed image sequence including a first plurality of 2D images parallel to the landmark plane. The reconstructing process of the native image sequence may introduce blurring and reconstructed imaging artifacts into the first reconstructed image sequence, thereby mimicking a reconstructed image sequence that may be encountered at inference time, because the reconstructed blurring and artifacts are a function of the resolution of the image data being reconstructed, wherein lower resolution imaging data produces more or more severe reconstructed imaging artifacts. Image reconstructing may be performed using one or more MPR algorithms known in the art of medical imaging, and may include resampling image data from the native image sequence along the landmark plane. In some embodiments, MPR may be performed by "stacking" image data from the native image sequence and reslicing the image data along the landmark plane.

[0070] At operation 508, the imaging system reassembles the high-resolution native image sequence along the landmark plane to produce a second reassembled image sequence including a second plurality of 2D images parallel to the landmark plane. The reassembly process of the high-resolution native image sequence may introduce little or no blur or reassembly imaging artifacts into the second reassembled image sequence because the degree of blur and artifacts introduced by MPR is inversely proportional to the resolution of the reassembled imaging data. Therefore, the second reassembled image sequence is a higher resolution version of the first reassembled image sequence and may include attenuated reassembly artifacts or no reassembly artifacts, thereby enabling the second reassembled image sequence to be used as a ground truth high-resolution image sequence. Image reassembly may be performed using one or more MPR algorithms known in the art of medical imaging and may include resampling image data from the high-resolution image sequence along the landmark plane. In some embodiments, MPR may be performed by "stacking" image data from the high-resolution image sequence and reslicing the image data along the landmark plane.

[0071] At operation 510, the imaging system stores the landmark planes, the first recombined image sequence, and the native image sequence as input data, and stores the second recombined image sequence as the corresponding ground truth image sequence of the training data pair. In one embodiment, the imaging system may also store additional parameters or metadata associated with the image sequence in a manner associated with the training data pair. For example, the system may store information such as the imaging modality used (e.g., MRI, CT, ultrasound), the patient's demographic information, or the specific settings used during image acquisition. This additional information can be used to train the image enhancement network to adapt to changes in imaging modalities or patient characteristics. In another embodiment, the imaging system may store multiple landmark planes and corresponding recombined image sequences generated from the same high-resolution native image sequence as distinct training data pairs. Multiple landmark planes can be derived from different anatomical regions captured by the high-resolution native image sequence, thereby providing a more diverse training data set. After operation 510, method 500 can end.

[0072] Reference now Figure 6 , shows an example 600 of the training data generation method 500. The example 600 includes a high-resolution native image sequence 602 overlaid with a landmark plane 604. Figure 6 As can be seen in FIG. 6 , the high-resolution native image sequence 602 includes a plurality of 2D images having a first orientation (i.e., parallel to a first plane). The landmark plane 604 is not parallel to the plurality of 2D images including the high-resolution native image sequence 602 because the landmark plane 604 intersects the 2D images of the high-resolution native image sequence 602.

[0073] Downsampling of the high resolution native image sequence 602 produces a low resolution native image sequence 606 including a plurality of 2D images also parallel to the first plane. A landmark plane 604 is also shown overlaid on the low resolution native image sequence 606. MPR of the low resolution native image sequence 606 along the landmark plane 604 produces a first recombined image sequence 612 including a plurality of 2D images parallel to the landmark plane 604. The first recombined image sequence 612 may include a recombined artifact because the recombined artifact is derived from the low resolution native image sequence 606 and the presence of the recombined artifact is negatively correlated with the resolution of the recombined imaging data.

[0074] The MPR performed on the high-resolution native image sequence 602 along the landmark plane 604 produces a second reconstructed image sequence 610 including a second plurality of 2D images parallel to the landmark plane 604. The reconstructing process of the high-resolution native image sequence 602 may introduce little or no blur or reconstructing artifacts into the second reconstructed image sequence 610 because the degree of blur and artifacts introduced by MPR is inversely proportional to the resolution of the reconstructed imaging data. Therefore, the second reconstructed image sequence 610 is a higher resolution version of the first reconstructed image sequence 612 and may include attenuated or no reconstructing artifacts, thereby enabling the second reconstructed image sequence 610 to be used as a ground truth high-resolution image sequence.

[0075] refer to Figure 7 , a method 700 of training an image enhancement network to selectively denoise and enhance the resolution of a reconstructed image sequence based on a plane used to generate the reconstructed image sequence is shown. The method 700 may be implemented by an imaging system such as Figure 2 Method 700 may be performed to train an image enhancement network (such as the image enhancement network employed in method 900) using training data generated via methods 300 and / or 500.

[0076] Method 700 begins at operation 702, where the imaging system selects a training data pair including input data (including a plane, a reorganized image sequence, and a native image sequence) and a ground truth image sequence. The reorganized image sequence is generated by reorganizing the data of the native image sequence along the plane, or in some embodiments, by reorganizing the native image sequence along a perturbed version of the plane. The reorganized image sequence may include blurring and / or reorganization artifacts (such as step edges), which may be generated by MPR. In other words, the input data includes a pre-reorganized image sequence (native image sequence), a post-reorganized image sequence (reorganized image sequence), and a plane for performing the reorganization. The ground truth image sequence is a high-resolution image sequence that captures the same volume / imaging subject as the reorganized image sequence, but has a higher spatial resolution and has attenuated reorganization-induced imaging artifacts or no reorganization-induced imaging artifacts. In some embodiments, the orientation of the 2D image comprising the ground truth image sequence relative to the imaging volume / imaging subject may be the same as the 2D image of the reorganized image sequence.

[0077] At operation 704, the imaging system uses an image enhancement network to map the input data to a predicted resolution-enhanced image sequence. In some embodiments, the image enhancement network includes a plurality of 3D convolution filters, wherein the plurality of 3D convolution filters receive spatially continuous data from spatially corresponding sub-regions of both the reorganized image sequence and the native image sequence or from a subsequently generated 3D feature map. The image enhancement network may include one or more convolution layers, the convolution layer including one or more 3D convolution filters. The 3D convolution filters may be used to detect features in the reorganized image sequence and the native image sequence, such as edges, corners, and shapes. In some embodiments, a plurality of 3D convolution filters are convolved with the reorganized image sequence and the native image sequence to detect features, which may be output as feature maps. The filters of the image enhancement network may also be configured to receive a plane as an additional feature channel of the input data, for example by encoding the plane as a numerical value or vector, and connecting the numerical value or vector to each voxel of the reorganized image sequence or the native image sequence. In some embodiments, the plane information may be provided to the image enhancement network as a 3D binary matrix corresponding to the reorganized / landmark plane parameters.

[0078] The predicted enhanced resolution image sequence includes a plurality of 2D images corresponding to the reassembled image sequence, wherein each of the plurality of 2D images is parallel to the plane. In some embodiments, there may be a one-to-one correspondence between the 2D images of the reassembled image sequence and the predicted enhanced resolution image sequence (e.g., for a first image of the reassembled image sequence, there may be exactly one corresponding image in the predicted enhanced resolution image sequence that captures the same or substantially similar anatomical features).

[0079] At operation 706, the imaging system determines a loss based on the predicted resolution-enhanced image sequence and the ground-truth high-resolution image sequence. In some embodiments, the difference in spatially corresponding voxel intensity values ​​between the predicted resolution-enhanced image sequence and the ground-truth high-resolution image sequence can be used to determine the loss. In some embodiments, the loss function may include a mean square error (MSE) loss function, in which the mean square difference between the predicted voxel value and the ground-truth voxel value is used as the current predicted loss of the image enhancement network. In another embodiment, the loss function may include a mean absolute error (MAE) loss function, in which the mean absolute difference between the predicted voxel value and the ground-truth voxel value is used as the current predicted loss of the image enhancement network. The above two loss functions are provided as non-limiting examples, and it should be understood that the present disclosure covers the use of other loss functions known in the field of image processing and deep learning.

[0080] At operation 708, the imaging system adjusts the weights and biases of the image enhancement network based on the loss. In some embodiments, the parameters of the image enhancement network may be adjusted to reduce the loss on the training data set. In some embodiments, back propagation of the loss may occur according to a gradient descent algorithm, wherein the gradient (first-order derivative or approximation of the first-order derivative) of the loss function is determined for each weight and bias of the image enhancement network. Each weight (and bias) of the image enhancement network is then updated by adding the negative of the gradient product determined (or approximated) for the weight (or bias) to a predetermined step size. After operation 708, method 700 may end.

[0081] It should be noted that method 700 may be repeated for each of the plurality of training data pairs in the training data set, and the process may be repeated until a stop condition is met, wherein in some embodiments, the stop condition includes one or more of the loss decreasing below a threshold loss, the loss change rate decreasing below a loss change threshold rate, the validation loss determined on the validation data set, reaching a minimum value, etc.

[0082] refer to Figure 8, an illustration 800 of the training method 700 is shown. As can be seen, input data including pairs of training data of a reorganized image sequence 802, a native image sequence 804, and a landmark plane 806 is fed to an image enhancement network 808. The image enhancement network 808 maps the input data to a predicted resolution-enhanced image sequence 810, and determines a loss 816 for the predicted resolution-enhanced image sequence 810 based on the difference between the ground truth high-resolution image sequence 812 and the predicted resolution-enhanced image sequence 810. Once determined, the loss 816 is back-propagated through the image enhancement network 808 and used to update parameter values ​​of the image enhancement network.

[0083] refer to Fig. 9 , shows a first embodiment of a method 900 for performing machine learning-assisted MPR of an image sequence using a trained image enhancement network. The method 900 may use a trained image enhancement network (such as a Figure 7 The method 700 shown in FIG. 7 is used to train an image enhancement network (such as a Figure 2 The present invention may be performed by the imaging system 200 shown in the figure, wherein the training data is generated by one or more of the methods 300 and / or 500.

[0084] Method 900 begins at operation 902, where the imaging system acquires a three-plane locator image of the volume. A three-plane locator image is a 2D image that provides a three-dimensional view of the volume to be imaged. In some embodiments, the three-plane locator image is taken from an axial plane, a sagittal plane, and a coronal plane that are perpendicular to each other. The three-plane locator image can be used to plan a diagnostic scan by providing a comprehensive view of the volume to be imaged (e.g., the body of the imaging subject). The three-plane locator image can be used to identify the location of the region of interest and the size and shape of the region of interest. This information can then be used to determine the scanning parameters for the diagnostic scan, such as the scan type, the scan angle, the number of slices to be taken, and the slice thickness. The three-plane locator image can also be used to identify potential obstacles that may interfere with the scan, such as bones, organs, or implants.

[0085] At operation 904, the imaging system determines a landmark plane from the three-plane locator image. In some embodiments, a landmark plane is determined based on the three-plane locator image of the volume using a landmark plane segmentation model, wherein the landmark plane intersects with a predetermined anatomical region. The landmark plane segmentation model may include a deep learning model configured to receive the three-plane locator image as an input feature and output one or more landmark planes (also referred to herein as planes of interest). In some embodiments, the landmark plane segmentation model includes one or more learned convolution filters. In some embodiments, at operation 904, multiple landmark planes may be determined. As an example, a diagnostic scan may capture an anatomical region of interest, and operation 904 may include determining multiple planes of a standard view that captures the anatomical region of interest, wherein the standard view may include more than one orientation of the anatomical region of interest (e.g., obtaining both a sagittal view and a coronal view of the same anatomical region of interest). In another example, a diagnostic scan may capture multiple anatomical regions of interest, wherein at operation 904, the imaging system may determine at least one landmark plane for each anatomical region of interest, wherein the landmark plane captures the corresponding anatomical region of interest in one or more standard views.

[0086] At operation 906, the imaging system acquires an image sequence based on the landmark plane. The image sequence includes a plurality of 2D images, wherein each of the plurality of 2D images is parallel to a first plane, and wherein the landmark plane is not parallel to the first plane. The image sequence acquired at operation 906 may be referred to as a native image sequence because the plurality of 2D images comprising the image sequence have not been reassembled. In ordinary cases, the first plane (i.e., the orientation of the image of the diagnostic scan) is parallel to the landmark plane, and thus the image sequence may include an expected standard view of the anatomical region of interest, however, in most cases, the diagnostic imaging sequence is not parallel to the landmark plane, or if there are multiple non-parallel landmark planes, the imaging sequence may not be parallel to all landmark planes, and therefore a retrospective MPR is required. This is a non-ordinary case of retrospective MPR targeted by the present disclosure.

[0087] At operation 908, the imaging system reassembles the image sequence along the landmark plane to produce a first reassembled image sequence. The first reassembled image sequence includes a plurality of 2D images, wherein each of the plurality of 2D images is parallel to the landmark plane. By reassembling the image sequence along the landmark plane, wherein the landmark plane intersects the anatomical region of interest at a predetermined orientation (e.g., to produce a standard view of the anatomical region), a desired standard view of the anatomical region of interest is included in the first reassembled image sequence. The reassembly process performed at operation 908 may introduce artifacts (such as step edges or blurring) into the first reassembled image sequence.

[0088] At operation 910, the imaging system maps the first reorganized image sequence, the image sequence, and the landmark plane to a resolution enhanced image sequence using a trained image enhancement network. In some embodiments, the trained image enhancement network includes a plurality of learned three-dimensional (3D) convolutional filters, wherein the plurality of learned 3D convolutional filters receive spatially continuous data from spatially corresponding sub-regions of both the image sequence and the first reorganized image sequence or from a subsequently generated 3D feature map. The image enhancement network may include one or more convolutional layers including one or more learned 3D convolutional filters. By applying a plurality of learned 3D filters to the image sequence and the first reorganized image sequence, the 3D convolutional filters can be used to detect features such as edges, corners, and shapes in the image sequence (i.e., the diagnostic image sequence) and the first reorganized image sequence.

[0089] In some embodiments, a plurality of 3D convolution filters are convolved with the image sequence and the first reorganized image sequence to detect learned features, which can be output as a feature map. In some embodiments, a 3D convolution filter configured to receive the image sequence, the first reorganized image sequence, and the landmark plane may include a four-dimensional matrix, wherein three dimensions of the matrix correspond to spatial dimensions of the image sequence, and the fourth dimension corresponds to different input channels (e.g., the image sequence can be considered a first input channel, the first reorganized image sequence can be considered a second input channel, and the landmark plane can be considered a third input channel). In a specific example, an embodiment of the 3D convolution filter may include a 3×3×3×n matrix, wherein the matrix is ​​configured to receive spatially contiguous 3×3×3 "blocks" of input data from n different input channels. In another specific example, a first convolutional layer configured to receive as input an image sequence, a first reorganized input sequence, and a landmark plane may include m distinct 3D convolutional filters, the m distinct 3D convolutional filters including m distinct four-dimensional matrices, wherein the four-dimensional matrices have the form of A×A×A×3, wherein A is the size of a spatially contiguous “block” of 3D data taken from the corresponding image sequence as input, and wherein 3 corresponds to three input channels (a first channel for the image sequence, a second channel for the first reorganized image sequence, and a third channel for the landmark plane). After convolving the input data, each of the m 3D convolutional filters may produce a corresponding plurality of features, thereby generating a feature map of three spatial dimensions and one channel dimension of size m (one channel for each convolutional filter in the first layer). Thus, the convolution filters in subsequent layers may have the form B×B×B×m, where B is the size of a spatially contiguous “block” of input data obtained as input for each of the m distinct features determined by the previous m distinct convolution filters. The last layer of the image enhancement network may receive a feature map of size X×Y×Z×o, where X×Y×Z corresponds to the resolution of the feature map in the X, Y, and Z spatial dimensions, respectively, and where o corresponds to the number of feature channels produced by the penultimate layer of the image enhancement network. In some embodiments, the last layer of the image enhancement network may map the feature map of size X×Y×Z×o to an image sequence of size X×Y×Z×l, for example by performing a convolution using l distinct convolution filters each of size 1×1×1×o, where l is the number of color channels in the resolution enhanced image sequence.

[0090] The enhanced resolution image sequence includes a plurality of 2D images corresponding to the first recombined image sequence, wherein each 2D image in the plurality of 2D images is parallel to the landmark plane. In some embodiments, there may be a one-to-one correspondence between the 2D images of the first recombined image sequence and the enhanced resolution image sequence (e.g., for a first image of the first recombined image sequence, there may be exactly one corresponding image in the enhanced resolution image sequence that captures the same or substantially similar anatomical features).

[0091] At operation 912, the imaging system displays the resolution enhanced image sequence via a display device. In some embodiments, images of the resolution enhanced image sequence corresponding to the landmark planes may be displayed at operation 912, thereby enabling a radiologist to quickly view the anatomical region of interest in a standard orientation, which may facilitate assessment of the imaging subject. After operation 912, method 900 may end.

[0092] refer to Fig.10 , an illustration 1000 of the method 900 for performing machine learning-assisted MPR of an image sequence is shown. The illustration 1000 shows a diagnostic image sequence 1002 of an image volume, such as the diagnostic image sequence acquired at operation 906 of the method 900. The diagnostic image sequence 1002 includes a plurality of 2D images parallel to a first plane. The illustration 1000 also shows a reassembled image sequence 1004, which includes a plurality of 2D images parallel to a landmark plane 1006, wherein the landmark plane 1006 is automatically determined as described above at operation 904 of the method 900. The reassembled image sequence 1004 is generated from the diagnostic image sequence 1002 by performing MPR on the diagnostic image sequence along the landmark plane 1006, as described above at operation 908. The diagnostic image sequence 1002, the reconstructed image sequence 1004, and the landmark planes 1006 are fed as inputs to a trained image enhancement network 1008, which maps the reconstructed image sequence to a resolution enhanced image sequence 1010 based on the diagnostic image sequence 1002 and the landmark planes 1006. The resolution enhanced image sequence 1010 may capture the same portion of the imaging volume as the reconstructed image sequence 1004, and may include the same number and orientation of image slices as the reconstructed image sequence 1004, however the resolution of the resolution enhanced image sequence 1010 is greater than the resolution of the reconstructed image sequence 1004, at least within the image planes including the resolution enhanced image sequence 1010.

[0093] refer to Fig.11, shows a second embodiment of a method 1100 for performing machine learning-assisted MPR of an image sequence using a trained image enhancement network. The method 1100 may use a trained image enhancement network (such as using Figure 7 The method 700 shown in FIG. 7 is used to train an image enhancement network (such as a Figure 2 The present invention may be performed by the imaging system 200 shown in the figure, wherein the training data is generated by one or more of the methods 300 and / or 500.

[0094] Method 1100 begins at operation 1102, where the imaging system acquires a three-plane locator image of the volume. A three-plane locator image is a 2D image that provides a three-dimensional view of the volume to be imaged. In some embodiments, the three-plane locator image is taken from an axial plane, a sagittal plane, and a coronal plane that are perpendicular to each other. The three-plane locator image can be used to plan a diagnostic scan by providing a comprehensive view of the volume to be imaged (e.g., the body of the imaged subject). The three-plane locator image can be used to identify the location of the region of interest and the size and shape of the region of interest. This information can then be used to determine the scanning parameters for the diagnostic scan, such as the scan type, the scan angle, the number of slices to be taken, and the slice thickness. The three-plane locator image can also be used to identify potential obstacles that may interfere with the scan, such as bones, organs, or implants.

[0095] At operation 1104, the imaging system determines a landmark plane from the three-plane locator image. In some embodiments, a landmark plane is determined based on the three-plane locator image of the volume using a landmark plane segmentation model, wherein the landmark plane intersects with a predetermined anatomical region. The landmark plane segmentation model may include a deep learning model configured to receive the three-plane locator image as an input feature and output one or more landmark planes (also referred to herein as planes of interest). In some embodiments, the landmark plane segmentation model includes one or more learned convolution filters. In some embodiments, at operation 1104, multiple landmark planes may be determined. As an example, a diagnostic scan may capture an anatomical region of interest, and operation 1104 may include determining multiple planes of a standard view that captures the anatomical region of interest, wherein the standard view may include more than one orientation of the anatomical region of interest (e.g., obtaining both a sagittal view and a coronal view of the same anatomical region of interest). In another example, a diagnostic scan may capture multiple anatomical regions of interest, wherein at operation 1104, the imaging system may determine at least one landmark plane for each anatomical region of interest, wherein the landmark plane captures the corresponding anatomical region of interest in one or more standard views.

[0096] At operation 1106, the imaging system acquires an image sequence based on the landmark plane. The image sequence includes a plurality of 2D images, wherein each of the plurality of 2D images is parallel to a first plane, and wherein the landmark plane is not parallel to the first plane. The image sequence acquired at operation 1106 may be referred to as a native image sequence because the plurality of 2D images comprising the image sequence have not been reassembled. In ordinary cases, the first plane (i.e., the orientation of the image of the diagnostic scan) is parallel to the landmark plane, and thus the image sequence may include an expected standard view of the anatomical region of interest, however, in most cases, the diagnostic imaging sequence is not parallel to the landmark plane, or if there are multiple non-parallel landmark planes, the imaging sequence may not be parallel to all landmark planes, and therefore a retrospective MPR is required. This is a non-ordinary case of retrospective MPR targeted by the present disclosure.

[0097] At operation 1108, the imaging system reassembles the image sequence along the landmark plane to produce a first reassembled image sequence. The first reassembled image sequence includes a plurality of 2D images, wherein each 2D image of the plurality of 2D images is parallel to the landmark plane. By reassembling the image sequence along the landmark plane, wherein the landmark plane intersects the anatomical region of interest at a predetermined orientation (e.g., to produce a standard view of the anatomical region), a desired standard view of the anatomical region of interest is included in the first reassembled image sequence. The reassembly process performed at operation 1108 may introduce artifacts (such as step edges or blurring) into the first reassembled image sequence.

[0098] At operation 1110, the imaging system perturbs the landmark plane to produce a perturbed landmark plane. The perturbation of the landmark plane may include a small (e.g., less than 5%) change in one or more cosine angles of the landmark plane to produce a perturbed landmark plane that is substantially similar to the landmark plane. In some embodiments, the imaging system may perturb the landmark plane by generating one or more random numbers within a predetermined range and adding the one or more random numbers to one or more plane parameters of the landmark plane to produce a perturbed set of plane parameters that define the perturbed landmark plane.

[0099] At operation 1112, the imaging system reassembles the first reassembled image sequence along the perturbed landmark plane to produce a second reassembled image sequence. The second reassembled image sequence includes a plurality of 2D images, wherein each of the plurality of 2D images is parallel to the perturbed landmark plane. By reassembling the first reassembled image sequence along the perturbed landmark plane, wherein the perturbed landmark plane is substantially similar to the landmark plane and thus intersects the anatomical region of interest within a threshold of a predetermined orientation (e.g., to produce a standard view of the anatomical region), a desired standard view of the anatomical region of interest is included in the second reassembled image sequence. The reassembly process performed at operation 1112 may not introduce artifacts that are substantially newer than artifacts in the first reassembled image sequence because the degree of perturbation is small relative to the landmark plane.

[0100] At operation 1114, the imaging system maps the second reorganized image sequence, the image sequence, and the landmark plane to the resolution enhanced image sequence using the trained image enhancement network. In some embodiments, the trained image enhancement network includes a plurality of learned three-dimensional (3D) convolution filters, wherein the plurality of learned 3D convolution filters receive spatially continuous data from spatially corresponding sub-regions of both the image sequence and the second reorganized image sequence or from a subsequently generated 3D feature map. The resolution enhanced image sequence includes a plurality of 2D images corresponding to the second reorganized image sequence, wherein each of the plurality of 2D images is parallel to the perturbed landmark plane. In some embodiments, there may be a one-to-one correspondence between the 2D images of the second reorganized image sequence and the resolution enhanced image sequence.

[0101] At operation 1116, the imaging system displays the resolution enhanced image sequence via a display device. In some embodiments, images of the resolution enhanced image sequence corresponding to the landmark planes may be displayed at operation 1116, thereby enabling a radiologist to quickly view the anatomical region of interest in a standard orientation, which may facilitate the assessment of the imaging subject. After operation 1116, method 1100 may end.

[0102] refer to Fig.12, an illustration 1200 of the method 1100 for performing machine learning-assisted MPR of an image sequence is shown. The illustration 1200 shows a diagnostic / native image sequence 1202 of an image volume, such as a diagnostic image sequence acquired at operation 1106 of the method 1100. The diagnostic image sequence 1202 includes a plurality of 2D images parallel to a first plane. The illustration 1200 also shows a second recombined image sequence 1204, the second recombined image sequence including a plurality of 2D images parallel to a perturbed landmark plane, wherein the perturbed landmark plane is automatically determined as described above at operation 1110 of the method 1100. The second recombined image sequence 1204 is generated by recombining the first recombined image sequence along a perturbed landmark plane, wherein the first recombined image sequence is generated by recombining the native image sequence 1202 along a landmark plane 1206, and wherein the perturbed landmark plane is substantially similar to the landmark plane 1206.

[0103] The native image sequence 1202, the second recombined image sequence 1204, and the landmark planes 1206 are fed as inputs to a trained image enhancement network 1208, which maps the second recombined image sequence 1204 to a resolution-enhanced image sequence 1210 based on the native image sequence 1202 and the landmark planes 1206. The resolution-enhanced image sequence 1210 may capture the same portion of the imaging volume as the second recombined image sequence 1204, and may include the same number and orientation of image slices as the second recombined image sequence 1204, however, the resolution of the resolution-enhanced image sequence 1210 is greater than the resolution of the recombined image sequence 1204, at least within the image planes including the resolution-enhanced image sequence 1210.

[0104] In this way, a diagnostic scan can be performed to acquire an image sequence comprising a first plurality of images in a first orientation (i.e., parallel to a first plane), and retrospective MPR can be used to generate one or more standard views of one or more anatomical regions of interest from the image sequence, wherein the one or more standard views of the one or more anatomical regions may have reduced visibility of blurring and reconstruction artifacts compared to conventional methods without imposing strict resolution constraints on the diagnostic scan, thereby enabling the diagnostic scan to be performed rapidly without sacrificing diagnostic image resolution of the anatomical region of interest.

[0105] The present disclosure also provides support for a method, the method comprising: determining a landmark plane intersecting a volume; acquiring an image sequence; reassembling the image sequence along the landmark plane to produce a first reassembled image sequence; perturbing the landmark plane to produce a perturbed landmark plane; reassembling the first reassembled image sequence along the perturbed landmark plane to produce a second reassembled image sequence; using a trained image enhancement network to map the second reassembled image sequence, the image sequence, and the landmark plane to a resolution enhanced image sequence; and displaying the resolution enhanced image sequence via a display device. In a first example of the method, a landmark plane is determined based on a three-plane locator image of the volume using a landmark plane segmentation model, and wherein the landmark plane intersects a predetermined anatomical region. In a second example of the method (optionally including the first example), the image sequence includes a plurality of two-dimensional (2D) images, wherein each of the plurality of 2D images is parallel to a first plane, and wherein the landmark plane is not parallel to the first plane. In a third example of the method (optionally including one or both of the first and second examples), the first reorganized image sequence includes a plurality of two-dimensional (2D) images, wherein each of the plurality of 2D images is parallel to the landmark plane. In a fourth example of the method (optionally including one or more or each of the first to third examples), the resolution-enhanced image sequence includes a plurality of two-dimensional (2D) images corresponding to the second reorganized image sequence, wherein each of the plurality of 2D images is parallel to the perturbed landmark plane. In a fifth example of the method (optionally including one or more or each of the first to fourth examples), the trained image enhancement network includes a plurality of learned convolution filters. In a sixth example of the method (optionally including one or more or each of the first to fifth examples), the plurality of learned convolution filters receive the second reorganized image sequence.

[0106] The present disclosure also provides support for a method for training an image enhancement network, the method comprising: selecting a training data pair including input data and a ground truth image sequence, wherein the input data includes: a landmark plane, a reorganized image sequence, and a native image sequence including a first plurality of images parallel to a first plane; and using the image enhancement network to map the input data to a predicted resolution-enhanced image sequence; determining a loss based on a difference between the predicted resolution-enhanced image sequence and the ground truth image sequence; and updating parameters of the image enhancement network based on the loss to produce a trained image enhancement network. In a first example of the method, the landmark plane intersects a predetermined anatomical region, and wherein the landmark plane is automatically determined using a landmark plane segmentation model. In a second example of the method (optionally including the first example), the method further includes: generating a training data pair by: receiving a native image sequence of a volume; selecting a landmark plane that intersects the volume; recombining the native image sequence along the landmark plane to produce a second image sequence including a second plurality of images parallel to the landmark plane; determining an inverse plane of the landmark plane; recombining the second image sequence along the inverse plane to produce a recombined image sequence including a third plurality of images parallel to the first plane; determining an overlapping region of the native image sequence and the recombined image sequence; storing the overlapping region as a ground truth image sequence of the training data pair; and storing the landmark plane, the recombined image sequence, and the native image sequence as input data of the training data pair. In a third example of the method (optionally including one or both of the first and second examples), determining the inverse plane of the landmark plane includes determining a set of transformations that map from the landmark plane to the first plane. In a fourth example of the method (optionally including one or more or each of the first to third examples), determining the overlapping area of ​​the native image sequence and the recombined image sequence includes masking the area of ​​the native image sequence outside the intersection between the foreground of the native image sequence and the foreground of the recombined image sequence. In a fifth example of the method (optionally including one or more or each of the first to fourth examples), the method also includes: generating a training data pair by the following steps: receiving a high-resolution native image sequence of the volume; downsampling the high-resolution native image sequence to produce a native image sequence; recombining the native image sequence along a landmark plane to produce a recombined image sequence including a second plurality of images parallel to the landmark plane; recombining the high-resolution native image sequence along the landmark plane to produce a ground truth image sequence; and storing the ground truth image sequence and input data including the landmark plane, the recombined image sequence, and the native image sequence as a training data pair.In a sixth example of the method (optionally including one or more or each of the first to fifth examples), downsampling the high-resolution native image sequence to generate the native image sequence further includes: adding noise to the high-resolution native image sequence. In a seventh example of the method (optionally including one or more or each of the first to sixth examples), the high-resolution native image sequence includes a third plurality of images, and wherein downsampling the high-resolution native image sequence to generate the native image sequence further includes: reducing the spatial resolution of each of the third plurality of images, and reducing the number of the third plurality of images.

[0107] The present disclosure also provides support for an imaging system, the imaging system comprising: an imaging device, a display device, and an image processing system, the image processing system being configured with a processor and executable instructions stored in a non-transitory memory, wherein when the instructions are executed, the processor causes the imaging system to: determine a landmark plane intersecting a volume; acquire an image sequence of the volume based on the landmark plane; reassemble the image sequence along the landmark plane to produce a first reassembled image sequence; map the first reassembled image sequence, the image sequence, and the landmark plane to a resolution-enhanced image sequence using a trained image enhancement network; and display the resolution-enhanced image sequence via a display device. In a first example of the system, a landmark plane segmentation model is used to determine a landmark plane based on a three-plane locator image of the volume acquired via the imaging device, and wherein the landmark plane intersects a predetermined anatomical region. In a second example of the system (optionally including the first example), the image sequence comprises a plurality of two-dimensional (2D) images, wherein each of the plurality of 2D images is parallel to a first plane, and wherein the landmark plane is not parallel to the first plane. In a third example of the system (optionally including one or both of the first and second examples), the first recombined image sequence includes a plurality of two-dimensional (2D) images, wherein each of the plurality of 2D images is parallel to the landmark plane. In a fourth example of the system (optionally including one or more or each of the first to third examples), the resolution-enhanced image sequence includes a plurality of two-dimensional (2D) images corresponding to the first recombined image sequence, wherein each of the plurality of 2D images is parallel to the landmark plane.

[0108] As used herein, the elements or steps listed in the singular and beginning with the word "one" or "an" should be understood as not excluding a plurality of said elements or steps, unless such exclusion is explicitly stated. In addition, the reference to "one embodiment" of the present invention is not intended to be interpreted as excluding the existence of additional embodiments that also include the cited features. In addition, unless explicitly stated to the contrary, "comprising", "including" or "having" an embodiment of an element or multiple elements with a specific characteristic may include additional such elements that do not have the characteristic. The terms "including" and "in..." are used as the concise language equivalents of the corresponding terms "comprising" and "wherein". In addition, the terms "first", "second" and "third" etc. are only used as marks, and are not intended to impose numerical requirements or specific positional order on their objects.

[0109] This written description uses examples to disclose the invention, including the best mode, and also to enable a person of ordinary skill in the relevant art to practice the invention, including making and using any devices or systems and performing any included methods. The patentable scope of the invention is defined by the claims, and may include other examples that occur to a person of ordinary skill in the art. Such other examples are intended to fall within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insignificant differences from the literal language of the claims.

Claims

1. A method, comprising: determining a landmark plane that intersects the volume; Get image sequence; recombining the image sequence along the landmark plane to generate a first recombined image sequence; perturbing the landmark plane to produce a perturbed landmark plane; recombining the first recombined image sequence along the perturbed landmark plane to generate a second recombined image sequence; mapping the second reorganized image sequence, the image sequence, and the landmark plane to a resolution enhanced image sequence using a trained image enhancement network; as well as The resolution enhanced image sequence is displayed via a display device. 2 . The method of claim 1 , wherein the landmark planes are determined based on the three-plane localizer image of the volume using a landmark plane segmentation model, and wherein the landmark planes intersect a predetermined anatomical region. 3 . The method of claim 1 , wherein the sequence of images comprises a plurality of two-dimensional (2D) images, wherein each of the plurality of 2D images is parallel to a first plane, and wherein the landmark plane is not parallel to the first plane. 4 . The method of claim 1 , wherein the first reorganized image sequence comprises a plurality of two-dimensional (2D) images, wherein each 2D image of the plurality of 2D images is parallel to the landmark plane.

5. The method of claim 1, wherein the resolution enhanced image sequence comprises a plurality of two-dimensional (2D) images corresponding to the second reorganized image sequence, wherein each 2D image of the plurality of 2D images is parallel to the perturbed landmark plane.

6. The method of claim 1, wherein the trained image enhancement network comprises a plurality of learned convolutional filters.

7. The method of claim 6, wherein the plurality of learned convolutional filters receive the second reorganized image sequence.

8. A method for training an image enhancement network, the method comprising: A training data pair comprising input data and a ground truth image sequence is selected, wherein the input data comprises: Landmark plane; the reassembled image sequence; and a native image sequence comprising a first plurality of images parallel to a first plane; and Mapping the input data to a predicted resolution-enhanced image sequence using the image enhancement network; determining a loss based on a difference between the predicted resolution enhanced image sequence and the ground truth image sequence; and Parameters of the image enhancement network are updated based on the loss to generate a trained image enhancement network.

9. The method of claim 8, wherein the landmark plane intersects a predetermined anatomical region, and wherein the landmark plane is automatically determined using a landmark plane segmentation model.

10. The method according to claim 8, further comprising: The training data pairs are generated by the following steps: receiving a sequence of native images of a volume; selecting the landmark plane that intersects the volume; recombining the native image sequence along the landmark plane to generate a second image sequence including a second plurality of images parallel to the landmark plane; determining an inverse plane of the landmark plane; recombining the second sequence of images along the inverse plane to produce the recombined sequence of images comprising a third plurality of images parallel to the first plane; Determining an overlapping area between the original image sequence and the recombined image sequence; storing the overlapping regions as the ground truth image sequences of the training data pairs, and The landmark plane, the reorganized image sequence, and the native image sequence are stored as the input data of the training data pair. The method of claim 10 , wherein determining the inverse plane of the landmark plane comprises determining a set of transformations that map from the landmark plane to the first plane.

12. The method of claim 10, wherein determining an overlapping area of ​​the native image sequence and the reassembled image sequence comprises masking an area of ​​the native image sequence outside of an intersection between a foreground of the native image sequence and a foreground of the reassembled image sequence.

13. The method according to claim 8, further comprising: Generate training data pairs by following the steps below: A high-resolution native image sequence of the receiving volume; Downsampling the high-resolution native image sequence to generate the native image sequence; recombining the native image sequence along the landmark plane to produce the recombined image sequence including a second plurality of images parallel to the landmark plane; recombining the high-resolution native image sequence along the landmark plane to generate the ground truth image sequence; as well as The ground truth image sequence and the input data including the landmark plane, the reorganized image sequence and the native image sequence are stored as the training data pairs.

14. The method according to claim 13, wherein downsampling the high-resolution native image sequence to generate the native image sequence further comprises: Noise is added to the high-resolution native image sequence.

15. The method of claim 13, wherein the high-resolution native image sequence comprises a third plurality of images, and wherein downsampling the high-resolution native image sequence to produce the native image sequence comprises: reducing the spatial resolution of each image in the third plurality of images; as well as The number of the third plurality of images is reduced.

16. An imaging system, comprising: Imaging equipment; Display devices; and An image processing system configured with a processor and executable instructions stored in a non-transitory memory, wherein when executing the instructions, the processor causes the imaging system to: determining a landmark plane that intersects the volume; acquiring a sequence of images of the volume based on the landmark planes; recombining the image sequence along the landmark plane to generate a first recombined image sequence; mapping the first reorganized image sequence, the image sequence, and the landmark plane to a resolution enhanced image sequence using a trained image enhancement network; as well as The resolution enhanced image sequence is displayed via the display device.

17. The imaging system of claim 16, wherein the landmark plane is determined based on a three-plane localizer image of the volume acquired via the imaging device using a landmark plane segmentation model, and wherein the landmark plane intersects a predetermined anatomical region.

18. The imaging system of claim 16, wherein the sequence of images comprises a plurality of two-dimensional (2D) images, wherein each of the plurality of 2D images is parallel to a first plane, and wherein the landmark plane is not parallel to the first plane.

19. The imaging system of claim 16, wherein the first recombined image sequence comprises a plurality of two-dimensional (2D) images, wherein each 2D image of the plurality of 2D images is parallel to the landmark plane.

20. The imaging system of claim 16, wherein the resolution enhanced image sequence comprises a plurality of two-dimensional (2D) images corresponding to the first recombined image sequence, wherein each 2D image of the plurality of 2D images is parallel to the landmark plane.

Images