Distortion artifact removal and enhancement in magnetic resonance imaging
By combining super-resolution neural networks and image filter modules, the problem of image distortion artifacts in magnetic resonance imaging is solved, thereby improving image quality and resolution, and making it suitable for magnetic resonance imaging systems.
Patent Information
- Application Number
- CN202380032551.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-02-08
- Filing Date
- 2023-02-02
- Publication Date
- 2024-11-26
AI Technical Summary
The presence of image distortion artifacts in magnetic resonance imaging leads to information loss and image quality degradation. Traditional methods struggle to improve image resolution while removing artifacts.
A super-resolution neural network combined with an image filter module is used to train the neural network to simultaneously remove image distortion artifacts and improve image resolution. Supervised learning methods and natural image training data are used to improve network performance.
It effectively removes image distortion artifacts and improves image quality, preserves the true information in the image, improves the resolution and signal-to-noise ratio of MRI images, and reduces the impact of noise.
Smart Images

Figure CN119032289A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to magnetic resonance imaging, and in particular to artifact removal of magnetic resonance images and enhancement of magnetic resonance images. Background Art
[0002] Magnetic resonance imaging (MRI) scanners use a large static magnetic field to align the nuclear spins of atoms as part of the procedure for producing images inside a patient's body. This large static magnetic field is called the B0 field or main magnetic field. MRI can be used to measure various quantities or properties of an object spatially and to image it. The difficulty in performing magnetic resonance imaging is that it takes time to acquire the k-space data necessary to reconstruct the magnetic resonance imaging. The longer the acquisition time, the more likely the object is to move or have some other problem. Another difficulty in performing magnetic resonance imaging is that there are various things that can cause spatial distortion of the magnetic resonance image.
[0003] U.S. Patent Application Publication US2020 / 0126190 A1 discloses selective denoising of medical images. In one method, one or more deep learning networks are trained to map a damaged image to a first type of artifact and a second type of artifact present in a corresponding damaged image. Then one or more trained learning networks are used to pick out the first type of artifact and the second type of artifact from a particular medical image. The first type of artifact is removed to a first degree, and the second type of artifact is removed to a second degree. The first and second degrees may be different. For example, one type of artifact may be completely suppressed, while another type of artifact may be partially removed from the medical image. Summary of the invention
[0004] The invention relates to a medical system, a method and a computer program product in the independent claims. Embodiments are given in the dependent claims.
[0005] Spatial distortions of magnetic resonance images contain data or describe objects. Embodiments of the present invention use super-resolution neural networks to simultaneously enhance magnetic resonance and remove or reduce image distortion artifacts. These two operations are complementary. Image distortion artifacts herein encompass image artifacts that contain information describing an object or a true or undistorted magnetic resonance image. If image distortion artifacts are removed or mitigated prior to enhancement, this information may be lost. Performing removal of image distortion artifacts while enhancing can provide better image distortion artifact removal and / or higher quality (more accurate) enhancement.
[0006] In one aspect, the present invention provides a medical system comprising a memory storing machine executable instructions and a super-resolution neural network. As used herein, a super-resolution neural network encompasses a neural network for image processing that receives an image having a first resolution and outputs a version of the image having an increased resolution (in other words, a second resolution higher than the first resolution). Such neural networks are typically used to enhance images, for example, to increase resolution for viewing video or other purposes.
[0007] The super-resolution neural network is configured to receive an initial magnetic resonance image describing an object. That is, there is an initial magnetic resonance image detecting at least a portion of the object. The initial magnetic resonance image has a first resolution and contains image distortion artifacts. As used herein, an image distortion artifact is an artifact in the magnetic resonance image that contains information about a true or undistorted image of the object. The term distortion means that there is information contained in the image distortion artifact.
[0008] The super-resolution neural network is configured to output an enhanced magnetic resonance image describing the object in response to receiving the initial magnetic resonance image. The enhanced magnetic resonance image has a second resolution and reduces or removes the image distortion artifacts. The second resolution is higher than the first resolution.
[0009] In this example, the super-resolution neural network is used in an unconventional manner. In addition to upscaling or improving the resolution of the initial magnetic resonance image, the super-resolution neural network is also used to perform a reduction or removal of the image distortion artifacts. This may be beneficial because image distortion artifacts contain information about the undistorted or true image of the object. Therefore, by using this information in the upscaling or super-resolution neural network, this information present in the image distortion artifacts can be used to perform upscaling or improving the resolution of the initial magnetic resonance image.
[0010] The medical system further includes a computing system. Execution of the machine executable instructions causes the computing system to first receive the initial magnetic resonance image and then receive an enhanced magnetic resonance image in response to inputting the initial magnetic resonance image into the super-resolution neural network.
[0011] As already mentioned before, this may be beneficial since image distortion artifacts may contain information describing the object, for example.If image filters or other techniques were to be applied to remove image distortion artifacts before performing the upscaling, this may result in an enhanced magnetic resonance image with lower image quality.
[0012] Neural network architectures that can be used to implement super-resolution neural networks include: VDSR (Very deep super-resolution network), SRResNet (Super-resolution residual network), SRDenseNet (Super-resolution densely connected network), EDSR (Enhanced Deep Super-Resolution Network), MDSR (Multi-Scale Deep Super-Resolution System), or other types of non-(fully) convolutional feed-forward neural networks.
[0013] Training of the super-resolution neural network can be straightforward. The magnetic resonance image at the second resolution can be obtained from a clinical study or a dedicated acquisition. The image is then downgraded to the first resolution and image distortion artifacts can be artificially added in image space or in k-space. The downgrading operation itself can be chosen so that it already introduces the distortion artifacts by itself. For example, cropping in k-space will degrade the image and introduce Gibbs ringing artifacts. The downgraded image with added artifacts and the original image at the second resolution can be used to train the neural network.
[0014] In another embodiment, the image distortion artifact is a Gibbs ringing image artifact.Combining the removal of Gibbs ringing artifacts with lifting may be highly beneficial and may produce better results than sequentially removing Gibbs ringing artifacts and then performing lifting.
[0015] Removal of Gibbs ringing artifacts is normally performed by using filtering and zero padding in k-space. The effect of this type of Gibbs ringing artifact removal is that the high frequency information in the image is severely attenuated. Lifting is essentially an interpolation process. Lifting an image that has had Gibbs ringing artifacts removed in this way results in a lifted image with reduced sharpness and loss of effective resolution.
[0016] This embodiment may be beneficial because it can provide an enhanced image that not only removes Gibbs ringing artifacts but also improves sharpness and image quality.
[0017] In another embodiment, the super-resolution neural network is trained according to a supervised learning method.
[0018] In another embodiment, the super-resolution neural network is trained using a supervised learning method, the supervised learning method comprising receiving a reference truth image set having the second resolution. The supervised learning method further comprises constructing a test image set from the reference truth image set. Constructing the test image set comprises: providing a downsampled image set by downsampling the reference truth image to the first resolution; providing a set of k-space representations of the downsampled image set by transforming the downsampled reference truth image set to k-space; providing a set of modified k-space representations by cropping or truncating the set of k-space representations of the downsampled image set; and providing the test image set by transforming the set of modified k-space representations to image space. The supervised learning method further comprises training the super-resolution neural network using the reference truth image set and the test image set.
[0019] In another embodiment, the supervised learning method includes using a reference truth image set that at least partially includes photographic images. This method may be particularly beneficial because non-medical images are used to train the super-resolution neural network. This enables a large number of reference truth images to be provided, thereby reducing the cost of training the super-resolution neural network. It may also have the effect of providing a super-resolution neural network that is better trained and does a better job in both removing Gibbs ringing artifacts and upscaling images from a first resolution to a second resolution. Another benefit of using natural or photographic images is that it enables the use of higher quality images. For example, photographic images may have a better signal-to-noise ratio SNR than magnetic resonance images. Therefore, it can cause the super-resolution neural network to perform better than a super-resolution neural network trained using magnetic resonance images.
[0020] In another embodiment, the image distortion artifact is any one of: truncation artifact, motion artifact, patient motion artifact, pulsation artifact, cardiac artifact (such as motion artifact), aliasing artifact, winding artifact, and Gibbs ringing image artifact. These artifacts may be beneficial because they contain information describing the object. Training a super-resolution neural network to remove or reduce these artifacts while improving or increasing the resolution can provide an enhanced magnetic resonance image with higher quality or better interpolation.
[0021] In another embodiment, the image distortion artifact is a halo artifact. A halo artifact is an image artifact in magnetic resonance images that is a sensitivity artifact that may appear in the presence of paramagnetic materials. It may also be beneficial to train the super-resolution neural network to remove the halo artifact because the halo artifact is seen around certain compounds such as hemosiderin or calcifications. It is also visible in the presence of metal or gas (e.g., due to air embolism). Halo refers to the fact that a lesion or object (e.g., medical or traumatic debris) may cause a halo artifact or a lesion to appear larger than it actually is. It may be beneficial to have the super-resolution neural network correct for it because it can make the lesion or object become the appropriate size during lifting.
[0022] In another embodiment, the image distortion artifacts comprise information describing the object. This may be beneficial because the information contained in the image distortion artifacts may be useful when performing an upscaling of the initial magnetic resonance image into the enhanced magnetic resonance image.
[0023] In another embodiment, the memory also stores an image filter module configured to remove random image errors from an image having a first resolution. Random image errors as used herein are image errors that occur due to random or stray measurements in k-space or in image space. For example, data or errors or random noise in k-space may result in regular or patterned images in a magnetic resonance image in image space. Although this may result in regular patterns in image space, it is still random in k-space. For example, an example of an image error would be noise in k-space or in image space. In this case, the random image error is unrelated to the image content and does not contain information describing the object.
[0024] Execution of the machine executable instructions further causes the computing system to remove random image errors from the initial magnetic resonance image by inputting the initial magnetic resonance image into the image filter module before inputting the initial magnetic resonance image into the super-resolution neural network.
[0025] In this embodiment, the image filter module is applied to the initial magnetic resonance image before the initial image magnetic resonance image is enhanced using the super resolution neural network. This is done in a particularly beneficial manner because the image filter module removes these random image errors, or in some cases noise, before the enhancement. This is effective because the random image errors have no association or connection with the true or undistorted image of the object. Therefore, these image artifacts or noise can be removed. The method is beneficial because it avoids undesirable enhancement of the random image errors by the super resolution neural network. Therefore, this embodiment may be beneficial because it can provide the enhanced magnetic resonance image of improved quality.
[0026] In another embodiment, the image filter module is an algorithmic image filter module. That is, the image filter module is not constructed using a neural network. There is an algorithm applied and used in the image filter module. Preferably, the image filter module is configured to denoise an image having a first resolution. For example, the algorithmic image filter module can be an algorithmic denoising filter.
[0027] Examples of algorithmic image filter modules include: linear smoothing filters, such as convolution with a Gaussian filter kernel; anisotropic diffusion; non-local means (NLM), other nonlinear filters such as median filtering; wavelet thresholding; 3D block matched filtering (BM3D).
[0028] In another embodiment, the denoising filter module is an image filtering neural network configured to remove the random image errors from an image having a first resolution. The image filtering neural network is a denoising neural network. For example, the image filtering neural network can be trained to remove noise from an image. The denoising filter module can be trained in a variety of ways. In one example, a magnetic resonance image having the first resolution (which does not have any noise or random image errors) can be artificially increased with noise or random image errors. The image with the artificial or random image artifacts added can be used as an input to the neural network, while the original image without errors can be used for deep learning. This can be done using a combination of many different images.
[0029] Training can be performed for various types of random image errors. For example, the image can be transformed into k-space, random image artifacts or noise can then be added to k-space, and then the image can be reconstructed or reconstructed in image space. Various types of neural networks can be used to implement the image filtering neural network. Most types of neural networks suitable for image processing can be used.
[0030] Examples of image filtering neural networks include: feed-forward convolutional neural networks, which may be unbiased or have skip connections, such as DnCNN, U-Net, ResNet, DenseNet, encoder-decoder networks (for denoising, encoder-decoder networks are simply called autoencoders).
[0031] In another embodiment, the super-resolution neural network is configured to use a predetermined super-resolution neural network architecture with increased capacity to achieve removal of magnetic resonance image artifacts and / or to achieve partial reconstruction of the enhanced magnetic resonance image using the image artifacts. For example, there are a relatively large number of neural network architectures that may be suitable for implementing a super-resolution neural network. The capacity may be increased in several different ways so as to directly train the super-resolution neural network so as to also perform reduction or removal of the image distortion artifacts.
[0032] In another embodiment, the increased capacity of the super-resolution neural network is increased by providing the predetermined super-resolution neural network architecture with an increased number of cores per convolutional layer. Additionally or alternatively, this may also be performed by providing the predetermined neural network architecture with an increased number of convolutional layers. Thus, in essence, a neural network architecture suitable for super-resolution or boosting and increasing the number of cores per convolutional layer and / or increasing the number of convolutional layers may be employed.
[0033] In another embodiment, the predetermined neural network includes neural network building blocks. The increased capacity is increased by providing the predetermined super-resolution neural network with additional repetitions of one or more of the neural network building blocks. For example, the neural network may have multiple structures within it. One way to increase capacity is to increase the number of these macro structures within the neural network architecture.
[0034] In another embodiment, the medical system further comprises a magnetic resonance imaging system. The memory further comprises pulse sequence commands, the pulse sequence commands being configured to control the magnetic resonance imaging system to acquire k-space data. The execution of the machine executable instructions further causes the computing system to acquire the k-space data by controlling the magnetic resonance imaging system using the pulse sequence commands. The execution of the machine executable instructions further causes the computing system to reconstruct the initial magnetic resonance image based on the k-space data.
[0035] In another embodiment, the reconstruction of the initial magnetic resonance image results in the image distortion artifact. For example, the image distortion artifact may be an image reconstruction artifact, such as curling or Gibbs ringing. This embodiment may be beneficial because the super-resolution neural network may be able to correct artifacts generated by the reconstruction of the k-space data into the initial magnetic resonance image in image space.
[0036] In another aspect, the present invention provides a method of medical imaging. The method includes receiving an initial magnetic resonance image. The method also includes receiving an enhanced magnetic resonance image in response to inputting the initial magnetic resonance image into a super-resolution neural network. The super-resolution neural network is configured to receive the initial magnetic resonance image describing the object. The initial magnetic resonance image has a first resolution and includes image distortion artifacts. The super-resolution neural network is configured to output an enhanced magnetic resonance image describing the object in response to receiving the initial magnetic resonance image. The enhanced magnetic resonance image has a second resolution and has a reduction or removal of the image distortion artifacts. The second resolution is higher than the first resolution.
[0037] In another aspect, the present invention provides a computer program product comprising machine executable instructions for execution by a computing system. The computing system may be used, for example, to control a medical system as described herein. Execution of the machine executable instructions causes the computing system to receive an initial magnetic resonance image describing an object. Execution of the machine executable instructions also causes the computing system to receive an enhanced magnetic resonance image in response to inputting the initial magnetic resonance image into a super-resolution neural network. The super-resolution neural network is configured to receive the initial magnetic resonance image. The initial magnetic resonance image has a first resolution and includes image distortion artifacts. The super-resolution neural network is configured to output an enhanced magnetic resonance image describing the object in response to receiving the initial magnetic resonance image. The enhanced magnetic resonance image has a second resolution and has a reduction or removal of the image distortion artifacts. The second resolution is higher than the first resolution.
[0038] The computer program product may also include a super-resolution neural network.
[0039] It should be understood that one or more of the aforementioned embodiments of the present invention may be combined as long as the combined embodiments are not mutually exclusive.
[0040] Those skilled in the art will recognize that aspects of the present invention may be embodied as devices, methods, or computer program products. Accordingly, aspects of the present invention may take the form of an embodiment that is hardware in its entirety, a software embodiment (including firmware, mainstream software, microcode, etc.) in its entirety, or a combination of software and hardware aspects, all of which may be collectively referred to herein as "circuits," "modules," or "systems." In addition, aspects of the present invention may take the form of a computer program product embedded with one or more computer-readable media (with computer executable code embedded thereon).
[0041] Any combination of one or more computer-readable media can be utilized. Computer-readable media can be computer-readable signal media or computer-readable storage media. "Computer-readable storage media" used herein covers any tangible storage media that can store instructions, which can be run by a processor or computing system of a computing device. Computer-readable storage media can be referred to as computer-readable non-temporary storage media. Computer-readable storage media can also be referred to as tangible computer-readable mechanisms. In some implementations, computer-readable storage media can also be capable of storing data, which can be accessed by a computing system of a computing device. Examples of computer-readable storage media include, but are not limited to: floppy disks, magnetic hard disk drives, solid-state hard disks, flash memory, USB thumb drives, random access memory (RAM), read-only memory (ROM), optical disks, magneto-optical disks, and register files of computing systems. Examples of optical disks include compact disks (CDs) and digital versatile disks (DVDs), such as CD-ROMs, CD-RWs, CD-Rs, DVD-ROMs, DVD-RWs, or DVD-R disks. The term computer-readable storage media also refers to various types of recording media that can be accessed by a computer device via a network or communication link. For example, data can be obtained over a modem, over the Internet, or over a local area network. Computer executable code embodied on a computer readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
[0042] A computer readable signal medium may include a propagated data signal having computer executable code embodied therein (e.g., in baseband or as part of a carrier wave). Such a propagated signal may take any of a variety of forms, including, but not limited to, electromagnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that is capable of communicating, propagating, or transmitting a program for use by an instruction execution system, device, or apparatus or in connection with an execution execution system, device, or apparatus.
[0043] "Computer memory" or "memory" is an example of a computer-readable storage medium. Computer memory is any memory directly accessible to a computing system. "Computer storage" or "storage" is another example of a computer-readable storage medium. Computer storage is any non-volatile computer-readable storage medium. In some implementations, computer storage may also be computer memory and vice versa.
[0044] As used herein, "computing system" encompasses an electronic component capable of running a program or computer executable instructions or computer executable code. Reference to a computing system (including examples of "computing system") should be interpreted as possibly including more than one computing system or processing core. A computing system may be, for example, a multi-core processor. A computing system may also refer to a collection of computing systems within a single computing system or distributed across multiple computer systems. The term computing system should also be interpreted as possibly referring to a collection or network of multiple computing devices, each of which includes a processor or computing system. Machine executable code or instructions may be run by multiple computing systems or processors, which may be within the same computing device or they may even be distributed across multiple computing devices.
[0045] Machine executable instructions or computer executable code may include instructions or programs that cause a processor or other computing system to perform an aspect of the present invention. Computer executable code for performing operations of various aspects of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages (e.g., Java, Smalltalk, C++, etc.) and traditional procedural programming languages (e.g., "C" programming languages or similar programming languages), and compiled into machine executable instructions. In some cases, the computer executable code may be in the form of a high-level language or in a precompiled form, and may be used in conjunction with an interpreter that generates machine executable instructions online. In other cases, the machine executable instructions or computer executable code may be in the form of programming for a programmable logic gate array.
[0046] The computer executable code may run entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (e.g., over the Internet using an Internet service provider).
[0047] Aspects of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present invention. It is to be understood that each frame or a portion of frames in a flowchart, diagram and / or block diagram can be implemented by computer program instructions in the form of computer executable code when applicable. It is also to be understood that, when mutually non-exclusive, the combination of frames in different flowcharts, diagrams and / or block diagrams can be combined. These computer program instructions can be provided to a computing system of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that the instructions run via the computing system of a computer or other programmable data processing device create a means for implementing the function / action specified in a flowchart and / or one or more block diagram frames.
[0048] These executable instructions or computer program instructions may also be stored in a computer-readable medium, which can direct a computer, other programmable data processing apparatus, or other device to function in a specific manner so that the instructions stored in the computer-readable medium produce an article of manufacture including instructions for implementing the functions / actions specified in the flowchart and / or one or more block diagram blocks.
[0049] Machine executable instructions or computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process, so that the instructions running on the computer or other programmable apparatus provide a process for implementing the functions / actions specified in the flowchart and / or one or more block diagram blocks.
[0050] "User interface" used herein is an interface that allows a user or operator to interact with a computer or computer system. "User interface" may also be referred to as "human-machine interface device". A user interface may provide information or data to an operator and / or receive information or data from an operator. A user interface may enable a computer to receive input from an operator, and may provide output to a user from a computer. In other words, a user interface may allow an operator to control or manipulate a computer, and an interface may allow a computer to indicate the effect of operator control or manipulation. The display of data or information on a display or graphical user interface is an example of providing information to an operator. The reception of data by a keyboard, mouse, trackball, touchpad, pointing stick, drawing board, joystick, game controller, webcam, earphone, pedal, wired gloves, remote controller, and accelerometer is an example of a user interface component that enables receiving information or data from an operator.
[0051] As used herein, "hardware interface" encompasses an interface that enables a computing system of a computer system to interact with and / or control an external computing device and / or apparatus. A hardware interface may allow a computing system to send a control signal or instruction to an external computing device and / or apparatus. A hardware interface may also enable a computing system to exchange data with an external computing device and / or apparatus. Examples of hardware interfaces include, but are not limited to: a universal serial bus, an IEEE 1394 port, a parallel port, an IEEE 1284 port, a serial port, an RS-232 port, an IEEE-488 port, a Bluetooth connection, a wireless LAN connection, a TCP / IP connection, an Ethernet connection, a control voltage interface, a MIDI interface, an analog input interface, and a digital input interface.
[0052] As used herein, a "display" or "display device" encompasses an output device or user interface suitable for displaying images or data. A display may output visual, auditory, and / or tactile data. Examples of displays include, but are not limited to, computer monitors, television screens, touch screens, tactile electronic displays, Braille screens, cathode ray tubes (CRTs), memory tubes, bi-stable displays, electronic paper, vectorscopes, flat panel displays, vacuum fluorescent displays (VFs), light emitting diode (LED) displays, electroluminescent displays (ELDs), plasma display panels (PDPs), liquid crystal displays (LCDs), organic light emitting diode displays (OLEDs), projectors, and head mounted displays.
[0053] K-space data is defined herein as the measurements of radio frequency signals emitted by atomic spins using the antenna of the magnetic resonance apparatus recorded during a magnetic resonance imaging scan.
[0054] A magnetic resonance imaging (MRI) image or magnetic resonance (MR) image is defined herein as a two-dimensional or three-dimensional visualization of the reconstruction of anatomical data contained within k-space data. The visualization may be performed using a computer. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] Preferred embodiments of the present invention will be described below by way of example only with reference to the accompanying drawings, in which:
[0056] Figure 1 The illustration shows an example of a medical system;
[0057] Figure 2 The diagram shows the use of Figure 1 A flowchart of a method of a medical system;
[0058] Figure 3 The illustration shows an example of a medical system;
[0059] Figure 4 The diagram shows the use of Figure 3 A flowchart of a method of a medical system; and
[0060] Figure 5 Illustration of the collaborative use of the image filter module and the super-resolution neural network.
[0061] Reference numerals
[0062] 100 Medical System
[0063] 102 Computer
[0064] 104 Computing Systems
[0065] 106 Hardware Interface
[0066] 108 User Interface
[0067] 110 Memory
[0068] 112 machine executable instructions
[0069] 114 Initial MRI images
[0070] 114' Magnetic resonance image with random image errors removed or reduced
[0071] 116 Image Filter Module
[0072] 118 Super-resolution neural network
[0073] 120 Enhanced MRI
[0074] 200 Receive initial MRI image
[0075] 202 removing random image errors from the initial magnetic resonance image by inputting the initial magnetic resonance image to an image filtering module before inputting the initial magnetic resonance image to the super-resolution neural network 204 receiving an enhanced magnetic resonance image in response to inputting the initial magnetic resonance image to the super-resolution neural network
[0076] 300 Medical System
[0077] 302 Magnetic Resonance Imaging System
[0078] 304 Magnet
[0079] 306 Magnet Cavity
[0080] 308 Imaging Area
[0081] 309 Field of View
[0082] 310 Magnetic Field Gradient Coil
[0083] 312 Magnetic Field Gradient Coil Power Supply
[0084] 314 RF Coil
[0085] 316 Transceiver
[0086] 318 objects
[0087] 320 Object Support
[0088] 332 k-space data
[0089] 400 collects k-space data by controlling the MRI system using pulse sequence commands
[0090] 402 Reconstructing an initial magnetic resonance image based on k-space data
[0091] 500 Noise or random image error DETAILED DESCRIPTION
[0092] The same reference numerals in these figures may be equivalent elements or perform the same function. Elements discussed previously may not necessarily be discussed in subsequent figures if the functions are equivalent.
[0093] Figure 1 An example of a medical system 100 is illustrated. The medical system 100 is shown to include a computer 102 having a computing system 104. For example, computer 102 is intended to represent one or more computers or computer systems that may be located in one or more locations. Computing system 104 is also intended to represent one or more computing systems 104 that may be in one or more locations. Computing system 104 is shown in communication with an optional hardware interface and an optional user interface 108. If there are other components of the medical system 100, the computing system 104 can communicate and control them using hardware interface 106. User interface 108 can provide a means for an operator to operate and control the medical system 100.
[0094] Computing system 104 is also shown in communication with memory 110. Memory 110 is intended to represent any type of memory or storage device with which computing system 104 may communicate. In some examples, memory 110 may be a non-transitory storage medium.
[0095] Memory 110 is shown as containing machine-executable instructions 112. Machine-executable instructions 112 enable computing system 104 to perform basic computing and data analysis tasks. Machine-executable instructions 112 may also enable computing system 104 to communicate with and / or control other components and devices.
[0096] The memory 110 is also shown as containing an initial magnetic resonance image 114 having a first resolution. The memory 110 is also shown as containing an optional image filter module 116, which may be configured to remove random image errors from the initial magnetic resonance image 114. The memory 110 is also shown as containing a super-resolution neural network 118, which is configured to receive the initial magnetic resonance image 114 and output an enhanced magnetic resonance image 120. The enhanced magnetic resonance image 120 has a second resolution greater than the first resolution. The enhanced magnetic resonance image 120 also has an image from which image distortion artifacts that were present in the initial magnetic resonance image 114 have been removed or reduced.
[0097] Figure 2 The diagram shows the operation Figure 1 FIG. 1 is a flow chart of a method of a medical system 100 of the present invention. First, in step 200, an initial magnetic resonance image 114 is received. Next, step 202 is optionally performed. In step 202, the initial magnetic resonance image 114 is input into an image filter module 116. The effect of doing so is to remove or reduce random image errors present in the initial magnetic resonance image 114. Next, in step 204, the initial magnetic resonance image is input into a super-resolution neural network 118. In response, the super-resolution neural network 118 outputs an enhanced magnetic resonance image 120.
[0098] Figure 3 Another example of a medical system 300 is shown. The medical system 300 is similar to Figure 1 , except that it additionally includes a magnetic resonance imaging system 302 controlled by the computing system 104 . Figure 3 The example illustrated in FIG. 3 is a magnetic resonance imaging system 302 .
[0099] The magnetic resonance imaging system 302 includes a magnet 304. The magnet 304 is a superconducting cylindrical magnet having a bore 306 passing through it. It is also possible to use different types of magnets; for example it is also possible to use both a split cylindrical magnet and a so-called open magnet. A split cylindrical magnet is similar to a standard cylindrical magnet, except that the cryostat is divided into two parts to allow access to the isoplane of the magnet, such a magnet can be used in conjunction with charged particle beam therapy, for example. An open magnet has two magnet sections, one above the other, with a space large enough in between to accommodate the subject: the arrangement of the two sections is similar to that of a Helmholtz coil. Open magnets are popular because the subject is less confined. Inside the cryostat of the cylindrical magnet there is a series of superconducting coils.
[0100] An imaging zone 308 is present within the bore 306 of the cylindrical magnet 304, wherein the magnetic field is strong and uniform enough to perform magnetic resonance imaging. A field of view 309 is shown within the imaging zone 308. The acquired k-space data is typically acquired for the field of view 309. The region of interest may be the same as the field of view 309 or it may be a sub-volume of the field of view 309. An object 318 is shown supported by an object support 320 such that at least a portion of the object 318 is within the imaging zone 308 and the field of view 309.
[0101] There is also a set of magnetic field gradient coils 310 in the bore 306 of the magnet, which is used to acquire preliminary k-space data to spatially encode magnetic spins in the imaging zone 308 of the magnet 304. The magnetic field gradient coils 310 are connected to a magnetic field gradient coil power supply 312. The magnetic field gradient coils 310 are intended to be representative. Typically, the magnetic field gradient coils 310 include three independent sets of coils for spatial encoding in three orthogonal spatial directions. The magnetic field gradient power supply supplies current to the magnetic field gradient coils. The current supplied to the magnetic field gradient coils 310 is controlled as a function of time and can be ramped or pulsed.
[0102] Adjacent to the imaging zone 308 is a radio frequency coil 314, which is used to manipulate the orientation of magnetic spins within the imaging zone 308 and to receive radio transmissions from spins also within the imaging zone 308. The radio frequency antenna may include multiple coil elements. The radio frequency antenna may also be referred to as a channel or antenna. The radio frequency coil 314 is connected to a radio frequency transceiver 316. The radio frequency coil 314 and the radio frequency transceiver 316 may be replaced by separate transmit and receive coils and separate transmitters and receivers. It should be understood that the radio frequency coil 314 and the radio frequency transceiver 316 are representative. The radio frequency coil 314 is intended to also represent a dedicated transmit antenna and a dedicated receive antenna. Similarly, the transceiver 316 may also represent a separate transmitter and receiver. The radio frequency coil 314 may also have multiple receive / transmit elements, and the radio frequency transceiver 316 may have multiple receive / transmit channels.
[0103] The transceiver 316 and the gradient controller 312 are shown connected to the hardware interface 106 of the computer system 102. These two components may provide the sensor data 126, as well as other components such as the object support that provides position data.
[0104] The memory is shown as containing pulse sequence commands 330. The pulse sequence commands 330 are instructions, or data that can be converted into instructions, that the computing system 104 can use to control the magnetic resonance imaging system 302 to acquire k-space data 332 from the field of view 309. The memory 110 is also shown as containing k-space data 332. The k-space data 332 can be reconstructed into the initial magnetic resonance image 114.
[0105] Figure 4The diagram shows the operation Figure 3 Flow chart of a method of a medical system 300 of the present invention. First, in step 400, a pulse sequence command 330 is used to control the magnetic resonance imaging system 302 to acquire k-space data 332. Next, in step 402, an initial magnetic resonance image 114 is reconstructed based on the k-space data 332. The method then proceeds to step 200, optional step 202, and step 204, as shown in FIG. Figure 2 Figure 2.1 shows the diagram and discusses it in the text.
[0106] Image quality (IQ) is an important aspect of MRI as it has a substantial impact on the radiologist's diagnostic confidence and reading time. Higher image quality (e.g., characterized by a high signal-to-noise ratio (SNR) and perceived image clarity) can potentially be achieved with longer acquisition procedures. For MRI, long acquisition times are very costly and are therefore often not practiced. Typically, fast acquisition techniques are used (e.g., using specialized sampling techniques), which often result in lower image quality. Noise, Gibbs ringing, and loss of resolution are common problems that are traditionally addressed using filtering and zero-filling in k-space followed by filtering in the spatial domain. However, zero-filling greatly limits the efficiency of subsequent filtering operations, and in general, the IQ improvements achieved are limited. In addition, classical anti-ringing filters in k-space suppress the measured high-frequency information at the expense of clarity and resolution.
[0107] As discussed herein, a denoising neural network (image filter module) is optionally used before the initial magnetic resonance image is input to a super-resolution neural network configured to remove or reduce image distortion artifacts.
[0108] Image quality improvement (IQ enhancement) for MRI is an important approach for imaging systems, since high quality images are required to derive reliable diagnoses. However, the long acquisition times and therefore high costs of MRI acquisitions have led to the widespread adoption of fast acquisition methods at the expense of IQ. Typically, images are affected by noise and Gibbs ringing artifacts, they have low resolution and low perceived clarity, resulting in reduced IQ. One way to improve IQ is to apply post-processing methods that remove artifacts and improve the resolution of the acquired images by better preserving the measured information.
[0109] The traditional way to improve IQ after acquisition is to use a series of processing steps to increase the number of voxels and remove artifacts. The number of voxels in the image in the spatial domain can be increased by applying zero padding to k-space before transforming the image into the spatial domain. Zero padding in k-space leads to interpolation and a larger number of voxels in the spatial domain, which results in a slightly improved IQ. However, the interpolation step changes the characteristics of the image, especially Gibbs ringing artifacts. These artifacts are difficult to remove after interpolation.
[0110] Furthermore, dedicated neural networks have been proposed for isolated problems such as MRI noise removal and MRI super-resolution. However, the chain of processing steps and potential interactions are not considered, and these methods usually require expensive, high-quality MRI scans for training. One method is described below:
[0111] Step 1) Train a NN (a possible implementation of an image filter module) to detect and predict noise (random image errors) in MR images. The NN can be trained on complex images or magnitude images. Furthermore, the training images do not necessarily have to be MR images, but other types of images can also be used, since image noise is specific to MRI. In the training procedure, the NN receives images that are believed to be artificially corrupted by noise and predicts the noise added to the image. The training data is obtained by adding noise to the image. The original image serves as the ground truth, while the noisy image is used as the input for training. Different types of noise can be used in this procedure. Specifically, by including undersampled MR scans during training, noise artifacts caused by undersampling techniques are taken into account.
[0112] Step 2) Train the NN for super-resolution and in order to remove image distortion artifacts such as Gibbs ringing. Once again, the NN can be trained on different types of images, although it is ultimately applied to MR. During training, the NN receives low-resolution images that are upsampled to a higher resolution. The training data can be generated by using, for example, high-resolution natural images that are downsampled by a certain factor. The high-resolution images serve as ground truth, while the downsampled images are shown to the network as input during training.
[0113] Figure 5 An example of generating an upsampled magnetic resonance image 120 is illustrated. Initially, an initial magnetic resonance image 114 is received. The initial magnetic resonance image 114 is then input into an optional image filter module 116, which outputs identified noise or random image errors 500. The random image errors 500 may be subtracted from the initial magnetic resonance image 114 to generate an initial magnetic resonance image 114' with the random image errors removed or reduced. The initial magnetic resonance image 114' is then input into a super-resolution neural network 118, which then outputs an enhanced magnetic resonance image 120. Figure 5 is an illustration of the application of the IQ improvement method. First, the NN predicts the noise in the image, which is then removed. Second, the NN predicts the residual ringing artifacts, which are also removed and upsampled to a higher resolution. The super-resolution network upsamples the image without random noise to a second resolution, while also removing image distortion artifacts such as Gibbs ringing.
[0114] The proposed IQ improvement method is very flexible and can be applied on any MR acquisition pipeline. The method operates only on reconstructed images. Therefore, after appropriate adjustments, the method can be applied to images where the reconstruction is performed based on full sampling or accelerated scans such as SENSE, Compressed SENSE or HyperSense scans. This means that both old and new systems can be equipped with the method to improve IQ.
[0115] In particular, low-cost MR systems that suffer more from image artifacts will benefit greatly from this approach. Furthermore, it should be noted that the computation time of the IQ boosting package is not too long and only results in a slight increase in the runtime, since the IQ boosting NN can be small.
[0116] For the design and application of the described invention, examples may include one or more of the following features:
[0117] - In one example, one of the neural networks can be omitted or replaced with an alternative technique. For example, denoising can be performed using a conventional filtering system.
[0118] - Another example is the combination of some of the neural networks. The training of separate networks for different artifacts can be merged into a single network.
[0119] While the invention has been illustrated and described in the foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments.
[0120] Other variations of the disclosed embodiments may be understood and implemented by those skilled in the art in practicing the claimed invention, based on a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or other unit may implement the functions of multiple items recited in the claims. The mere fact that specific measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be utilized. The computer program may be stored / distributed on a suitable medium, such as an optical storage medium or solid-state medium provided together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems. Any figure marks in the claims should not be interpreted as limiting the scope.
Claims
1. A medical system (100, 300), comprising: a memory (110) storing machine executable instructions (116) and a super resolution neural network (118), wherein the super resolution neural network is configured to receive an initial magnetic resonance image (114, 114') depicting an object (318), wherein the initial magnetic resonance image has a first resolution and includes image distortion artifacts, wherein the image distortion artifacts are Gibbs ringing image artifacts, wherein the super resolution neural network is configured to output an enhanced magnetic resonance image depicting the object in response to receiving the initial magnetic resonance image, wherein the enhanced magnetic resonance image has a second resolution and has the image distortion artifacts reduced or removed, wherein the second resolution is higher than the first resolution; - a computing system (104), wherein execution of the machine-executable instructions causes the computing system to: - receiving (200) said initial magnetic resonance image; and - receiving (204) the enhanced magnetic resonance image in response to inputting the initial magnetic resonance image to the super-resolution neural network.
2. The medical system of claim 1, wherein: The super-resolution neural network is configured to use information contained in the image distortion artifacts to partially reconstruct the enhanced magnetic resonance image.
3. The medical system according to claim 1 or 2, wherein: The image distortion artifacts include information describing the object.
4. A medical system as claimed in any one of the preceding claims, wherein: The memory also stores an image filter module (116) configured to remove random image errors (500) from the image having the first resolution, wherein execution of the machine executable instructions further causes the computing system to remove (202) random image errors from the initial magnetic resonance image by inputting the initial magnetic resonance image to the image filter module before inputting the initial magnetic resonance image to the super-resolution neural network.
5. The medical system of claim 4, wherein: The image filter module is an algorithmic image filter module, preferably, the image filter module is configured to denoise the image having the first resolution.
6. The medical system of claim 5, wherein: The denoising filter module is an image filtering neural network, which is configured to remove the random image errors from the image with the first resolution. Preferably, the image filtering neural network is a denoising neural network.
7. A medical system as claimed in any one of the preceding claims, wherein: The medical system further comprises a magnetic resonance imaging system, wherein the memory further comprises pulse sequence commands configured to control the magnetic resonance imaging system to acquire k-space data (332), wherein execution of the machine executable instructions further causes the computing system to: - acquiring (400) the k-space data by controlling the magnetic resonance imaging system using the pulse sequence commands; and - reconstructing (402) the initial magnetic resonance image from the k-space data.
8. The medical system of claim 7, wherein: The reconstruction of the initial magnetic resonance image results in image distortion artifacts.
9. A medical system as claimed in any one of the preceding claims, wherein: The super-resolution neural network is trained according to a supervised learning method.
10. The medical system of any one of claims 1 to 8, wherein: The super-resolution neural network is trained using a supervised learning method, which includes: - receiving a set of ground truth images having said second resolution; - constructing a test image set according to the reference true value image set, wherein constructing the test image set comprises: - providing a set of downsampled images by downsampling the reference truth images to the first resolution; - providing a set of k-space representations of the downsampled image set by transforming the downsampled ground truth image set into k-space; - providing a set of modified k-space representations by cropping or truncating the set of k-space representations of the downsampled image set; and - providing said set of trial images by transforming said set of modified k-space representations into image space; and -Training the super-resolution neural network using the benchmark true image set and the test image set.
11. The method of claim 10, wherein: The reference truth image set includes photographic images.
12. A method of medical imaging, the method comprising: - receiving (200) an initial magnetic resonance image (114, 114'); as well as - receiving (204) an enhanced magnetic resonance image (120) in response to inputting the initial magnetic resonance image to a super-resolution neural network (118), wherein the super-resolution neural network is configured to receive the initial magnetic resonance image depicting an object (318), wherein the initial magnetic resonance image has a first resolution and includes image distortion artifacts, wherein the image distortion artifacts are Gibbs ringing image artifacts, wherein the super-resolution neural network is configured to output an enhanced magnetic resonance image depicting the object in response to receiving the initial magnetic resonance image, wherein the enhanced magnetic resonance image has a second resolution and has the image distortion artifacts reduced or removed, wherein the second resolution is higher than the first resolution.
13. The method of claim 12, wherein: The super-resolution neural network is trained according to a supervised learning method.
14. The method of claim 12, wherein: The super-resolution neural network is trained using a supervised learning method, which includes: - receiving a set of ground truth images having said second resolution; - constructing a test image set according to the reference true value image set, wherein constructing the test image set comprises: - providing a set of downsampled images by downsampling the reference truth images to the first resolution; - providing a set of k-space representations of the downsampled image set by transforming the downsampled ground truth image set into k-space; - providing a set of modified k-space representations by cropping or truncating the set of k-space representations of the downsampled image set; and - providing said set of trial images by transforming said set of modified k-space representations into image space; and -Training the super-resolution neural network using the benchmark true image set and the test image set.
15. The method of claim 14, wherein: The reference truth image set includes photographic images.
16. A computer program product comprising machine executable instructions (112) for execution by a computing system, wherein: Execution of the machine-executable instructions causes the computing system to: - receiving (200) an initial magnetic resonance image (114, 114') depicting an object (318); and - receiving (204) an enhanced magnetic resonance image (120) in response to inputting the initial magnetic resonance image to a super-resolution neural network (118), wherein the super-resolution neural network is configured to receive the initial magnetic resonance image, wherein the initial magnetic resonance image has a first resolution and includes image distortion artifacts, wherein the image distortion artifacts are Gibbs ringing image artifacts, wherein the super-resolution neural network is configured to output the enhanced magnetic resonance image depicting the object in response to receiving the initial magnetic resonance image, wherein the enhanced magnetic resonance image has a second resolution and has the image distortion artifacts reduced or removed, wherein the second resolution is higher than the first resolution.
17. The method of claim 16, wherein: The super-resolution neural network is trained according to a supervised learning method.
18. The method of claim 16, wherein: The super-resolution neural network is trained using a supervised learning method, which includes: - receiving a set of ground truth images having said second resolution; - constructing a test image set according to the reference true value image set, wherein constructing the test image set comprises: - providing a set of downsampled images by downsampling the reference truth images to the first resolution; - providing a set of k-space representations of the downsampled image set by transforming the downsampled ground truth image set into k-space; - providing a set of modified k-space representations by cropping or truncating the set of k-space representations of the downsampled image set; and - providing said set of trial images by transforming said set of modified k-space representations into image space; and - Using the benchmark truth image set and the test image set to train the super-resolution neural network.
19. The method of claim 18, wherein: The reference truth image set includes photographic images.
