Training data for 3D image cleaning ML models

Abstract

Disclosed herein is a method for generating training data for a machine learning model. The machine learning model is configured to improve the quality of acquired 3D medical images. The method includes selecting a video having desired values ​​of one or more image parameters and preprocessing the video of a dataset. The preprocessing includes downsampling the video by a predetermined reduction factor, cropping the downsampled video in space and / or time to obtain video patches, modifying the clean video patches to reduce their quality, and generating a training dataset having the modified video patches as inputs and the associated clean video patches as learning targets.

Description

[Technical Field]

[0001] The present invention relates to magnetic resonance imaging, and more particularly to generating and using data for training machine learning models for cleaning 3D medical image data. [Background technology]

[0002] The selection of training data for healthcare artificial intelligence (AI) models is a fundamental and important part of the entire workflow. Summary of the Invention [Problem to be solved by the invention]

[0003] However, training AI models on magnetic resonance (MR) scans can be problematic as the training data provided may not be suitable for obtaining an improved model.

[0004] The present invention provides a medical system, a computer program, a data structure and a method in the independent claims. Embodiments are set out in the dependent claims. [Means for solving the problem]

[0005] Training AI models on MR scans can be problematic because the amount of training data is small and the data quality and representation can be poor. This is because acquiring high-quality MR data can be difficult, time-consuming, and sometimes impossible. Embodiments can use natural, high-quality video as a training data source to train three-dimensional (3D) AI image quality improvement models for 3D MR scans. The advantage of this approach can be the diversity of the data. The video can be of high quality without compression, noise, or interpolation artifacts and can serve as good reference data. To enable the use of such data for 3D image quality improvement, special preprocessing and data augmentation techniques can be provided. This approach can be advantageous for denoising, deringing, and super-resolution tasks of MR scans in Image Quality (IQ)Boost3D and can be extended to other modalities and tasks.

[0006] In one aspect, the present invention provides a medical system having a memory storing machine-executable instructions and a computing system, where execution of the machine-executable instructions causes the computing system to perform the steps of receiving video from one or more sources, selecting from the received video a video having desired values ​​of one or more image parameters, generating a dataset (initial dataset) including the selected video, and preprocessing the video of the dataset. The preprocessing includes downsampling the video by a predetermined reduction factor, cropping the downsampled video in space and / or time to obtain video patches, referred to herein as clean video patches, and modifying the clean video patches to reduce their quality. A training dataset can be generated. The training dataset has the modified video patches as input and the associated clean video patches as learning targets.

[0007] The video can be downsampled, for example, along the spatial dimension. Alternatively, the video can be downsampled along the time dimension. This can be particularly advantageous in the case of slow-motion video.

[0008] The machine learning model can be configured to improve the quality of acquired 3D image data (e.g., 3D MR image data). The quality of the image data can include the level of noise in the image data, the resolution of the image data, the number of artifacts in the image data, etc. The enhancement can include noise removal, ringing removal, super-resolution, motion artifact reduction, ghost artifact removal, or undersampling artifact removal. The machine learning model can be, for example, a fully convolutional neural network with a 3D kernel, a ResNet, a DenseNet, an EfficientNet, or a vision transformer with a kernel tuned for 3D processing.

[0009] The video can be received from one or more sources. The video can be treated as a three-dimensional volume, with the third dimension being time. Sources can include web sources, such as social media websites or public databases with appropriate licensing policies. The received video can represent a non-medical field. For example, the received video does not include medical images, such as magnetic resonance (MR) images. For example, the received video can be natural high-quality videos (NV). Training on natural videos can increase the generalizability of machine learning models. Machine learning models can learn more valuable features from non-MRI content. Using the received video, the machine learning model can avoid overfitting to any anatomical structures, which can therefore increase confidence that the anatomical structures remain unchanged during inference. Furthermore, received videos, such as NV, can serve as a good baseline for machine learning models because a large volume is available, many of which may exist in high resolution. Additionally, the received video may be easier to obtain for research purposes because it poses fewer risks of data privacy issues compared to medical data. Machine learning models can be trained using many high-quality, non-medical examples, allowing them to perform well on data from other fields during inference. Furthermore, using videos from non-medical fields may be advantageous for the following reasons: Acquiring high-quality medical images (e.g., 3D MR images) for training AI models can be very expensive and sometimes impossible. For example, it may be impossible to acquire completely noise-free medical images. Using multiple medical image acquisitions and averaging can reduce noise to some extent. However, in the case of in vivo scans, this can result in subject motion and therefore degrade image quality. This video can overcome these issues associated with 3D MR images.

[0010] The present subject matter can further improve the training of a machine learning model based on the received video by selecting relevant videos, downsampling the selected videos, and generating video patches from the downsampled videos. The downsampling step and the subsequent cropping step can form a preprocessing (or processing) step.

[0011] Downsampling of a video can be performed along the spatial dimension of the video or along the temporal dimension of the video. Downsampling of a video can include, for example, downsampling each frame of the video to a smaller size. For example, a frame can have an initial size X0×Y0 and can be downsampled to a size of X1×Y1, where X1 < X0 and Y1 < Y0. This saves the memory space required to store each frame of the video and can provide 3D image data having a quality better than or at least similar to the quality of the acquired medical image data. Another purpose of downsampling can be to match the scale of the image content to the image content such as MRI. For example, a 50×50 patch cropped from a 4k resolution image can contain less structure and content than a 50×50 crop from a full HD resolution image. Therefore, the downsampling factor can be selected such that the resulting image content and scale are similar to the MRI image content.

[0012] Each downsampled video can be cropped. This may result in a set of video patches, called a set of clean video patches. The video patches can be randomly cropped from each downsampled video. A video patch is a video. A video patch can be completely contained within its source video. Video patches obtained from the same video may or may not overlap. In one example, the overlap between video patches obtained from the same video may not exceed 20%. This may allow for covering the entire 3D space occupied by the video and therefore all features in the video. For example, three different types of video patches can be obtained from each downsampled video of the dataset. Spatial video patches can be obtained by spatially cropping within a given video such that the spatial video patches have the same duration as the given video but are cropped to, for example, 30% of the spatial dimension. Temporal video patches can be obtained by temporally cropping a given video such that the temporal video patches have the same spatial size as the given video but are cropped to, for example, 30% of the duration. Spatio-temporal video patches can be obtained by cropping a given video in time and space, such that cropping is performed along all three dimensions of the given video, for example, by 30%. Video patches can augment training data and capture local quality features of the video. This can improve medical image cleaning, as patch-based training can result in more generalizable AI models.

[0013] Thus, cropping the downsampled video can provide a set of clean video patches. Due to their improved quality, these clean video patches can be advantageously used as targets or labels for training a machine learning model. Inputs for training the machine learning model can be obtained by modifying the set of clean video patches. The preprocessing can further include a modification step. The modification of the set of clean video patches can result in a set of modified video patches associated with each clean video patch. The present subject matter can modify the video patches based on the purpose of the machine learning model. For example, if the machine learning model is for denoising medical images, the modification can be performed by adding noise to the video patches. A training dataset can be generated using the set of modified video patches as input and the associated set of clean video patches as targets.

[0014] The present subject matter may be advantageous because it may enable improved image quality, for example, by more accurately denoising images using a machine learning model after being trained using the present training dataset. That is, when used, the present training dataset may enable improved cleaning of 3D medical images. The present subject matter may provide a desired amount of reliable training dataset without relying on medical images. Such medical images may be time-consuming to acquire and may not provide the very high-quality medical images that may be required for training. Cleaning of 3D medical images may include, for example, noise removal, super-resolution, sharpening, motion artifact reduction, ghost artifact removal, or undersampling artifact removal.

[0015] The present subject matter avoids the use of 2D AI image quality improvement models and can apply them to 3D MRI scans on a slice-by-slice basis. Indeed, due to the very small slice thickness of 3D MRI scans, applying a 2D model slice-by-slice can result in inconsistencies between slices, thereby reducing quality and degrading the scan. Therefore, an AI image quality improvement model for a 3D MRI acquisition may require a 3D model trained on the 3D data, as provided by the present subject matter.

[0016] According to one embodiment, execution of the machine-executable instructions causes the computing system to permute dimensions within selected video patches of the set of clean video patches before modifying the set of clean video patches. For example, before modifying the set of clean video patches, one or more subsets of video patches may be randomly selected from the set of clean video patches. The dimensions of each selected video patch may be permuted. The resulting set of clean video patches may include permuted and unpermuted video patches. The resulting set of clean video patches may be modified. This may result in a set of modified video patches each associated with the set of clean video patches. This embodiment may allow for mixing dimensions such that the machine learning model is not biased toward the time dimension. This may also allow for all data in different dimensions to be treated equally.

[0017] According to one embodiment, execution of the machine-executable instructions causes the computing system to flip (flip left-to-right or up-to-down) selected video patches from the set of clean video patches before modifying the set of clean video patches. For example, one or more subsets of video patches can be randomly selected from the set of clean video patches before modifying the set of clean video patches. Each selected video patch can be flipped. The resulting set of clean video patches includes flipped and non-flipped video patches. The resulting set of clean video patches can be modified.

[0018] Random permuting and / or flipping (dimensions) can prevent machine learning models from overfitting to the characteristics of the time dimension. Permuting and / or flipping can make the video more similar to 3D MR data characteristics, where each image dimension has similar characteristics.

[0019] In one example, execution of the machine-executable instructions causes the computing system to both permute and flip dimensions of a subset of selected video patches of the set of clean video patches before modifying the resulting set of clean video patches. Alternatively, execution of the machine-executable instructions causes the computing system to permute dimensions of a first subset of selected video patches of the set of clean video patches and flip a different second subset of selected video patches of the set of clean video patches before modifying the resulting set of clean video patches. These examples can further improve the data representation of the training dataset.

[0020] According to one embodiment, each video in the initial dataset can have at least one of the following characteristics: a resolution higher than the minimum resolution, missing artifacts from compression or interpolation, a noise amount smaller than the maximum amount, and a gradient variation greater than the minimum amount. For example, this embodiment can define the following image parameters: resolution, number of artifacts, amount of noise, and gradient in each direction. The target (or desired) value for the resolution parameter may be a resolution higher than the minimum resolution. The target value for the number of artifacts may be zero. The target value for the noise amount may be a noise amount smaller than the maximum amount. The gradient variation for each direction is greater than the minimum amount. The last point means that the video should contain some kind of motion and not contain still images over time. This initial selection of videos may enable obtaining a reliable training dataset for accurate training of machine learning models. This may enable using such trained models to improve medical image cleaning. Cleaning may include, for example, noise removal, super-resolution, sharpening, motion artifact reduction, ghost artifact removal, or undersampling artifact removal.

[0021] According to one embodiment, execution of the machine-executable instructions causes the computing system to modify the video patch by adding random Gaussian noise to the video patch or by reducing the resolution of the video patch. Resolution reduction can be performed, for example, by applying a crop in k-space of the video patch while preserving a certain percentage of the k-space information. K-space cropping of the video patch can be performed to obtain a correspondence between the k-space characteristics of the acquired MR data and the video. For example, k-space cropping can enable mapping between data intervals in k-space and the cropped video patch. Applying a k-space crop can result in low-resolution video and ringing artifacts in the video, which serves as input to a machine learning model. Resolution reduction can also be performed by downsampling and interpolation (e.g., using b-splines).

[0022] This modification is made based on the purpose of the machine learning model. As an example, if the machine learning model is a 3D AI denoiser, the machine learning model can be trained by adding random Gaussian noise to the preprocessed video, which is then used as input to the machine learning model. The learning target is the clean, noise-free, preprocessed video. In this case, training can be performed using standard gradient descent. In another example, if the machine learning model is a 3D AI de-ringing and super-resolution model, the model can be trained by applying a k-space crop to the preprocessed video, resulting in a low-resolution video with ringing artifacts that serves as input to the model. The learning target is the clean, preprocessed video.

[0023] According to one embodiment, execution of the machine-executable instructions causes a computing system to randomly perform cropping such that each video patch has the same size but smaller than a maximum size. This embodiment can handle huge volume sizes in a systematic way using small but fixed patch sizes. Random cropping can prevent machine learning models from overfitting to specific features.

[0024] According to one embodiment, execution of the machine-executable instructions causes the computing system to apply an interpolator to downsample the video, which may allow maintaining good quality of the video after downsampling. The interpolator may be, for example, a high-quality interpolator such as Lanczos or other high-quality interpolator that can be used to downsample received video.

[0025] According to one embodiment, execution of the machine-executable instructions causes the computing system to augment the dataset by resampling each video in the initial dataset to obtain different versions of the video, where each version of the video has a different size. The obtained videos are added to the initial dataset before the videos in the dataset are preprocessed. This may further increase the size and diversity of the training dataset, thus enabling accurate medical image cleaning. Cleaning may include, for example, noise removal, super-resolution, sharpening, motion artifact reduction, ghost artifact removal, or undersampling artifact removal.

[0026] According to one embodiment, the reduction factor is defined according to the resolution of the medical images used by the machine learning model. For example, the video can be downsampled to a size that is the size of the 3D medical images used by the machine learning model during inference. The 3D medical images can be, for example, MRI images or computed tomography (CT) images. For example, the reduction factor can be 2, 3, or 4. These values ​​can be, for example, user-defined or can be empirically determined values ​​that provide the most optimal downsampling results. Having a predetermined fixed reduction factor allows for systematic downsampling of the video and can save resources that may be required to calculate them dynamically.

[0027] According to one embodiment, execution of the machine-executable instructions causes a computing system to train a machine learning model using a training dataset. In one example, the trained machine learning model can be inferred to improve the quality of an input 3D medical image. The input 3D medical image can be an MR image, an ultrasound image, or a CT image.

[0028] According to one embodiment, the medical system further comprises a magnetic resonance imaging system, wherein the memory comprises pulse sequence commands configured to control the magnetic resonance imaging system to acquire k-space data according to a magnetic resonance imaging protocol, and execution of the machine-executable instructions causes the computing system to control the magnetic resonance imaging system to acquire k-space data, reconstruct a 3D magnetic resonance image from the k-space data, and use the trained model to improve the quality of the 3D magnetic resonance image. Use of the trained machine learning model includes inferring the machine learning model using the reconstructed 3D MR image, which can enable a reliable and accurate MR imaging system.

[0029] While natural images (e.g., received video) may not have complex parts, MR scans do. To overcome this during inference, the real and imaginary parts of the MR scan are treated and processed by the model as individual images, and both outputs are finally combined to obtain the result.

[0030] According to one embodiment, the received video is non-medical video. This may be advantageous compared to medical images for the following reasons: The received video is descriptive, high-quality, and can provide a sufficient amount of training data. However, in the medical field of MRI, existing MRI datasets may lack quality and quantity, and are usually biased. For example, there are no scanners capable of collecting data at 4K resolution, so high-resolution MRI data may not exist at all. MR scans suffer from noise and Gibbs ringing artifacts, resulting in poor image quality due to low resolution and poor perceived clarity. Training on MRI data may also be unsafe because models learn anatomical structures (e.g., brain / knee structures) and can be performed on anatomical structures during inference, resulting in unexpected core changes. These issues can be overcome by using videos such as natural video (NV). The resolution of the data is higher, therefore, more information is contained in a single data set. The amount of data may be 10 to 100 times greater than MRI data. Natural video (NV) can include all possible structures and geometries, not just those specific to MRI, and therefore provides much more descriptive data.

[0031] In another aspect, the present invention relates to a computer program having machine-executable instructions for execution by a computing system, wherein execution of the machine-executable instructions causes the computing system to perform the following steps: receiving video from one or more sources; selecting from the received video a video having desired values ​​of one or more image parameters; generating a dataset having the selected video; preprocessing the video of the dataset, the preprocessing including downsampling the video by a predetermined reduction factor; cropping the downsampled video in space and / or time to obtain video patches (referred to herein as clean video patches); and modifying the clean video patches to reduce their quality; and generating a training dataset having the modified video patches as input and the associated clean video patches as learning targets. The video can be downsampled, for example, along the spatial dimension of the video or along the temporal dimension of the video.

[0032] In another aspect, the present invention relates to a method for generating training data for a machine learning model configured to improve the quality of acquired 3D medical images. The method includes receiving video from one or more sources, selecting from the received video a video having desired values ​​of one or more image parameters, generating a dataset including the selected video, preprocessing the video of the dataset, the preprocessing including downsampling the video by a predetermined reduction factor, cropping the downsampled video in space and / or time to obtain video patches, referred to herein as clean video patches, and modifying the clean video patches to reduce their quality, and generating a training dataset having the modified video patches as inputs and the associated clean video patches as learning targets. The video can be downsampled, for example, along the spatial dimension of the video or along the temporal dimension of the video.

[0033] In another aspect, the present invention relates to a computer-implemented data structure for training a machine learning model for cleaning 3D medical images, the data structure having modified video patches and associated clean video patches, the modified video patches having modifications obtained according to a cleaning objective of the machine learning model. Cleaning the 3D medical images can include, for example, noise removal, super-resolution, sharpening, motion artifact reduction, ghost artifact removal, or undersampling artifact removal from the 3D medical images.

[0034] It should be understood that one or more of the above-described embodiments of the invention may be combined as long as the combined embodiments are not mutually exclusive. As will be appreciated by those skilled in the art, aspects of the invention may be embodied as an apparatus, a method, or a computer program product. Accordingly, aspects of the invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, all of which may be referred to generally as a "circuit," "module," or "system" herein. Furthermore, aspects of the invention may take the form of a computer program product embodied in one or more computer-readable medium(s) having computer-executable code embodied thereon.

[0035] Any combination of one or more computer-readable media may be utilized. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. As used herein, "computer-readable storage medium" encompasses any tangible storage medium capable of storing instructions executable by a processor or computing system of a computing device. The computer-readable storage medium may also be referred to as a computer-readable non-transitory storage medium. The computer-readable storage medium may also be referred to as a tangible computer-readable medium. In some embodiments, the computer-readable storage medium may also store data accessible by the computing system of a computing device. Examples of computer-readable storage media include, but are not limited to, floppy disks, magnetic hard disk drives, solid-state hard disks, flash memory, USB thumb drives, random access memory, read-only memory (ROM), optical disks, magneto-optical disks, and computing system register files. Examples of optical disks include compact discs (CDs) and digital versatile discs (DVDs), such as CD-ROM, CD-RW, CD-R, DVD-ROM, DVD-RW, or DVD-R discs. The term computer-readable storage medium also refers to various types of storage media that can be accessed by a computer device over a network or communications link. For example, data can be retrieved via a modem, over the Internet, or over a local area network. Computer-executable code embodied on a computer-readable medium can be transmitted using any suitable medium, including, but not limited to, wireless, wired, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

[0036] A computer-readable signal medium may include a propagated data signal having computer-executable code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electromagnetic, optical, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium, and such computer-readable medium is not a computer-readable storage medium, but is capable of communicating, propagating, or transmitting a program for use by or in connection with an instruction execution system, apparatus, or device.

[0037] "Computer memory" or "memory" is one example of a computer-readable storage medium. Computer memory is any memory directly accessible by a computing system. "Computer storage" or "storage" is another example of a computer-readable storage medium. Computer storage is any non-volatile computer-readable storage medium. In some embodiments, computer storage may be computer memory, or vice versa.

[0038] As used herein, a "computing system" encompasses electronic components capable of executing programs, machine-executable instructions, or computer-executable code. References to a computing system, including examples of a "computing system," should be interpreted as including two or more computing systems or processing cores, as the case may be. A computing system may be, for example, a multi-core processor. A computing system can also refer to a collection of computing systems within a single computer system or distributed among multiple computer systems. The term computing system should also be interpreted as referring to a collection or network of computing devices, possibly each having a processor or computing system. Machine-executable code or instructions can be executed by multiple computing systems or processors, which may be within the same computing device or may be distributed across multiple computing devices.

[0039] Machine-executable instructions or computer-executable code may comprise instructions or programs that cause a processor or other computing system to perform aspects of the present invention. Computer-executable code for performing operations for aspects of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages, such as the "C" programming language or similar programming languages, compiled into machine-executable instructions. In some examples, the computer-executable code may be in the form of a high-level language or pre-compiled, and may also be used in conjunction with an interpreter that generates machine-executable instructions on the fly. In other examples, the machine-executable instructions or computer-executable code may be in the form of a program for a programmable logic gate array.

[0040] The computer executable code may run entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter situation, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider).

[0041] Aspects of the present invention will be described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block or portion of a block in the flowcharts, diagrams, and / or block diagrams, where applicable, can be implemented by computer program instructions in the form of computer-executable code. It will also be understood that blocks in different flowcharts, diagrams, and / or block diagrams can be combined, if not mutually exclusive. These computer program instructions can be provided to a general-purpose computer, special-purpose computer, or other programmable data processing device computing system to generate machine-generated means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0042] These machine-executable instructions or computer program instructions can be stored on a computer-readable medium that can instruct a computer, other programmable data processing apparatus, or other device to function in a particular manner, such that the instructions stored on the computer-readable medium generate instructions that implement the function / act specified in a block or blocks of the flowcharts and / or block diagrams.

[0043] Machine-executable instructions or computer program instructions can also be loaded onto a computer, other programmable data processing device, or other device to cause the computer to execute a series of operational steps that execute on the computer, other programmable device, or other device, creating a computer-implemented process, such that the instructions executing on the computer or other programmable device provide a process for implementing the function / operation specified in the flowchart and / or block diagram block or blocks. As used herein, a "user interface" is an interface that allows a user or operator to interact with a computer or computer system. A "user interface" is also called a "human interface device," and a user interface can provide information or data to an operator and / or receive information or data from an operator. A user interface can allow a computer to receive input from an operator and can provide output from a computer to a user. In other words, a user interface can allow an operator to control or operate a computer, and an interface can allow a computer to indicate the effects of the operator's control or operation. The display of data or information on a display or graphical user interface is an example of providing information to an operator. A keyboard, mouse, trackball, touchpad, pointing stick, graphics tablet, joystick, gamepad, webcam, headset, pedals, wired gloves, remote control, and receiving data via an accelerometer are all examples of user interface components that allow for the reception of information or data from an operator. As used herein, a "hardware interface" includes an interface that allows the computing system of a computer system to interact with and / or control external computing devices and / or equipment. A hardware interface may allow the computing system to send control signals or instructions to external computing devices and / or equipment.A hardware interface may also enable a computing system to exchange data with external computing devices and / or equipment. Examples of hardware interfaces include, but are not limited to, a universal serial bus, an IEEE 1394 port, a parallel port, an IEEE 1284 port, a serial port, an RS-232 port, an IEEE-488 port, a Bluetooth connection, a wireless local area network connection, a TCP / IP connection, an Ethernet connection, a control voltage interface, a MIDI interface, an analog input interface, and a digital input interface.

[0044] As used herein, a "display" or "display device" encompasses an output device or user interface adapted to display images or data. A display can output visual, auditory, and / or tactile data. Examples of displays include, but are not limited to, computer monitors, television screens, touch screens, tactile electronic displays, Braille screens, cathode ray tubes (CRTs), memory tubes, bi-stable displays, electronic paper, vector displays, flat panel displays, vacuum fluorescent displays (VFs), light-emitting diode (LED) displays, electroluminescent displays (ELDs), plasma display panels (PDPs), liquid crystal displays (LCDs), organic light-emitting diode displays (OLEDs), projectors, and head-mounted displays.

[0045] K-space data is defined herein as the recorded measurements of radio frequency signals emitted by atomic spins using the antenna of a magnetic resonance machine during a magnetic resonance imaging scan. Magnetic resonance data is an example of tomographic medical image data.

[0046] A magnetic resonance image or MR image is defined herein as a reconstructed two-dimensional or three-dimensional visualization of the anatomical data contained within the k-space data, which visualization can be performed, for example, using a computer.

[0047] Preferred embodiments of the present invention will now be described, by way of example only, with reference to the drawings in which: [Brief explanation of the drawings]

[0048] [Figure 1] FIG. 1 is a diagram illustrating an example of a medical system. [Figure 2] FIG. 2 shows a flowchart of a method of using the medical system of FIG. 1. [Figure 3] FIG. 1 is a diagram illustrating an example of a medical system. [Figure 4] FIG. 4 shows a flowchart of a method of using the medical system of FIG. 3. [Figure 5] FIG. 2 shows a flowchart of a method of using the medical system of FIG. 1. [Figure 6] FIG. 1 shows several example medical images before and after being processed by a trained machine learning model. DETAILED DESCRIPTION OF THE INVENTION

[0049] Like numbered components in these figures are either equivalent components or perform the same function. An aforementioned component is not necessarily described in a subsequent figure if the functionality is equivalent.

[0050] FIG. 1 illustrates an example of a medical system 100. In this example, the medical system 100 includes a computer 102 having a computing system 104. The computer 102 is intended to represent one or more computers or computer systems. The computing system 104 is intended to represent one or more computing systems or computing cores. The computer 102 is further shown to include an optional hardware interface 106 connected to the computing system 104. If other components of the medical system 100, such as a magnetic resonance imaging system, are present or included, the hardware interface 106 can be used to exchange data and commands with these other components. The medical system 100 is further shown to include an optional user interface 108, which can provide various means for relaying data and receiving data and commands from an operator.

[0051] The medical system 100 is further shown as having memory 110 connected to the computing system 104. The memory 110 is intended to represent various types of memory accessible by the computing system 104. The memory 110 is shown as including machine-executable instructions 120. The machine-executable instructions 120 are instructions that enable the computing system 104 to perform various control, data processing, and image processing tasks. The memory 110 is further shown as including machine learning models 122 and videos 124 obtained from one or more web sources.

[0052] The machine learning model 122 can be configured to receive a 3D medical image as input and to clean the 3D medical image, which can include, for example, denoising, deringing, or super-resolution.

[0053] FIG. 2 is a flowchart of a method for generating training data for a machine learning model according to an example of the present subject matter. The machine learning model is configured to obtain high-quality images from acquired 3D medical images. The method can be performed, for example, by the medical system of FIG. 1. In step 201, videos can be received from one or more sources, such as a web source. In step 203, videos having target values ​​of image parameters can be selected from the received videos. A dataset including the selected videos can be generated in step 205, which may be referred to as an initial dataset. In step 207, the videos of the initial dataset can be downsampled by a predetermined reduction factor. Downsampling can be performed, for example, along the spatial dimension of the video or along the temporal dimension of the video. In step 209, the downsampled videos of the initial dataset can be cropped in space and / or time to obtain video patches, referred to herein as clean video patches. In step 211, the clean video patches can be modified to reduce the quality of the video patches. In step 213, a training dataset can be generated. The training dataset has the modified video patches as inputs and the associated clean video patches as learning targets. Step 213 can further include, for example, storing the training dataset, for example, on the medical system 100 or an accessible remote system.

[0054] Figure 3 shows another example of a medical system 300. The medical system of Figure 3 is similar to the medical system of Figure 1, except that it further includes a magnetic resonance imaging system 302 controlled by the computing system 104. The example shown in Figure 3 is a magnetic resonance imaging system 302.

[0055] The magnetic resonance imaging system 302 includes a magnet 304. The magnet 304 is a cylindrical superconducting magnet with a bore 306 extending therethrough. Different types of magnets can be used. For example, both split cylindrical magnets and so-called open magnets can be used. Split cylindrical magnets are similar to standard cylindrical magnets except that the cryostat is divided into two sections to allow access to the magnet's equal surface. Such magnets can be used, for example, in conjunction with charged particle beam therapy. Open magnets have two magnet sections, one above the other, with a space between them large enough to accommodate a subject. The arrangement of the two magnet sections resembles a Helmholtz coil. Open magnets are popular because the subject is less enclosed. Inside the cryostat of the cylindrical magnet is a collection of superconducting coils.

[0056] Within the bore 306 of the cylindrical magnet 304 is an imaging zone 308 where the magnetic field is strong and uniform enough to perform magnetic resonance imaging. A field of view 309 is shown within the imaging zone 308. K-space data is typically acquired for the field of view 309. The region of interest may be identical to the field of view 309 or may be a subvolume of the field of view 309. A subject 318 is shown supported by a subject support 320 such that at least a portion of the subject 318 is within the imaging zone 308 and the field of view 309.

[0057] Also located within the magnet bore 306 is a set of magnetic field gradient coils 310 used for preliminary k-space data acquisition to spatially encode magnetic spins within the imaging zone 308 of the magnet 304. The magnetic field gradient coils 310 are connected to a magnetic field gradient coil power supply 312. The magnetic field gradient coils 310 are intended to be an example. Typically, the magnetic field gradient coils 310 have three separate coil sets for spatial encoding in three orthogonal spatial directions. The magnetic field gradient power supply supplies current to the magnetic field gradient coils 310. The current supplied to the magnetic field gradient coils 310 is controlled as a function of time and may be ramped or pulsed.

[0058] Adjacent to the imaging zone 308 is a radio frequency coil 314 for manipulating the orientation of magnetic spins within the imaging zone 308 and for receiving radio transmissions from the spins within the imaging zone 308. A radio frequency antenna can have multiple coil elements. A radio frequency antenna can also be referred to as a channel or an antenna. The radio frequency coil 314 is connected to a radio frequency transceiver 316. The radio frequency coil 314 and the radio frequency transceiver 316 can be replaced with separate transmit and receive coils and separate transmitters and receivers. It is understood that the radio frequency coil 314 and the radio frequency transceiver 316 are exemplary. The radio frequency coil 314 is also intended to represent a dedicated transmit antenna and a dedicated receive antenna. Similarly, the transceiver 316 can also represent a separate transmitter and receiver. Additionally, the radio frequency coil 314 can have multiple receive / transmit elements, and the radio frequency transceiver 316 can have multiple receive / transmit channels.

[0059] The transceiver 316 and the gradient controller 312 are shown connected to the hardware interface 106 of the computer system 102 .

[0060] The memory is further shown as having pulse sequence commands 330. The pulse sequence commands 330 are commands or data that can be converted into commands that enable the computing system 104 to control the magnetic resonance imaging system to acquire k-space data 332. The memory 110 shows the k-space data 332 acquired by controlling the magnetic resonance imaging system with the pulse sequence commands 330.

[0061] 4 is a flowchart of a method for acquiring MR image data according to an example of the present subject matter. This method can be performed, for example, by the system of FIG. 3. For example, a machine learning model can be trained using the training dataset obtained by the method of FIG. 2. The machine learning model can be configured to clean the 3D MR images.

[0062] In step 401, k-space data can be acquired by controlling a magnetic resonance imaging system. In step 403, a 3D magnetic resonance image can be reconstructed from the k-space data. The trained machine learning model can be used to improve the quality of the 3D magnetic resonance image in step 405. FIG. 6 shows an example of a reconstructed image 601 before denoising and a corresponding image 605 that is denoised using the trained machine learning model. That is, the denoised image is obtained by inference of the trained machine learning model. While natural images (e.g., received videos) may not have complex parts, MR scans do have complex parts. To overcome this for inference, the real and imaginary parts of the MR scan can be treated as individual images, and both outputs are finally summed to obtain the result.

[0063] FIG. 5 is a flowchart of a method for generating training data for a machine learning model according to an example of the present subject matter.

[0064] Natural high-quality video (initial video) can be obtained from a web source and used to train a machine learning model. The machine learning model can be incorporated, for example, into the IQ Boost 3D system. The machine learning model can be a convolutional neural network that can be trained for denoising, deringing, and super-resolution tasks. In step 501, data selection can be performed. For example, a video can be selected from the initial video. The video can be selected to have high resolution, should not contain artifacts from compression or interpolation, and should contain a limited amount of noise. Also, the gradient in each direction of the selected video should not be zero, which means that the video should contain some kind of movement and not contain still images over time. Data can also be selected based on its license policy so that it can be used for product development purposes.

[0065] Preprocessing of the selected video can be performed. In particular, the following steps may be one embodiment of preprocessing, although there may be several variations of preprocessing: The video is treated as a three-dimensional volume with the third dimension being time. As an example, preprocessing steps can be used for IQ Boost 3D, but preprocessing is not limited to these specific steps. In step 503, all videos can be downsampled to a smaller matrix size, which results in a structural appearance more similar to an MRI scan. The time dimension is left intact. To maintain good quality, a high-quality interpolator such as Lanczos can be used. As additional data augmentation, in step 505, more versions of the video can be generated by resampling to different matrix sizes.

[0066] In step 507, small-sized random patches from each dimension are cropped from the normalized video. This step may allow for handling large volume sizes. In some embodiments, a small but fixed patch size may be used. In step 509, random permuting and flipping of dimensions is applied to reduce the influence of the time dimension. This may be important because the time dimension is very different from the spatial dimension. Random permuting and flipping can prevent the AI ​​model from overfitting to the characteristics of the time dimension. For example, these steps may make the video more similar to the MR data properties.

[0067] The preparation of input data for the machine learning model may include a modification step and therefore may depend on the objective of the model. The video patches may be modified according to the objective of the machine learning model in step 511. A training dataset may be generated from the video patches, and the machine learning model may be trained using the training dataset in step 513.

[0068] Machine learning models are sometimes called 3D AI models. A 3D AI model can be a fully convolutional neural network with 3D kernels, or a ResNet, DenseNet, EfficientNet, or Vision Transformer with kernels tuned for 3D processing. As an example of an ML model, a 3D AI denoiser can be trained by adding random Gaussian noise to clean preprocessed videos and using them as input to the model. The training target is the clean preprocessed video without noise. Training is performed using standard gradient descent. Another example is a 3D AI deringing and super-resolution model. The model can be trained by applying a k-space crop to the clean preprocessed video and providing a low-resolution video with ringing artifacts that serves as input to the model. The training target is the clean preprocessed video.

Claims

1. 1. A medical system for generating training data for a machine learning model configured to improve the quality of acquired 3D medical images, the medical system comprising: a memory storing machine-executable instructions; and a computing system, wherein execution of the machine-executable instructions causes the computing system to: receiving video from one or more sources; selecting, from the received videos, a video having desired values ​​of one or more image parameters; generating a dataset containing the selected videos; Pre-processing the videos of the dataset, the pre-processing comprising: downsampling the video by a predetermined reduction factor; cropping the downsampled video in space and / or time to obtain a video patch, referred to herein as a clean video patch; modifying the clean video patch to reduce its quality; generating a training dataset having the modified video patches as inputs and the associated clean video patches as learning targets; A medical system that performs the above.

2. 2. The medical system of claim 1, wherein execution of the machine-executable instructions further causes the computing system to perform the step of permuting the dimension within a selected video patch of the clean video patch prior to the modification of the clean video patch.

3. 3. The medical system of claim 1 or 2, wherein execution of the machine-executable instructions causes the computing system to further include flipping a selected video patch of the clean video patch before the modification of the clean video patch.

4. 4. The medical system of claim 1, wherein the desired values ​​of the image parameters include at least one of: a resolution higher than a minimum resolution; missing artifacts from compression or interpolation; an amount of noise less than a maximum amount; and a gradient variation in each direction in the video greater than a minimum amount.

5. 5. The medical system of claim 1, wherein execution of the machine-executable instructions further causes the computing system to perform the step of modifying the video patch by adding random Gaussian noise to the video patch and / or by reducing the resolution of the video patch.

6. 6. The medical system of claim 1, wherein execution of the machine-executable instructions further causes the computing system to perform the step of randomly performing the cropping such that each video patch has a size smaller than a maximum size.

7. The medical system of claim 1 , wherein the received video is a non-medical video.

8. 8. The medical system of claim 1, wherein execution of the machine-executable instructions further causes the computing system to perform the step of applying an interpolator to perform the downsampling of the video.

9. Execution of the machine-executable instructions further causes the computing system to: augmenting the dataset by resampling each video of the dataset to obtain different versions of the video, each version of the video having a different size; adding the captured video to the dataset before preprocessing; The medical system according to claim 1 , wherein the medical system executes the following:

10. 10. The medical system of claim 1, wherein the reduction factor is defined such that the resolution obtained after downsampling the video is similar to or equal to the resolution of the 3D medical image of the machine learning model.

11. 11. The medical system of claim 1, wherein execution of the machine-executable instructions further causes the computing system to perform the step of training the machine learning model using the training dataset.

12. The medical system further comprises a magnetic resonance imaging system, the memory further comprising pulse sequence commands configured to control the magnetic resonance imaging system to acquire k-space data according to the magnetic resonance imaging protocol, and execution of the machine-executable instructions further causes the computing system to: acquiring the k-space data by controlling the magnetic resonance imaging system; reconstructing a 3D magnetic resonance image from the k-space data; and using the trained model to improve the quality of the 3D magnetic resonance image. The medical system of claim 11 , comprising:

13. 1. A computer program having machine-executable instructions, the execution of which causes a computing system to: receiving video from one or more sources; selecting, from the received videos, a video having desired values ​​of one or more image parameters; generating a dataset comprising the selected videos; Pre-processing videos of the dataset, said pre-processing comprising: downsampling the video by a predetermined reduction factor; cropping the downsampled video in space and / or time to obtain a video patch, referred to herein as a clean video patch; modifying the clean video patch to reduce its quality; generating a training dataset having the modified video patches as input and the associated clean video patches as learning targets; A computer program that executes

14. 1. A method for generating training data for a machine learning model configured to improve the quality of acquired 3D medical images, comprising: receiving video from one or more sources; selecting, from the received videos, a video having desired values ​​of one or more image parameters; generating a dataset comprising the selected videos; Pre-processing the videos of the dataset, the pre-processing comprising: downsampling the video by a predetermined reduction factor; cropping the downsampled video in space and / or time to obtain a video patch, referred to herein as a clean video patch; modifying the clean video patch to reduce its quality; generating a training dataset having the modified video patches as inputs and the associated clean video patches as learning targets; 1. A computer program comprising:

15. 1. A computer-implemented data structure for training a machine learning model for cleaning 3D medical images, the data structure having modified video patches and associated clean video patches, the modified video patches including modifications obtained according to a cleaning objective of the machine learning model.

16. The data structure of claim 15 , wherein the modified video patch comprises a noisy video patch having ringing artifacts or a low-resolution video patch.

Images