Medical image monitoring using neural network generated image evaluation.

Abstract

Disclosed herein is a medical system (100, 300) comprising a memory storing machine executable instructions and a plurality of neural networks. The plurality of neural networks comprises a noise estimation neural network and at least one image quantification neural network (124). The noise estimation neural network is configured to output a noise estimate (128) of the medical image (126, 126') in response to receiving the medical image as an input. The at least one image quantification neural network is configured to output image attributes of the medical image in response to receiving the medical image as an input. The medical system further comprises a computing system.

Description

[Technical field]

[0001] The present invention relates to medical imaging, and more particularly to monitoring image attributes during medical imaging. [Background technology]

[0002] A variety of cross-sectional medical imaging techniques, such as magnetic resonance imaging (MRI), computed tomography, positron emission tomography, and single photon emission tomography, allow for detailed visualization of a subject's anatomy. Summary of the Invention [Problem to be solved by the invention]

[0003] The quality of the images can be degraded if the medical imaging device has a malfunction or other problem.

[0004] US Patent Application Publication No. 2019 / 0340470(A1) discloses a method for real-time assessment of image information using a computer including one or more processors, the method including acquiring magnetic resonance (MR) images of a region of interest via a scanner and providing data corresponding to at least a portion of the MR images to a deep learning model, the deep learning model having been pre-trained based on one or more training data sets. The method further includes evaluating the data using the deep learning model and the data acquired from an image quality database to obtain an assessed quality value, and formulating an image quality classification in response to the assessed image quality value. The method further includes outputting an image quality classification within a predetermined time from an initial scan of the region of interest.

[0005] US patent application US2016 / 042499 discloses user identification of an image quality index and identification of an acquisition / reconstruction method based on the image quality index. [Means for solving the problem]

[0006] The invention provides a medical system, a computer program and a method in the independent claims. Embodiments are given in the dependent claims.

[0007] The embodiments may provide a means for more effectively determining image attributes. Various characteristics of an image, such as noise or image resolution, may be interrelated. The embodiments use a neural network, referred to herein as an image quantification neural network, to determine a noise estimate for the medical image. After the noise estimate is obtained, it is used during the determination of at least one other image attribute, using a neural network, referred to herein as an image quantification neural network. The noise estimate is used to control or modify the determination of the image attribute. The image attribute and the noise estimate are then used to determine whether the image quality meets a predetermined criterion or criteria. In response to the image evaluation, or in response to meeting the predetermined criteria, a warning signal is provided. The warning signal may take different forms in different embodiments. In some examples, the warning signal may include a visual warning signal, an audible warning signal, an operator command, or a maintenance command. In other examples, the warning signal may include a machine command or modify the function of the medical imaging device. Because a neural network is used, this warning signal may be provided quickly, even while the medical imaging device is acquiring medical images of the subject.

[0008] In one aspect, the present invention provides a medical system comprising a memory storing machine executable instructions and a plurality of neural networks. Both the machine executable instructions and the plurality of neural networks are executable by a computing system. The plurality of neural networks comprises a noise estimation neural network and at least one image quantification neural network. The noise estimation neural network is a label to indicate a particular neural network. The label "at least one image quantification neural network" is a label to indicate a particular neural network. Thus, each of the at least one image quantification neural network is a neural network.

[0009] The noise estimation neural network is configured to output a noise estimate for the medical image in response to receiving the medical image as an input. That is, the noise estimation neural network is configured to receive the medical image and has an input capable of outputting a noise estimate. The noise estimate may take different forms in different examples. In one example, the noise estimate may be a noise estimate for the entire medical image, in which case it will be a single number. The noise estimate may be for a portion or quadrant of the medical image. In yet another example, the noise estimate may be for each pixel or voxel of the medical image, in which case the noise estimate may be a multi-dimensional mapping of the noise in the medical image.

[0010] Thus, the noise estimation neural network can be any of a variety of neural networks configured to receive an image. In particular, the Unet and ResNet architectures work particularly well. If the noise estimate is an image or image data, then for example, the Unet architecture, DeepLab-based architecture, DnCnn, or FNet can be used. If the noise estimate is the number of the entire image or a section of the image, then a classification neural network can be used. For example, the neural network can be a ResNet, DenseNet, or VGG net.

[0011] Training the noise estimation neural network to output noise estimates is provided in more detail below. In short, the noise estimation neural network can be trained by inputting images with a known noise level and then using the resulting noise estimates output to compare with the known noise level. For example, images with low or non-existent noise can have noise artificially added to them to provide this training data.

[0012] At least one image quantification neural network is configured to output image attributes of the medical image in response to receiving the medical image as input. Similarly, the output image attributes may be for the entire image, for a portion or quadrant of the medical image, or may be a pixel-wise or voxel-wide map of image attributes in the medical image. The same neural network architectures described above for the noise estimation neural network may also be used to implement the image quantification neural network. In particular, the Unet and ResNet architectures perform particularly well. If the noise estimate is an image or image data, then for example, the Unet architecture, DeepLab-based architecture, DnCnn, or FNet may be used. If the noise estimate is the number of the entire image or a section of the image, a classification neural network may be used. For example, the neural network may be a ResNet, DenseNet, or VGG net.

[0013] The image quantification neural network can be trained, for example by performing deep learning, using input images with known or artificially added image attributes and then comparing them in the output with the known or added image attributes.

[0014] The medical system further comprises a computing system. Execution of the machine-executable instructions causes the computing system to receive the medical image. The medical image can be received in various ways, for example, it can be transmitted to the computing system over a network or other telecommunication system, the medical image can be retrieved from a memory where it is stored, or it can be received by the medical system acquiring the medical image, if the medical system comprises a medical imaging device such as an MRI or CT system.

[0015] Execution of the machine executable instructions further causes the computing system to receive a noise estimate in response to inputting the medical image to the noise estimation neural network. Execution of the machine executable instructions further causes the computing system to receive image attributes in response to inputting the medical image to the image quantification neural network. The determination of the image attributes by the image quantification neural network depends on the noise estimate. As will be described in more detail below, this may have a number of different possibilities.

[0016] In one example, the noise estimate is used to select a particular image quantification neural network. For example, various image quantification neural networks can be trained using images with a particular range of noise in the medical image. In yet another example, the image quantification neural network can be configured to receive a noise estimate, and during training of the image quantification neural network, the known noise of the image used for training is also input, in which case the image quantification neural network is adapted to receive and adjust its response depending on the noise estimate. In yet another example, the noise estimate can be used to trigger noise removal from the medical image before it is input to the image quantification neural network. These various possibilities provide a very flexible means of improving the quality of determining attributes using the image quantification neural network.

[0017] In another embodiment, execution of the machine executable instructions further causes the computing system to add noise estimates and image attributes from each of the at least one image quantification neural network to the image evaluation. Execution of the machine executable instructions further causes the computer system to provide a warning signal in response to the image evaluation using a predetermined criterion. For example, if the evaluation meets the predetermined criterion, a warning signal can be provided. The warning signal can take different forms in different examples. For example, it can be an audible signal, a visual signal, or a tactile signal provided to an operator. In other examples, the warning signal can be a signal provided to other software modules or executable code to modify behavior. In other examples, the warning signal can affect the operation of a medical imaging device used to acquire the medical image, for example, causing either a reacquisition or a change.

[0018] In still other cases, the warning signal may be used to modify the behavior of the reconstruction of the medical image. For example, it may trigger the use of an alternative reconstruction method for reconstructing the medical image from the raw medical data acquired by the medical imaging device. The warning signal may be an indication of the image evaluation or part of the image evaluation on the display. The warning signal may also be used to label the medical image in a medical database.

[0019] The predetermined criteria can take different forms in different embodiments. For example, it can be used to check whether the value of the image rating is within a particular range or within a particular statistical range. For example, if one or more of the elements or contents of the image rating are image data, these images can be processed for average, mean, high and low values, any number of which can be used to be the predetermined criteria.

[0020] This embodiment may be advantageous as it may provide an effective means of ensuring that medical images are properly reconstructed, or may be used to adjust the operation of a medical system imaging system.

[0021] In another embodiment, the medical system further comprises a medical imaging device configured to acquire medical data describing the subject. Execution of the machine executable instructions further causes the computing system to control the medical imaging device to acquire the medical data. Execution of the machine executable instructions further causes the computing system to reconstruct a medical image from the medical data. Execution of the machine executable instructions further causes the computing system to modify operation of the medical imaging device if a warning signal is provided. Modifying operation of the medical imaging device includes any one of providing a user warning, adding a warning signal to the medical image, generating a medical imaging device repair request, triggering reacquisition of the medical data, displaying a resolution estimate, displaying at least one image attribute, providing an operational instruction, providing a repair instruction, and combinations thereof.

[0022] This embodiment may be particularly beneficial because it may provide a means of monitoring the medical imaging device on the fly and ensuring that it is operating properly. For example, this may then automatically trigger a service or repair request. This may also be beneficial because it may limit the amount of preventative maintenance that needs to occur. It may also be beneficial because it may eliminate the need for time-consuming testing of the medical imaging device on a periodic basis.

[0023] An example of this is a magnetic resonance imaging system where noise and resolution estimation is performed semi-periodically using measurements on a phantom to ensure that the magnetic resonance imaging is functioning properly. This embodiment can eliminate or reduce the need for such time consuming testing.

[0024] In another embodiment, the medical device is a magnetic resonance imaging system, which, as mentioned above, can be beneficial as it can provide a means of continuously monitoring that the magnetic resonance imaging system is functioning properly without or reducing noise and the need to perform, for example, resolution checks.

[0025] In another embodiment, the plurality of neural networks further comprises a uniformity estimation neural network. The uniformity estimation neural network is configured to output an image uniformity estimate in response to inputting a medical image into the uniformity estimation neural network. The uniformity estimation neural network is a neural network. The uniformity estimation neural network is configured to take a medical image as an input and then output an image uniformity estimate. In the context of magnetic resonance imaging, the uniformity of the image may vary, for example, due to improper magnetic shimming. This may result in contrast variations across the medical image. The uniformity estimation neural network may be implemented, for example, using the neural network architecture described above for the noise estimation neural network. For training, an estimated neural network magnetic resonance image with a known image uniformity may be used as an input during training, and this known image uniformity may be used, for example, as ground truth data during deep learning training of the uniformity estimation neural network.

[0026] Execution of the machine executable instructions further causes the computing system to receive the uniformity estimation response for inputting the medical image into a uniformity estimation neural network. Execution of the machine executable instructions further causes the computing system to add the uniformity estimation to the image evaluation. This may be beneficial, for example, because it may provide a means of indicating a malfunction in the magnetic resonance imaging system if the uniformity is outside a particular range.

[0027] In another embodiment, the uniformity estimate is received before inputting the medical image to the noise estimation neural network. Execution of the machine executable instructions further causes the computing system to, in one option, apply an image uniformity correction algorithm to the medical image before inputting the medical image to the noise estimation neural network. The image uniformity correction algorithm is controlled by the uniformity estimate. For example, the contrast of the medical image can be adjusted using an algorithmic image uniformity correction algorithm. As an alternative option, or in conjunction with the first option, execution of the machine executable instructions further causes the computing system to trigger a reacquisition of the medical image data if the uniformity estimate meets a predetermined uniformity criterion.

[0028] As a further option, which may be performed alone or in conjunction with the first two options, execution of the machine executable instructions further causes the computing system to request magnetic reshimming of the magnet of the magnetic resonance imaging system if it meets a predetermined uniformity criterion. These various options may be beneficial as they provide a means to improve the uniformity of medical images. Image uniformity may indicate (im)proper field shimming in magnetic resonance imaging. In particular, it may be beneficial to perform this before performing noise estimation, as this may make the determination of noise estimation and other image attributes more accurate.

[0029] In another embodiment, the plurality of neural networks further comprises a magnetic resonance weighting estimation neural network. The magnetic resonance weighting estimation neural network is configured to estimate the magnetic resonance weighting dependency in response to receiving a medical image as an input. As used herein, a magnetic resonance weighting estimation neural network encompasses a neural network configured to receive an image and configured to output a magnetic resonance weighting estimation. Weighting in this respect refers to a type of contrast or a type of weighting for magnetic resonance imaging. For example, this may be T1, T2, T2 star, diffusion weighting, or other types of weighting for the magnetic resonance image. This weighting implements a contrast weighting or contrast encoding weighting of the medical image or image data from which the medical image is reconstructed, respectively. As mentioned above, this can be trained by acquiring a magnetic resonance image with a known image weighting type and then using this as input and ground truth data for training the magnetic resonance weighting estimation neural network using, for example, deep learning.

[0030] As applies to the noise estimation neural network, the magnetic resonance weighting estimation neural network may be any of a variety of neural networks configured to receive an image. In particular, the Unet and ResNet architectures work particularly well. If the noise estimate is an image or image data, then for example, the Unet architecture, DeepLab-based architecture, DnCnn, or FNet can be used. If the noise estimate is the entire image or a number of sections of the image, a classification neural network can be used. For example, the neural network can be a ResNet, DenseNet, or VGG net.

[0031] Execution of the machine-executable instructions further causes the computing system to receive magnetic resonance weighting dependencies in response to inputting the medical image into the magnetic resonance weighting estimation neural network and add the magnetic resonance weighting dependencies to the image evaluation. In some examples, the determination of the magnetic resonance weighting dependencies is performed after an image uniformity correction is performed and / or before a noise estimation or correction is performed.

[0032] In another embodiment, the medical imaging device is a tomographic medical imaging device.

[0033] In another embodiment, the medical imaging device is a computed tomography system.

[0034] In another embodiment, the medical imaging device is a C-arm computed tomography system, which is typically a portable computed tomography system having a detector and an x-ray source mounted on a C-arm.

[0035] In another embodiment, the medical imaging device is a planar x-ray system.

[0036] In another embodiment, the medical imaging device is a fluoroscopy system or a digital fluoroscopy system.

[0037] In another embodiment, the medical imaging device is a positron emission tomography system.

[0038] In another embodiment, the medical imaging device is a single photon emission computed tomography system.

[0039] In another embodiment, the medical imaging device is an ultrasound system.

[0040] In another embodiment, execution of the machine executable instructions further causes the computing system to apply a denoising algorithm to the medical image after receiving the noise estimate and before inputting the medical image into at least one image quantification neural network. This embodiment may be beneficial as it may allow the image quantification neural network to perform more accurately.

[0041] In another embodiment, at least one image quantification neural network is further configured to receive the noise estimate as an input. Execution of the machine executable instructions further causes the computing system to input the noise estimate into the image quantification neural network before receiving the image attributes. This embodiment may be advantageous as it may provide a means to more accurately determine image quantification (Note: insert standard text here describing how to input multiple inputs into a neural network).

[0042] In another embodiment, each of the at least one image quantification neural network is selected from a group of image attribute specific image quantification neural networks using the noise estimate, in this embodiment, for each individual image quantification, there is a group or pool of image quantification neural networks, which are trained using different amounts of noise, so that when the noise estimate is used to select an appropriate image quantification neural network from the image attribute specific image quantification neural networks, an accurate value of the image quantification is obtained.

[0043] In another embodiment, at least one image quantification neural network comprises a resolution estimation neural network configured to output a resolution estimate of the medical image in response to receiving the medical image as an input. This embodiment may be beneficial because the resolution estimate may be much more accurate after the noise estimate is taken into account. The above-mentioned training and neural network architecture details for the image quantification neural network are applicable to the resolution estimation neural network. Images with known resolution (or known spatially dependent resolution) may be used for training.

[0044] In another embodiment, the at least one image quantification neural network comprises an artifact estimation neural network configured to output image artifact quantification in the medical image in response to receiving the medical image as input. For example, the artifact estimation neural network may provide the location of artifacts in the magnetic resonance image or indicate areas likely to contain image artifacts. This may be beneficial as it may be more accurate after noise estimation is taken into account. The details of the training and neural network architecture described above for the image quantification neural network are applicable to the resolution estimation neural network. Images with known levels of artifacts or motion artifacts may be used for training. If the output is an identification of artifact location, labeled images with labeled artifact locations may be used for training.

[0045] The resolution estimation neural network can be trained by taking an image with a known resolution and then using this as input and using the known resolution as ground truth during training. Similarly, the artifact estimation neural network can be trained by using a magnetic resonance image with known artifacts therein and then performing training using this knowledge of the artifacts as ground truth. The artifact estimation neural network can be trained to return image attributes in the form of a quantification of the amount of blur in the input (source) image. To that end, the artifact estimation can be trained on a ground truth image with a predetermined degree of blur. For that purpose, a magnetic resonance image is provided that is reconstructed from magnetic resonance signals with a predetermined degree of (e.g. transverse) signal attenuation, for example. Alternatively, an intentionally blurred version of a natural (video) image with a controlled degree of resolution loss can be used as a target image for training.

[0046] In another embodiment, the memory further comprises a foreign object detection neural network. The foreign object detection neural network is configured to output a foreign object segmentation in the medical image in response to input of the medical image. Each of the plurality of neural networks is further configured to receive an object mask to identify regions in the medical image to be ignored. Execution of the machine executable instructions further causes the computing system to receive the foreign object segmentation in response to input of the medical image to the foreign object detection neural network. Execution of the machine executable instructions further causes the computing system to construct an image mask from the foreign object segmentation. Execution of the machine executable instructions further causes the computing system to input the image mask to each of the plurality of neural networks.

[0047] For example, the foreign object detection neural network may be a convolutional neural network trained to locate objects in images. This is a very classic and typical use of convolutional neural networks. They may be trained by providing medical images with foreign objects labeled in them. The labeled images may then be used to train the neural network by performing deep learning on the foreign object detection neural network. The following are exemplary neural network architectures that may be used, for example, the Unet architecture, DeepLab-based architecture, DnCnn, or FNet may be used.

[0048] The neural networks can be trained to also receive and ignore image masks that include portions of the image, thus allowing noise and other image attributes to be determined without being affected by extraneous objects.

[0049] In another embodiment, the memory further includes an evaluation neural network configured to output a warning signal if the image assessment meets a predetermined criterion. Execution of the machine executable instructions further causes the computer system to input the image assessment to the evaluation neural network and receive a warning signal if the predetermined criterion is met. In some examples, the predetermined criterion may be determined by using a look-up table or ranges acceptable for noise estimation and various image quantifications. In this embodiment, the neural network may receive the image assessment and then be trained to provide a warning signal if the value of the image assessment is unacceptable. This may be trained, for example, by taking various combinations of image assessments from actual measurements and then assigning them as either acceptable or unacceptable. The values ​​may be put into the neural network and then a decision of whether they should trigger an alert may be used as ground truth. The evaluation neural network may be implemented, for example, using standard neural networks used to perform classification tasks, such as ResNet, DenseNet, or VGG net.

[0050] In another aspect, the present invention provides a computer program comprising machine executable instructions for execution by a computing system and a plurality of neural networks. The plurality of neural networks comprises a noise estimation neural network and at least one image quantification neural network. The noise estimation neural network is configured to output a noise estimate for the medical image in response to receiving the medical image as an input. The at least one image quantification neural network is configured to output image attributes for the medical image in response to receiving the medical image as an input. Execution of the machine executable instructions causes the computing system to receive the medical image.

[0051] Execution of the machine-executable instructions further causes the computing system to receive a noise estimate in response to inputting the medical image to the noise estimation neural network. Execution of the machine-executable instructions further causes the computing system to receive image attributes in response to inputting the medical image to the image quantification neural network. Determination of the image attributes by the image quantification neural network is dependent on the noise estimate. Execution of the machine-executable instructions further causes the computing system to add the noise estimates and the image attributes from each of the at least one image quantification neural network to an image assessment. Execution of the machine-executable instructions further causes the computer system to provide a warning signal in response to the image assessment meeting a predetermined criterion.

[0052] In another aspect, the present invention provides a method for training a noise estimation neural network. The method includes receiving an untrained noise estimation neural network. The method further includes receiving training data. The training data includes pairs of input images and ground truth noise values. At least some of the pairs of input images are modified optical images. The modified optical images are generated by adding noise to the optical images and applying an image mask to blacken parts of the optical images. For example, when performing magnetic resonance imaging, most of the image is dark or black. The noise estimation neural network can be trained to ignore these dark or black areas of the magnetic resonance image. Typically, normal optical images, such as those acquired by optical cameras, cannot be used to train neural networks to estimate noise in magnetic resonance images or other medical images because of these dark or information-free areas. Optical images, as used herein, encompass images acquired using optical cameras. Optical images may also be referred to as natural images or photographs (or digital photographs).

[0053] The image mask does not have to be binary, and parts of the image are completely blacked out. A smoothed mask can be used instead of a binary mask to mimic the signal drop-off in a magnetic resonance image at the transition between parts showing anatomy and its mostly dark background. There is a gradual transition between the image and the background, rather than an abrupt or binary transition. For example, one can take a regular binary mask and smooth it with a Gaussian kernel. When multiplying a natural image with a smoothed mask, a similar signal drop-off can be achieved at the mask boundaries. This can provide a more robustly functioning neural network. In this example, the image mask can include a subject portion that simulates a subject in a medical image, and a background portion that simulates a darkened portion of the medical image. In this example, there is a smooth transition between the subject portion and the background portion.

[0054] However, the use of these optical images can be used to train these neural networks by applying image masks in addition to adding a controlled amount of noise to these images. This is extremely advantageous as it allows the ability to generate large amounts of training data so that the noise estimation neural network performs much better than if it were trained using magnetic resonance images. In addition, modifying these optical images by adding noise allows for very accurate training of the neural network. When intrinsic magnetic resonance images are used, there is an interaction between noise and resolution. It can be very difficult to accurately train a noise estimation neural network using only magnetic resonance images.

[0055] It will be understood that one or more of the above-described embodiments of the present invention may be combined, unless the combined embodiments are mutually exclusive.

[0056] As will be appreciated by one of ordinary skill in the art, aspects of the present invention may be embodied as an apparatus, a method, or a computer program product. Accordingly, aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, all of which may be referred to generally herein as a "circuit," "module," or "system," and further, aspects of the present invention may take the form of a computer program product embodied in one or more computer-readable medium(s) having computer-executable code embodied thereon.

[0057] Any combination of one or more computer readable media may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. As used herein, a "computer readable storage medium" encompasses any tangible storage medium capable of storing instructions executable by a processor or a computing system of a computing device. The computer readable storage medium may be referred to as a computer readable non-transitory storage medium. The computer readable storage medium may also be referred to as a tangible computer readable medium. In some embodiments, the computer readable storage medium may also be capable of storing data that can be accessed by the computing system of a computing device. Examples of computer readable storage media include, but are not limited to, floppy disks, magnetic hard disk drives, solid state hard disks, flash memory, USB thumb drives, random access memory (RAM), read only memory (ROM), optical disks, magneto-optical disks, and register files of a computing system. Examples of optical disks include compact disks (CDs) and digital versatile disks (DVDs), such as CDROM, CDRW, CDR, DVDROM, DVDRW, or DVDR disks. The term computer-readable storage medium also refers to various types of recording media that can be accessed by a computer device over a network or communication link. For example, data can be retrieved over a modem, over the Internet, or over a local area network. Computer executable code embodied on a computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wireline, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

[0058] A computer-readable signal medium may include a propagated data signal having computer-executable code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electromagnetic, optical, or any suitable combination thereof. A computer-readable signal medium is not a computer-readable storage medium but may be any computer-readable medium that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0059] "Computer memory" or "memory" is one example of a computer-readable storage medium. Computer memory is any memory directly accessible to a computing system. "Computer storage" or "storage" is a further example of a computer-readable storage medium. Computer storage is any non-volatile memory computer-readable storage medium. In some embodiments, computer storage may be computer memory, and vice versa.

[0060] As used herein, a "computing system" encompasses electronic components capable of executing programs, machine-executable instructions, or computer-executable code. References to a computing system, including examples of a "computing system," should be interpreted as including more than one computing system or processing core, as the case may be. A computing system may be, for example, a multi-core processor. A computing system may also refer to a collection of computing systems within a single computer system or distributed among multiple computer systems. The term computing system should also be interpreted as referring to a collection or network of computing devices, each possibly comprising a processor or computing system. Machine-executable code or instructions may be executed by multiple computing systems or processors, which may be within the same computing device or may be distributed across multiple computing devices.

[0061] Machine-executable instructions or computer-executable code may comprise instructions or programs that cause a processor or other computing system to perform aspects of the present invention. Computer-executable code for performing operations for aspects of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and traditional procedural programming languages ​​such as the "C" programming language or similar programming languages, and compiled into machine-executable instructions. In some cases, the computer-executable code may be in the form of a high-level language or in a pre-compiled form, and may be used in conjunction with an interpreter that generates machine-executable instructions on the fly. In other cases, the machine-executable instructions or computer-executable code may form a program for a programmable logic gate array.

[0062] The computer executable code may run entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be to an external computer (e.g., via the Internet using an Internet Service Provider).

[0063] Aspects of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It should be understood that each block or portion of a block in the flowcharts, diagrams, and / or block diagrams may be implemented by computer program instructions in the form of computer executable code, where applicable. Furthermore, it should be noted that combinations of blocks in different flowcharts, diagrams, and / or block diagrams may be combined, if not mutually exclusive. These computer program instructions may be provided to a general purpose computer, special purpose computer, or other programmable data processing device computing system to generate a machine such that the instructions, executed via the computer or other programmable data processing device computing system, create means for performing the functions / operations specified in the flowchart and / or block diagram blocks or blocks.

[0064] These machine-executable instructions or computer program instructions may be stored on a computer-readable medium that can instruct a computer, other programmable data processing apparatus, or other device to function in a particular manner, such that the instructions stored on the computer-readable medium produce an article of manufacture that includes instructions that implement the function / act specified in a block or blocks of the flowcharts and / or block diagrams.

[0065] The machine-executable instructions or computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps executed on the computer, other programmable apparatus, or other device to produce a computer-implemented process, such that the instructions executing on the computer or other programmable apparatus provide a process for implementing the functions / operations specified in the flowchart and / or block diagram blocks or blocks.

[0066] A "user interface" as used herein is an interface that allows a user or operator to interact with a computer or computer system. A "user interface" is sometimes referred to as a "human interface device" and can provide information or data to an operator and / or receive information or data from an operator. A user interface can allow input from an operator to be received by a computer and can provide output from the computer to a user. In other words, a user interface can allow an operator to control or manipulate a computer and an interface can allow a computer to show the effect of the operator's control or manipulation. The display of data or information on a display or graphical user interface is an example of providing information to an operator. Receiving data via a keyboard, mouse, trackball, touchpad, pointing stick, graphics tablet, joystick, gamepad, webcam, headset, pedals, wired gloves, remote control, and accelerometer are all examples of user interface components that allow for the reception of information or data from an operator.

[0067] As used herein, a "hardware interface" encompasses an interface that allows a computing system of a computer system to interact with and / or control external computing devices and / or equipment. A hardware interface may allow a computing system to send control signals or instructions to an external computing device and / or equipment. A hardware interface may also allow a computing system to exchange data with an external computing device and / or equipment. Examples of hardware interfaces include, but are not limited to, a universal serial bus, an IEEE 1394 port, a parallel port, an IEEE 1284 port, a serial port, an RS232 port, an IEEE 488 port, a Bluetooth connection, a wireless local area network connection, a TCP / IP connection, an Ethernet connection, a control voltage interface, a MIDI interface, an analog input interface, and a digital input interface.

[0068] As used herein, a "display" or "display device" encompasses an output device or user interface adapted to display images or data. A display can output visual, audio, and / or tactile data.

[0069] Examples of displays include, but are not limited to, computer monitors, television screens, touch screens, tactile electronic displays, Braille screens, cathode ray tubes (liquid), memory tubes, bi-stable displays, electronic paper, vector displays, flat panel displays, vacuum fluorescent displays (VF), light emitting diode (LED) displays, electroluminescent displays (ELD), plasma display panels (PDP), liquid crystal displays (LCD), organic light emitting diode displays (OLED), projectors, and head mounted displays.

[0070] Medical data is defined herein as recorded measurements made by medical imaging equipment that describe a subject. Medical image data may be reconstructed into a medical image. A medical image is defined herein as a reconstructed two- or three-dimensional visualization of anatomical data contained within the medical data. This visualization may be performed using a computer.

[0071] K-space data is defined herein as the recorded measurements of radio frequency signals emitted by atomic spins using the antenna of a magnetic resonance machine during a magnetic resonance imaging scan. Magnetic resonance data is an example of medical data.

[0072] A magnetic resonance imaging (MRI) image or MR image is defined herein as a reconstructed two- or three-dimensional visualization of the anatomical data contained within the k-space data. This visualization can be performed using a computer.

[0073] In the following, preferred embodiments of the invention will be described, by way of example only, with reference to the drawings in which: [Brief description of the drawings]

[0074] [Figure 1] 1 shows an example of a medical system. [Diagram 2] 2 shows a flow chart illustrating a method of using the medical system of FIG. [Diagram 3] 1 shows an example of a medical system. [Figure 4] 4 shows a flow chart illustrating a method of using the medical system of FIG. [Diagram 5] 1 illustrates an example of how various neural networks in a medical system can be deployed in a coordinated manner. [Figure 6] 1 illustrates a further example of how various neural networks in a medical system can be collaboratively arranged. [Figure 7]1 illustrates a further example of how various neural networks in a medical system can be deployed in a coordinated manner. [Figure 8] 1 illustrates a further neural network that can be used to identify foreign objects in medical images. [Figure 9] 1 illustrates an optical image that may be used to train a neural network to determine the noise level or resolution in a medical image. [Figure 10] 13 illustrates further optical images that may be used to train a neural network to determine noise levels or resolution in medical images. [Figure 11] Referring now to a method for training a neural network, FIG. 12 shows a neural network topology suitable for outputting a single numerical value such as the signal-to-noise level or the resolution of an image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0075] Like numbered elements in these figures are equivalent elements or perform the same function. An element described above is not necessarily discussed in a subsequent figure if there is functional equivalence.

[0076] An example of a medical system is shown in FIG. 1. The medical system 100 is shown to include a computer 102 having a computing system 104. The computer 102 may represent, for example, one or more networked computers. Similarly, the computing system 104 may represent one or more computing systems, such as multiple processors or multiple cores operating in cooperation. The computing system 104 is shown to be connected to an optional hardware interface 106 and an optional user interface 108. The hardware interface 106 may, for example, allow the computing system 104 to communicate with and control other components of the medical system 100. For example, the medical system 100 may further include a medical imaging device.

[0077] The computing system 104 is further shown to be in communication with a memory 110. The memory 110 is intended to represent different types of memory or storage accessible to the computing system 104. The memory 110 is shown to include machine-executable instructions 120. The machine-executable instructions 120 enable the computing system 104 to perform basic computational and image processing tasks. The memory 110 is further shown to include a noise estimation neural network 122. The memory is further shown to include an image quantification neural network 124.

[0078] The memory is further shown as including a medical image 126. The memory 110 is further shown as including a noise estimate. The noise estimate 128 is obtained by inputting the medical image into a noise estimation neural network 122 and receiving the noise estimate 128 as an output. The memory is further shown as including image attributes 130. The image attributes 130 are obtained by inputting the medical image 126 and / or the noise estimate 128 into an image quantification neural network 124 and receiving the image attributes 130 in response. The noise estimate 128 is used to control how the image quantification neural network 124 provides the image attributes 130. This can be done in a variety of different ways. In one case there are multiple image quantification neural networks 124 for a given image attribute, which are trained using different amounts of noise. The noise estimate 128 can then be used to select the appropriate image quantification neural network 122 to use for a particular value of the noise estimate 128.

[0079] In another variation, the image quantification neural network 124 is trained using known noise during the training process. In this case, the noise estimate 120 is input directly to the image quantification neural network 124. In another example, there may be an image filter that is controlled using the noise estimate 128. For example, a medical image is first input to the noise estimation neural network 122, and then the resulting noise estimate 128 is used to control an image filter. The filtered image is then input to the image quantification neural network 124.

[0080] The memory 132 is further shown as including an image assessment 132. The image assessment 132 is a collection of the noise estimate 128 and any determined image attributes 130. The memory is further shown as including a predetermined criterion 136, which is then used to determine whether the image assessment 132 is within acceptable boundaries. For example, the predetermined criterion 136 may be used to determine whether the medical imaging device used to acquire the medical image 126 is functioning properly. This can be done in a variety of ways. In one example, there may be a decision module 134 that compares the predetermined criterion 136 to the image assessment 132. This may essentially be equivalent to a look-up table, or may compare values ​​in the image assessment 132 to various ranges or boundaries. The decision module 134 may then provide a signal 138 if the image assessment does not meet the predetermined criterion. Alternatively, the decision module 134 may be an assessment neural network that is trained to receive the image assessment 132 and provide the signal 138 if the predetermined criterion is not met.

[0081] Figure 2 is a flow chart illustrating a method of operating the medical system of Figure 1. First, in step 200, a medical image 126 is received. Next, in step 202, in response to inputting the medical image 126 to the noise estimation neural network 122, a noise estimate 128 is received. Next, in step 204, in response to inputting the medical image 126 to the image quantification neural network 124, image attributes 130 are received. As previously mentioned, the determination of the image attributes by the image quantification neural network depends on the noise estimate 128. Next, in step 206, the noise estimate and the image attributes 130 are added to the image assessment 132. Finally, in step 208, an alert signal 138 is provided in response to the image assessment meeting a predetermined criterion 136.

[0082] FIG. 3 shows a further example of a medical system 300. Embodiments of the present invention can be used with a variety of different medical imaging devices. The magnetic resonance imaging system 302 is intended as an example. The medical system 300 of FIG. 3 is similar to the medical system 100 of FIG. 1, except that it further includes a magnetic resonance imaging system 302.

[0083] The example shown in Figure 3 is a magnetic resonance imaging system 302. However, the magnetic resonance imaging system can be replaced by other types of tomographic medical imaging devices. Examples can be a computed tomography system, a C-arm computed tomography system, a single photon emission tomography system, a positron emission tomography system, a fluoroscopy system, an x-ray system, or an ultrasound system.

[0084] The magnetic resonance imaging system 302 comprises a magnet 304. The magnet 304 is a superconducting cylindrical magnet with a type bore 306. It is also possible to use different types of magnets, for example both split cylindrical magnets and so-called open magnets. Split cylindrical magnets are similar to standard cylindrical magnets, except that the cryostat is split into two sections to allow access to the magnet's isoplane, and such magnets can be used, for example, in conjunction with charged particle beam therapy. Open magnets have two magnet sections, one above the other, with a space between them large enough to accommodate the subject, i.e. an arrangement of the two section areas similar to the area of ​​a Helmholtz coil. Open magnets are popular because they are less confining to the subject. Inside the cryostat of the cylindrical magnet is a collection of superconducting coils.

[0085] Within the bore 306 of the cylindrical magnet 304 is an imaging zone 308 where the magnetic field is strong and sufficiently uniform to perform magnetic resonance imaging. A field of view 309 is shown within the imaging zone 308. Magnetic resonance data that is typically acquired for the field of view 309. The region of interest may be identical to the field of view 309 or may be a sub-volume of the field of view 309. A subject 318 is shown supported by a subject support 320 such that at least a portion of the subject 318 is within the imaging zone 308 and the field of view 309.

[0086] Also within the magnet bore 306 are a set of magnetic field gradient coils 310 used for preliminary magnetic resonance data acquisition for spatially encoding magnetic spins within the imaging zone 308 of the magnet 304. The magnetic field gradient coils 310 are connected to a magnetic field gradient coil power supply 312. The magnetic field gradient coils 310 are intended to be representative. Typically, the magnetic field gradient coils 310 include three separate coil sets for spatially encoding in three orthogonal spatial directions. The magnetic field gradient power supply supplies current to the magnetic field gradient coils 310. The current supplied to the magnetic field gradient coils 310 is controlled as a function of time and may be ramped or pulsed.

[0087] Adjacent to the imaging zone 308 is a radio frequency coil 314 for manipulating the orientation of magnetic spins in the imaging zone 308 and for receiving radio transmissions from the spins in the imaging zone 308. A radio frequency antenna may include multiple coil elements. A radio frequency antenna may also be referred to as a channel or antenna. The radio frequency coil 314 is connected to a radio frequency transceiver 316. The radio frequency coil 314 and the radio frequency transceiver 316 may be replaced by separate transmit and receive coils, as well as separate transmitters and receivers. It is understood that the radio frequency coil 314 and the radio frequency transceiver 316 are representative. The radio frequency coil 314 is also intended to represent a dedicated transmit antenna and a dedicated receive antenna. Similarly, the transceiver 316 may also represent a separate transmitter and receiver. The radio frequency coil 314 may also have multiple receive / transmit elements, and the radio frequency transceiver 316 may have multiple receive / transmit channels.

[0088] The transceiver 316 and the tilt controller 312 are shown as being connected to the hardware interface 106 of the computer system 102 .

[0089] The memory 110 is further shown as containing pulse sequence commands 330. The pulse sequence commands are either commands or data that can be converted into commands that the computing system 104 uses to control the magnetic resonance imaging system to acquire k-space data. The memory 110 is shown as containing k-space data 332 acquired by the magnetic resonance imaging system for the field of view 309. The medical image 126 in this example is a magnetic resonance image. The medical image 126 was reconstructed from the k-space data 332.

[0090] FIG. 4 shows a flow chart illustrating a method of operating the medical system 300 of FIG. 3. First, in step 400, a medical imaging device, or in this case a magnetic resonance imaging system 302, is controlled to acquire medical data. In this example, the medical data is k-space data 332. Next, in step 402, a medical image 126 is reconstructed from the medical data 332. The method after step 302 then proceeds and executes steps 200, 202, 204, 206, and 208 as shown in FIG. 2. After step 208 is executed, step 404 is executed. In step 404, if a warning signal 138 is provided, the operation of the medical imaging device 302 is modified. A particularly useful example is when the image attributes 130 include a resolution estimate. The combination of the resolution estimate and the noise estimate can be useful to determine whether the magnetic resonance imaging system is functioning correctly. The method shown in FIG. 4 can be executed repeatedly, for example, after each image acquisition. For example, if both are out of bounds, this may indicate the need for a shimming operation or adjustment of the magnetic shims of the magnetic resonance imaging system.

[0091] 5 shows an example of how various neural networks of a medical system can be arranged in a coordinated manner. First, the medical image 126 is optionally inserted or input into a uniformity estimation neural network 502. This returns an image uniformity estimate 503. This may then optionally be input into an image uniformity correction algorithm 504. In this example, the image uniformity correction algorithm 504 takes the image uniformity estimate 503 and the medical image 126 as input, and then outputs a medical image 126' whose uniformity has been corrected. The uniformity corrected medical image 126' is then input into a noise estimation neural network 122. Optionally, if the uniformity estimation neural network 502 and the image uniformity correction algorithm 504 are not present, the medical image 126 is directly input into the noise estimation neural network 122.

[0092] The noise estimation neural network 122 then outputs a noise estimate 128. The medical image 126' and the noise estimate 128 are then input to a noise filter 506. The noise estimate 128 is used to control the function of the noise filter 506.

[0093] The noise filter 506 then outputs a filtered medical image 126'', with noise reduced by the noise filter 506, which is input to various other neural networks, the filtered medical image 126'' is shown as being input to an optional magnetic resonance weighting estimation neural network 508 which returns a magnetic resonance weighting dependency 510. This identifies the type of magnetic resonance weighting, for example T1 or T2, or T2 star weighting. Two examples of image quantification neural networks 124 are also shown, in this example having a resolution estimation neural network 512 and an artifact estimation neural network 516. The filtered medical image 126'' is input to the resolution estimation neural network 512 which in response outputs a resolution estimate 514, and the filtered medical image 126'' is input to the artifact estimation neural network 516 which returns an image artifact quantification 518. This can be either a segmentation or a mask, identifying the location of the artifact or providing a measurement such as the number of voxels likely to contain an image artifact.

[0094] The noise estimate, the resolution estimate, and the image artifact quantification 518 are input to the decision module 134. The magnetic resonance weighting dependency 510 may also optionally be input to the decision module 134. The decision module 134 then determines whether the medical image 126 meets a predetermined criterion 136. This can be done by looking at or providing various value boundaries for the values ​​in the image assessment 132, or by implementing the decision module 134 as a trained neural network. The image assessment 132 is not shown directly, but is a combination of the values ​​128, 510, 514, and 518.

[0095] 6 shows a further way in which neural networks of a medical system can be combined cooperatively. In this example, the noise estimate 128 is used to select the magnetic resonance weighting estimation neural network 508, the resolution estimation neural network 512, and the artifact estimation neural network 516. In essence, these neural networks are tuned to work well for a particular range of the noise estimate 128. Thus, it can be seen that these neural networks have the uniformity corrected medical image 126' input directly to them. In the absence of any uniformity estimation neural network 502 and image uniformity correction algorithm 504, the medical image 126 is input directly to the magnetic resonance weighting estimation neural network 508, the resolution estimation neural network 512, and the artifact estimation neural network 516.

[0096] 7 illustrates another alternative way in which neural networks in a medical system may be used collaboratively. In this example, a magnetic resonance weighting estimation neural network 508, a resolution estimation neural network 512, and an artifact estimation neural network 516 are trained to accept a noise estimate 128 as an input in addition to receiving a medical image 126 or 126'. By training these neural networks 508, 512, and 516 with various noise levels, the neural networks 508, 512, 516 are able to accurately provide their output values ​​510, 514, 518 when the noise estimate 128 is provided.

[0097] FIG. 8 illustrates further neural networks that may be used to identify foreign objects, such as pacemakers, metal bits, or other structures that are not part of the subject and that cause errors in the determination of the warning signal. The medical image 126 is shown as being input to a foreign object identification neural network 800. This may be a standard convolutional neural network used for image segmentation and identification of objects within the image. The foreign object identification neural network 800 then outputs a foreign object segmentation 802, which then enters a mask construction module 804. The mask construction module 804 may be used, for example, to identify different portions or sections of the foreign object segmentation 802 and construct an image mask 806 that identifies areas of the medical image 126 that contain foreign objects. The image mask 806 may then be provided as an additional input to any or all of the neural networks shown in FIGS. 5, 6, and 7. In this case, those neural networks that receive the image mask 806 have been trained during their training to ignore areas indicated by the image mask 806 as containing foreign objects. The technical result is that the neural networks of FIGS. 5, 6 and 7 ignore regions of the image mask when preparing the image assessment 132.

[0098] Image quality is a critical aspect for magnetic resonance imaging (MRI) and other imaging modalities, as it substantially impacts the evaluation of medical scans by radiologists. High image quality is characterized, for example, by a high signal-to-noise ratio (SNR) and high resolution. Image quality can be compromised by a wide range of factors and identifying those factors is crucial since harming patients by diagnosing them based on low quality and potentially flawed images must be avoided at all costs. Assessment of (global) image quality is traditionally performed by using phantom scans and analysis of the phantom images with respect to different image quality parameters. In the case of MRI this can be very time and resource consuming and is not routinely feasible.

[0099] An example could address this issue using a combination of several deep learning models (neural networks) each evaluating a specific image quality parameter such as resolution and SNR. A major advantage of this technique is that image quality can be monitored in real time, and thus the method of the present invention can inform the user in case of potential problems compromising image quality, which can be addressed directly.

[0100] Maintaining the highest possible image quality standards in MRI is crucial for diagnostic reliability and to enable, for example, multicenter studies. However, there are several factors that can lead to poor image quality, such as inappropriate or suboptimal scanning parameters, patient motion, system wear and tear, and technical defects within the MRI scanner. One way to detect poor image quality caused by inappropriate scanning parameters, system degradation, or technical defects is to continuously monitor the reconstructed images with respect to image quality parameters and notify the user in real time as soon as image quality is compromised for any reason. Quality assurance is also an important preparatory step before using the system after it has been installed at the site.

[0101] The traditional method of investigating the overall quality characteristics of an MRI scanner is by using phantom measurements and semi-automatically assessing phantom images for image quality parameters such as spatial resolution, signal-to-noise ratio (SNR), and image uniformity. However, this approach is very time-consuming and resource-intensive. While this may be acceptable when initially setting up the system, it is not feasible for monitoring image quality on a routine basis.

[0102] An example may overcome the need for phantom measurements using a set of specific neural networks (NNs) that are trained to analyze the reconstructed images and determine a wide range of image quality parameters such as resolution, SNR, and image uniformity, thus making it possible to evaluate the overall quality of the reconstructed images. Based on this evaluation, the system can inform the user if it detects a degradation in image quality for any reason. The tool can operate continuously in a fully automatic manner, for example monitoring image quality in real time.

[0103] An example is training an image-based NN to directly estimate the respective quality parameters on the reconstructed images. These parameters may be image resolution and SNR, but also other parameters such as image uniformity. The NN can be trained on real MR images, but also high-resolution low-noise photos (optical images from a camera) can be used as training data. When using natural photos for training, it is important to use an appropriate statistical noise model when simulating different SNR levels. To train the NN, a dataset is used with inputs containing images with varying resolution and SNR, respectively, as shown in Figure 9 below. A typical network topology for such a task is shown in Figure 12 below.

[0104] 9 shows various training images 900. In the top row 902, the training images show varying noise levels (signal to noise ratio). In the bottom row 904, the images show varying resolution. These are non-medical images, but can still be used to train the noise estimation neural network 122 and the resolution estimation neural network 512. Image quality degrades artificially from left to right. A neural network can be trained on such natural or optical images, so that an example can monitor MR image quality on the fly in clinical practice.

[0105] FIG. 10 shows the same training images 1000 with an image mask 1002 added. This blacks out parts of these images. However, the shape and amount of the image simulates what would be present in a magnetic resonance image. This helps the neural network to be better adapted to inspect magnetic resonance images during training. The optical image and the image mask 1000 can be modified. This is a very cheap and easy way to provide training data for training the image quantification neural network 124 or the noise estimation neural network 122.

[0106] FIG. 11 illustrates a method of using training images 900 with an image mask 1000. First, in step 1100, an untrained neural network is received. Next, in step 1102, training data is received, which may include natural or optical images with known noise levels or corresponding resolutions (or other image attributes). At least some of the input image pairs in some examples are modified optical images with an image mask as shown in FIG. 10. The modified optical image 1000 is generated, for example, by adding noise or other transformations to adjust the resolution to the optical image and applying the image mask 1002 to blacken parts of the optical image.

[0107] 12 shows an exemplary neural network architecture for the noise estimation neural network 122. A medical image 126 is shown as being input to a convolutional layer 1200. There are multiple convolutional layers stacked in sequence. The final convolutional layer 1200 outputs a signal to noise ratio 130. A similar neural network can be used for the image quantification neural network 124.

[0108] Once trained, the NNs are capable of estimating the resolution and SNR of any given MRI (or other modality) scan. Due to their short execution time, the NNs can be used continuously on the system to assess the image quality of all reconstructed images and notify the user as soon as an anomaly is detected with respect to any of the monitored parameters. In addition to the NN quantifying image quality, we envision another AI-based algorithm that analyzes the output of the NN and determines if quality is compromised by inappropriate scan parameters or technical defects.

[0109] For image-based measurements of SNR, instead of training a CNN to directly determine the noise level, it is also possible to acquire a series of images with potentially low SNR and apply a CNN-based AI denoising algorithm to these images. In a well-calibrated system, the pixel-based variance between denoised images, as well as the average distance to ground truth (obtained over longer scans), should both be minimal. This may be used as a quality test before using the system after it has been installed at the site.

[0110] This approach can also be used in an on-system learning manner: whenever the network detects degraded quality, the user can accept or reject the finding (or corrective action can be measured), so that the network receives additional training data to better adapt to the actual degradation. As an additional embodiment of the present invention, we envision a NN that is trained to quantify contrast (i.e., dependence on T1, T2, diffusion, etc.) in an image-based manner as well.

[0111] The presented example can allow monitoring of image quality in real time, is very versatile and can be used with all MRI scanners, but can also be transferred, for example, to CT scanners or cone-beam CT systems (IGT, RT) or ultrasound devices (practically all imaging systems). More specifically, it can be applied already to the projection data before reconstruction (CT). This makes iterative scanning and expensive troubleshooting processes with phantom measurements obsolete and ensures the best possible image quality at a single point in time. It can also be used for automatic alerts (services). In addition, it can be used for marketing purposes, for example, to compare the image quality of reconstructed images from different manufacturers.

[0112] While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered as illustrative or illustrative and not restrictive, and the invention is not limited to the disclosed embodiments.

[0113] Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. A computer program can be stored / distributed on a suitable medium, such as an optical storage medium or a solid-state medium provided together with or as part of other hardware, but also in other forms, such as via the Internet or other wired or wireless telecommunication systems. Any reference signs in the claims should not be interpreted as limiting the scope. [Explanation of symbols]

[0114] 100 Medical Systems 102 Computer 104 Computing Systems 106 Hardware Interface 108 User Interface 110 Memory 120 Machine Executable Instructions 122 Noise Estimation Neural Network 124 Neural Network for Image Quantification 126 Medical Imaging 126' Uniformity Corrected Medical Images 126'' Filtered Medical Image 128 Noise Estimation 130 Image Attributes 132 Image Evaluation 134 Decision Module 136 Prescribed standards 138 Warning Signal 200 medical images received 202 receiving a noise estimate in response to inputting a medical image to the noise estimation neural network; 204 Receive image attributes in response to inputting medical images into an image quantification neural network 206 Adding the resolution estimates and image attributes from each of at least one image quantification neural network to the image assessment. 208 Provide a warning signal in response to image evaluation that meets predetermined criteria 300 Medical Systems 302 Magnetic Resonance Imaging Equipment 304 Magnet 306 Magnet Bore 308 Imaging Zone 309 Field of view 310 Magnetic Gradient Coil 312 Gradient coil power supply 314 High Frequency Coil 316 Transceiver 318 Subject 320 Subject support part 330 Pulse Sequence Commands 332 k-space data (medical data) 400 Control medical imaging devices to acquire medical data 402 Reconstructing Medical Images from Medical Data 404 If a warning signal is provided, modify the operation of the medical imaging device. 502 Uniformity Estimation Neural Network 503 Image Uniformity Estimation 504 Image Uniformity Correction Algorithm 506 Noise Filter 508 Magnetic Resonance Weighting Estimation Neural Network 510 Magnetic Resonance Weighting Dependence 512 Resolution Estimation Neural Network 514 Resolution Estimate 516 Artifact Estimation Neural Network 518 Quantification of Image Artifacts 800 Foreign object identification neural network 802 Foreign object segmentation 804 Mask Configuration Module 806 Image Mask 900 Training Optical Images 902 Varying noise levels 904 Changing Resolution Optical image with 1000 image masks added 1002 Image Mask 1100 Received an untrained neural network 1102 Receive training data 1104 A noise estimation neural network or an image quantification neural network is provided by training an untrained neural network using training data. 1200 convolutional layers

Claims

1. A memory for storing machine-executable instructions and a plurality of neural networks, wherein the plurality of neural networks include a noise estimation neural network and at least one image quantification neural network, the noise estimation neural network is configured to output an estimated noise value of the medical image in response to receiving a medical image as input, and the at least one image quantification neural network is configured to output an image attribute representing one of the following in response to receiving the medical image as input: resolution, amount of blur, image uniformity indicating (in)appropriate field shimming in magnetic resonance imaging, magnetic resonance image contrast weighting, or identification of foreign objects in the medical image, A computing system wherein the execution of the machine-executable instructions is performed by the computing system. The steps include receiving the aforementioned medical image, The steps include receiving the noise estimate in response to inputting the medical image into the noise estimation neural network, A step of receiving the image attributes in response to inputting the medical image into the image quantification neural network, wherein the determination of the image attributes by the image quantification neural network depends on the noise estimate. The steps include adding image attribute and noise estimates from each of the at least one image quantification neural networks to the image evaluation, A step of providing a warning signal based on the image evaluation that meets predetermined criteria. A computing system that executes and A medical system having [a certain feature].

2. The medical system further includes a medical imaging device configured to acquire medical data describing a subject, and the execution of the machine-executable instructions is further performed by the computing system. The steps include controlling the medical imaging device to acquire the aforementioned medical data, The steps include: reconstructing the medical image from the medical data; If the aforementioned warning signal is provided, the steps of correcting the operation of the medical imaging device include: providing a user warning; adding the warning signal to the medical image; generating a medical imaging device repair request; triggering the reacquisition of the medical data; displaying the estimated resolution; displaying at least one image attribute; providing an operation command; providing a repair command; and any combination thereof. A medical system according to claim 1, which causes the following to be performed.

3. The medical imaging device is a magnetic resonance imaging system, according to claim 2.

4. The plurality of neural networks further comprises a uniformity estimation neural network, which is configured to output an image uniformity estimate in response to inputting the medical image into the uniformity estimation neural network, and the execution of the machine-executable instructions is further performed by the computing system. The steps include receiving the uniformity estimation in response to inputting the medical image into the uniformity estimation neural network, The steps include adding the uniformity estimation to the image evaluation. A medical system according to any one of claims 1 to 3, which causes the following to be performed.

5. The execution of the machine-executable instruction further involves the computing system, A step of applying an image uniformity correction algorithm to the medical image before inputting the medical image to the noise estimation neural network, wherein the image uniformity correction algorithm is controlled by the uniformity estimation, If the uniformity estimation satisfies a predetermined uniformity criterion, the step of triggering the reacquisition of the medical image data, The steps include requesting readjustment of the magnetic shim of the magnet of the medical imaging device, Those combinations and The medical system according to claim 4, which causes any one of the following to be performed.

6. The plurality of neural networks further comprises a magnetic resonance weighting estimation neural network, which is configured to estimate magnetic resonance weighting dependencies in response to receiving the medical image as input, and the execution of the machine-executable instructions is further performed by the computing system. The process involves receiving the magnetic resonance weighting dependency in response to inputting the medical image into the magnetic resonance weighting estimation neural network, The steps include adding the magnetic resonance weighting dependence to the image evaluation. A medical system according to any one of claims 1 to 3, which causes the following to be performed.

7. The medical imaging device is a computed tomography system, a C-arm computed tomography system, a planar X-ray system, a fluorescence fluoroscopy system, a positron emission tomography system, a single-photon emission computed tomography system, or an ultrasound system, according to claim 1 or 2.

8. The medical system according to any one of claims 1 to 3, wherein the execution of the machine-executable instructions further causes the computing system to apply a noise reduction algorithm to the medical image after receiving the noise estimate and before inputting the medical image to the at least one image quantification neural network.

9. The medical system according to any one of claims 1 to 3, wherein the at least one image quantification neural network is further configured to receive the noise estimate as input, and the execution of the machine-executable instruction further causes the computing system to input the noise estimate to the image quantification neural network before receiving the image attributes.

10. The medical system according to any one of claims 1 to 3, wherein each of the at least one image quantification neural networks is selected from a group of image attribute-specific image quantification neural networks using the noise estimate.

11. The aforementioned at least one image quantification neural network is A resolution estimation neural network configured to output an estimated resolution of a medical image in response to receiving the medical image as input, An artifact estimation neural network configured to output quantitative values ​​of image artifacts within a medical image in response to receiving the aforementioned medical image as input, Those combinations and A medical system according to any one of claims 1 to 3, having any one of the above.

12. The memory further comprises a foreign object detection neural network, which is configured to output a foreign object segmentation representing a foreign object in the medical image in response to the input of the medical image, each of the plurality of neural networks is further configured to receive an object mask for identifying areas to be ignored in the medical image, the object mask preferably having a smooth transition between the target area and the background area, and the execution of the machine-executable instructions is further performed by the computing system The steps include receiving the foreign object segmentation in response to inputting the medical image into the foreign object detection neural network, The steps include constructing an image mask from the aforementioned foreign object segmentation, The steps include inputting the image mask into each of the plurality of neural networks. A medical system according to any one of claims 1 to 3, which causes the following to be performed.

13. The memory further includes an evaluation neural network configured to output the warning signal when the image evaluation meets the predetermined criteria, and the execution of the machine-executable instruction is further performed by the computing system. The steps include inputting the aforementioned image evaluation into the evaluation neural network, If the predetermined criteria are met, the step of receiving the warning signal and A medical system according to any one of claims 1 to 3, which causes the following to be performed.

14. A computer program having machine-executable instructions for execution by a computing system and a plurality of neural networks, wherein the plurality of neural networks comprises a noise estimation neural network and at least one image quantification neural network, the noise estimation neural network is configured to output a noise estimate of the medical image in response to receiving a medical image as input, and the at least one image quantification neural network is configured to output an image attribute representing one of the resolution of the medical image, the amount of blur, image uniformity indicating the suitability of field shimming in magnetic resonance imaging, magnetic resonance image contrast weighting, or foreign object identification in response to receiving the medical image as input, and the execution of the machine-executable instructions is performed by the computing system. The steps include receiving the aforementioned medical image, The steps include receiving the noise estimate in response to inputting the medical image into the noise estimation neural network, A step of receiving the image attributes in response to inputting the medical image into the image quantification neural network, wherein the determination of the image attributes by the image quantification neural network depends on the noise estimate. The steps include adding the image attributes and noise estimates from each of the at least one image quantification neural networks to the image evaluation, A step of providing a warning signal based on the image evaluation that meets predetermined criteria. A computer program that executes something.

Images