Monitoring medical images using neural network generated image evaluation

Through the combination of image quantization neural network and noise estimation neural network, real-time monitoring of the noise and properties of medical images is solved, and the problem of low image quality monitoring efficiency in the prior art is achieved, and automatic adjustment and image quality improvement are achieved.

CN119072714APending Publication Date: 2024-12-03KONINKLIJKE PHILIPS NV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202380034061.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-04-14
Filing Date
2023-04-07
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Existing medical imaging technologies are inefficient in image quality monitoring and fault detection, making it difficult to effectively identify and solve image noise, resolution and uniformity problems in real time, resulting in a decline in image quality.

Method used

The image quantization neural network and noise estimation neural network are combined to monitor the noise and image properties of medical images in real time through deep learning technology, provide warning signals according to predetermined criteria, and automatically adjust the imaging equipment to ensure image quality.

Benefits of technology

It realizes rapid and accurate monitoring and automatic adjustment of medical image quality, reduces the need for time-consuming testing, and improves the accuracy of image reconstruction and the operation efficiency of equipment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119072714A_ABST
    Figure CN119072714A_ABST
Patent Text Reader

Abstract

A medical system (100, 300) is disclosed that includes a memory (110) storing machine executable instructions (120) and a plurality of neural networks. The plurality of neural networks includes a noise estimation neural network (122) and at least one image quantization neural network (124). The noise estimation neural network is configured to output a noise estimate (128) of a medical image (126, 126 ') in response to receiving the medical image as input. The at least one image quantization neural network is configured to output image attributes of the medical image in response to receiving the medical image as input (130). The medical system also includes a computing system (104). Execution of the machine executable instructions causes the computing system to: receive (200) the medical image; receiving (202) the noise estimate in response to inputting the medical image into the noise estimation neural network; receiving (204) the image attribute in response to inputting the medical image into the image quantization neural network, where a determination of the image attribute by the image quantization neural network depends on the noise estimate; appending the noise estimate and the image attribute from each of the at least one image quantization neural network to an image evaluation (132); a warning signal is provided (208) in accordance with the image evaluation that satisfies a predetermined criterion.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to medical imaging and, in particular, to monitoring image properties during medical imaging. Background Art

[0002] Various tomographic medical imaging techniques, such as magnetic resonance imaging (MRI), computed tomography, positron emission tomography, and single photon emission tomography, enable detailed visualization of the anatomical structure of a subject. If there is a malfunction or other problem with the medical imaging equipment, the quality of the image may be reduced.

[0003] U.S. patent application US2019 / 0340470 A1 discloses a method for evaluating image information in real time using a computer including one or more processors, comprising: obtaining a magnetic resonance (MR) image of a region of interest via a scanner, and providing data corresponding to at least a portion of the MR image to a deep learning model, wherein the deep learning model is previously trained based on one or more sets of training data. The method also includes evaluating the data using the deep learning model and data obtained from an image quality database to obtain an evaluated image quality value, and formulating an image quality classification in response to the evaluated image quality value. The method additionally includes outputting the image quality classification within a predetermined time period from the initial scan of the region of interest.

[0004] US patent application US2016 / 042499 discloses user identification of an image quality index and identification of an acquisition / reconstruction method based on the image quality index. Summary of the invention

[0005] The invention provides a medical system, a computer program and a method in the independent claims. Embodiments are given in the dependent claims.

[0006] Embodiments may provide a module for more efficiently determining image attributes. Various properties of an image, such as noise or resolution of the image, may be interrelated. Embodiments use a neural network referred to herein as an image quantization neural network to determine a noise estimate for a medical image. After obtaining the noise estimate, the noise estimate is used during determination of at least one other image attribute using the neural network referred to as the image quantization neural network. The noise estimate is used to control or modify the determination of the image attribute. The image attribute and the noise estimate are then used to determine whether the image quality meets one or more predetermined criteria. Depending on whether the image assessment reaches or meets the predetermined criteria, a warning signal is provided. In different examples, the warning signal may take different forms. In some examples, the warning signal may include a visual warning signal, an audible warning signal, an operator instruction, or a maintenance instruction may be provided. In other examples, the warning signal may include a machine command or modify the function of a medical imaging device. Due to the use of a neural network, the warning signal may be provided quickly even when the medical imaging device is acquiring a medical image of an object.

[0007] In one aspect, the present invention provides a medical system comprising a memory storing machine executable instructions and a plurality of neural networks. The machine executable instructions and the plurality of neural networks are executable by a computing system. The plurality of neural networks comprises a noise estimation neural network and at least one image quantization neural network. The noise estimation neural network is a label indicating a specific neural network. The label "at least one image quantization neural network" is a label indicating a specific neural network. Thus, each of the at least one image quantization neural network is a neural network.

[0008] The noise estimation neural network is configured to output a noise estimate of the medical image in response to receiving the medical image as an input. That is, the noise estimation neural network has an input configured to receive the medical image and is capable of outputting a noise estimate. In different examples, the noise estimate can take different forms. In one example, the noise estimate can be a noise estimate for the entire medical image, in which case it will be a single number. The noise estimate can be for a portion or quadrant of the medical image. In yet another example, the noise estimate can be for each pixel or voxel of the medical image, in which case the noise estimate can be a multi-dimensional map of the noise within the medical image.

[0009] Thus, the noise estimation neural network can be any of a variety of neural networks configured to receive an image. In particular, U-net and ResNet architectures will work particularly well. The noise estimate is an image or image data, then for example a U-net architecture, a DeepLab based architecture, DnCnn or F-Net can be used. If the noise estimate is a number for the entire image or a portion of an image, a classification neural network can be used. For example, the neural network can be a ResNet, DenseNet or VGG network.

[0010] The training of a noise estimation neural network that outputs a noise estimate is provided in more detail below. Briefly, the noise estimation neural network may be trained by inputting an image having a known noise level, and then using the resulting noise estimate of the output to compare to the known noise level. For example, an image having low or absent noise may have noise artificially added to it to provide the training data.

[0011] The at least one image quantization neural network is configured to output an image attribute of the medical image in response to receiving the medical image as input. Likewise, the output image attribute can be for the entire image, it can be for a portion or quadrant of the medical image, or it can be a pixel-by-pixel or voxel-by-voxel map of the image attribute in the medical image. The same neural network architecture mentioned above for the noise estimation neural network can also be used to implement the image quantization neural network. In particular, U-net and ResNet architectures will work particularly well. If the noise estimate is an image or image data, then a U-net architecture, a DeepLab-based architecture, DnCnn or F-Net can be used, for example. If the noise estimate is a number for the entire image or a portion of an image, a classification neural network can be used. For example, the neural network can be a ResNet, DenseNet or VGG network.

[0012] The image quantization neural network may be trained using an input image with known or artificially added image attributes and then comparing at the output with the known or added image attributes to perform, for example, deep learning.

[0013] The medical system further comprises a computing system. Execution of the machine executable instructions causes the computing system to receive the medical image. The medical image may be received in various ways, for example, it may be transmitted to the computing system via a network or other telecommunication system, the medical image may be retrieved from a memory where it is stored, or if the medical system comprises a medical imaging device, such as an MRI or CT system, the medical image may be received by the medical system that acquired the medical image.

[0014] Execution of the machine executable instructions further causes the computing system to receive the noise estimate in response to inputting the medical image into the noise estimation neural network. Execution of the machine executable instructions further causes the computing system to receive the image attribute in response to inputting the medical image into the image quantization neural network. The determination of the image attribute by the image quantization neural network is dependent on the noise estimate. As described in detail below, this may have several different possibilities.

[0015] In one example, the noise estimate is used to select a specific image quantization neural network. For example, various image quantization neural networks may be trained using images having a specific noise range within the medical image. In yet another example, the image quantization neural network may be configured to receive the noise estimate, and during the training of the image quantization neural network, known noise of the image used for training is also input, in which case the image quantization neural network is already adapted to receive and adjust its response based on the noise estimate. In yet another example, the noise estimate may be used to trigger noise removal from the medical image before the medical image is input into the image quantization neural network. These different possibilities provide a very flexible way of improving the quality of determining the attribute using the image quantization neural network.

[0016] In another embodiment, execution of the machine executable instructions further causes the computing system to append the noise estimate and the image attributes from each of the at least one image quantization neural network to the image evaluation. Execution of the machine executable instructions further causes the computing system to provide a warning signal based on the image evaluation using predetermined criteria. For example, if the evaluation meets one or more predetermined criteria, then the warning signal may be provided. In different examples, the warning signal may take different forms. For example, it may be an auditory, visual, or tactile signal provided to an operator. In other examples, the warning signal may be a signal provided to other software modules or executable code to modify behavior. In other examples, the warning signal may affect, for example, the operation of a medical imaging device used to acquire the medical image, resulting in either reacquisition or change.

[0017] In other cases, the warning signal can be used to modify the behavior of the reconstruction of the medical image. For example, it can trigger the use of an alternative reconstruction method for reconstructing the medical image based on the original medical data acquired by the medical imaging device. The warning signal can also be a display of the image evaluation or part of the image evaluation on the display. The warning signal can also be used to mark the medical image in a medical database.

[0018] In different examples, the predetermined criteria can take different forms. For example, it can be used to check whether the value of the image assessment is within a specific range or a specific statistical range. For example, if one or more of the elements or contents of the image assessment are image data, then these images can be processed for average values, mean values, high values, and low values, and many of these values ​​can be used as predetermined criteria.

[0019] This embodiment may be beneficial because it may provide an efficient way of ensuring that the medical image is correctly reconstructed or may be used to adjust the operation of the medical system and / or the medical image.

[0020] In another embodiment, the medical system further comprises a medical imaging device, which may also be referred to as a medical imaging device, configured to acquire medical data describing an object. Execution of the machine executable instructions further causes the computing system to control the medical imaging device to acquire the medical data. Execution of the machine executable instructions further causes the computing system to reconstruct the medical image based on the medical data. Execution of the machine executable instructions further causes the computing system to modify the operation of the medical imaging device if the warning signal is provided. The modification of the operation of the medical imaging device comprises any one of: providing a user alert, appending the warning signal to the medical image, generating a medical imaging device repair request, triggering re-acquisition of the medical data, displaying a resolution estimate, displaying the at least one image attribute, providing operating instructions, providing repair instructions, and combinations thereof.

[0021] This embodiment may be particularly beneficial because it may provide a way to monitor the medical imaging device in operation and ensure that it is operating properly. For example, this may then automatically trigger a service or repair request. This may also be beneficial because it may limit the amount of preventive maintenance that needs to be performed. It may also be beneficial because it may eliminate the need to periodically perform time-consuming testing of medical imaging equipment.

[0022] An example of this is a magnetic resonance imaging system where noise and resolution estimates are performed semi-regularly using measurements on a phantom to ensure that the magnetic resonance imaging is functioning properly. This embodiment may eliminate or reduce the need for such time consuming testing.

[0023] In another embodiment, the medical device is a magnetic resonance imaging system. As mentioned above, this may be beneficial because it may provide a method of continuously monitoring whether the magnetic resonance imaging system is functioning properly without requiring or reducing the need to perform noise and, for example, resolution checks.

[0024] In another embodiment, the plurality of neural networks further comprises a uniformity estimation neural network. The uniformity estimation neural network is configured to output an image uniformity estimate in response to inputting the medical image into the uniformity estimation neural network. The uniformity estimation neural network is a neural network. The uniformity estimation neural network is configured to take the medical image as input and then output an image uniformity estimate. In the context of magnetic resonance imaging, the image uniformity may change, for example due to inappropriate magnetic shimming. This may result in variations in contrast across the medical image. The uniformity estimation neural network may be implemented, for example, using the neural network architecture mentioned above for the noise estimation neural network. In order to train the estimation neural network, a magnetic resonance image with known image uniformity may be used as input during training, and this known image uniformity may be used as ground truth data during the deep learning training of the uniformity estimation neural network.

[0025] Execution of the machine executable instructions further causes the computing system to receive the uniformity estimate in response to inputting the medical image into the uniformity estimation neural network. Execution of the machine executable instructions further causes the computing system to append the uniformity estimate to the image evaluation. This may be beneficial, for example, because if the uniformity is outside certain limits, it may provide a way to indicate that there is a problem with the magnetic resonance imaging system.

[0026] In another embodiment, the uniformity estimate is received before the medical image is input to the noise estimation neural network. In one option, execution of the machine executable instructions further causes the computing system to apply an image uniformity correction algorithm to the medical image before the medical image is input to the noise estimation neural network. The image uniformity correction algorithm is controlled by the uniformity estimate. For example, an algorithmic image uniformity correction algorithm can be used to adjust the contrast of the medical image. As an alternative or in combination with the first option, execution of the machine executable instructions further causes the computing system to trigger reacquisition of the medical image data if the uniformity estimate satisfies a predetermined uniformity criterion.

[0027] As another option, which may be performed alone or in combination with the first two options, execution of the machine executable instructions further causes the computing system to request a magnetic shimming re-adjustment of the magnet of the magnetic resonance imaging system if it matches the predetermined uniformity criterion. These various options may be beneficial because they provide a way to improve the uniformity of medical images. Image uniformity may indicate proper shimming or improper shimming in magnetic resonance imaging. In particular, it may be beneficial to perform this operation before performing the noise estimation because this may make the noise estimation and determination of other image attributes more accurate.

[0028] In another embodiment, the plurality of neural networks further comprises a magnetic resonance weighted estimation neural network. The magnetic resonance weighted estimation neural network is configured to estimate magnetic resonance weighted correlations in response to receiving the medical image as input. The magnetic resonance weighted estimation neural network as used herein encompasses a neural network configured to receive an image and configured to output a magnetic resonance weighted estimate. The weighting in this regard refers to the type of contrast or weighting used for the magnetic resonance imaging. For example, this may be T1, T2, T2 star, diffusion weighted or other type of weighting for the magnetic resonance image. The weighting implements a weighting of the corresponding contrast weights or contrast encodings of the medical image or the imaging data reconstructing the medical image. As mentioned, this can be trained by taking a magnetic resonance image with a known image weighting type and then using it as input and ground truth data for training the magnetic resonance weighted estimation neural network using, for example, deep learning.

[0029] As is true for the noise estimation neural network, the magnetic resonance weighted estimation neural network may also be any of a variety of neural networks configured to receive an image. In particular, U-net and ResNet architectures will work particularly well. The noise estimate is an image or image data, and then for example a U-net architecture, a DeepLab based architecture, DnCnn or F-Net may be used. If the noise estimate is a number for the entire image or a portion of an image, a classification neural network may be used. For example, the neural network may be a ResNet, DenseNet or VGG network.

[0030] Execution of the machine executable instructions further causes the computing system to receive the magnetic resonance weighted correlation in response to inputting the medical image into the magnetic resonance weighted estimation neural network and append the magnetic resonance weighted correlation to the image evaluation. In some examples, determining the magnetic resonance weighted correlation is performed after performing image uniformity correction and / or before performing noise estimation or correction.

[0031] In another embodiment the medical imaging device is a tomographic medical imaging device.

[0032] In another embodiment, the medical imaging device is a computed tomography system.

[0033] In another embodiment, the medical imaging device is a C-arm computed tomography system. A C-arm computed tomography system is typically a portable computed tomography system having a detector and an X-ray source mounted on a C-arm.

[0034] In another embodiment, the medical imaging device is a planar X-ray system.

[0035] In another embodiment, the medical imaging device is a fluoroscopy system or a digital fluoroscopy system.

[0036] In another embodiment, the medical imaging device is a positron emission tomography system.

[0037] In another embodiment, the medical imaging device is a single photon emission tomography system.

[0038] In another embodiment, the medical imaging device is an ultrasound system.

[0039] In another embodiment, execution of the machine executable instructions further causes the computing system to apply a noise removal algorithm to the medical image after receiving the noise estimate and before inputting the medical image into the at least one image quantization neural network. This embodiment may be beneficial because it may enable more accurate execution of the image quantization neural network.

[0040] In another embodiment, the at least one image quantization neural network is further configured to receive the noise estimate as an input. Execution of the machine executable instructions further causes the computing system to input the noise estimate into the image quantization neural network prior to receiving the image attribute. This embodiment may be beneficial because it may provide a more accurate way to determine the quantization of an image. (Note to self: insert standard text describing how to put multiple inputs into a neural network here)

[0041] In another embodiment, each of the at least one image quantization neural networks is selected from a set of image attribute specific image quantization neural networks using the noise estimate. In this embodiment, for each individual image quantization, there is a set or population of image quantization neural networks that have been trained using different amounts of noise, so once the noise estimate is used to select the appropriate image quantization neural network from the image attribute specific image quantization neural networks, an accurate value for the image quantization is obtained.

[0042] In another embodiment, the at least one image quantization neural network comprises a resolution estimation neural network configured to output a resolution estimate of the medical image in response to receiving the medical image as input. This embodiment may be beneficial because the resolution estimate may be much more accurate after taking into account the noise estimate. The training and neural network architecture previously described for the image quantization neural network is applicable to the resolution estimation neural network. Images with known resolution (or known spatially correlated resolution) may be used for training.

[0043] In another embodiment, the at least one image quantization neural network includes an artifact estimation neural network configured to output a quantification of image artifacts in the medical image in response to receiving the medical image as input. For example, the artifact estimation neural network may provide the location of an artifact within the magnetic resonance image, or may indicate an area that may contain image artifacts. This may be beneficial because it may also be more accurate after taking into account the noise estimate. The training and neural network architecture previously described for the image quantization neural network is applicable to the resolution estimation neural network. Images with known artifact or motion artifact levels may be used for training. In the case where the output is an identification of an artifact location, a labeled image with the location of the artifact marked may be used for training.

[0044] The resolution estimation neural network may be trained by taking an image with a known resolution and then using this as the input during training and using the known resolution as the ground truth. Similarly, the artifact estimation neural network may be trained by using a magnetic resonance image with known artifacts in it and then using this knowledge about the artifacts as the ground truth to perform the training. The artifact estimation neural network may be trained to return the image properties in the form of a quantization of the amount of blur in the input (source) image. For this purpose, the artifact estimation may be trained on ground truth images with a predetermined degree of blur. For this purpose, for example, magnetic resonance images reconstructed from magnetic resonance signals with a predetermined degree of (e.g. lateral) signal attenuation. Alternatively, intentionally blurred versions of natural (video) images with a controllable degree of loss of resolution may be used as target images for training.

[0045] In another embodiment, the memory further comprises a foreign body detection neural network. The foreign body detection neural network is configured to output a foreign body segmentation in the medical image in response to inputting the medical image. Each of the plurality of neural networks is further configured to receive an object mask to identify areas to be ignored in the medical image. The execution of the machine executable instructions further causes the computing system to receive the foreign body segmentation in response to inputting the medical image into the foreign body detection neural network. The execution of the machine executable instructions further causes the computing system to construct the image mask based on the foreign body segmentation. The execution of the machine executable instructions further causes the computing system to input the image mask into each of the plurality of neural networks.

[0046] For example, the foreign body detection neural network can be a convolutional neural network that is trained to locate objects within the image. This is a very classic and typical use of convolutional neural networks. They can be trained by providing medical images in which foreign bodies have been marked. The marked images can then be used to train the neural network by performing deep learning on the foreign body detection neural network. The following are example neural network architectures that can be used: For example, a U-net architecture, a DeepLab-based architecture, a DnCnn, or an F-Net can be used.

[0047] Multiple neural networks can be trained so that they also receive image masks that ignore parts of the image. In this way, noise and other image properties can be determined without them being affected by foreign objects.

[0048] In another embodiment, the memory further comprises an evaluation neural network configured to output the warning signal if the image assessment satisfies a predetermined criterion. The execution of the machine executable instructions further causes the computing system to input the image assessment into the evaluation neural network and receive the warning signal if the predetermined criterion is met. In some examples, the predetermined criterion can be determined by using a lookup table or range acceptable for noise estimation and various image quantizations. In this embodiment, the neural network can be trained to receive the image assessment and then provide the warning signal if the value of the image assessment is unacceptable. For example, this can be trained by taking various combinations of image assessments from actual measurement results and then assigning them as acceptable or unacceptable. The values ​​can be input into the neural network and then the decision of whether they should cause a warning can be used as the truth. The evaluation neural network can be implemented, for example, using a standard neural network for performing classification tasks, such as a ResNet, DenseNet, or VGG network.

[0049] In another aspect, the invention provides a computer program comprising machine executable instructions and a plurality of neural networks for execution by a computing system. The plurality of neural networks comprises a noise estimation neural network and at least one image quantization neural network. The noise estimation neural network is configured to output a noise estimate of the medical image in response to receiving the medical image as input. The at least one image quantization neural network is configured to output an image attribute of the medical image in response to receiving the medical image as input. Execution of the machine executable instructions causes the computing system to receive the medical image.

[0050] Execution of the machine executable instructions further causes the computing system to receive the noise estimate in response to inputting the medical image into the noise estimation neural network. Execution of the machine executable instructions further causes the computing system to receive the image attribute in response to inputting the medical image into the image quantization neural network. Determination of the image attribute by the image quantization neural network is dependent on the noise estimate. Execution of the machine executable instructions further causes the computing system to append the noise estimate and the image attribute from each of the at least one image quantization neural network to an image assessment. Execution of the machine executable instructions further causes the computing system to provide a warning signal based on the image assessment meeting a predetermined criterion.

[0051] In another aspect, the present invention provides a method for training a noise estimation neural network. The method includes receiving an untrained noise estimation neural network. The method also includes receiving training data. The training data includes an input image pair and a true noise value. At least a portion of the input image pair is a modified optical image. The modified optical image is generated by adding noise to the optical image and applying an image mask to the light-shielded portion of the optical image. For example, when performing magnetic resonance imaging, most of the image is dark or black. The noise estimation neural network can be trained so that it ignores these dark or black areas of the magnetic resonance image. Typically, normal optical images, such as images that would be acquired by an optical camera, cannot be used to train a neural network that will estimate the noise of a magnetic resonance image or other medical image because there are these areas that are dark or do not contain any information. As used herein, an optical image encompasses an image acquired using an optical camera. An optical image may also be referred to as a natural image or a photograph (or digital photograph).

[0052] The image mask need not be binary, where portions of the image are completely blacked out. To simulate the signal drop at the transition between portions of a magnetic resonance image depicting anatomical structures and its mostly dark background, one can use a smoothing mask rather than a binary mask. There is a gradual transition between the image and the background rather than an abrupt or binary transition. For example, one can take an ordinary binary mask and smooth it with a Gaussian kernel. When multiplying a natural image with a smoothing mask, one can achieve a similar signal drop at the mask boundaries. This can provide a neural network that operates more reliably. In this example, the image mask may include an object portion that simulates the object in the medical image and a background portion that simulates the darkened portion of the medical image. In this example, there is then a smooth transition between the object portion and the background portion.

[0053] However, in addition to inputting controlled amounts of noise into the images, the use of these optical images can be used to train the neural networks by applying image masks. This is extremely advantageous as it enables the ability to generate large amounts of training data so that the noise estimation neural network is much better than if it had been trained using magnetic resonance images. Additionally, modifying the optical images by adding noise enables very accurate training of the neural network. If natural magnetic resonance images are used, there is an interaction between noise and resolution. It can be very difficult to accurately train the noise estimation neural network using only magnetic resonance images.

[0054] It should be understood that one or more of the aforementioned embodiments of the present invention may be combined as long as the combined embodiments are not mutually exclusive.

[0055] As will be appreciated by those skilled in the art, aspects of the present invention may be implemented as an apparatus, method, or computer program product. Accordingly, aspects of the present invention may take the form of a fully hardware embodiment, a fully software embodiment (including firmware, resident software, microcode, etc.), or a combination of software and hardware aspects (all of which may be generally referred to herein as "circuits," "modules," or "systems"). In addition, aspects of the present invention may take the form of a computer program product implemented in one or more computer-readable media having computer executable code implemented thereon.

[0056] Any combination of one or more computer-readable media may be utilized. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. As used herein, "computer-readable storage medium" encompasses any tangible storage medium that can store instructions that can be executed by a processor or a computing system of a computing device. A computer-readable storage medium may be referred to as a computer-readable non-transient storage medium. A computer-readable storage medium may also be referred to as a tangible computer-readable medium. In some embodiments, a computer-readable storage medium may also be capable of storing data that can be accessed by a computing system of a computing device. Examples of computer-readable storage media include, but are not limited to, floppy disks, magnetic hard disk drives, solid-state hard disks, flash memories, USB thumb drives, random access memories (RAM), read-only memories (ROM), optical disks, magneto-optical disks, and register files of computing systems. Examples of optical disks include compact disks (CDs) and digital versatile disks (DVDs), such as CD-ROMs, CD-RWs, CD-Rs, DVD-ROMs, DVD-RWs, or DVD-R disks. The term computer-readable storage medium also refers to various types of recording media that can be accessed by a computer device via a network or communication link. For example, data may be retrieved on a modem, the Internet, or a local area network. Computer executable code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0057] A computer readable signal medium may include a propagated data signal having computer executable code implemented therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including but not limited to electromagnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and is capable of conveying, propagating, or transmitting a program used by or in conjunction with an instruction execution system, apparatus, or device.

[0058] "Computer memory" or "memory" is an example of a computer-readable storage medium. Computer memory is any memory that is directly accessible by a computing system. "Computer storage" or "storage" is another example of a computer-readable storage medium. A computer storage is any non-volatile computer-readable storage medium. In some embodiments, a computer storage may also be computer memory, or vice versa.

[0059] As used herein, "computing system" encompasses an electronic component capable of executing a program or machine executable instructions or computer executable code. References to computing systems including examples of "computing system" should be interpreted as possibly including more than one computing system or processing core. The computing system may be, for example, a multi-core processor. A computing system may also refer to a collection of computing systems within a single computer system or distributed among multiple computer systems. The term computing system should also be interpreted as possibly referring to a collection or network of computing devices that each include a processor or computing system. Machine executable code or instructions may be executed by multiple computing systems or processors that may be within the same computing device or may even be distributed across multiple computing devices.

[0060] Machine executable instructions or computer executable code may include instructions or programs that cause a processor or other computing system to perform aspects of the present invention. Computer executable code for performing operations for various aspects of the present invention may be written in any combination of one or more programming languages ​​and compiled into machine executable instructions, and the one or more programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, C++, etc. and conventional procedural programming languages ​​such as "C" programming language or similar programming languages. In some instances, the computer executable code may take the form of a high-level language or take a precompiled form and be used in conjunction with a decoder that generates machine executable instructions at work. In other instances, the machine executable instructions or computer executable code may take the form of programming for a programmable logic gate array.

[0061] The computer executable code may run entirely on the user's computer, partially on the user's computer (as a stand-alone software package), partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0062] Aspects of the present invention are described with reference to the flowchart, diagram and / or block diagram of the method, device (system) and computer program product according to an embodiment of the present invention. It should be understood that, when applicable, each frame or part of the frame of the flowchart, diagram and / or block diagram can be implemented by a computer program instruction in the form of a computer executable code. It should also be understood that, when mutually non-exclusive, the combination of frames in different flowcharts, diagrams and / or block diagrams can be combined. These computer program instructions can be provided to the computing system of a general-purpose computer, a special-purpose computer or other programmable data processing device that produces a machine, so that the instruction running via the computing system of a computer or other programmable data processing device creates a unit for implementing the function / action specified in a flowchart and / or one or more block diagram frames.

[0063] These machine-executable instructions or computer program instructions may also be stored in a computer-readable medium, which can direct a computer, other programmable data processing apparatus, or other device to work in a specific manner, so that the instructions stored in the computer-readable medium produce an article of manufacture including instructions for implementing the functions / actions specified in the flowchart and / or one or more block diagram blocks.

[0064] The machine-executable instructions or computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device, thereby producing a computer-implemented process, so that the instructions running on the computer or other programmable apparatus provide a process for the functions / actions specified in the flowchart and / or one or more block diagram blocks.

[0065] As used herein, "user interface" is an interface that allows a user or operator to interact with a computer or computer system. "User interface" may also be referred to as a "human-machine interface device". A user interface may provide information or data to an operator and / or receive information or data from an operator. A user interface may enable the input from an operator to be received by a computer and may provide output to a user from a computer. In other words, the user interface may allow an operator to control or manipulate a computer, and the interface may allow a computer to indicate the effect of the control or manipulation of an operator. The display of data or information on a display or graphical user interface is an example of providing information to an operator. The reception of data by a keyboard, mouse, tracking ball, touch pad, pointing stick, graphic input board, joystick, game controller, webcam, earphone, pedal, wired gloves, remote controller, and accelerometer are all examples of user interface components that implement the reception of information or data from an operator.

[0066] As used herein, "hardware interface" encompasses an interface that enables a computing system of a computer system to interact with and / or control an external computing device and / or apparatus. A hardware interface may allow a computing system to send a control signal or instruction to an external computing device and / or apparatus. A hardware interface may also enable a computing system to exchange data with an external computing device and / or apparatus. Examples of hardware interfaces include, but are not limited to, a universal serial bus, an IEEE 1394 port, a parallel port, an IEEE 1284 port, a serial port, an RS-232 port, an IEEE-488 port, a Bluetooth connection, a wireless LAN connection, a TCP / IP connection, an Ethernet connection, a control voltage interface, a MIDI interface, an analog input interface, and a digital input interface.

[0067] As used herein, a "display" or "display device" encompasses an output device or user interface suitable for displaying images or data. A display may output visual, audio, and / or tactile data. Examples of displays include, but are not limited to, computer monitors, television screens, touch screens, tactile electronic displays, Braille screens, cathode ray tubes (CRTs), memory tubes, bi-stable displays, electronic paper, vectorscopes, flat panel displays, vacuum fluorescent displays (VFs), light emitting diode (LED) displays, electroluminescent displays (ELDs), plasma display panels (PDPs), liquid crystal displays (LCDs), organic light emitting diode displays (OLEDs), projectors, and head mounted displays.

[0068] Medical data is defined herein as recorded measurements made by a medical imaging device describing an object. Medical imaging data can be reconstructed into a medical image. A medical image is defined herein as a reconstructed two-dimensional or three-dimensional visualization of anatomical data contained within the medical data. The visualization can be performed using a computer.

[0069] K-space data is defined herein as measurements of radio frequency signals emitted by atomic spins recorded using the antenna of a magnetic resonance apparatus during a magnetic resonance imaging scan. Magnetic resonance data is an example of medical data.

[0070] A Magnetic Resonance Imaging (MRI) image or MR image is defined herein as a reconstructed two-dimensional or three-dimensional visualization of anatomical data contained within k-space data. The visualization may be performed using a computer. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In the following preferred embodiments, the present invention will be described, by way of example only, with reference to the accompanying drawings, in which:

[0072] Figure 1 An example of a medical system is illustrated;

[0073] Figure 2The diagram shows the use of Figure 1 A flowchart of a method of a medical system;

[0074] Figure 3 An example of a medical system is illustrated;

[0075] Figure 4 The diagram shows the use of Figure 3 A flowchart of a method of a medical system;

[0076] Figure 5 illustrates an example of how various neural networks of a medical system may be arranged in cooperation;

[0077] Figure 6 illustrates another example of how various neural networks of a medical system may be arranged in cooperation;

[0078] Figure 7 illustrates another example of how various neural networks of a medical system may be arranged in cooperation;

[0079] Figure 8 Another neural network that can be used to identify foreign objects in medical images is illustrated;

[0080] Fig. 9 illustrates an optical image that can be used to train a neural network to determine noise levels or resolution in a medical image;

[0081] Fig.10 illustrates additional optical images that may be used to train a neural network to determine noise levels or resolution in a medical image;

[0082] Fig.11 illustrates a method for training a neural network; and

[0083] Fig.12 A neural network topology suitable for outputting a single numerical value such as the signal-to-noise level or the resolution of an image is illustrated.

[0084] Reference numerals list

[0085] 100 Medical Systems

[0086] 102 Computer

[0087] 104 Computing System

[0088] 106 Hardware Interface

[0089] 108 User Interface

[0090] 110 Memory

[0091] 120 machine executable instructions

[0092] 122 Noise Estimation Neural Network

[0093] 124 Image Quantization Neural Network

[0094] 126 Medical Images

[0095] 126'Uniformity Corrected Medical Images

[0096] 126" filtered medical image

[0097] 128 Noise Estimation

[0098] 130 Image properties

[0099] 132 Image Evaluation

[0100] 134 Decision-making Module

[0101] 136 Booking Criteria

[0102] 138 Warning Signs

[0103] 200 receiving a medical image;

[0104] 202 Receiving a noise estimation in response to inputting a medical image into a noise estimation neural network

[0105] 204 Receiving image attributes in response to inputting the medical image into the image quantization neural network

[0106] 206 appending the resolution estimate and image attributes from each of the at least one image quantization neural networks to the image evaluation

[0107] 208 Providing a warning signal based on image evaluation that meets predetermined criteria

[0108] 300 Medical Systems

[0109] 302 Magnetic Resonance Imaging System

[0110] 304 magnet

[0111] 306 magnet bore

[0112] 308 Imaging Area

[0113] 309 Field of View

[0114] 310 magnetic field gradient coil

[0115] 312 Magnetic Field Gradient Coil Power Supply

[0116] 314 RF Coil

[0117] 316 transceiver

[0118] 318 objects

[0119] 320 object support

[0120] 330 pulse sequence command

[0121] 332k spatial data (medical data)

[0122] 400 Controlling medical imaging equipment to acquire medical data

[0123] 402Reconstructing medical images from medical data

[0124] 404 If a warning signal is provided, modify the operation of the medical imaging device

[0125] 502 Uniformity Estimation Neural Network

[0126] 503 Image Uniformity Estimation

[0127] 504 Image Uniformity Correction Algorithm

[0128] 506 Noise Filter

[0129] 508 Magnetic Resonance Weighted Estimation Neural Network

[0130] 510 MRI weighted correlation

[0131] 512 resolution estimation neural network

[0132] 514 Resolution Estimation

[0133] 516 Artifact Estimation Neural Network

[0134] 518 Image Artifact Quantification

[0135] 800 foreign body recognition neural network

[0136] 802 Foreign body segmentation

[0137] 804 Mask Construction Module

[0138] 806 Image Mask

[0139] 900 optical images for training

[0140] 902 Variation of noise level

[0141] 904 Change resolution

[0142] 1000 optical images with image masks added

[0143] 1002 Image Mask

[0144] 1100 Receive untrained neural network

[0145] 1102 Receiving training data

[0146] 1104 providing a noise estimation neural network or an image quantization neural network by training an untrained neural network using training data 1200 convolutional layers DETAILED DESCRIPTION

[0147] Like numbered elements in these figures are either equivalent elements or perform the same function. If the function is equivalent, an element that has been discussed previously will not necessarily be discussed in a later figure.

[0148] Figure 1 An example of a medical system is illustrated. Medical system 100 is shown to include a computer 102 having a computing system 104. Computer 102 may, for example, represent one or more networked computers. Likewise, computing system 104 may represent one or more computing systems, such as multiple processors or multiple cores working in concert. Computing system 104 is shown connected to an optional hardware interface 106 and an optional user interface 108. Hardware interface 106 may, for example, enable computing system 104 to communicate and control other components of medical system 100. For example, if medical system 100 also includes a medical imaging device.

[0149] The computing system 104 is also shown in communication with a memory 110. The memory 110 is intended to represent different types of memory or storage devices accessible to the computing system 104. The memory 110 is shown as containing machine executable instructions 120. The machine executable instructions 120 enable the computing system 104 to perform basic computational and image processing tasks. The memory 110 is also shown as containing a noise estimation neural network 122. The memory is also shown as containing an image quantization neural network 124.

[0150] The memory is also shown as containing a medical image 126. The memory 110 is also shown as containing a noise estimate. The noise estimate 128 is obtained by inputting the medical image into the noise estimation neural network 122 and receiving the noise estimate 128 as an output. The memory is also shown as containing an image attribute 130. The image attribute 130 is obtained by inputting the medical image 126 and / or the noise estimate 128 into the image quantization neural network 124 and receiving the image attribute 130 in response. The noise estimate 128 is used to control how the image quantization neural network 124 provides the image attribute 130. This can be done in a variety of different ways. In one case, for a given image attribute, there are multiple image quantization neural networks 124 and they have been trained using different amounts of noise. The noise estimate 128 can then be used to select an appropriate image quantization neural network 122 for a particular value of the noise estimate 128.

[0151] In other variations, the image quantization neural network 124 is trained using known noise during the training process. In this case, the noise estimate 120 is directly input into the image quantization neural network 124. In other examples, there may be an image filter that is controlled using the noise estimate 128. For example, a medical image is first input into the noise estimation neural network 122, and then the resulting noise estimate 128 is used to control the image filter. The filter image is then input into the image quantization neural network 124.

[0152] The memory 132 is also shown as containing an image assessment 132. The image assessment 132 is a collection of any image attributes 130 and noise estimates 128 that have been determined. The memory is also shown as containing predetermined criteria 136, which are then used to determine whether the image assessment 132 is within acceptable limits. For example, the predetermined criteria 136 can be used to determine whether the medical imaging device used to acquire the medical image 126 is operating properly. This can be done in various ways. In one example, there can be a decision module 134 that compares the predetermined criteria 136 to the image assessment 132. This can be essentially equivalent to a lookup table, or comparing the values ​​within the image assessment 132 to various ranges or limits. If the image assessment does not meet the predetermined criteria, then the decision module 134 can provide a signal 138. Alternatively, the decision module 134 can be an evaluation neural network that is trained to receive the image assessment 132 and then provide a signal 138 if the predetermined criteria are not met.

[0153] Figure 2 The diagram shows the operation Figure 1 FIG. 1 is a flow chart of a method of a medical system for detecting a medical image 126. First, in step 200, a medical image 126 is received. Next, in step 202, a noise estimate 128 is received in response to inputting the medical image 126 into the noise estimation neural network 122. Then, in step 204, an image attribute 130 is received in response to inputting the medical image 126 into the image quantization neural network 124. As previously described, the determination of the image attribute by the image quantization neural network depends on the noise estimate 128. Then, in step 206, the noise estimate and the image attribute 130 are appended to the image evaluation 132. Finally, in step 208, a warning signal 138 is provided based on the image evaluation meeting the predetermined criteria 136.

[0154] Figure 3 Another example of a medical system 300 is illustrated. Embodiments of the present invention may be used with a variety of different medical imaging devices. Magnetic resonance imaging system 302 is intended as an example. In addition to including magnetic resonance imaging system 302, Figure 3 The medical system 300 in is similar to Figure 1 Medical system 100 in.

[0155] Figure 3 The example illustrated in is a magnetic resonance imaging system 302. However, the magnetic resonance imaging system may be replaced by other types of tomographic medical imaging devices. Examples may be a computed tomography system, a C-arm computed tomography system, a single photon emission tomography system, a positron emission tomography system, a fluoroscopy system, an X-ray system, or an ultrasound system.

[0156] The magnetic resonance imaging system 302 includes a magnet 304. The magnet 304 is a superconducting cylindrical magnet having a bore 306 passing through it. The use of different types of magnets is also possible; for example, it is also possible to use both a split cylindrical magnet and a so-called open magnet. A split cylindrical magnet is similar to a standard cylindrical magnet, except that the cryostat has been split into two sections to allow access to the isoplane of the magnet, which magnet can be used, for example, in conjunction with charged particle beam therapy. An open magnet has two magnet sections, one above the other, with a space large enough in between to receive a subject: the arrangement of the two sections is similar to the arrangement of Helmholtz coils. Open magnets are common because the subject is less constrained. Inside the cryostat of the cylindrical magnet there is a series of superconducting coils.

[0157] Within the bore 306 of the cylindrical magnet 304 there is an imaging zone 308 in which the magnetic field is strong enough and uniform enough to perform magnetic resonance imaging. A field of view 309 is shown within the imaging zone 308. Magnetic resonance data is typically acquired for the field of view 309. A region of interest may be the same as the field of view 309, or it may be a sub-volume of the field of view 309. An object 318 is shown supported by an object support 320 so that at least a portion of the object 318 is within the imaging zone 308 and the field of view 309.

[0158] There is also a set of magnetic field gradient coils 310 within the bore 306 of the magnet, which are used to acquire preliminary magnetic resonance data to spatially encode magnetic spins within the imaging zone 308 of the magnet 304. The magnetic field gradient coils 310 are connected to a magnetic field gradient coil power supply 312. The magnetic field gradient coils 310 are intended to be representative. Typically, the magnetic field gradient coils 310 include three independent coil sets for spatial encoding in three orthogonal spatial directions. The magnetic field gradient coil power supply supplies current to the magnetic field gradient coils 310. The current supplied to the magnetic field gradient coils 310 is controlled according to time, and the current can be ramped or pulsed.

[0159] Adjacent to the imaging zone 308 is a radio frequency coil 314 for manipulating the orientation of magnetic spins within the imaging zone 308 and for receiving radio transmissions from spins also within the imaging zone 308. The radio frequency antenna may include multiple coil elements. The radio frequency antenna may also be referred to as a channel or antenna. The radio frequency coil 314 is connected to a radio frequency transceiver 316. The radio frequency coil 314 and the radio frequency transceiver 316 may be replaced by separate transmit and receive coils and separate transmitters and receivers. It should be understood that the radio frequency coil 314 and the radio frequency transceiver 316 are representative. The radio frequency coil 314 is also intended to represent a dedicated transmit antenna and a dedicated receive antenna. Similarly, the transceiver 316 may also represent a separate transmitter and receiver. The radio frequency coil 314 may also have multiple receive / transmit elements, and the radio frequency transceiver 316 may have multiple receive / transmit channels.

[0160] The transceiver 316 and the gradient controller 312 are shown connected to the hardware interface 106 of the computer system 102 .

[0161] The memory 110 is also shown as containing pulse sequence commands 330. Pulse sequence commands are commands or data that can be converted into commands that the computing system 104 uses to control the magnetic resonance imaging system to acquire k-space data. The memory 110 is shown as containing k-space data 332 that has been acquired by the magnetic resonance imaging system for the field of view 309. The medical image 126 in this example is a magnetic resonance image. The medical image 126 is reconstructed based on the k-space data 332.

[0162] Figure 4 The diagram shows the operation Figure 3 Flow chart of a method of a medical system 300 of the present invention. First, in step 400, a medical imaging device, or in this case a magnetic resonance imaging system 302, is controlled to acquire medical data. In this example, the medical data is k-space data 332. Next, in step 402, a medical image 126 is reconstructed based on the medical data 332. The method after step 302 then continues and executes steps 200, 202, 204, 206 and 208, as shown in FIG. Figure 2 After executing step 208, step 404 is executed. In step 404, if the warning signal 138 is provided, the operation of the medical imaging device 302 is modified. A particularly useful example is when the image attribute 130 includes a resolution estimate. The combination of the resolution estimate and the noise estimate can be useful in determining whether the magnetic resonance imaging system is operating properly. For example, Figure 4 The method shown may be repeated after each image acquisition. For example, if both are out of bounds, it may indicate that an adjustment of the shimming operation or of the magnetic shimming of the magnetic resonance imaging system is required.

[0163] Figure 5An example of how the various neural networks of a medical system may be arranged in cooperation is illustrated. First, the medical image 126 is optionally inserted or input into the uniformity estimation neural network 502. This returns an image uniformity estimate 503. This may then optionally be input into an image uniformity correction algorithm 504. In this example, the image uniformity correction algorithm 504 receives as input the image uniformity estimate 503 and the medical image 126; it then outputs a medical image 126' that has been corrected for uniformity. Next, the uniformity-corrected medical image 126' is input into the noise estimation neural network 122. Optionally, if the uniformity estimation neural network 502 and the image uniformity correction algorithm 504 are not present, then the medical image 126 is directly input into the noise estimation neural network 122.

[0164] The noise estimation neural network 122 then outputs a noise estimate 128. The medical image 126' and the noise estimate 128 are then input into a noise filter 506. The noise estimate 128 is used to control the function of the noise filter 506.

[0165] The noise filter 506 then outputs a filtered medical image 126" which has its noise reduced by the noise filter 506. These are input into various other neural networks. The filtered medical image 126' is shown as being input into an optional magnetic resonance weight estimation neural network 508 which returns magnetic resonance weight correlations 510. For example, this identifies the type of magnetic resonance weighting, such as T1 or T2, or T2 star weighting. Two examples of image quantization neural networks 124 are also shown. In this example, we have a resolution estimation neural network 512 and an artifact estimation neural network 516. The filtered medical image 126" is input into the resolution estimation neural network 512 and in response, a resolution estimate 514 is output. The filtered medical image 126" is then input into the artifact estimation neural network 516 and an image artifact quantification 518 is returned. This can be either a segmentation or mask identifying the location of artifacts or providing a number of voxels that may contain image artifacts.

[0166] The noise estimate, resolution estimate, and image artifact quantification 518 are input into the decision module 134. The magnetic resonance weighted correlation 510 may also optionally be input into the decision module 134. The decision module 134 then decides whether the medical image 126 meets the predetermined criteria 136. This may be by looking at or providing various value boundaries for the values ​​in the image assessment 132, or by implementing the decision module 134 as a trained neural network. The image assessment 132 is not directly shown, but will be a combination of the values ​​128, 510, 514, and 518.

[0167] Figure 6Another way in which the neural networks of a medical system can be synergistically combined is illustrated. In this example, the noise estimate 128 is used to select the magnetic resonance weighted estimation neural network 508, the resolution estimation neural network 512, and the artifact estimation neural network 516. In essence, these neural networks have been tailored to work appropriately for the specific range of the noise estimate 128. In this way, it can be seen that for these neural networks, the uniformity corrected medical image 126' is directly input into them. If the optional uniformity estimation neural network 502 and the image uniformity correction algorithm 504 are not present, then the medical image 126 is directly input into the magnetic resonance weighted estimation neural network 508, the resolution estimation neural network 512, and the artifact estimation neural network 516.

[0168] Figure 7 Another alternative way in which neural networks of a medical system can be used in conjunction is shown. In this example, a magnetic resonance weight estimation neural network 508, a resolution estimation neural network 512, and an artifact estimation neural network 516 have been trained to accept as input, in addition to receiving a medical image 126 or 126', a noise estimate 128. By training these neural networks 508, 512, and 516 with various noise levels, the neural networks 508, 512, 516 can correctly provide their output values ​​510, 514, 518 when provided with a noise estimate 128.

[0169] Figure 8 Another neural network is illustrated which may be used to identify foreign bodies such as pacemakers, metal drill bits, or other structures which are not part of the object and which would result in an erroneous determination of a warning signal. A medical image 126 is shown as being input into a foreign body identification neural network 800. This may be a standard convolutional neural network which is used for image segmentation and identifying objects in an image. The foreign body identification neural network 800 then outputs a foreign body segmentation 802 which then enters a mask construction module 804. The mask construction module 804 may, for example, be used to identify different portions or compartments of the foreign body segmentation 802 and construct an image mask 806 which identifies the areas in the medical image 126 that contain the foreign body. The image mask 806 may then be provided to a Figure 5 , 6 and 7 as additional inputs to any or all of the neural networks shown. In this case, the neural networks receiving image mask 806 have been trained during their training to ignore areas indicated by image mask 1806 as containing foreign matter. The technical result is that Figure 5 , 6 The neural network in and 7 then ignores regions of the image mask when preparing image evaluation 132 .

[0170] Image quality is a critical aspect of magnetic resonance imaging (MRI) and other imaging modalities, as it has a significant impact on the radiologist's evaluation of medical scans. High image quality is characterized, for example, by a high signal-to-noise ratio (SNR) and high resolution.

[0171] Image quality can be impaired by various factors, and identifying these factors is crucial, as harming the patient by basing a diagnosis on low-quality, possibly even flawed images must be avoided at all costs. Traditionally, the evaluation of (overall) image quality is performed by employing phantom scanning and analysis of the phantom images with respect to different quality parameters. For MRI, this can be very time- and resource-consuming and is not feasible on a routine basis.

[0172] An example may utilize a combination of several deep learning models (neural networks) to solve the problem, each evaluating a specific image quality parameter such as, for example, resolution and SNR. A key advantage of this technique is that image quality can be monitored in real time, and therefore our approach can notify the user in the event of potential problems compromising image quality, which can then be addressed directly.

[0173] Maintaining the highest possible standards for image quality in MRI is crucial for diagnostic confidence and to allow, for example, multicenter studies. However, there are several factors that may lead to reduced image quality, such as insufficient or suboptimal scanning parameters, patient motion, system wear and tear, and technical defects within the MRI scanner. One way to detect degradation of image quality caused by insufficient scanning parameters, system degradation or technical defects is to continuously monitor the reconstructed images with respect to image quality parameters and notify the user in real time when the image quality is compromised for some reason. Moreover, quality assurance is an important preparatory step before the system is used after its installation in the field.

[0174] The traditional way to study the overall quality characteristics of an MRI scanner is to take phantom measurements and semi-automatically evaluate the phantom images towards image quality parameters such as spatial resolution, signal-to-noise ratio (SNR), and image uniformity. However, this approach is very time and resource consuming. This may be acceptable when initially setting up a system, but is not feasible for monitoring image quality on a daily basis.

[0175] An example may overcome the need for phantom measurements with a series of specific neural networks (NNs) trained to analyze the reconstructed images and determine various image quality parameters, such as resolution, SNR and image uniformity, and thus allow an assessment of the overall quality of the reconstructed image. Based on this assessment, the system may notify the user if it detects a degradation in image quality for some reason. The tool may run continuously in a fully automated manner, which allows, for example, real-time monitoring of image quality.

[0176] Example An image-based neural network can be trained to estimate the corresponding quality parameters directly on the reconstructed image. These parameters can be image resolution and SNR, but also other parameters like image uniformity. The NN can be trained on real MR images, but high-resolution low-noise photos (such as optical images from a camera) can also be used as training data. When using natural photos for training, it is crucial to use the right statistical noise model when simulating different SNR levels. To train the NN, a dataset is used as input containing images with different resolutions and SNRs, respectively, as follows Fig. 9 As shown below Fig.12 A typical network topology for such a task is depicted in .

[0177] Fig. 9 Various training images 900 are shown. In the top row 902, the training images show different noise levels (signal-to-noise ratios). In the bottom row 904, the images show different resolutions. Although these are non-medical images, they can still be used to train the noise estimation neural network 122 and the resolution estimation neural network 512. The image quality is artificially further degraded from left to right. Examples can train neural networks on such natural or optical images to be able to monitor MR image quality on the fly in clinical practice.

[0178] Fig.10 The same training images 1000 are shown with an image mask 1002 added. Portions of these images are now blacked out. However, the shape and amount of the images simulate what would be present in a magnetic resonance image. This helps the neural network better adapt to examining magnetic resonance images during training. The optical images and image mask 1000 can vary. This is a very cheap and simple way to provide training data for training the image quantization neural network 124 or the noise estimation neural network 122.

[0179] Fig.11 A method using training images 900 with image masks 1000 is illustrated. First, in step 1100, an untrained neural network is received. Next, in step 1102, training data is received. It may include natural images or optical images, where a known noise level or corresponding resolution (or other image properties) is known. In some examples, at least part of the input image pair is a modified optical image with an image mask, such as Fig.10 A modified optical image 1000 is generated by adding noise or other transformations to adjust, for example, the resolution of the optical image and applying an image mask 1002 to the light-blocked portions of the optical image.

[0180] Fig.12An exemplary neural network architecture for the noise estimation neural network 122 is illustrated. A medical image 126 is shown as being input into a convolutional layer 1200. There are many convolutional layers stacked in sequence. The final convolutional layer 1200 outputs a signal-to-noise ratio 130. A similar neural network can also be used for the image quantization neural network 124.

[0181] Once the NN is trained, it will be able to estimate the resolution and SNR of any given MRI (or other modality) scan. Due to its short runtime, the NN can be used continuously on the system to evaluate the image quality of each reconstructed image and can notify the user once an abnormality is detected with respect to any monitored parameter. In addition to the NN that quantifies image quality, we envision another AI-based algorithm that analyzes the output of the NN and determines whether the quality is compromised by inadequate scan parameters or technical flaws.

[0182] Regarding image-based measurements of SNR, as an alternative to training a CNN to determine noise levels directly, it is also possible to acquire a series of images with potentially low SNR and apply a CNN-based AI denoising algorithm to these images. In a well-calibrated system, both the pixel-based variance between denoised images and the average distance to the truth (acquired in a longer scan) should be minimal. This can be used as a quality test before using the system after it has been installed in the field.

[0183] The method can also be used in a systematically learned fashion. Each time the network detects degraded quality, the user can accept or reject the finding (or can measure corrective action), and the network thus receives additional training data to better adapt to the actual degradation.

[0184] As a conventional embodiment of our invention, we envision that the NN is trained to also quantify contrast in an image-based manner (ie, correlation to T1, T2, diffusion, etc.).

[0185] The presented example can enable real-time monitoring of image quality which is very versatile and can be used with all MRI scanners and can be transferred to, for example, CT scanners or cone beam CT systems (IGT, RT) or ultrasound devices (practically all imaging systems). More specifically, it can already be applied to the projection data before reconstruction (CT). It makes repeated scans and expensive troubleshooting processes by phantom measurements obsolete and ensures the highest image quality at any given time. It can also be used for automatic alarms (services). In addition, it can be used for marketing purposes, for example to compare the image quality of reconstructed images from different manufacturers.

[0186] While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary only and not restrictive; the invention is not limited to the disclosed embodiments.

[0187] By studying the drawings, the specification and the appended claims, those skilled in the art can understand and implement other variations of the disclosed embodiments when practicing the claimed invention. In the claims, the word "comprising" does not exclude other elements or steps, and the word "one" or "an" does not exclude multiple. A single processor or other unit can implement the functions of several items recorded in the claims. Although specific measures are recorded in mutually different dependent claims, this does not indicate that a combination of these measures cannot be used to advantage. The computer program can be stored / distributed on a suitable medium, such as an optical storage medium or solid-state medium provided together with other hardware or as part of other hardware, but the computer program can also be distributed in other forms, such as distributed via the Internet or other wired or wireless telecommunications systems. Any figure mark in the claims should not be interpreted as limiting the scope.

Claims

1. A medical system (100, 300), comprising: A memory (110) storing machine executable instructions (120) and a plurality of neural networks, wherein the plurality of neural networks include a noise estimation neural network (122) and at least one image quantization neural network (124); wherein the noise estimation neural network is configured to output a noise estimate (128) of a medical image in response to receiving a medical image (126, 126') as input; wherein the at least one image quantization neural network is configured to output an image attribute (130) representing any one of the following in response to receiving the medical image as input: resolution of the medical image, amount of blur, image uniformity indicating proper shimming or improper shimming in magnetic resonance imaging, magnetic resonance image contrast weighting, or identification of foreign matter; A computing system (104), wherein execution of the machine executable instructions causes the computing system to: receiving (200) the medical image; receiving (202) the noise estimate in response to inputting the medical image into the noise estimation neural network; receiving (204) the image attribute in response to inputting the medical image into the image quantization neural network, wherein determination of the image attribute by the image quantization neural network is dependent on the noise estimate; appending the noise estimate and the image attribute from each of the at least one image quantization neural network to an image evaluation (132); A warning signal is provided (208) based on the image evaluation satisfying predetermined criteria.

2. The medical system according to claim 1, wherein: The medical system also includes a medical imaging device (302) configured to acquire medical data (332) describing a subject (318), wherein execution of the machine-executable instructions further causes the computing system to: controlling (400) the medical imaging device to acquire the medical data; reconstructing (402) the medical image based on the medical data; and If the warning signal is provided, the operation of the medical imaging device is modified (404), wherein the modification of the operation of the medical imaging device includes any of the following: providing a user alert, appending the warning signal to the medical image, generating a medical imaging device repair request, triggering re-acquisition of the medical data, displaying a resolution estimate, displaying at least one image attribute, providing operating instructions, providing repair instructions, and combinations thereof.

3. The medical system according to claim 2, wherein: The medical device is a magnetic resonance imaging system (302).

4. The medical system according to claim 1, 2 or 3, wherein: The plurality of neural networks further comprises a uniformity estimation neural network (502), wherein the uniformity estimation neural network is configured to output an image uniformity estimate in response to inputting the medical image into the uniformity estimation neural network, wherein execution of the machine-executable instructions further causes the computing system to: receiving the uniformity estimate in response to inputting the medical image into the uniformity estimation neural network; and The uniformity estimate is appended to the image evaluation.

5. The medical system according to claim 4, wherein: receiving the uniformity estimate prior to inputting the medical image to the noise estimation neural network, wherein execution of the machine-executable instructions further causes the computing system to perform any of the following: applying an image uniformity correction algorithm (504) to the medical image before inputting the medical image to the noise estimation neural network, wherein the image uniformity correction algorithm is controlled by the uniformity estimate; and triggering reacquisition of the medical image data if the uniformity estimate satisfies a predetermined uniformity criterion; requesting a readjustment of the magnetic shimming of a magnet of the medical device, and Its combination.

6. A medical system according to any one of the preceding claims, wherein: The plurality of neural networks further comprises a magnetic resonance weighted estimation neural network, wherein the magnetic resonance weighted estimation neural network is configured to estimate a magnetic resonance weighted correlation in response to receiving the medical image as input, wherein execution of the machine-executable instructions further causes the computing system to: receiving the magnetic resonance weighted correlation in response to inputting the medical image into the magnetic resonance weighted estimation neural network; and The magnetic resonance weighted correlation is appended to the image evaluation.

7. The medical system according to claim 1 or 2, wherein: The medical imaging device is a computed tomography system, a C-arm computed tomography system, a planar X-ray system, a fluoroscopy system, a positron emission tomography system, a single photon emission computed tomography system or an ultrasound system.

8. The medical system according to any one of the preceding claims, wherein: Execution of the machine-executable instructions further causes the computing system to apply a noise removal algorithm to the medical image after receiving the noise estimate and before inputting the medical image into the at least one image quantization neural network.

9. The medical system according to any one of claims 1 to 7, wherein: The at least one image quantization neural network is further configured to receive the noise estimate as an input, wherein execution of the machine-executable instructions further causes the computing system to input the noise estimate into the image quantization neural network prior to receiving the image attribute.

10. The medical system according to any one of claims 1 to 7, wherein: Each of the at least one image quantization neural network is selected from a set of image attribute specific image quantization neural networks using the noise estimate.

11. The medical system according to any one of the preceding claims, wherein: The at least one image quantization neural network comprises an artifact estimation neural network configured to output a quantification of image artifacts in the medical image in response to receiving the medical image as input, Wherein, the at least one image quantization neural network includes any one of the following: a resolution estimation neural network configured to output a resolution estimate of the medical image in response to receiving the medical image as input, an artifact estimation neural network configured to output a quantification of image artifacts in the medical image in response to receiving the medical image as input, and Its combination.

12. The medical system according to any one of the preceding claims, wherein: The memory further comprises a foreign body detection neural network (800), wherein the foreign body detection neural network is configured to output a foreign body segmentation (802) in response to inputting the medical image, the foreign body segmentation representing the foreign body in the medical image, wherein each of the plurality of neural networks is further configured to receive an object mask to identify areas to be ignored in the medical image, wherein the object mask preferably has a smooth transition between an object portion and a background portion, wherein execution of the machine executable instructions further causes the computing system to: receiving the foreign body segmentation in response to inputting the medical image into the foreign body detection neural network (802); constructing the image mask according to the foreign body segmentation; and The image mask is input into each of the plurality of neural networks.

13. A medical system according to any one of the preceding claims, wherein: The memory further comprises an evaluation neural network (134) configured to output the warning signal if the image assessment satisfies the predetermined criterion, and wherein execution of the machine executable instructions further causes the computing system to: inputting the image assessment into the evaluation neural network; and If the predetermined criteria are met, the warning signal is received.

14. Operation of a computing system (104), wherein: The plurality of neural networks include a noise estimation neural network (122) and at least one image quantization neural network (124); wherein the noise estimation neural network is configured to output a noise estimate (128) of the medical image in response to receiving the medical image as input; wherein the at least one image quantization neural network is configured to output an image attribute (130) of the medical image in response to receiving the medical image as input; wherein execution of the machine-executable instructions causes the computing system to: receiving (200) the medical image; receiving (202) the noise estimate in response to inputting the medical image into the noise estimation neural network; receiving (204) the image attribute in response to inputting the medical image into the image quantization neural network, wherein determination of the image attribute by the image quantization neural network is dependent on the noise estimate; appending the noise estimate and the image attribute from each of the at least one image quantization neural network to an image evaluation (132); A warning signal is provided (208) based on the image evaluation meeting predetermined criteria.

15. A method for training a neural network, wherein: The method comprises: receiving (1100) an untrained neural network, wherein the untrained neural network has an input configured to receive a medical image, wherein the untrained neural network has an output configured to output a noise estimate or an image property, receiving (1102) training data, wherein the training data comprises an input image pair and ground truth data, wherein the ground truth data is noise ground truth data or image attribute ground truth data, wherein at least a portion of the input image pair is a modified optical image (900, 1000), wherein the modified optical image is generated by adding noise to an optical image or modeling the image attribute in the input image, wherein the modified optical image is further modified by applying an image mask (1002) to an obscured portion of the modified optical image; and A noise estimation neural network (122) or an image quantization neural network (124) is provided (1104) by training the untrained neural network using the training data.