Improved medical scan protocol for in-scanner patient data acquisition analysis

Abstract

To provide a method of performing a medical scan of a subject using a medical imaging system.SOLUTION: There is provided a method of performing a medical scan of a subject using a medical imaging system. The method comprises the steps of: a) initiating a medical scan session; b) performing, using a medical imaging system, a first image acquisition sequence to obtain first image scan data; c) performing, using a computer system, image analysis on the first image scan data acquired during the first image acquisition sequence to identify one or more quantitative indicators of pathology and / or image quality; d) on the basis of the identification of one or more quantitative indicators at the step c), determining, whether any additional image acquisition sequences are required or not; and, if so required: e) determining a second image acquisition sequence on the basis of the one or more quantitative indicators; and f) performing a second image acquisition sequence, using the medical imaging system during the medical scan session to obtain second image scan data.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an improved method and apparatus for in-scanner patient data acquisition and analysis. More particularly, the present invention relates to an improved method and apparatus for scanning, selecting and acquiring in response to one or more indicators. At least one indicator may include a metric or quantitative parameter indicative of pathology. [Background technology]

[0002] A non-invasive technique for imaging the brain and other body areas is the computed tomography (CT) scan, which combines a series of X-ray images taken from different angles around the body and uses computational methods to produce cross-sectional pictures (slices) of the bones, blood vessels, and soft tissues inside the body.

[0003] Another technique is Positron Emission Tomography (PET). PET can be used to produce detailed three-dimensional images of the inside of the body. PET scans utilize radioactive tracers. Radioactive tracers are molecules that contain small amounts of radioactive material that can be detected by a PET scan. Radioactive tracers are designed to accumulate in cancerous tumors or areas of inflammation. Radioactive tracers can also be made to bind to specific proteins in the body.

[0004] A more preferred technique is structural magnetic resonance imaging (MRI). MRI is a non-invasive technique that examines the physical structure of the body (e.g., calculating tissue volume). It is of high value for monitoring tissue damage, especially in neurodegenerative diseases.

[0005] As is well known, MRI is based on the magnetization of atomic nuclei: a large, uniform external magnetic field orients proteins within the aqueous nuclei of the tissue under examination. The orientation is then perturbed by an external radio frequency (RF) signal.

[0006] The nuclei return to their resting orientation through a number of different relaxation processes as the RF signal is emitted. By varying the emission and detection sequence of the RF pulses, different properties of the tissue under examination can be measured. The repetition time (TR) is the amount of time between successive pulse sequences applied to the same slice. The echo time (TE) is the time between the delivery of the RF pulse and the receipt of the echo signal.

[0007] Many MRI techniques are available. T1-weighted and T2-weighted scans are common. T1 (longitudinal relaxation time) is the time constant that determines the rate at which excited protons reorient with an applied external magnetic field. T2 (transverse relaxation time) is the time constant that determines the rate at which excited protons lose phase coherence with nuclei whose spins are orthogonal to the applied external magnetic field. T1-weighted images are characterized by short TE and TR times. Conversely, T2-weighted images are produced by using longer TE and TR times.

[0008] The more widely used sequence is fluid-attenuated inversion recovery (FLAIR). FLAIR is similar to T2-weighted imaging, but has much longer TE and TR times. Using this technique, abnormalities remain bright, but cerebrospinal fluid is attenuated and therefore dark in the acquired image. Such systems are also called T2 FLAIR scans.

[0009] Diffusion weighted imaging (DWI) is a form of MRI based on measuring the random Brownian motion of water molecules within a voxel (volume pixel) of tissue being measured. In general, tissues with higher cell density or cellular swelling exhibit lower diffusion coefficients. Diffusion is particularly useful in tumor characterization and cerebral ischemia.

[0010] Contrast-enhanced MRI (CE-MRI) and dynamic contrast-enhanced MRI (DEC-MRI) are also available techniques. In CE-MRI, a gadolinium contrast agent is used to improve the clarity of the available images. CE-MRI can improve the visibility of, for example, inflammation, blood vessels, and tumors. In DCE-MRI, multiple MR images are acquired sequentially following administration of the contrast agent. This allows the contrast of the contrast agent ("wash-in" and "wash-out") to be monitored, allowing for improved detection of, for example, vascular lesions and tumors.

[0011] A further technique is the Gradient Echo (GRE) MRI scan. This is produced by a single RF pulse in combination with a gradient reversal. Following the RF pulse, the initial negative part of the gradient lobe causes phase dispersion of the precessing spins. When this gradient is reversed, the spins refocus, forming a gradient (recalled) echo. GRE scans typically include short TR and TE values, thus providing fast signal acquisition. Thus, GRE sequences can provide a fast imaging and MR angiography technique.

[0012] Susceptibility weighted imaging (SWI) is a 3D high spatial resolution velocity-corrected gradient echo MRI sequence that utilizes compounds with paramagnetic, diamagnetic and ferromagnetic properties. Such paramagnetic compounds include deoxyhemoglobin, ferritin and hemosiderin. Diamagnetic compounds include bone mineral and dystrophic calcification.

[0013] A variation of GRE is susceptibility weighted angiography (SWAN), which allows high-resolution visualization of both cerebral veins and arteries in one sequence without the administration of contrast agents and with scan times much shorter than other configurations.

[0014] Turbo Spin Echo (TSE) (also known as Fast Spin Echo - FSE) is an adapted form of the regular Spin Echo (SE) acquisition technique designed to reduce imaging times. In a standard SE sequence, a single echo is measured during each repetition time (TR). In TSE, multiple echoes are recorded after each TR. This is achieved by transmitting a series of 180 degree inversion pulses at predetermined intervals according to different phase encoding gradients and recording the corresponding echo signals. In this way, multiple phase encoding steps can be encoded after a single 90 degree pulse.

[0015] The above is a non-exhaustive list and other techniques are contemplated.

[0016] A scan may include one or more of the above tests. Often the techniques are combined to provide enhanced medical data in a process known as co-restoration or image fusion. Indeed, scanners capable of performing multiple diagnostic procedures in a single scan are available, such as single photon emission computed tomography / computed tomography (SPECT / CT) and positron emission tomography / computed tomography (PET / CT) units capable of performing multiple imaging examinations simultaneously.

[0017] Regardless of the technique used, there are several problems with existing scanning arrangements: Patient scanning is expensive for the service provider and potentially inconvenient and stressful for the patient; therefore, it is desirable to minimize the number of times a patient needs to be scanned.

[0018] However, in many cases, when patient scan data is acquired and later analyzed, it is found that the scan data cannot be used in the diagnostic process, either due to inaccurate or insufficient acquisition, or due to quality control deficiencies that result in image quality that is too poor for medical diagnostic purposes. In such an event, it is necessary to later recall the patient for a further scan. This is clearly undesirable and can result in exposing the patient to additional and unnecessary risks (e.g., when using PET or when contrast agents are utilized). Summary of the Invention [Problem to be solved by the invention]

[0019] Thus, there is a need in the art for an improved process that reduces the likelihood of having to recall a patient for an additional scan so that additional images can be acquired. Technical problems exist in that existing methods of acquiring scan data are too slow and inadequately adapted to allow acquisition of all necessary data during a single scanning session and to prevent unnecessary scanning steps from being performed. [Means for solving the problem]

[0020] The present invention addresses the above problems in embodiments. The invention as described and claimed below may be used in conjunction with all of the techniques described above, without limitation.

[0021] According to a first aspect of the present invention there is provided a method of performing a medical scan of an object using a medical imaging system comprising: a) initiating a medical scanning session; b) during the medical scanning session performing a first image acquisition sequence using the medical imaging system to acquire first image scan data; c) using a computer system performing image analysis on the first image scan data acquired during said first image acquisition sequence to identify one or more quantitative indicators of pathology and / or image quality; d) based on said identification of the one or more quantitative indicators in step c) using a computer system determining whether any additional image acquisition sequences are required during said medical scanning session and if so, e) using the computer system determining a second image acquisition sequence based on the one or more quantitative indicators; and f) during the medical scanning session performing a second image acquisition sequence using the medical imaging system to acquire second image scan data.

[0022] In one embodiment, the first image scan data and / or the second image scan data comprises two-dimensional image scan data.In one embodiment, the first image scan data and / or the second image scan data comprises three-dimensional image scan data.

[0023] In one embodiment, the first image scan data and / or the second image scan data include one or more images of the selected body part of the subject. In one embodiment, the first image scan data and / or the second image scan data include one or more 2D and / or 3D images of the selected body part of the subject.

[0024] In one embodiment, the quantitative indices are derived from a prior training process that utilizes unrelated image scan data.

[0025] In one embodiment, step c) is performed using a classifier and the one or more quantitative indicators comprise voxel classifications. In one embodiment, step c) further comprises g) generating a voxel classification map from the first image scan data. In one embodiment, step g) further comprises h) aggregating the voxel classification map into an overall classification of the first image scan data.

[0026] In one embodiment, the method further comprises the steps of: i) identifying any blobs in the voxel classification map that belong to a predetermined voxel classification; j) comparing the identified blobs to a predetermined threshold; and k) identifying a positive indication of potential pathology if the predetermined threshold is exceeded.

[0027] In one embodiment, step i) comprises: l) performing one or more of anisotropic Gaussian blob detection, isotropic Gaussian blob detection and / or ridge detection to identify said blobs.

[0028] In one embodiment, the computer system includes a neural network algorithm. In one embodiment, the neural network algorithm includes a convolutional neural network algorithm. In one embodiment, the neural network algorithm includes a U-net. In one embodiment, the convolutional neural network includes an encoder and a decoder. In one embodiment, the encoder includes a convergent path and the decoder includes a divergent path. In one embodiment, the encoder includes multiple convolutions, each followed by a normalized linear unit and a pooling operation.

[0029] In one embodiment, the method further comprises, prior to step a), l) training a neural network algorithm to identify said quantitative indicators using a training set comprising a set of medical scan tuples. In one embodiment, step l) comprises utilizing relevant ground truth annotations. In one embodiment, step c) is operable to blindly predict said quantitative indicators of pathology and / or image quality.

[0030] In one embodiment, step c) does not require prior image scan data of said particular object.In one embodiment, the first image acquisition sequence comprises a plurality of different scan types.

[0031] In one embodiment, the medical imaging system includes an MRI scanner and the multiple scan types include T2 FLAIR, DWI, 3D SWI, T2 * It is selected from the group of gradient echo and T1 TSE.

[0032] In one embodiment, step b) includes performing a first image scan of a first type and then performing a second image scan of a second type, and in one embodiment, step c) is performed on image scan data from the first image scan while the second image scan is being performed.

[0033] In one embodiment, the quantitative index provides an indication of one or more of the following potential pathologies: tumor (general), glioma, granuloma, abscess, hemorrhage, microhemorrhage, infarction, ischemic changes, demyelination, neoplasm, neurodegenerative disease, and hydrocephalus.

[0034] In one embodiment, step d) further comprises: m) determining, using a computer system based on said identification of one or more of said quantitative indicators of pathology in step c), whether a notification should be generated, and if determined to be generated, the method further comprises n) generating, during the scanning session, a notification corresponding to the identified quantitative indicator.

[0035] In one embodiment, step d) further comprises: m) determining, using a computer system, based on said identification of one or more of said quantitative indicators of pathology in step c), whether a notification should be generated or not, and if determined to be generated, the method further comprises n) generating, during the medical scanning session, a notification regarding pathology likely to be associated with said identified quantitative indicators.

[0036] In one embodiment, the notification is provided to a medical imaging system operator and / or a medical professional during a medical scanning session, hi one embodiment, the notification indicates that the subject is likely in need of emergency treatment.

[0037] In one embodiment, the one or more quantitative indicators identified in step c) provide one or more indications of hemorrhage, microhemorrhage or infarction and emergency treatment is for stroke. In one embodiment, the notification indicates that thrombolytic therapy, anticoagulant medication or treatment to avoid anticoagulant medication is likely required.

[0038] In one embodiment, step m) comprises: o) correlating one or more values ​​of said one or more identified quantitative indicators with values ​​of said one or more identified quantitative indicators associated with normal healthy subjects and / or with values ​​associated with known pathologies.

[0039] According to a second aspect of the present invention there is provided a method of performing a medical scan of a subject using a medical imaging system comprising: a) initiating a medical scanning session; b) during the medical scanning session performing an image acquisition sequence using the medical imaging system to acquire image scan data; c) performing, using a computer system, image analysis on image scan data acquired during said image acquisition sequence to identify one or more quantitative indicators of pathology; d) based on said identification of one or more of said quantitative indicators of pathology in step c), determining, using the computer system, whether a notification should be generated, and if it is determined that a notification should be generated, further comprising: e) generating, during the medical scanning session, a notification regarding pathology likely to be associated with said identified quantitative indicator.

[0040] In one embodiment, the notification is provided to a medical imaging system operator and / or a medical professional during a medical scanning session, hi one embodiment, the notification indicates that the subject is likely in need of emergency treatment.

[0041] In one embodiment, the one or more quantitative indicators identified in step c) provide one or more indications of hemorrhage, microhemorrhage or infarction and emergency treatment is for stroke. In one embodiment, the notification indicates that thrombolytic therapy, anticoagulant medication or treatment to avoid anticoagulant medication is likely required.

[0042] In one embodiment, step c) further comprises: f) using a computer system, performing image analysis on image scan data acquired during said image acquisition sequence to identify one or more quantitative indicators of image quality.

[0043] In one embodiment, step d) further comprises: g) determining, using a computer system based on said identification of one or more quantitative indicators in step c), whether any additional image acquisition sequences are required during said medical scanning session, and if so, h) determining, using a computer system, a second image acquisition sequence based on the one or more quantitative indicators; and i) performing a second image acquisition sequence using a medical imaging system during the medical scanning session to acquire second image scan data.

[0044] In one embodiment, the first image scan data and / or the second image scan data comprise two-dimensional image scan data. In one embodiment, the first image scan data and / or the second image scan data comprise three-dimensional image scan data. In one embodiment, step c) is performed using a classifier and the one or more quantitative indicators comprise voxel classifications. In one embodiment, step c) further comprises j) generating a voxel classification map from the image scan data.

[0045] In one embodiment, step g) further comprises k) aggregating the voxel classification map into an overall classification of the image scan data.

[0046] In one embodiment, the method further comprises the steps of: l) identifying any blobs in the voxel classification map that belong to a predetermined voxel classification; m) comparing the identified blobs to a predetermined threshold; and n) identifying a positive indication of potential pathology if the predetermined threshold is exceeded.

[0047] In one embodiment, the computer system includes a neural network algorithm. In one embodiment, the neural network algorithm includes a convolutional neural network algorithm. In one embodiment, the neural network algorithm includes a U-net.

[0048] In one embodiment, the method further comprises, prior to step a), o) training a neural network algorithm to identify said quantitative indicators using a training set comprising a set of medical scan tuples. In one embodiment, step o) comprises utilizing relevant ground truth annotations. In one embodiment, step c) is operable to blindly predict said quantitative indicators of pathology and / or image quality.

[0049] In one embodiment, step c) does not require prior image scan data of said particular object.In one embodiment, the image acquisition sequence comprises a plurality of different scan types.

[0050] In one embodiment, the medical imaging system includes an MRI scanner and the multiple scan types include T2 FLAIR, DWI, 3D SWI, T2 * It is selected from the group of gradient echo and T1 TSE.

[0051] In one embodiment, step b) includes performing an image scan of a first type and thereafter performing a second image scan of a second type, and in one embodiment, step c) is performed on image scan data from said image scan while the second image scan is being performed.

[0052] In one embodiment, the medical imaging system includes an MRI scanner and the image scan data is produced by magnetic resonance imaging (MRI). In one embodiment, the medical imaging system includes an MRI scanner and the image scan data is produced by magnetic resonance imaging (MRI) using one or more techniques selected from the group of T1-weighted and T2-weighted MRI scans, FLAIR scans, diffusion weighted imaging (DWI), contrast-enhanced MRI (CE-MRI), dynamic contrast-enhanced MRI (DEC-MRI), gradient echo (GRE), susceptibility weighted imaging (SWI) and turbo spin echo (TSE).

[0053] In one embodiment, the medical imaging system includes a CT scanner and the image scan data includes computed tomography (CT) data, hi one embodiment, the medical imaging system includes a PET scanner and the image scan data includes PET data.

[0054] In an embodiment, steps a) to f) of the first aspect and steps a) to e) of the second aspect are all performed while the subject remains within the medical imaging system. In an embodiment, the (first) image scan data and / or the second image scan data relate to imaging of a body part of the subject. In an embodiment, the (first) image scan data and / or the second image scan data relate to imaging of a brain of the subject.

[0055] In one embodiment, step c) and / or step f) further comprises performing skull stripping of the first image scan data to obtain a voxel map of brain tissue and cerebrospinal fluid. In one embodiment, said skull stripping is performed by a neural network algorithm. In one embodiment, said neural network algorithm is trained to identify and remove skull portions of said first image scan data and / or second image scan data using a training set comprising a set of image scan tuples.

[0056] According to a third aspect of the present invention there is provided a computer system including a processing device configured to perform the method of the first or second aspect.

[0057] According to a fourth aspect of the present invention there is provided a computer readable medium comprising instructions which, when executed, are configured to perform the methods of the first and second aspects.

[0058] According to a fifth aspect of the present invention there is provided a computer system comprising a processing device, a storage device and the computer readable medium of the third aspect.

[0059] Embodiments of the present invention will now be described in detail with reference to the accompanying drawings. [Brief description of the drawings]

[0060] [Figure 1] 1 illustrates an example computing system forming part of the present invention; [Diagram 2] The computing system of FIG. 1 is shown in relation to a scan sequence workflow process. [Diagram 3] The schematic configuration of the U-net neural network is shown below. [Figure 4] 1 shows a flow chart of the method of the present invention. [Diagram 5] A simplified workflow of the Level 1 scan acquisition process is shown. [Figure 6] 1 shows a schematic diagram of the scan acquisition process including level 2 and subsequent scan acquisition. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0061] The present invention relates in embodiments to a novel non-invasive medical imaging processing application intended for the automatic labeling and visualization of candidate findings from a set of Magnetic Resonance (MR) images, where the output consists of labeling the visualization of candidate pathologies with quantitative indicators.

[0062] The user should use the tools provided for validation of candidate findings. The present invention is intended to provide the skilled medical professional with information complementary to the evaluation and assessment of MR images, to assist the skilled medical professional in determining additional MR sequences appropriate for diagnosis, and in determining prioritized workflow. The present invention provides a post-processing workstation or cloud solution that works on routed Digital Imaging and Communications in Medicine (DICOM) images.

[0063] A smart protocol according to an embodiment of the present invention is operable to classify data acquired during a scan and provide an indication to the medical professional as to whether any further scans are recommended. In other words, a smart protocol is a configurable item that outputs a recommended next action (e.g., a new scan) for a given input tuple of a medical scan. The configuration is situation specific.

[0064] A unique feature of the present invention is that the method does not require prior knowledge of the patient's past or previous scan data, therefore the present invention can be used with patients who are being scanned for the first time or who may be investigated for medical complications different from previous scans.

[0065] The smart protocol is realized as a decision tree, where the decisions at each node of the tree are made automatically by an algorithm.

[0066] The method includes a machine learning algorithm that identifies parameters of interest in the scan data. The parameters are identified by a training process that allows for classification of specific features in the images by one or more quantitative indicators or parameters indicative of one or more pathologies or image quality. The computational process thus trained can, during a medical scanning session, identify quantitative indicators or parameters indicative of specific pathologies and / or image quality, recommend any additional scan sequences that may be required, and / or provide quantitative indicators that, in certain embodiments, provide an indication of likely pathologies that may require immediate diagnosis and / or treatment by a medical professional.

[0067] This is a significant improvement from known arrangements, which require analysis of the scan data to be performed by a medical professional after the scanning session is completed.

[0068] Figure 1 illustrates an example computing system 100 forming part of the present invention. Figure 2 illustrates the computing system 100 of Figure 1 in the context of a scan sequence workflow process.

[0069] The present invention relates to a medical imaging system 10. The medical imaging system 10 includes a computing system 100. The computing system 100 may take any suitable form and may include, for example, a remote computer, a cloud-based computing system, a local workstation, or a medical imaging system console.

[0070] The computing system 100 includes one or more processors 102, computer readable physical memory 104, and a non-transitory storage device 106, such as a hard disk drive or solid state drive. The storage device 106 may take any suitable form and may include storage devices local to the computing system 100 and / or storage measures external to the computing system 100. An interface 108 is provided to enable the computing system 100 to communicate with other components within the medical imaging system 10. A computing application 110 executes on the computing system and is operable to cause the computing system 100 to perform the methods of the present invention.

[0071] The computing application, as described below, includes a classifier 112 and a classifier database 114. The image processor 116 is further operable to interact with the classifier 112, as described below.

[0072] The computing application 110 is operable to communicate through the interface 108 with a medical imaging device 118, which in this embodiment takes the form of an MRI scanner, however this is not intended to be limiting and other scanner types, such as CT or PET, or a combination thereof, may also be used.

[0073] The interface 108 and scanner 116 are in data communication with a storage device 120 in the form of a Picture Archiving and Communication System (PACS). The PACS 120 is an industry standard device and format for medical imaging. However, this is not a limitation and other configurations can be used. The PACS 120 is operable to handle images and other data in Digital Imaging and Communications in Medicine (DICOM) format. However, other proprietary or non-proprietary formats can be used.

[0074] A read-out station 122 communicates with the PACS 120 and can be used by medical practitioners to read, interpret, and analyze the image data.

[0075] It should be noted that the computing system 100, in this embodiment, communicates through a radiologist information system or hospital information system (HIS), generally designated 124. However, this is not critical to the invention and other systems may be used as appropriate.

[0076] Here, the operation method of the present invention will be described with reference to FIGS.

[0077] Overall, the invention comprises a control and analysis method that, for a given input tuple of a medical scan, outputs a recommended next action (e.g., a new scan is performed). Configurations are situation specific and are provided for use by clinics in smart protocols. The smart protocols are realized as decision trees. The decisions at each node of this tree are made automatically by an algorithm.

[0078] Step 200: Providing the algorithm The classifier 114 includes a machine learning algorithm that in one embodiment utilizes aggregate voxel-wise classification from the U-Net deep learning algorithm for medical image segmentation described in reference [1a], however, this is not intended to be limiting and other voxel-level classifiers may be used instead.

[0079] The building blocks of a deep neural network are artificial neurons or nodes. Each input has an associated weight. The sum of all the weighted inputs is then passed through a nonlinear activation function f that converts the preactivation level of the neuron into an output. The output then serves as the input to the nodes in the next layer.

[0080] Several activation functions are available. They differ in the way they map pre-activation levels to output values. The most common activation functions used are the rectification function (neurons using the rectification function are called rectified linear units (ReLU)), the hyperbolic sine function, the sigmoid function and the softmax function.

[0081] The softmax function is commonly used in the output layer because it can compute the probability of multi-class labels. For each mode j in the first hidden layer, a nonlinear function is applied to a weighted sum of the inputs. The result of this transformation serves as the input for the second hidden layer. The information propagates through the network up to the output layer, where the softmax function yields the probability of a given observation belonging to each class.

[0082] Typically, convolutional networks are used for classification tasks, where the output for an image is a single class label. However, in many visual tasks, especially biomedical image processing, the desired output should include localization, i.e., a class label assigned to each voxel.

[0083] In one exemplary embodiment, the network architecture of the U-net algorithm is shown in Figure 3, which is taken from reference [1a]. The algorithm corresponds to a Convolutional Neural Network (CNN) and includes an encoding part and a decoding part.

[0084] The encoding side includes the convergent path (shown on the left) and the decoding part includes the divergent path (shown on the right).

[0085] The convergence path follows the typical architecture of convolutional networks and consists of the repeated application of two 3 × 3 convolutions, each followed by a rectified linear unit (ReLU) operation and a 2 × 2 max-pooling operation with stride 2 for downsampling.

[0086] At each downsampling step, the number of feature channels is doubled. Every step in the diffusion path consists of upsampling a feature map, followed by a 2x2 convolution ("deconvolution") that halves the number of feature channels, concatenation with the correspondingly cropped feature map from the convergence path, and two 3x3 convolutions, each followed by a ReLU.

[0087] In an embodiment, if a CNN using a U-net algorithm is utilized as the classifier 114, this may include different forms. For example, a conventional 3D U-net may be used with different parameters from the example above. Any suitable number and size of convolutions with any suitable kernel size and stride size may be utilized. For example, a stride of 1 may be used.

[0088] The ReLU may include, in non-limiting examples, a soft plus approximation, a noisy ReLU (including Gaussian noise), a leaky ReLU (allowing small positive gradients when the ReLU is not active), a parametric ReLU (where the leakage coefficient in the leaky ReLU is a learned parameter of the network) and / or an exponential linear unit (ELU).

[0089] Finally, the pooling layer may utilize max pooling with any suitable filter and stride size.

[0090] Alternatives to the traditional U-net algorithm may be used, for example a variational encoder (VAE) may also be used. An example of a VAE is given in reference [1b].

[0091] As a further alternative, one can use a cascaded U-net VAE, in which two sets of U-net VAE models are interlinked such that the output of the first U-net is concatenated with the input image and fed into the second U-net. An example can be found in reference [1c].

[0092] Step 202: Training the algorithm In the context of deep machine learning neural net solutions, classification involves two main steps. In the first step, the training phase, a subset of the available data, known as the training set, is used to optimize the parameters of the network to perform the desired task (classification). The model can be trained from scratch without any pre-training.

[0093] In the second step, the so-called testing phase, the remaining subset, known as the test set, is used to assess whether the trained model is able to blindly predict the class of a new observation. When the amount of available data is limited, it is also possible to perform the training and testing phases several times with different training and testing splits of the original data and then estimate the average performance of the model - a technique known as cross-validation.

[0094] The algorithm of the present invention is trained using a training set that includes a set of medical scan tuples with associated ground truth annotations to classify each voxel in a medical scan into one of several candidate classes, which may be from any suitable image acquisition technique or method.

[0095] A number of input images forming a training set comprising a set of medical scan tuples with associated ground truth annotations are utilized to classify each voxel in the medical scan into one of several candidate classes.

[0096] The data may be pre-processed. For example, the input image may be resampled to achieve an isotropic voxel size. To address problems such as infarct segmentation, the diffusion sequence may be resampled so that the image is first isotropic, and then the fluid attenuated inversion recovery (FLAIR) and / or susceptibility weighted imaging (SWI) sequences may be reformatted into the resampled diffusion space. The apparent diffusion coefficient may then be calculated.

[0097] Similarly, for hemorrhage segmentation problems, the SWI space may be used. After resampling, the images may be z-regularized and fed to a neural network for training and reference. After resampling, the data may be cropped to regions of non-zero values, for example using Otsu median filtering.

[0098] The above examples are intended to be illustrative and non-limiting. Those of ordinary skill in the art will readily recognize other input images and / or pre-processing methods that can be used in conjunction with the present invention.

[0099] In addition, U-Net training is improved by artificial augmentation using data augmentation, which can be desirable to make the network sufficiently trainable when limited input images are available.

[0100] Some transformations may be used to augment the dataset of training images. For example, the input images may be augmented with a transformation based on a random velocity field [2]. Additionally or alternatively, a random selection of transformations (which may not include augmentations or transformations) may be used for each training step.

[0101] The input image so transformed may be weighted using a cost function having a predetermined factor that accounts for the inconsistencies and artifacts. In one embodiment, the predetermined factor is 0.75 and the artifacts.

[0102] Alternative transformations may be used, which may include one or more of rotation, elastic transformation, Gaussian noise, Gaussian blur, contrast, gamma correction, and mirroring. For models that use only the VAE branch, only intensity transformations may be used due to the image reconstruction rules.

[0103] Finally, post-processing of the images may be utilized to improve image classification performance. In an embodiment, morphological filtering may be used to remove random predictions in the images.

[0104] In an architectural choice, blobs smaller than a certain size may be removed. In an embodiment, blobs smaller than 5 mm 3 Blobs smaller than 125 mm may be removed. Other sizes may be used. For example, classification performance may be improved by 3 This can be optimized by removing blobs smaller than

[0105] The classifier 112 is trained using a training set of images in a classifier database 114. In general, the classifier 112 works by taking the values ​​of particular features (independent variables or predictors in regression) for an example (a set of independent variable values) and predicting the class (dependent variable) that the example belongs to.

[0106] In the present context of medical imaging, features include voxels (volume pixels). Classes may indicate a particular likely pathology. The classifier 112 needs to learn some parameters from the training data in the database 114. A classifier is essentially a model of the relationship between features in the training set and class labels. More formally, given an example x, the classifier

number

[0107] Typically, neural networks can be trained through gradient descent-based algorithms, which aim to find the values ​​of the network weights that best minimize the error (difference) between the estimated and true outputs.

[0108] Note that these methods are not actual classifiers themselves, but instead networks that are pre-trained to learn useful patterns in the data and then fed into the actual classifier in the last layer.

[0109] In one exemplary embodiment, a training set of input images and corresponding segmentation maps are used to train an algorithmic network using stochastic gradient descent. Below is a general example of this from reference [1a]:

[0110] The energy function is computed by pixel-wise softmax over the final feature maps in combination with a cross-entropy loss function. Softmax is calculated as follows:

number

number

number

number

number

[0111] Weight maps are pre-computed for each ground truth segmentation to compensate for the different frequencies of pixels from particular classes in the training dataset and to force the network to learn small separation boundaries that are introduced between adjacent test cells.

[0112] The weight map is then calculated as:

number

number

number

number

[0113] In deep networks with many convolutional layers and different paths through the network, it is important to initialize the weights correctly; otherwise, parts of the network may contribute excessive activation while other parts never contribute. Ideally, the initial weights should be adapted so that each feature map in the network has approximately unit variance.

[0114] In a network with alternating convolutional and ReLU layers, this means that the standard deviation

number

[0115] However, alternative approaches can be used. In one embodiment, a U-net or similar neural network is trained using Dice as the loss function. In one example, a kernel weight regularization of 1e-6 may be used for all layers.

[0116] We can use a Dice loss function similar in function to that in reference [1d], which is given by Equation 4:

number

[0117] Both the predicted segmentation and the ground truth are one-hot coded. A sigmoid is used as the activation function. In one embodiment, optimization may be achieved through the use of adaptive moment estimation or adaptive learning rate techniques such as Adam. The Adam process utilizes estimates of the first and second moments of the gradient to adapt the learning rate for each weight of the neural network.

[0118] In one embodiment, the Adam methodology is utilized as the optimizer with a learning rate of 1e-1. In an embodiment, the model may be trained for 100 epochs, where one epoch includes 750 steps.

[0119] In a non-limiting embodiment, a learning rate scheduler may be used to reduce the learning rate by 5% if there were no increases in validation dice in the previous 10 epochs. The epoch with the best overall validation dice may then be used to make the final guess.

[0120] In that case, the verification dice score consists of two scores: a) the whole dice in the foreground, and b) the dice on only images with foreground. The average of these two dice scores constitutes the whole dice.

[0121] Other methods may be used, and such configurations are non-limiting, but such a combination has the advantage that there is a large collection of non-foreground images with diffuse pathology that may result in a large number of false positives.

[0122] Step 204: Testing the Algorithm Once trained, the classifier can be used to determine whether the features used contain information about the class of an example. This relationship is tested by using the trained classifier on a different set of test data.

[0123] In the present embodiment, infarct identification in brain MRI was performed based on a tuple of T2-weighted fluid-attenuated inversion recovery (FLAIR), diffusion-weighted imaging (DWI) and susceptibility-weighted angiography (SWAN) or gradient echo (GRE) MRI sequences.

[0124] Manually annotated findings on medical scans, annotated by in-house radiologists, were used to train the classifier 114. Medical scans were annotated by consulting existing clinical reports with radiological findings and annotating the voxels that correspond to these findings on the scan.

[0125] The technical feasibility of the classifier 114 was investigated by calculating the Dice overlap with annotations made by the same in-house radiologist on medical scans that were not part of the training set.

[0126] The clinical feasibility of the classifier 114 was investigated by calculating the subject-level per-finding (1 vs. rest) sensitivity and specificity for known radiological findings according to clinical reports. Clinical feasibility indicates the performance of the classifier 114 as a decision algorithm to be used in the present method.

[0127] The classifier 114 was trained on 29 subjects (16 with infarction and 13 with hemorrhage). The trained classifier 114 was technically evaluated using 4 independent subjects with infarction and clinically evaluated using 80 independent subjects (60 with infarction and 20 without findings). The results are shown in Table 1 below.

[0128] [Table 1]

[0129] Once the algorithms forming the classifier 114 have been fully tested, the classifier 114 may be utilized as part of an assisted automated scanning session as described below with reference to FIGS.

[0130] Step 250: Preparing for a Scan Session In step 250, the scanner is prepared for the scan, which may include determining the scan sequence and protocol required for a particular part of the body region of interest.

[0131] This may be derived automatically, or the medical practitioner may specify a particular scan sequence. In one embodiment, the calculation application 110 may calculate what is deemed the most likely through scan sequence that may be required for a particular patient, based on the initial input of the scan sequence. This may be provided by other data, such as empirical data. For example, semantic or natural language analysis of patient records, empirical knowledge of the most common pathologies for a given body part, or other data may be used to provide an initial "best guess." This serves as a placeholder for subsequent analysis or further scan sequences.

[0132] The following exemplary embodiments refer to brain scanning, however this is not intended to be limiting and in principle any suitable body part or region can be scanned using the described methods.

[0133] Step 252: Performing a First Scan Sequence In step 252, a first scan sequence is performed. In this embodiment, the first scan sequence includes multiple different image acquisition sequences that form part of the first or level 1 scan sequence. It should be noted that the following sequence order shown and described in this embodiment may be modified as appropriate and a different order or different scanning techniques may be used as appropriate.

[0134] In this embodiment, the first scan sequence includes the following techniques. T2 FLAIR DWI 3D SWI T1 TSE

[0135] These techniques are selected to provide a crude indication of the following underlying pathologies: Tumors (general), Gliomas (the most common type of tumor), ·Granuloma, ·abscess, ·bleeding, ·Minor bleeding, ·infarction, Ischemic changes, ·Demyelination, ·Neoplasm, ·Hydrocephalus.

[0136] The order of acquisition sequences can be determined based on importance. For example, if T2 FLAIR and SWI are acquired in 3D, reformatted FLAIR can be used to assess aqueductal stenosis for hydrocephalus and SWI can be used to assess calcification and hemorrhage / hemosiderin (iron). Therefore, other sequences are not needed for crude indication of potential pathology as listed above.

[0137] FIG. 5 shows a flow diagram of the time sequence of image acquisition during a level 1 scan sequence. As shown, first, a localizer is utilized for 20-30 seconds. Then, a T2 FLAIR scan acquisition is performed. This process takes approximately 3-5 minutes. The acquired data is transmitted from the scanner 118 via a DICOM router to the computing device 100 via the interface 108.

[0138] The next scan acquisition is a DWI scan process. This process lasts for approximately 2 minutes. Concurrent with this scan, image processing is performed by the image processor 116, including classification using the classifier 112, as described in the next step. However, steps 252 and 254 may occur simultaneously in time or at least partially simultaneously. Usefully, scan interpretation data from a T2 FLAIR process may be acquired before the end of the DWI sequence.

[0139] Once the scan is complete, the data is sent via a DICOM router to the image processor 116 and classifier 112 for processing and identification of potential pathology.

[0140] The next step in the level 1 scan acquisition according to this embodiment is the determination of the SWI scan. This typically takes about 3-4 minutes. If desired, a further scan in the form of a T1 TSE (not shown in the timeline) can be performed.

[0141] Image scan data is acquired with each scan acquisition, which may include any suitable form. It is noted, for example, that the scan data may include image data for a single image, image data for multiple images, or a combination of image data and metadata. The image scan data may include two-dimensional images (either acquired directly as two-dimensional images or acquired as two-dimensional images acquired from three-dimensional data) or three-dimensional images. Alternatively, the image scan data may include data and metadata not directly related to the image but utilized to form an image of the subject's body part.

[0142] Step 254: Analyze and classify the image As noted in step 252, step 254 may be performed simultaneously with step 252, thus providing image analysis and classification results before the level 1 scan sequence is completed, allowing the transition between level 1 and level 2 to be virtually seamless.

[0143] As noted above, the U-net algorithm of classifier 112 is trained to classify each voxel in a medical scan into one of several candidate classes.

[0144] When performing a level 1 scan sequence, the classifier 112 and image processor 116 take as input all medical scans at each measurement as a tuple and produce a voxel classification map as output. The method may include a voxel classification step using region of interest analysis and feature selection for each structure.

[0145] In this example, once the T2 FLAIR dataset tuples are acquired in step 252, processing is performed on these datasets to produce a voxel classification map. The voxel classification map is then summarized into an overall classification of the medical scan tuple (i.e., the presence of one or several findings or the absence of findings). This is accomplished by processing the voxel classification map for each type of finding. If a candidate finding is identified, the tuple is flagged as having this finding present.

[0146] The process uses a connected components algorithm to identify blobs (or defects) that belong to a particular class, and if the smallest blob identified exceeds a threshold, it is considered a positive finding and the tuple is flagged. Finding-specific thresholds can be predetermined as part of building a particular protocol, and can be determined as needed.

[0147] The defects can be analyzed in a number of ways. For example, feature detection can be used on the voxel classification map to identify defects that are indicative of a particular pathology. In addition, defects can be characterized and flagged for image quality issues.

[0148] Alternatively, in an embodiment, this involves performing anisotropic Gaussian blob detection, as described in the exemplary embodiment above. Further alternative methods may be used. For example, isotropic Gaussian blob detection and / or ridge detection may be applied to the voxel classification map to identify defects indicative of particular pathologies flagged in the training process.

[0149] Step 256: Deciding Next Steps Based on the findings flagged in the medical scan tuples in step 254, it is determined whether any further sequencing is necessary. Alternatively or additionally, it is determined whether any potential pathology identified via the analysis in step 254 may require immediate diagnosis and / or treatment by a medical professional. If the latter applies, the next step may include providing a notification during the scanning session that a likely pathology has been identified that may allow for urgent diagnosis and / or treatment by a medical professional, as described below in step 260.

[0150] The next step to be selected will depend on the potential pathology indicated by the analysis in step 254. Set forth below is a non-exhaustive list of pathologies and potential indicators to those pathologies that can be flagged by the present invention. 1) Tumor: T2 FLAIR: usually high signal; T1: usually low signal; Diffusion: limited by solid area; GRE: low signal if bleeding; GD enhancement: usually solid area 2) Granuloma: T2 FLAIR: hypointense (tuberculoma); T2 FLAIR: eccentric nodule (NCC - neurocysticercosis cyst); Diffuse: + / -; GRE: hypointense if calcified; GD enhancement: marginal 3) Abscess: T2 FLAIR: low signal; T1: low signal; Diffusion: centrally restricted; GD enhancement: thick margin 4) Infarction

[0151] [Table 2]

[0152] 5) Ischemic changes: T2 FLAIR: high signal; T1: low signal; GRE: may be associated with microbleeds 6) Demyelination: T2 FLAIR: high signal; T1: low signal; Diffusion: restricted + / -; GRE: -; GD enhancement: peripheral / solid plaque-like / broken ring 7) Hydrocephalus: T2 FLAIR: Periventricular confluent hyperintensity + / -; T1: + / -; Diffusion: -; GRE: -; GD enhancement: Leptomeningeal enhancement + / - 8) Bleeding

[0153] [Table 3]

[0154] 9) Microhemorrhage: T2 FLAIR: - T1: - Diffusion: + / -; GRE: Yes (3D GRE is better)

[0155] Based on the indication and the indication derived from the analysis in the level 1 scan acquisition process, the next step can be determined.

[0156] As noted above, based on level 1 scan acquisitions, T2 FLAIR and DWI provide crude indications of the presence of 8 of 10 pathologies (excluding hemorrhages and microhemorrhages).

[0157] In step 256, steps are taken to determine what subsequent sequences, if any, are required, with or without contrast enhancement. If none are found, a T2 or SWI is acquired to estimate the probability of diagnosis of pathology.

[0158] If quantitative indicators indicative of infarction, microbleeds and / or hemorrhage in the brain are identified, the next step may include notifying a medical professional of the likely pathology identified so that diagnosis and / or treatment can proceed urgently during the scanning session, as described below in step 260.

[0159] There are two possible outcomes from this step: nothing is found or something is found, as described below.

[0160] Step 258: Nothing Found If no indications of potential pathology are identified in step 256, the process may end. Optionally, however, based on natural language analysis of the clinical reference, the computer application 110 may suggest an end to the examination or may continue with additional sequences based on severity.

[0161] Step 260: Something is Found If a positive indication of potential pathology is identified in step 256, based on the learning algorithm, the computer application 110 may suggest what other actions should be taken to improve and support the additional imaging material required to enable the medical practitioner to arrive at a diagnosis.

[0162] A list of potential next sequences is shown in Figure 6. Briefly listed, the following sequences were derived as further tests at level 2 on the pathologies identified above. As shown in Figure 6, the available sequences are: Neoplasms, tumors (including gliomas) and abscesses: T1 TSE+3D T1 MPR_GD+(SOS perfusion_GD+spectroscopy_GD) Granuloma:T1 TSE+3D T1 MPR_GD Infarction: T2 TSE+TOF angiography or PC venography (based on DWI results) Ischemic changes / microbleeds: 3D T1 MPR+T2 TSE+SWI+T1 TSE+TOF angiography or PC venography (based on NLP results) Demyelination: T2 TSE+T1 TSE+T2sag (whole spinal cord screening)+3D T1 MPR_GD Hypomyelination: T2 TSE+T2sag (total spinal cord screening)+spectroscopy Bleeding: T2 TSE+T1 TSE+T2 * / SWI+TOF angiography or PC venography (based on T2 FLAIR and DWI results) Congenital anomalies (not mentioned in medical report): T2 TSE+3D T1 MPR Uncovering uncertain anomalies: Continuing the sequence based on NLP analysis

[0163] Once the level 2 scan is complete, the process proceeds to level 3, where the process ends unless any additional other pathology is identified in the level 2 scan, in which case a further scan may be performed.

[0164] Additionally, if a positive indication of probable pathology is identified in step 256, based on the learning algorithm, the computer application 110 may suggest additional or alternative actions, such as further scanning.

[0165] For example, a notification may be provided during the medical scanning session if a positive indication of probable pathology is identified in step 256. In other words, a corresponding or indicative notification regarding the probable pathology may be shown during the medical scanning session while the patient is still within the medical imaging device 118.

[0166] The notification may be provided in any suitable manner, for example, the notification may be provided as an audible signal or on a display or displays, or the notification may be provided to a medical professional or an operator of the medical imaging device 10.

[0167] In an embodiment, the notification may indicate a high probability of need for emergency treatment of the subject. Thus, the notification may provide instructions or information to a medical professional or operator. The medical professional or operator may then use the notification to make an appropriate diagnosis as to whether emergency treatment of the subject is required while the subject is within the medical imaging device 118.

[0168] In embodiments, the one or more quantitative indicators may provide one or more indications of hemorrhage, microhemorrhage, or infarction. In that case, a suitable notification may indicate to a medical professional that the subject is likely to have a stroke. The medical professional may then make a diagnosis of the condition and any necessary treatment.

[0169] In embodiments, the notification may indicate that thrombolytic therapy, anticoagulant medication, or treatment to avoid anticoagulant medication is likely required. By providing an indication of the likely need for such treatment, the present invention provides a notification that alerts a medical practitioner and facilitates a prompt diagnosis and appropriate treatment for a subject still within the medical imaging device 118.

[0170] Combinations of the above notifications and subsequent scan recommendations may be used and would be readily envisioned by one of ordinary skill in the art. EXAMPLES

[0171] [Example 1] In Example 1, different encoder-decoder architectures suitable for use with the method of the present invention are evaluated that seek to use sets of hyper-parameters across two different tasks of medical image segmentation: a) stroke lesion segmentation, and b) hemorrhage segmentation.

[0172] [Example 1: Data] A total of 997 subjects were considered for this study. All studies had the following sequences: fluid-attenuated inversion recovery sequences (FLAIR: mean shape (512;512;20), and mean voxel size (0.46 mm;0.46 mm;7 mm)), diffusion-weighted imaging sequences (DWI: mean shape (256;256;20), and mean voxel size (1.05 mm;1.05 mm;7 mm)), and susceptibility-weighted imaging sequences (SWI) or T2 * Gradient echo sequence (GRE) (SWI / GRE: mean shape (512; 512; 72), and mean voxel size (0.46 mm; 0.46 mm; 2 mm)).

[0173] The first dataset (Dataset A) has 156 cases with infarction, 67 cases with hemorrhage, and the rest is a mix of normal cases and cases with other pathologies (20% were reserved for performance evaluation). Dataset A is used for model selection. Once an architecture is selected, another dataset (with the same demographics) is used to train the selected architecture and evaluate a multi-class version of the architecture. The second dataset (Dataset B) has a total of 773 cases (125 cases with tumor, 135 cases with hemorrhage, and 189 cases with hemorrhage), of which 98 cases were reserved for validation and reporting of classification performance.

[0174] In this example we consider three models. All models have the same base network, called U-net, as described above. Additional modifications are made to the models as follows:

[0175] [Example 1-1] The 3D U-Net has an encoder-decoder structure with skip connections between matching scales in the encoder and decoder. It uses two convolutional layers (12 filters each) with dropout (rate 0.25) between the input and the U-Net. Each scale in the encoder represents a max pooling layer of size 2. Each scale in the decoder represents an upsampling layer. Trilinear interpolation is used for upsampling.

[0176] Between the two scales, two convolutional layers followed by an instance regularization layer are used. Each convolutional layer uses leaky ReLU (negative gradient 1e -2 ) A patch size of 192x192x192 is used. 12 filters are used in the first layer, followed by 24, 48 and 56 filters in the encoder. A similar number of filters are used in the decoder, but the order is reversed.

[0177] All convolutions are performed with a kernel size of 3 and a stride of 1. Finally, a sigmoid activation function is used to convert the logits into probabilities of belonging to the foreground and background.

[0178] [Example 1-2 - U-Net VAE] In this model, which roughly corresponds to the one disclosed in reference [1b], a variational autoencoder (VAE) is used to regularize the latent space. The VAE branch starts from the last layer of the encoder. The first layer of the encoder generates the mean and standard deviation of a normal distribution from which a 32-dimensional vector is sampled. The vector is then used to reconstruct the input. The sampled vector layer is then connected to a layer of size matching the last layer of the encoder. Subsequently, four convolutional layers with kernel size 1 and four upsampling (factor 2) layers are used to reconstruct the input.

[0179] [Example 1-3 - Cascaded U-Net VAE] We use two sets of U-Net VAE models with the same architecture as in Example 1-2. The output of the first U-Net is concatenated with the input image and fed into the second network. In the first U-Net, the input image is scaled by a factor of 0.5 with a scaling layer at the input. The output layer is then scaled by 2 to match the size of the input. The networks are trained sequentially.

[0180] Note that the first network is fixed while the second network is trained. Only the U-Net part of the first model is used while training the second model, and the VAE branch is discarded since it has no impact on segmentation during inference.

[0181] [Network Selection] To find the right architecture, we used only the hemorrhage and infarction datasets. Once a comparison between the models was made, the best architecture was used to train a multi-class network, i.e., each network has targets with multiple labels (tumor, infarction, and hemorrhage) instead of binary labels. Three such networks were trained. Each network is used to segment hemorrhage, tumor, and infarction separately.

[0182] In other words, all three segmentations can be obtained from the same network, but the hemorrhage model was used to extract only the hemorrhage segmentation.

[0183] Training is performed as described above in relation to optimization using the Dice loss function and Adam.

[0184] [Example 1: Result] We evaluated the models on two fronts: a) detection ability using Dice score, and b) sensitivity and specificity scores. Table 1 in Example 1 summarizes the results of infarct segmentation on Dataset A, and Table 2 in Example 1 summarizes the results of hemorrhage segmentation on Dataset A.

[0185] The best performing methods are highlighted in bold. For both problem statements, the baseline U-Net model outperforms the other two models.

[0186] Tables 3 and 4 in Example 1 show the performance on Dataset B using a larger blob size threshold. The evidence is clear that the multi-class variation only helps with classification accuracy (especially for features) of hemorrhage, while on other problems the U-Net binary network performs exactly the same.

[0187] Table 1: Summary of infarct segmentation results on dataset A. Only hyperacute, acute and subacute infarcts are considered. Sensitivity and specificity are reported only for the best performing method in terms of DICE, in this case the standard U-Net model. Blob filtering threshold 5mm 3 applies.

[0188] [Table 4]

[0189] Table 2: Summary of hemorrhage segmentation results on dataset A. All ages of cerebral hemorrhage are considered. Sensitivity and specificity are reported only for the best performing method in terms of DICE, in this case the standard U-Net model. Blob filtering threshold 5mm 3 applies.

[0190] [Table 5]

[0191] Table 3: Summary of infarct, hemorrhage and tumor segmentation results using binary standard U-Net on dataset B. Blob filtering threshold 125 mm 3 applies.

[0192] [Table 6]

[0193] Table 4: Summary of infarct, hemorrhage and tumor segmentation results using multi-class standard U-Net on dataset B. Blob filtering threshold 125 mm 3 applies.

[0194] [Table 7]

[0195] The experimental example represents an analysis that may be seen in the daily clinic, i.e. the dataset is composed of images without clinically relevant abnormalities as well as all other pathologies such as tumors, multiple sclerosis lesions, abscesses, age-related ischemic changes.

[0196] As shown, the performance of all model types is good, but the standard U-net model outperforms the other two methods, even though the cascade model has twice as many parameters.

[0197] [Example 2] In another example, natural language processing for pathology is used to automatically scan 2 million radiology reports to select 5,000 brain MRI studies reflecting the most prominent pathologies: infarcts (hyperacute and acute) and tumors.

[0198] Infarct and tumor pathology were annotated pixel-by-pixel by trained annotators under radiologist supervision and quality control. Two sets of MRI brain protocols were applied to clinically normal patients, patients with tumors and patients with infarcts (and patients with both) and established. These include 1) controls utilizing a standard clinical protocol and 2) a smart protocol according to the invention utilizing four base sequences and up to two additional dedicated pathology-specific sequences.

[0199] [Example 2-Result] In an independent dataset of 88 scans, the turnaround time from scanning the sequence to reporting results to the hospital system was less than 60 seconds.Specificity and sensitivity for detection were 95% (88-99%), 78% (52-94%) for tumor, and 75% (63-85%), 100% (83-100%) for infarction.

[0200] On average, this method acquired 1.25 fewer sequences per patient, with an overall loss of 0.23 dedicated sequences for patients with pathology, demonstrating the advantage of the inventive approach.

[0201] Although embodiments of the present invention have been described with particular reference to the examples shown, it will be understood that variations and modifications to the described examples can be made within the scope of the present invention.

[0202] For example, in embodiments, the training of U-Net can be further improved by using advanced data augmentation to artificially generate more training data, for example by using an alternative PADDIT algorithm [3].

[0203] Furthermore, for some findings (e.g. anatomical) or in case of abundant training data, the neural net algorithm / U-net can be directly trained on global findings by inputting medical scan tuples as input, and instead of training on voxel-wise annotation, it can be directly trained on clinical findings in the tuples.

[0204] Furthermore, bias field correction can be applied to reduce noise due to bias fields in medical scans, for example by using differential bias correction methods [4, 5] towards templates or by training a U-Net to bias correct. The latter requires proper training data and can provide a very fast bias correction method. The training set can be raw MRI scans as input and N4 (the state-of-the-art bias field correction method) bias corrected scans as targets. Furthermore, efficient utilization of 3D information through the use of multi-planar U-Net [6] and / or random projection augmentation [7].

[0205] Finally, the voxel-level results can be post-processed by applying skull stripping to the original medical scan to obtain a map of brain tissue and cerebrospinal fluid voxels, which is used to filter any mis-segmented voxels outside the brain. Fast skull stripping can be obtained, for example, by training a U-Net to perform skull stripping based on skull-stripped ground truth obtained from the ROBEX algorithm [8] or manually segmented ground truth.

[0206] In an aspect, the embodiments described herein relate to a method for extracting information from a digital image, however, the embodiments described herein are equally applicable as a set of computer instructions for carrying out said method or as a suitably programmed computer.

[0207] The methods described herein are, in use, implemented on a suitable computer system or device executing one or more computer programs formed in software and / or firmware and operable to perform the methods described above. A suitable computer system generally includes hardware and an operating system.

[0208] The term "computer program" shall be taken to mean, but is not necessarily limited to, any application program, middleware, operating system, firmware or device driver, or any other medium supporting executable program code.

[0209] The term "hardware" may be interpreted to mean any one or more of the collection of physical elements that make up a computer system / device, such as, but not limited to, processors, memory devices, communication ports, input / output devices, etc. The term "firmware" may be interpreted to mean any persistent memory and program code / data stored therein, such as, but not limited to, embedded systems. The term "operating system" may be interpreted to mean one or more, often a collection of software, that manages computer hardware and provides general services to computer programs.

[0210] The comparison step may also be performed utilizing previous measurements against a data set whose values ​​or averages are stored in a database or memory location within such a computer. The computer may be programmed to display the results of the comparison as a readout.

[0211] The methods described herein may be implemented in one or more of software and / or hardware. The software is preferably stored or otherwise encoded in a memory device such as, but not limited to, a hard disk drive, RAM, ROM, solid state memory, or any one or more of other suitable memory devices or components configured in software. The methods may be realized by executing / running the software. Additionally or alternatively, the methods may be hardware encoded.

[0212] The methods encoded in the software or hardware are preferably executed using one or more processors. The memory, hardware and / or processors are preferably included as or at least part of one or more servers and / or other suitable computing systems.

[0213] References 1.a Ronneberger O.,Fischer P.,Brox T.(2015)“U-Net:Convolutional Networks for Biomedical Image Segmentation”In:Navab N.,Hornegger J.,In Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015.Lecture Notes in Computer Science,vol 9351.Springer,Cham 1.b Myronenko,A(2018)“3D MRI brain tumor segmentation using autoencoder regularization”,arXiv:abs / 1810.11654 1.c.Hongying Liu,Xiongjie Shen,Fanhua Shang,Feihang Ge,and Fei Wang,“CU-net:Cascaded U-net with loss weighted sampling for brain tumor segmentation,Multimodal Brain Image Analysis and Mathematical Foundations of Computational Anatomy,Pages 102-111,Cham,2019.Springer International Publishing.ISBN 978-3-030-33226-6 1.d.Fabian Isensee,Jens Petersen,Andre Klein,David Zimmerer,Paul F.Jaeger,Simon Kohl,Jakob Wasserthal,Gregor Koehler,Tobias Norajitra,Sebastian Wirkert,and Klaus H.Maier-Hein.“Nnu-net:Self-adapting framework for u-net-based medical image segmentation”ArXiv,abs / 1809.10486,2018 2.Patrice Y.Simard,Dave Steinkraus,John C.Platt(2003)Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis,In Seventh International Conference on Document Analysis and Recognition 3.Mauricio Orbes-Ortega,Lauge Sorensen,Jorge Cardoso,Marc Modat,Sebastien Ourselin,Stefan Sommer,Mads Nielsen,Christian Igel,Akshay Pai(2019)PADDIT:Probabilistic Augmentation of Data using Diffeomorphic Image Transformation,In Medical Imaging 2019:Image Processing,SPIE Medical Imaging 2019. 4.Bin Zou,Akshay Pai,Lauge Sorensen,Mads Nielsen,Simultaneous registration and bilateral differential bias correction in brain MRI,in MICCAI 2014 Workshop on Intelligent Imaging:linking MR acquisition and processing,2014 5.Bin Zou,Akshay Pai,Lauge Sorensen,Mads Nielsen,Bias Correction in Images,U.S.patent application US20170243336A1 6.Kaisar Kushibar,Sergi Valverde,Sandra Gonzalez-Villa,Jose Bernal,Mariano Cabezas,Arnau Oliver,Xavier Llado,Automated sub-cortical brain structure segmentation combining spatial and deep convolutional features,Medical Image Analysis 48,2018,177-186. 7.Perslev,Mathias;Pai,Akshay Sadananda Uppinakudru;Igel,Christian;Dam,Erik Bjornager(2018),Knee Segmentation by Multiplanar Deep Learning Network-with data from OAI,In 12th International Workshop on Osteoarthritis Imaging. 8.Iglesias JE,Liu CY,Thompson P,Tu Z:“Robust Brain Extraction Across Datasets and Comparison with Publicly Available Methods”,IEEE Transactions on Medical Imaging,30(9),2011,1617-1634.

Claims

1. 1. A method of performing a medical scan of a subject using a medical imaging system, comprising: a) initiating a medical scanning session; b) performing a first image acquisition sequence using the medical imaging system during the medical scanning session to obtain first image scan data; c) using a computer system to perform image analysis on the first image scan data acquired during the first image acquisition sequence to identify one or more quantitative indicators of pathology and / or image quality; d) determining, using the computer system based on said identification of one or more quantitative indicators in step c), whether any additional image acquisition sequences are required during said medical scanning session, and if so, e) determining, using the computer system, a second image acquisition sequence based on the one or more quantitative indicators; f) performing a second image acquisition sequence using said medical imaging system during said medical scanning session to obtain second image scan data; The method includes:

2. The method of claim 1 , wherein the first image scan data and / or the second image scan data comprises two-dimensional image scan data.

3. The method of claim 1 , wherein the first image scan data and / or the second image scan data comprises three-dimensional image scan data.

4. The method of claim 2 or 3, wherein step c) is performed using a classifier and the one or more quantitative measures comprise voxel classifications.

5. Step c) is g) generating a voxel classification map from the first image scan data. The method of claim 4 further comprising:

6. Step g) is h) aggregating the voxel classification map into an overall classification of the first image scan data. The method of claim 5 further comprising:

7. i) identifying any blobs in the voxel classification map that belong to a given voxel classification; j) comparing the identified blobs with a predetermined threshold; k) identifying a positive indication of potential pathology if said predetermined threshold is exceeded; The method of claim 5 or 6, further comprising:

8. The method of any one of claims 1 to 7, wherein the computer system includes a neural network algorithm.

9. The method of claim 8 , wherein the neural network algorithm comprises a convolutional neural network algorithm.

10. The method of claim 8 or 9, wherein the neural network algorithm comprises U-net.

11. Before step a), l) training said neural network algorithm to identify a quantitative indicator using a training set comprising a set of medical scan tuples. The method according to any one of claims 8 to 10, further comprising:

12. The method of claim 11 , wherein step l) comprises utilizing relevant ground truth annotations.

13. The method of any one of claims 1 to 12, wherein step c) is operable to blindly predict quantitative indicators of pathology and / or image quality.

14. The method according to any one of claims 1 to 13, wherein step c) does not require pre-image scan data of the particular object.

15. The method of any one of claims 1 to 14, wherein the first image acquisition sequence comprises a plurality of different scan types.

16. The medical imaging system includes an MRI scanner, and the multiple scan types include T2 FLAIR, DWI, 3D SWI, T2 ＊ 16. The method of claim 15, selected from the group of gradient echo and T1 TSE.

17. 17. The method of claim 15 or 16, wherein step b) comprises performing a first image scan of a first type and then performing a second image scan of a second type.

18. 20. The method of claim 17, wherein step c) is performed on image scan data from the first image scan while the second image scan is occurring.

19. 19. The method of any one of claims 1 to 18, wherein the quantitative index provides an indication of one or more of the following underlying pathologies: tumor (general), glioma, granuloma, abscess, hemorrhage, microhemorrhage, infarction, ischemic changes, demyelination, neoplasm, neurodegenerative disease and hydrocephalus.

20. Step d) is m) determining, using said computer system, whether a notification should be generated based on said identification of one or more quantitative indicators of pathology in step c). and if it is determined that the image should be generated, the method further comprises: n) generating a notification during said medical scanning session regarding a pathology likely associated with said one or more identified quantitative indicators. The method of any one of claims 1 to 19, further comprising:

21. The method of claim 20 , wherein the notification is provided to an operator of the medical imaging system and / or a medical professional during the medical scanning session.

22. 22. The method of claim 20 or 21, wherein the notification indicates that the subject is likely in need of emergency treatment.

23. 23. The method of claim 22, wherein the one or more quantitative indicators identified in step c) provide one or more indications of hemorrhage, microhemorrhage or infarction and that the emergency treatment is for a stroke.

24. 24. The method of claim 23, wherein the notification indicates a high likelihood of need for thrombolytic therapy, anticoagulant medication, or therapy to avoid anticoagulant medication.

25. 1. A method of performing a medical scan of a subject using a medical imaging system, comprising: a) initiating a medical scanning session; b) performing an image acquisition sequence using the medical imaging system during the medical scanning session to obtain image scan data; c) using a computer system to perform image analysis on the image scan data acquired during the image acquisition sequence to identify one or more quantitative indicators of pathology; d) determining, using the computer system, whether a notification should be generated based on the identification of one or more of the quantitative indicators of pathology in step c); If it is determined that the e) generating a notification regarding a pathology likely associated with the one or more identified quantitative indicators during the medical scanning session. The method further comprises:

26. 26. The method of claim 25, wherein the notification is provided to an operator of the medical imaging system and / or a medical professional during the medical scanning session.

27. 27. The method of claim 25 or 26, wherein the notification indicates that the subject is likely in need of emergency treatment.

28. 28. The method of claim 27, wherein the one or more quantitative indicators identified in step c) provide one or more indications of hemorrhage, microhemorrhage or infarction and that the emergency treatment is for a stroke.

29. 30. The method of claim 28, wherein the notification indicates a high likelihood of need for thrombolytic therapy, anticoagulant medication, or therapy to avoid anticoagulant medication.

30. Step c) is f) using a computer system to perform image analysis on the image scan data acquired during the image acquisition sequence to identify one or more quantitative indicators of image quality. The method of any one of claims 25 to 29, further comprising:

31. Step d) is g) determining, using the computer system based on said identification of one or more quantitative indicators in step c), whether any additional image acquisition sequences are required during said medical scanning session, and if so, h) determining, using the computer system, a second image acquisition sequence based on the one or more quantitative indicators; i) performing a second image acquisition sequence using said medical imaging system during said medical scanning session to obtain second image scan data; The method of any one of claims 25 to 30, further comprising:

32. A method according to any one of claims 25 to 31, wherein the image scan data and / or the second image scan data comprises two-dimensional image scan data.

33. A method according to any one of claims 25 to 31, wherein the image scan data and / or the second image scan data comprises three-dimensional image scan data.

34. 34. The method of claim 32 or 33, wherein step c) is performed using a classifier and the one or more quantitative measures comprise voxel classifications.

35. Step c) is j) generating a voxel classification map from said image scan data.

35. The method of claim 34, further comprising:

36. Step g) is k) aggregating the voxel classification map into an overall classification of the image scan data.

36. The method of claim 35, further comprising:

37. l) identifying any blobs in the voxel classification map that belong to a given voxel classification; m) comparing the identified blobs with a predetermined threshold; n) identifying a positive indication of potential pathology if said predetermined threshold is exceeded; 37. The method of claim 35 or 36, further comprising:

38. The method of any one of claims 25 to 37, wherein the computer system includes a neural network algorithm.

39. 40. The method of claim 38, wherein the neural network algorithm comprises a convolutional neural network algorithm.

40. The method of claim 38 or 39, wherein the neural network algorithm comprises U-net.

41. Before step a), o) training said neural network algorithm to identify quantitative indicators using a training set comprising a set of medical scan tuples; The method of any one of claims 38 to 40, further comprising:

42. 42. The method of claim 41 , wherein step o) comprises utilizing relevant ground truth annotations.

43. The method of any one of claims 25 to 42, wherein step c) is operable to blindly predict quantitative indicators of pathology and / or image quality.

44. The method of any one of claims 25 to 43, wherein step c) does not require pre-image scan data of the particular object.

45. The method of any one of claims 25 to 44, wherein the image acquisition sequence comprises a plurality of different scan types.

46. The medical imaging system includes an MRI scanner, and the multiple scan types include T2 FLAIR, DWI, 3D SWI, T2 ＊ 46. ​​The method of claim 45, selected from the group of gradient echo and T1 TSE.

47. 47. A method according to claim 45 or 46, wherein step b) comprises performing an image scan of a first type and then performing a second image scan of a second type.

48. 48. The method of claim 47, wherein step c) is performed on image scan data from the image scan while the second image scan is being performed.

49. A method according to any preceding claim, wherein the medical imaging system comprises an MRI scanner and the image scan data is produced by Magnetic Resonance Imaging (MRI).

50. A method according to any preceding claim, wherein the medical imaging system comprises a CT scanner and the image scan data comprises computed tomography (CT) data.

51. A method according to any preceding claim, wherein the medical imaging system comprises a PET scanner and the image scan data comprises PET data.

52. The method of any one of claims 1 to 51, wherein the quantitative measures are derived from a prior training process that utilizes non-associated image scan data.

53. A computer system comprising a processing device configured to carry out the method of any one of claims 1 to 52.

54. A computer readable medium comprising instructions configured, when executed, to perform the method of any one of claims 1 to 52.

55. 55. A computer system comprising a processing device, a storage device, and the computer readable medium of claim 54.

Images