Improved medical scan protocol for patient data acquisition analysis within a scanner

Abstract

A method of performing a medical scan of a subject using a medical imaging system is provided, the method comprising: a) initiating a medical scan session; b) performing a first image acquisition sequence using the medical imaging system during the medical scan session to obtain first image scan data; c) performing image analysis using a computer system on the first image scan data acquired during the first image acquisition sequence to identify one or more quantitative indicators of pathology and / or image quality; d) based on the identification of the one or more quantitative indicators in step c), determining using a computer system whether any additional image acquisition sequences are needed during the medical scan session; and, if so: e) determining a second image acquisition sequence using the computer system based on the one or more quantitative indicators; and f) performing the second image acquisition sequence using the medical imaging system during the medical scan session to obtain second image scan data.

Description

Technical Field

[0001] This invention relates to an improved method and apparatus for in-scan patient data acquisition and analysis. More specifically, this invention relates to an improved method and apparatus for scan selection and acquisition in response to one or more indicators. At least one indicator may include a pathological indicator measure or a quantitative parameter. Background Technology

[0002] One non-invasive technique for imaging the brain and other areas of the body is computed tomography (CT) scans, which combine a series of X-ray images taken from different angles around the body and use computational methods to create cross-sectional images (slices) of bones, blood vessels, and soft tissues within the body.

[0003] Another technique is positron emission tomography (PET), which can be used to produce detailed three-dimensional images of the body's interior. PET scans use radioactive tracers, which are molecules containing small amounts of radioactive material that can be detected in a PET scan. These are designed to accumulate in cancerous tumors or areas of inflammation. They can also bind to specific proteins in the body.

[0004] The increasingly preferred technique is structural magnetic resonance imaging (MRI). MRI is a non-invasive technique used to examine the physical structure of the body (e.g., to calculate tissue volume). This is of great value for monitoring tissue damage, particularly neurodegenerative diseases.

[0005] As is well known, MRI is based on the magnetization properties of atomic nuclei. A large, uniform external magnetic field aligns proteins within the nuclei of the tissue being examined. This alignment is then disrupted by an external radio frequency (RF) signal.

[0006] The nucleus returns to rest alignment through many different relaxation processes, during which it emits RF signals. By changing the order of RF pulse emission and detection, different characteristics of the tissue under examination can be measured. Repetition time (TR) is the amount of time between consecutive pulse sequences applied to the same slice. Echo time (TE) is the time between the delivery of the RF pulse and the reception of the echo signal.

[0007] Many MRI techniques are available. T1-weighted and T2-weighted scans are common. T1 (longitudinal relaxation time) is a time constant that determines the rate at which excited protons rearrange themselves with the applied external magnetic field. T2 (transverse relaxation time) is a time constant that determines the rate at which excited protons lose phase coherence with a nucleus having a spin perpendicular to the applied external magnetic field. T1-weighted images are characterized by short TE and TR times. Conversely, T2-weighted images are produced by using longer TE and TR times.

[0008] An increasingly popular sequence is Fluid Attenuated Inversion Recovery (FLAIR). FLAIR is similar to T2-weighted imaging, except that the TE and TR times are much longer. Using this method, abnormalities remain bright, but are diluted by cerebrospinal fluid and thus appear darker in the acquired image. This system is also known as T2 FLAIR scanning.

[0009] Diffusion-weighted imaging (DWI) is a form of MRI that measures the random Brownian motion of water molecules within voxels (volume pixels) of the tissue being examined. Typically, highly cellular tissues or tissues with swollen cells exhibit a low diffusion coefficient. Diffusion is particularly useful in tumor features and in cases of localized cerebral ischemia.

[0010] Contrast-enhanced MRI (CE-MRI) and dynamic contrast-enhanced MRI (DEC-MRI) are also available techniques. In CE-MRI, gadolinium contrast agent is used to improve the clarity of available images. CE-MRI can, for example, improve the visibility of inflammation, blood vessels, and tumors. DCE-MRI acquires multiple MR images sequentially after the administration of a contrast agent. This allows for monitoring the contrast of the contrast agent (“wash-in” and “wash-out”), thereby enabling improved detection of, for example, vascular injury and tumors.

[0011] A more advanced technique is gradient echo (GRE) MRI scanning. This is generated by combining a single RF pulse with gradient inversion. After the RF pulse, the first negative portion of the gradient lobe causes phase dispersion of the precessing spin. When this gradient is inverted, the spin refocuses and forms a gradient (echo) echo. GRE scans typically involve short TR and short TE values, thus providing rapid signal acquisition. Therefore, GRE sequences can provide rapid imaging and MR angiography techniques.

[0012] Susceptibility-weighted imaging (SWI) is a 3D, spatially high-resolution, velocity-corrected gradient echo MRI sequence that uses compounds with paramagnetic, diamagnetic, and ferromagnetic properties. Such paramagnetic compounds include deoxyhemoglobin, ferritin, and hemosiderin. Diamagnetic compounds include bone minerals and dyscalcifications.

[0013] A variant of GRE is susceptibility-weighted angiography (SWAN). SWAN allows for high-resolution visualization of cerebral veins and arteries in a single sequence without the use of contrast agents, and the scan time is significantly shorter than other arrangements.

[0014] Turbine spin echo (TSE) (also known as fast spin echo - FSE) is an adaptation of conventional spin echo (SE) acquisition techniques designed to reduce imaging time. In a standard SE sequence, a single echo is measured during each repetition time (TR). In TSE, multiple echoes are recorded after each TR. This is achieved by sending a series of 180-degree out-of-phase pulses at predetermined intervals and recording the corresponding echo signals according to different phase encoding gradients. This allows for encoding multiple phase encoding steps after a single 90-degree pulse.

[0015] The above is a non-exhaustive list, and other techniques can be considered.

[0016] Scans can involve one or more of the tests mentioned above. Often, these techniques are combined to provide enhanced medical data in a process known as co-registration or image fusion. In practice, scanners are available that can perform multiple diagnostic procedures in a single scan; for example, single-photon emission computed tomography / computed tomography (SPECT / CT) and positron emission tomography / computed tomography (PET / CT) units can perform multiple imaging examinations simultaneously.

[0017] Regardless of the technology used, existing scanning devices have many problems. Patient scanning is expensive for service providers and potentially inconvenient and stressful for patients. Therefore, it is desirable to minimize the number of times patients need to be scanned.

[0018] However, patient scan data is often collected, and during subsequent analysis, it may be found that the scan data is unusable in the diagnostic process due to incorrect or insufficient acquisition, or due to deficiencies in quality control resulting in image quality that is too poor for medical diagnostic purposes. In such cases, the patient may have to be recalled for further scans. This is clearly undesirable and may (e.g., in the case of PET, or in the case of contrast agents) subject the patient to additional and unnecessary risks.

[0019] Therefore, there is a need in the art for an improved method that reduces the likelihood of patients being recalled for further scans, thereby enabling the acquisition of additional images. The existing technical problem is that existing methods for scan data acquisition are too slow and not adaptive enough to acquire all the necessary data during a single scan session and prevent unnecessary scanning steps. Summary of the Invention

[0020] The above-mentioned problems are solved in the embodiments of the present invention. The present invention described and claimed below can be used with all the above-described techniques, but is not limited thereto.

[0021] According to a first aspect of the present invention, a method for performing a medical scan of a subject using a medical imaging system is provided, the method comprising: a) initiating a medical scanning session; b) performing a first image acquisition sequence using the medical imaging system during the medical scanning session to obtain first image scan data; c) performing image analysis on the first image scan data acquired during the first image acquisition sequence using a computer system to identify one or more quantitative indicators of pathology and / or image quality; d) determining, based on the identification of one or more quantitative indicators in step c), whether any additional image acquisition sequence is required during the medical scanning session using the computer system; and, if required, e) determining a second image acquisition sequence using the computer system based on one or more quantitative indicators; and f) performing the second image acquisition sequence using the medical imaging system during the medical scanning session to obtain second image scan data.

[0022] In one embodiment, the first image scan data and / or the second image scan data include two-dimensional image scan data. In another embodiment, the first image scan data and / or the second image scan data include three-dimensional image scan data.

[0023] In one embodiment, the first image scan data and / or the second image scan data include one or more images of a selected body part of the subject. In another embodiment, the first image scan data and / or the second image scan data include one or more 2D and / or 3D images of a selected body part of the subject.

[0024] In one implementation, the quantitative metric is derived from a previous training process using unrelated image scan data.

[0025] In one embodiment, a classifier is used to perform step c), and the one or more quantitative indicators include voxel classification. In one embodiment, step c) further includes: g) generating a voxel classification map from the first image scan data. In one embodiment, step g) further includes: h) aggregating the voxel classification map into a total classification of the first image scan data.

[0026] In one embodiment, the method further includes the steps of: i) identifying any spots in a voxel classification map belonging to a predetermined voxel classification; and j) comparing the identified spots with a predetermined threshold; and k) if the predetermined threshold is exceeded, identifying a positive indication of potential pathology.

[0027] In one implementation, step i) includes: l) performing one or more of anisotropic Gaussian spot detection, isotropic Gaussian spot detection, and / or ridge detection to identify the spots.

[0028] In one embodiment, the computer system includes a neural network algorithm. In one embodiment, the neural network algorithm includes a convolutional neural network algorithm. In one embodiment, the neural network algorithm includes a U-network. In one embodiment, the convolutional neural network includes an encoding part and a decoding part. In one embodiment, the encoding part includes a shrinking path, and the decoding part includes an expanding path. In one embodiment, the encoding part includes multiple convolutions, each followed by a rectified linear unit and a merging operation.

[0029] In one embodiment, the method further includes, prior to step a), m) training a neural network algorithm using a training set comprising a set of medical scan tuples to identify the quantitative indicator. In one embodiment, step m) includes using relevant basic fact annotations. In one embodiment, step c) is operable for blindly predicting the quantitative indicator for pathology and / or image quality.

[0030] In one implementation, step c) does not require prior scan image data of the specific subject. In one implementation, the first image acquisition sequence includes multiple different scan types.

[0031] In one embodiment, the medical imaging system includes an MRI scanner, and the plurality of scan types are selected from the group consisting of T2FLAIR; DWI; 3D SWI; T2* gradient echo and T1 TSE.

[0032] In one implementation, step b) includes performing a first image scan of a first type, followed by performing a second image scan of a second type. In one implementation, step c) is performed on image scan data from the first image scan while the second image scan is being performed.

[0033] In one implementation, the quantitative indicators provide indications of one or more of the following potential pathologies: tumor (general); glioma; granuloma; abscess; hemorrhage; microbleed; infarction; localized ischemic changes; demyelination; vegetations; neurodegenerative diseases and hydrocephalus.

[0034] In one implementation, step d) further includes: m) using a computer system to determine whether a notification should be generated based on the identification of one or more of the quantitative indicators of the pathology in step c), and if so, the method further includes: n) generating a notification corresponding to the identified quantitative indicator during a medical scanning session.

[0035] In one implementation, step d) further includes: m) using a computer system to determine whether a notification should be generated based on the identification of one or more of the quantitative indicators of the pathology in step c), and if so, the method further includes: n) generating a notification related to a possible pathology during a medical scanning session that is associated with the identified quantitative indicators.

[0036] In one implementation, the notification is provided to the operator of the medical imaging system and / or a medical professional during a medical scanning session. In another implementation, the notification indicates the potential need for emergency treatment of the subject.

[0037] In one implementation, one or more quantitative indicators identified in step c) provide an indication of one or more of the following: bleeding; microbleeds; or infarction; emergency treatment involving stroke. In one implementation, the notification indication may require: thrombolytic therapy; blood-thinning drugs; or treatment to avoid blood-thinning drugs.

[0038] In one implementation, step m) includes: o) associating one or more values ​​of the one or more identified quantitative indicators with values ​​of the one or more identified quantitative indicators associated with normal healthy subjects and / or values ​​associated with known pathologies.

[0039] According to a second aspect of the present invention, a method for performing a medical scan of a subject using a medical imaging system is provided, the method comprising: a) initiating a medical scan session; b) performing an image acquisition sequence using the medical imaging system during the medical scan session to obtain image scan data; c) performing image analysis on the image scan data acquired during the image acquisition sequence using a computer system to identify one or more quantitative indicators of a pathology; d) determining, based on the identification of one or more quantitative indicators of a pathology in step c), whether a notification should be generated using the computer system, and if so, the method further comprising: e) generating a notification related to a possible pathology associated with the identified quantitative indicators during the medical scan session.

[0040] In one implementation, the notification is provided to the operator of the medical imaging system and / or a medical professional during a medical scanning session. In another implementation, the notification indicates the potential need for emergency treatment of the subject.

[0041] In one implementation, one or more quantitative indicators identified in step c) provide an indication of one or more of the following: bleeding; microbleeds; or infarction; emergency treatment involving stroke. In one implementation, the notification indication may require: thrombolytic therapy; blood-thinning drugs; or treatment to avoid blood-thinning drugs.

[0042] In one implementation, step c) further includes: f) using a computer system to perform image analysis on the image scan data acquired during the image acquisition sequence to identify one or more quantitative indicators of image quality.

[0043] In one implementation, step d) further includes g) using a computer system to determine whether any additional image acquisition sequence is required during the medical scanning session based on the identification of one or more quantitative indicators in step c); and, if required: h) using the computer system to determine a second image acquisition sequence based on the one or more quantitative indicators; and i) performing the second image acquisition sequence using the medical imaging system during the medical scanning session to obtain second image scan data.

[0044] In one embodiment, the first image scan data and / or the second image scan data include two-dimensional image scan data. In one embodiment, the first image scan data and / or the second image scan data include three-dimensional image scan data. In one embodiment, a classifier is used to perform step c), and the one or more quantitative indicators include voxel classification. In one embodiment, step c) further includes: j) generating a voxel classification map from the image scan data.

[0045] In one implementation, step g) further includes: k) aggregating the voxel classification map into a general classification of the image scan data.

[0046] In one implementation, the method further includes the steps of: l) identifying any spot in a voxel classification map belonging to a predetermined voxel classification; and m) comparing the identified spot with a predetermined threshold; and n) if the predetermined threshold is exceeded, identifying a positive indication of potential pathology.

[0047] In one embodiment, the computer system includes a neural network algorithm. In one embodiment, the neural network algorithm includes a convolutional neural network algorithm. In one embodiment, the neural network algorithm includes a U-network.

[0048] In one embodiment, prior to step a), the method further includes: o) training a neural network algorithm using a training set comprising a set of medical scan tuples to identify the quantitative indicator. In one embodiment, step o) includes using relevant basic fact annotations. In one embodiment, step c) is operable for blindly predicting the quantitative indicator for pathology and / or image quality.

[0049] In one implementation, step c) does not require prior image scan data of the specific subject. In one implementation, the image acquisition sequence includes multiple different scan types.

[0050] In one embodiment, the medical imaging system includes an MRI scanner, and the plurality of scan types are selected from the group consisting of T2FLAIR; DWI; 3D SWI; T2* gradient echo and T1 TSE.

[0051] In one implementation, step b) includes performing a first type of image scan, followed by a second type of second image scan. In one implementation, step c) is performed on the image scan data from the second image scan while the second image scan is being performed.

[0052] In one embodiment, the medical imaging system includes an MRI scanner, and the image scan data is generated by magnetic resonance imaging (MRI). In one embodiment, the medical imaging system includes an MRI scanner, and the image scan data is generated using magnetic resonance imaging (MRI) selected from one or more techniques selected from the group consisting of T1-weighted and T2-weighted MRI scans; FLAIR scans; diffusion-weighted imaging (DWI); contrast-enhanced MRI (CE-MRI); dynamic contrast-enhanced MRI (DEC-MRI); gradient echo (GRE); susceptibility-weighted imaging (SWI); and turbine spin echo (TSE).

[0053] In one embodiment, the medical imaging system includes a CT scanner, and the image scan data includes computed tomography (CT) data. In another embodiment, the medical imaging system includes a PET scanner, and the image scan data includes PET data.

[0054] In one embodiment, steps a) to f) of the first aspect and steps a) to e) of the second aspect are performed while the subject remains in the medical imaging system. In one embodiment, the (first) image scan data and / or the second image scan data relate to imaging a portion of the subject's body. In one embodiment, the (first) image scan data and / or the second image scan data relate to imaging the subject's brain.

[0055] In one embodiment, step c) and / or step f) further includes performing cranial dissection on the first image scan data to obtain voxel maps of brain tissue and cerebrospinal fluid. In one embodiment, the cranial dissection is performed by a neural network algorithm. In one embodiment, the neural network algorithm is trained using a training set containing a set of image scan tuples to identify and remove cranial portions of the first and / or second image scan data.

[0056] According to a third aspect of the invention, a computer system is provided, comprising processing equipment configured to perform the methods of the first or second aspect.

[0057] According to a fourth aspect of the invention, a computer-readable medium is provided comprising instructions that, when executed, are configured to perform the method of the first or second aspect.

[0058] According to a fifth aspect of the present invention, a computer system is provided, comprising: a processing device, a storage device, and a computer-readable medium as described in the third aspect. Attached Figure Description

[0059] Embodiments of the present invention will now be described in detail with reference to the accompanying drawings, wherein:

[0060] Figure 1 An exemplary computer system 100 that forms part of the present invention is shown;

[0061] Figure 2 This illustrates the context of the scan sequence workflow. Figure 1 Computer system 100;

[0062] Figure 3 A schematic arrangement of a U-network neural network is shown;

[0063] Figure 4 A flowchart of the method of the present invention is shown;

[0064] Figure 5 A schematic workflow of the Level 1 scanning acquisition process is shown; and

[0065] Figure 6 A schematic diagram of the scanning acquisition process, including two levels and subsequent scanning acquisition, is shown. Detailed Implementation

[0066] In embodiments of the present invention, the present invention relates to a novel non-invasive medical imaging processing application for the automatic labeling and visualization of candidate findings from a set of magnetic resonance (MR) images. The output consists of visualizations of candidate pathologies labeled with quantitative indicators.

[0067] Users need to use the provided tools to validate candidate findings. This invention aims to provide trained medical professionals with supplemental information for evaluating and assessing MR images, and to assist them in determining appropriate additional MR sequences for diagnosis and prioritizing workflows. This invention relates to a post-processing workstation or cloud solution for working with routed digital imaging and medical communications (DICOM) images.

[0068] The smart protocol according to embodiments of the invention is operable to classify data acquired during a scan and provide medical professionals with instructions on whether to recommend any further scans and which scans to recommend. In other words, the smart protocol is a configurable item that outputs a recommended next action (e.g., a new scan) for a given medical scan's input tuple. This configuration is context-specific.

[0069] A unique feature of this invention is that the method does not require prior knowledge of the patient's medical history or previous scan data. Therefore, this invention can be used for patients undergoing their first scan, or for patients investigating medical complications different from those observed in previous scans.

[0070] The smart protocol is implemented as a decision tree, and the decision at each node in the tree is made automatically by the algorithm.

[0071] This method includes a machine learning algorithm that identifies parameters of interest in scan data. These parameters are identified through a training process capable of classifying specific features in an image by one or more quantitative indicators or parameters indicative of one or more pathologies or image quality. Trained in this way, the computational process can identify quantitative indicators or parameters indicative of specific pathologies and / or image quality during a medical scanning session and recommend any additional scanning sequences that may be necessary, and / or, in a particular implementation, provide quantitative indicators that offer possible indications of pathologies that may require immediate diagnosis and / or treatment by a medical professional.

[0072] This is a significant improvement over known devices that require medical professionals to analyze scan data after the scan session is complete.

[0073] Figure 1 An exemplary computer system 100 that forms part of the present invention is shown. Figure 2 This illustrates the context of the scan sequence workflow. Figure 1 Computer system 100.

[0074] This invention relates to a medical imaging system 10. The medical imaging system 10 includes a computer system 100. The computer system 100 can take any suitable form and may include, for example, a remote computer, a cloud-based computing system, a local workstation, or a medical imaging system console.

[0075] Computer system 100 includes one or more processors 102, computer-readable physical memory 104, and non-transient storage devices 106 such as hard disk drives or solid-state drives. Storage device 106 can take any suitable form and may include storage devices local to computer system 100 and / or storage devices external to computer system 100. An interface 110 is provided to enable computer system 100 to communicate with other components in medical imaging system 10, computing applications 110 run on the computing system, and are operable to enable computer system 100 to perform the methods of the present invention.

[0076] As described below, the computational application includes a classifier 112 and a classifier database 114. The image processor 116 is also operable to interact with the classifier 112, as described below.

[0077] The computing application 110 is operable to communicate with a medical imaging device 118 via an interface 108, in this embodiment, the medical imaging device 118 being in the form of an MRI scanner. However, this is not intended to be limiting; other types of scanners, such as CT or PET, or combinations thereof, may be used.

[0078] Interface 108 and scanner 116 communicate with storage device 120 in the form of a Picture Archiving and Communication System (PACS). PACS 120 is an industry-standard device and format for medical imaging. However, this is not intended to be limiting; other configurations may be used. PACS 120 is operable for processing images and other data in Digital Imaging and Medical Communications (DICOM) format. However, other proprietary or non-proprietary formats may also be used.

[0079] Reading station 122 communicates with PACS 120 and can be used by licensed physicians to read, interpret and analyze image data.

[0080] It is worth noting that in this embodiment, the computer system 100 communicates via a radiologist information system or a hospital information system (HIS), as shown in 124. However, this is not essential to the present invention, and other systems may be appropriately used.

[0081] Now refer to Figures 3 to 6 The method of operation of the present invention is described.

[0082] In summary, this invention includes a control and analysis method that outputs a recommended next action (e.g., performing a new scan) for a given input tuple of a medical scan. This configuration is context-specific and provided by the clinic to use an intelligent protocol. The intelligent protocol is implemented as a decision tree, and decisions at each node in the tree are made automatically by an algorithm.

[0083] Step 200: Provide the algorithm

[0084] Classifier 114 includes a machine learning algorithm that, in one embodiment, utilizes a collection of voxel-wise classifications from a U-Net deep learning algorithm for medical image segmentation, as described in reference [1a]. However, this is not intended to be limiting, and other voxel-wise classifiers may also be used in their place.

[0085] The building blocks of deep neural networks are artificial neurons or nodes. Each input has an associated weight. The sum of all weighted inputs is then passed through a non-linear activation function f to transform the pre-activation level of the neuron into its output. This output is then used as the input to nodes in the next layer.

[0086] Several activation functions are available, and they differ in how they map pre-activation levels to output values. The most commonly used activation functions are the rectified function (where neurons using it are called rectified linear units (ReLU)), the hyperbolic tangent function, the sigmoid function, and the softmax function.

[0087] The latter is typically used in the output layer because it can compute probabilities for multiple class labels. For each pattern j in the first hidden layer, a nonlinear function is applied to the weighted sum of the inputs. The result of this transformation is used as the input to the second hidden layer. This information is propagated through the network to the output layer, where a flexible maximum transfer function generates the probability of a given observation belonging to each class.

[0088] Convolutional networks are typically used for classification tasks, where the output of an image is a single class label. However, in many vision tasks, particularly in biomedical image processing, the desired output should include localization, i.e., a class label assigned to each voxel.

[0089] In one exemplary embodiment, the network architecture of the U-Net algorithm is shown below. Figure 3 The algorithm is derived from reference [1a]. It corresponds to a convolutional neural network (CNN) and includes an encoding part and a decoding part.

[0090] The encoding side includes a shrinking path (as shown on the left), and the decoding side includes an expanding path (as shown on the right).

[0091] The shrinking path follows the typical architecture of a convolutional network and consists of repeated applications of two 3×3 convolutions, each followed by a rectified linear unit (ReLU) and a 2×2 max-merge operation, where the span of 2 is used for downsampling.

[0092] In each downsampling step, the number of feature channels is doubled. Each step in the expansion path includes upsampling of the feature map, followed by a 2×2 convolution (“upconvolution”) that halves the number of feature channels, concatenated with the corresponding cropped feature map from the contraction path, and two 3×3 convolutions, each followed by a ReLU.

[0093] In implementations, if a CNN using the U-Net algorithm is used as the classifier 114, this can include different forms. For example, a conventional 3D U-Net with different parameters than the embodiments described above can be used. Any suitable number and size of convolutions can be used, with any suitable kernel size and span size. For example, a 1-span convolution can be used.

[0094] In a non-limiting embodiment, ReLU may include: a SoftPlus approximation; a noisy ReLU (including Gaussian noise); a leaky ReLU (which allows small positive gradients when the ReLU is inactive); a parametric ReLU (where the leakage coefficients within the leaky ReLU are the network's learned parameters); and / or an exponential linear unit (ELU).

[0095] Finally, the pooling layer can utilize the largest pool with any suitable filter and span size.

[0096] Alternatives to the conventional U-Net algorithm can be used. For example, a variable autoencoder (VAE) can be used. An example of a VAE is described in reference [1b].

[0097] As an alternative, cascaded U-Net VAEs can be used, in which two sets of U-Net VAE models are interconnected, such that the output of the first U-Net is concatenated to the input image and fed into the second U-Net. See reference [1c] for an example.

[0098] Step 202: Training the Algorithm

[0099] In the context of deep machine learning neural network solutions, classification involves two main steps. In the first step, the training phase, a subset of available data, known as the training set, is used to optimize the network's parameters to perform the desired task (classification). The model can be trained from scratch without any pre-training.

[0100] In the second step, the so-called testing phase, where the remaining subset known as the test set is used to evaluate whether the trained model can blindly predict the class of a new observation. When the amount of available data is limited, the training and testing phases can also be run multiple times on different training and testing splits of the original data, and then the average performance of the model can be estimated—a method known as cross-validation.

[0101] The algorithm of this invention is trained using a training set to classify each voxel in a medical scan into one of several candidate categories. The training set comprises a set of medical scan tuples with associated basic fact annotations. These can come from any suitable image acquisition technique or method.

[0102] A training set is formed using multiple input images, containing a set of medical scan tuples with associated basic fact annotations, to classify each voxel in the medical scan into one of several candidate categories.

[0103] Data can be preprocessed. For example, the input image can be resampled to obtain isotropic voxel sizes. To address problems such as infarction segmentation, diffusion sequences can be resampled so that the image is initially isotropic, and then fluid attenuation inversion recovery (FLAIR) and / or susceptibility-weighted imaging (SWI) sequences can be reformatted to the resampled diffusion space. Apparent diffusion coefficients can then be calculated.

[0104] Similarly, for the bleeding segmentation problem, the SWI space can be used. After resampling, the image can be z-normalized and fed into a neural network for training and inference. After resampling using, for example, Otsu's median filter, the data can be cropped to regions of non-zero values.

[0105] The examples above are exemplary and non-limiting. Those skilled in the art will readily recognize other input image and / or preprocessing methods that can be used in conjunction with the present invention.

[0106] Additionally, U-Net training can be improved through artificial augmentation using data augmentation. This data augmentation may be desirable when limited input images are available, allowing the network to be adequately trained.

[0107] Multiple transformations can be used to augment the dataset of training images. For example, a transformation based on a random velocity field can be used to expand the input image [2]. Alternatively, a random selection of transformations (which may not include augmentation or transformation) can be used for each training step.

[0108] The transformed input image can be weighted using a cost function with predetermined factors to account for differences and artifacts. In one embodiment, the predetermined factors are 0.75 and artifacts.

[0109] Alternative transformations can be used, and these transformations can include one or more of the following: rotation, elastic transformation, Gaussian noise, Gaussian blur, contrast, gamma correction, and mirroring. For models with VAE branches, only intensity transformations can be used due to image reconstruction regularization.

[0110] Finally, image post-processing can be used to improve image classification performance. In one implementation, morphological filtering can be used to remove random predictions from the image.

[0111] Regarding architecture selection, spots smaller than a certain size can be removed. In this implementation, spots smaller than 5mm can be removed. 3 The spots. Other sizes can also be used. For example, spots smaller than 125mm can be removed. 3 The spots are used to optimize classification performance.

[0112] Classifier 112 is trained using a training set of images in classifier database 114. Typically, classifier 112 works by taking the values ​​of a specific feature (independent variable or predictor in regression) from an instance (a set of independent variable values) and predicting the category (dependent variable) to which that instance belongs.

[0113] In this medical imaging context, the features include voxels (volume pixels), and the categories can indicate specific possible pathologies. Classifier 112 needs to learn multiple parameters from training data in database 114. The classifier is essentially a model of the relationship between features and category labels in the training set. More formally, given an instance x, the classifier predicts the label. The function f.

[0114] Typically, neural networks can learn using gradient descent-based algorithms. The goal of gradient descent is to find network weight values ​​that optimally minimize the error (difference) between the estimated output and the true output.

[0115] It should be noted that these methods are not actual classifiers themselves; rather, they are networks that are pre-trained to learn useful patterns in the data and then fed into the final layer of the real classifier.

[0116] In an exemplary implementation, the algorithm network is trained using a training set of input images and corresponding segmentation maps via stochastic gradient descent. The following is a general example based on reference [1a].

[0117] The energy function is calculated using the soft-max transfer function for each voxel on the final feature map, combined with the cross-entropy loss function. The soft-max transfer function is defined as described in Equation 1):

[0118] 1)

[0119] Where a k (x) represents the feature channel k at pixel location x∈Ω. Activation, k is the number of classes, p kpk(x) is an approximate maximal function, meaning that for the value of k with the maximum activation ak(x), pk(x) ≈ 1, and for all other k, pk(x) ≈ 0. Then, cross-entropy compensates for the deviation of pk(x) from 1 at each position, as shown in 2) below:

[0120] 2)

[0121] Where l: Ω→{1,...,K} is the true label of each pixel, w: It is a weighted map that is introduced to give some pixels more importance during training.

[0122] Weight maps are pre-computed for each basic fact segment to compensate for the different frequencies of pixels from a certain class in the training dataset and to force the network to learn small separation boundaries introduced between adjacent test units.

[0123] Then, calculate the weighted graph according to equation 3):

[0124] 3)

[0125] Where w C : This is a weighted graph of balanced frequencies, d1: This represents the distance to the nearest cell boundary, and d2: This is the distance to the boundary of the second nearest unit.

[0126] In deep networks with many convolutional layers and different paths through the network, proper initialization of weights is crucial; otherwise, some parts of the network may give over-activation while others never contribute. Ideally, the initial weights should be adapted such that each feature map in the network has approximately unit variance.

[0127] For networks with alternating convolutional and ReLU layers, this can be achieved by extracting initial weights from a Gaussian distribution with a standard deviation of √2 / N, where N represents the number of incoming nodes for a neuron. For example, for a 3×3 convolution and 64 feature channels in the previous layer, N = 9·64 = 576.

[0128] However, alternative methods can be used. In one implementation, dice are used as the loss function to train a U-network or similar neural network. In one instance, kernel weight regularization of 1e-6 can be used on all layers.

[0129] A dice loss function that is functionally similar to that in reference [1d] can be used, as shown in equation [1d]:

[0130] 4)

[0131] Where u is the predicted segment, v is the basic fact, and k is the number of classes.

[0132] The predicted segmentation and the ground truth are both encoded with a single key. A sigmoid function is used as the activation function. In one implementation, optimization can be achieved using adaptive learning rate methods such as adaptive timing estimation or Adam. The Adam process utilizes the first and second time-series estimates of the gradient to adapt the learning rate of each weight in the neural network.

[0133] In one implementation, the Adam method with a learning rate of 1e-1 is used as the optimizer. In this implementation, the model can be trained for 100 epochs, with each epoch comprising 750 steps.

[0134] In a non-limiting implementation, if no increase is made in the verification dice over the first 10 periods, a learning rate scheduler can be used to reduce the learning rate by 5%. The period with the best overall verification dice can then be used for final inference.

[0135] Then, it is verified that the dice score consists of two scores: a) the total dice for the foreground, and b) the dice for only the image with foreground. The average of these two dice scores constitutes the total dice.

[0136] While other methods can be used and this configuration is not limiting, this combination has the following advantages: there are a large number of non-foreground images with diffuse pathology, which may lead to a large number of false positives.

[0137] Step 204: Test the algorithm

[0138] Once trained, the classifier can be used to determine whether the features used contain information about the class of the instance. This relationship is then tested by applying the learned classifier to different sets of test data.

[0139] In this embodiment, tuples of T2-weighted fluid attenuation inversion recovery (FLAIR), diffusion-weighted imaging (DWI), and magnetic susceptibility-weighted angiography (SWAN) or gradient echo (GRE) MRI sequences are used to identify infarcts in brain MRI.

[0140] The classifier 114 is trained using manually annotated findings from medical scans annotated by an in-house radiologist. Medical scans are annotated by referencing existing clinical reports and radiological findings, and the voxels corresponding to these findings in the scans are annotated.

[0141] The technical feasibility of classifier 114 was investigated by calculating the overlap between DICE annotations and annotations made by the same internal radiologist for medical scans that did not belong to the training set.

[0142] The clinical feasibility of classifier 114 was investigated by calculating the sensitivity and specificity (one-to-many) for each finding at the subject level for known radiological findings based on clinical reports. Clinical feasibility demonstrates the ability of classifier 114 as a decision algorithm used in this method.

[0143] Classifier 114 was trained on 29 subjects (16 with infarction and 13 with hemorrhage). The trained classifier 114 was used for technical evaluation on 4 independent subjects with infarction and for clinical evaluation on 80 independent subjects (60 with infarction and 20 without findings). The results are shown in Table 1 below.

[0144] dice infarction 0.7603 background 0.9983 No findings

[0145] Table 1

[0146] Once the algorithm constituting classifier 114 has been sufficiently tested, classifier 114 can be used as part of an auxiliary automatic scanning session, as shown below. Figures 4 to 6 As stated above.

[0147] Step 250: Prepare for the scanning session

[0148] In step 250, the scanner is prepared for scanning. This may include the scanning sequences and protocols needed to determine specific parts of the subject's body.

[0149] This can be automatically derived, or a physician can specify a particular scan sequence. In one implementation, computational application 110 can calculate, based on the initial input of the scan sequence, the most probable path considered to be necessary for a particular patient via the scan sequence. This can also be informed by other data, such as empirical data. For example, semantic or natural language analysis of patient records, empirical knowledge of the most common pathologies for a given body part, or other data can be used to inform the initial “best guess.” This acts as a placeholder for subsequent analysis or further scan sequences.

[0150] The following exemplary implementations relate to brain scanning. However, this is not limiting; in principle, the methods described can be used to scan any suitable part or region of the body.

[0151] Step 252: Execute the first scan sequence

[0152] In step 252, a first scan sequence is performed. In this embodiment, the first scan sequence includes multiple different image acquisition sequences that form part of a first or level 1 scan sequence. Note that the following order of the sequences shown and described in this embodiment can be appropriately changed, and different orders or different scanning techniques can be appropriately used.

[0153] In this embodiment, the first scan sequence includes the following techniques:

[0154] T2 FLAIR

[0155] DWI

[0156] 3D SWI

[0157] T1 TSE

[0158] These techniques were chosen to provide rough indicators of the following potential pathologies:

[0159] • Tumor (general)

[0160] • Glioma (the most common type of tumor)

[0161] Granuloma

[0162] abscess

[0163] Bleeding

[0164] Microbleeds

[0165] ·infarction

[0166] • Local ischemic changes

[0167] Demyelination

[0168] • Polyps

[0169] Hydrocephalus

[0170] The order of acquired sequences can be determined based on importance. For example, if T2FLAIR and SWI are obtained in 3D, reformatted FLAIR can be used to assess aqueductal stenosis associated with hydrocephalus, and SWI can be used to assess calcification and hemorrhage / hemose / hemoglobin (iron). Therefore, no additional sequences are needed for the rough indicators of the potential pathologies listed above.

[0171] Figure 5 A flowchart illustrating the time series of image acquisition during the Level 1 scan sequence is shown. As shown, the locator is initially used for 20-30 seconds. Then, a T2 FLAIR scan is performed, which takes approximately 3-5 minutes. The data thus acquired is transmitted from scanner 118 to computing device 100 via interface 108 through a DICOM router.

[0172] The next scan acquisition is a DWI scan procedure lasting approximately 2 minutes. Simultaneously with this scan, image processing is performed by image processor 116, and classification using classifier 112 also occurs. This will be described in the next step, although steps 252 and 254 may occur simultaneously or at least partially simultaneously in time. Advantageously, scan interpretation data from the T2 FLAIR procedure can be obtained before the end of the DWI sequence.

[0173] Once the scan is complete, the data is sent via a DICOM router to an image processor 116 and a classifier 112 for processing and identification of potential pathologies.

[0174] According to this embodiment, the next stage in the Level 1 scan acquisition is the determination of the SWI scan. This typically takes about 3-4 minutes. If necessary, further scans in the form of T1 TSE can also be performed (not shown in the timeline diagram).

[0175] Note that for each scan acquisition, image scan data is obtained. This can include any suitable form; for example, scan data can include image data of a single image, multiple images, or a combination of image data and metadata. Image scan data can include two-dimensional images (either obtained directly as two-dimensional images or as two-dimensional images obtained from three-dimensional data) or three-dimensional images. Alternatively, image scan data can include data and metadata that are not directly related to the image but are used to form images of body parts of the subject.

[0176] Step 254: Image Analysis and Classification

[0177] As noted in step 252, step 254 can be performed concurrently with step 252, thereby providing image analysis and classification results before the Level 1 scan sequence is completed. This enables efficient and seamless transitions between Level 1 and Level 2.

[0178] As described above, the U-Net algorithm of classifier 112 is trained to classify each voxel in a medical scan into one of several candidate categories.

[0179] When performing a Level 1 scan sequence, classifier 112 and image processor 116 take all medical scans from each measurement as tuple inputs and produce a voxel classification map as output. The method may include a voxel classification step, where region of interest analysis and feature selection are performed for each structure.

[0180] In this example, once the T2 FLAIR dataset tuples are obtained in step 252, these datasets are processed to generate voxel classification maps. The voxel classification maps are then aggregated into an overall classification of the medical scan tuples (i.e., the presence or absence of one or more findings). This is achieved by processing the voxel classification map for each type of finding, and if a candidate finding is identified, the tuple is labeled as containing that finding.

[0181] This process uses a connected component algorithm to identify blobs (or defects) belonging to a certain category, and if the minimum number of blobs identified is above a threshold, it is considered a positive result, and the tuple is labeled. The specificity threshold is predetermined as part of establishing a particular scheme and can be determined as needed.

[0182] Defects can be analyzed in a variety of ways. For example, feature detection can be used on voxel classification maps to identify defects indicating specific pathologies. Furthermore, defects related to image quality issues can be characterized and labeled.

[0183] Alternatively, in implementations, this includes performing anisotropic Gaussian spot detection, as described in the exemplary embodiments above. Other alternative methods may be used. For example, isotropic Gaussian spot detection and / or ridge detection may be applied to voxel classification maps to identify defects indicating specific pathologies marked during training.

[0184] Step 256: Determine the next step

[0185] Based on the marker findings in the medical scan tuple in step 254, determine whether any additional sequences are needed. Optionally or additionally, determine whether any potential pathologies identified via the analysis in step 254 may require immediate diagnosis and / or treatment by a medical professional. If the latter applies, the next step may include providing notification of the identified potential pathologies for urgent diagnosis and / or treatment by a medical professional during the scanning session, as described below in step 260.

[0186] The next step to be selected depends on the underlying pathology indicated by the analysis in step 254. Below is a non-exhaustive list of pathologies that can be tagged by this invention and potential indicators of those pathologies:

[0187] 1) Tumor: T2 FLAIR: usually high intensity; T1: usually low intensity; spread: confined to solid parts; GRE: low intensity if hemorrhage occurs; GD enhancement: usually solid parts

[0188] 2) Granuloma: T2 FLAIR: low intensity (tuberculoma); T2 FLAIR: eccentric nodule (NCC-neuronal cyst) spread: + / -; GRE: low intensity if calcified; GD enhancement: peripheral

[0189] 3) Abscess: T2 FLAIR: low intensity; T1: low intensity; diffusion: centrally confined; GD enhancement: peripheral thickness.

[0190] 4) Infarction

[0191]

[0192] 5) Local ischemic changes: T2 FLAIR: high intensity; 1: low intensity; GRE: may be related to microbleeds.

[0193] 6) Demyelination: T2 FLAIR: high intensity; T1: low intensity; diffusion: restricted + / - GRE: -GD enhancement: edge / solid patch-like / fractured ring.

[0194] 7) Hydrocephalus: T2 FLAIR: Periventricular fusion high intensity + / -; T1: + / -; Spread: -; GRE: -; GD enhancement: pia mater enhancement + / -

[0195] 8) Bleeding

[0196] hyperacute acute subacute chronic T2 low strength High strength High strength High strength FLAIR low strength High strength High strength High strength T1 Equal strength High strength High strength low strength diffusion Limit + / - + / - - - GRE + / - low strength low strength low strength

[0197] 9) Microbleeds: T2 FLAIR: -T1: -Diffusion: + / -GRE: Yes (better on 3D GRE)

[0198] Based on the above and the indications derived from the analysis during the Level 1 scan process, the next step can be determined.

[0199] As mentioned above, based on Level 1 scan acquisition, T2 FLAIR and DWI provide a rough indication of the presence of 8 / 10 pathological conditions (excluding hemorrhage and microbleeds).

[0200] In step 256, steps are taken to determine what subsequent sequences are needed with or without contrast enhancement. If no T2 or SWI is found, a pathological diagnosis approximating 10 / 10 is reached.

[0201] If the quantitative indicators of infarction are used to identify microbleeds and / or hemorrhages in the brain, the next step may include notifying medical professionals of the identified possible pathology so that urgent diagnosis and / or treatment can be performed during the scanning session, as described in step 260 below.

[0202] This step has two possible outcomes: as described below, nothing is found or something is found.

[0203] Step 258: Nothing was found.

[0204] If no indicators of potential pathology are identified in step 256, the process may be terminated. However, alternatively, based on natural language analysis of clinical referrals, the computational application 110 may suggest ending the examination or proceeding with additional sequences depending on the severity.

[0205] Step 260: Discover something

[0206] If a positive indication of potential pathology is identified in step 256, the application 110 calculates, based on the learning algorithm, what other actions it can suggest to refine and confirm the additional image material necessary for the physician to make a diagnosis.

[0207] A list of potential next sequences as follows Figure 6 As shown. The sequences listed below are briefly listed as further tests for the grade 2 pathology identified above. (See below for details.) Figure 6 As shown, the available sequences are:

[0208] Vesicles, tumors (including gliomas), and abscesses: T1 TSE+3D T1 MPR_GD+(SOS perfusion_GD+spectral GD)

[0209] Granuloma: T1 TSE+3D T1 MPR_GD

[0210] Infarction: T2 TSE+TOF angiography or PC venography (based on DWI results)

[0211] Localized ischemic changes / microbleeds: 3D T1 MPR+T2 TSE+SWI+T1 TSE+TOF angiography or PC venography (based on NLP results)

[0212] Demyelination: T2 TSE + T1 TSE + T2 sag (whole spine screening) + 3D T1 MPR_GD

[0213] Myelin abnormalities: T2 TSE + T2 sag (whole spine screening) + spectral analysis

[0214] Bleeding: T2 TSE+T1 TSE+T2* / SWI+TOF angiography or PC venography (based on T2 FLAIR and DWI results)

[0215] Congenital abnormality (not mentioned in the medical report): T2 TSE+3D T1 MPR

[0216] Uncertain anomalies discovered: Sequences for further NLP-based analysis

[0217] Once the Level 2 scan is completed, the process proceeds to Level 3. At Level 3, the process ends unless additional pathology is identified during the Level 2 scan, in which case further scans may be performed.

[0218] Additionally, if a positive indication of possible pathology is identified in step 256, then, based on the learning algorithm, the application 110 can suggest additional or alternative actions for further scanning.

[0219] For example, if a positive indication of a possible pathology is identified in step 256, a notification can be provided during the medical scanning session. In other words, during the medical scanning session, while the subject is still in the medical imaging device 118, a notification related to a possible pathology, corresponding to a possible pathology, or indicating a possible pathology can be provided.

[0220] The notification may be provided in any appropriate manner. For example, it may be provided as an audible signal, or it may be provided on one or more displays. The notification may be provided to medical professionals or operators of the medical imaging equipment 10.

[0221] In this implementation, the notification may indicate the potential need for emergency treatment of the subject. Therefore, the notification may provide instructions or information to a medical professional or operator who can then use these instructions or information to make an appropriate diagnosis regarding whether the subject requires emergency treatment while in the medical imaging device 118.

[0222] In an implementation, one or more quantitative indicators may provide an indication of one or more of the following: bleeding; microbleeds; or infarction, in which case appropriate notification may indicate to a medical professional that a stroke may be present in the subject, and the medical professional can then diagnose the necessary condition and provide treatment.

[0223] In implementation, the notification may indicate the need for treatment such as thrombolysis, blood-thinning agents, or treatment to avoid the use of blood-thinning agents. By providing indication of the potential need for such treatment, the present invention provides a notification that warns and facilitates a physician's rapid diagnosis and appropriate treatment of a subject still in the medical imaging device 118.

[0224] A combination of the above notification and recommendations for subsequent scanning can be used, and such combinations can be readily conceived by those skilled in the art.

[0225] Experimental Example

[0226] Example 1

[0227] In Example 1, different encoder-decoder architectures suitable for the method of the present invention are evaluated, which attempt to use a set of hyperparameters on two different tasks of medical image segmentation: a) stroke lesion segmentation and b) hemorrhage segmentation.

[0228] Example 1: Data

[0229] This study included 997 participants. All studies included the following sequences: Fluid Attenuated Inversion Recovery (FLAIR): mean shape (512; 512; 20) and mean voxel size (0.46 mm; 0.46 mm; 7 mm), Diffusion Weighted Imaging (DWI): mean shape (256; 256; 20) and mean voxel size (1.05 mm; 1.05 mm; 7 mm), and Susceptibility Weighted Imaging (SWI) with T2* Gradient Echo (GRE) (SWI / GRE: one of mean shape (512; 512; 72) and mean voxel size (0.46 mm; 0.46 mm; 2 mm)).

[0230] The first dataset (Dataset-A) contains 156 cases of infarction, 67 cases of hemorrhage, and the remainder is a mixture of normal cases and other pathological cases (20% used for performance evaluation). Dataset-A is used for model selection. Once an architecture is selected, another dataset (with the same demographics) is used to train and evaluate a multi-class version of the selected architecture. The second dataset (Dataset-B) contains a total of 773 cases (125 tumors, 135 hemorrhages, and 189 bleeds). Of these, 98 cases are reserved for validation and reporting classification performance.

[0231] This embodiment considers three models. All models have the same basic network as the U-Net described above. The models are further modified as follows:

[0232] Example 1-1

[0233] A 3D U-Net with an encoder-decoder architecture and jump connections between the matching scales of the encoder and decoder is used. Two convolutional layers (each with 12 filters) are employed, each with dropout (0.25% rate) between the input and the U-Net. Each scale in the encoder represents a max-merge layer of size 2, and in the decoder, it represents an upsampling layer. Trilinear interpolation is used for upsampling.

[0234] Between the two scales, two convolutional layers are used, followed by an instance normalization layer. Each convolutional layer is equipped with leaky ReLU (negative slope 1e). -2A patch size of 192×192×192 is used. 12 filters are used in the first layer, followed by 24, 48, and 56 filters in the encoder. A similar number of filters are used in the decoder, but in reverse order.

[0235] All convolutions are performed with a kernel size of 3 and a span of 1. Finally, the model (logits) is converted into probabilities belonging to the foreground and background using the sigmoid activation function.

[0236] Example 1-2 U-Network VAE

[0237] In this model, roughly corresponding to that disclosed in reference [1b] is a variational autoencoder (VAE) used to adjust the latent space. The VAE branch begins from the last layer of the encoder. The first layer of the encoder produces the mean and standard deviation of a normal distribution from its sampled 32-dimensional vectors. The vectors are then used to reconstruct the input. The sampled vector layer is then connected to a layer of the same size as the last layer of the encoder. Subsequently, the input is reconstructed using four convolutional layers with a kernel size of 1 and four upsampling (factor 2) layers.

[0238] Example 1-3 - Cascaded U-Network VAE

[0239] Two sets of U-Net VAE models with the same architecture as in Examples 1-3 are used. The output of the first U-Net is connected to the input image and fed into the second network. In the first U-Net, the input image is scaled by a factor of 0.5 through a scaling layer at the input. The output layer is then scaled by a factor of 2 to match the size of the input. The networks are trained sequentially.

[0240] Note that the first network is fixed when training the second network. When training the second model, only the U-network portion of the first model is used, and the VAE branch is discarded because it has no impact on segmentation during inference.

[0241] Network Selection

[0242] The correct architecture is found using only the hemorrhage and infarction datasets. Once the models are compared, the optimal architecture is used to train a multi-class network, where each network has a target with multiple labels (tumor, infarction, and hemorrhage) instead of binary labels. Three such networks are trained. Each network is used to segment hemorrhage, tumor, and infarction, respectively.

[0243] In other words, the model bleed is only used to extract bleed segments, even though all three segments can be obtained from the same network.

[0244] As mentioned above, the training was performed according to the dice loss function and the optimization using Adam.

[0245] Example 1: Results

[0246] The model is evaluated from two aspects: a) the dice score, and b) the detection capability using sensitivity and specificity scores. Table 1 of Example 1 summarizes the results of infarction segmentation on dataset A, and Table 2 of Example 1 summarizes the results of hemorrhage segmentation on dataset A.

[0247] The best practice is highlighted in bold. In both problem statements, the baseline U-net model outperforms the other two models.

[0248] Tables 3 and 4 of Example 1 show the performance on dataset-B with a large spot size threshold. The evidence suggests that multi-class variation only contributes to accurate (especially specific) classification of hemorrhage, while the U-Net binary network is equally effective for other problems.

[0249] Table 1: Summary of infarction segmentation results on dataset A. Only hyperacute, acute, and subacute infarctions are considered. Sensitivity and specificity are reported only for the best performance method represented by dice, in which case the standard U-net model is used. A 5mm... 3 The speckle filtering threshold.

[0250] method Total dice Non-zero dice TP FP TN FP Sensitivity Specificity U-Net 0.30 0.71 38 50 94 1 97％ 66％ UNet VAE 0.29 0.64 - - - - - - Cascaded U-Network VAE 0.26 0.69 - - - - - -

[0251] Table 1 of Example 1

[0252] Table 2: Summary of hemorrhage segmentation results on dataset A. Brain hemorrhage of all ages was considered. Sensitivity and specificity are reported only for the best performance representation using dice, in which case the standard U-net model is used. A 5mm... 3 The speckle filtering threshold.

[0253] method Total dice Non-zero dice TP FP TN FP Sensitivity Specificity U-Net 0.16 0.53 38 57 85 3 93％ 60％ UNet VAE 0.13 0.55 - - - - - - Cascaded U-Network VAE 0.13 0.46 - - - - - -

[0254] Table 2 of Example 1

[0255] Table 3: Summary of infarction, hemorrhage, and tumor segmentation results using a binary standard U-net on dataset B. A 125mm mesh was applied. 3 The speckle filtering threshold.

[0256] method Total dice Non-zero dice TP FP TN FP Sensitivity Specificity infarction 0.54 0.77 24 10 62 0 100％ 86％ tumor 0.06 0.66 7 62 29 0 100％ 32％ Bleeding 0.12 0.32 12 37 44 2 86％ 54％

[0257] Table 4: Summary of infarction, hemorrhage, and tumor segmentation results using multi-class standard U-nets on dataset B. 125mm was applied. 3 The speckle filtering threshold.

[0258] method Total dice Non-zero dice TP FP TN FP Sensitivity Specificity infarction 0.41 0.69 16 56 94 0 100％ 77％ tumor 0.01 0.11 6 49 42 1 86％ 46％ Bleeding 0.15 0.21 10 6 75 4 72％ 93％

[0259] Table 4 of Example 1

[0260] This experimental example represents an analysis that can be seen in routine outpatient settings. That is, in addition to images without clinically relevant abnormalities, the dataset includes all other pathologies such as tumors, multiple sclerosis, abscesses, and age-related ischemic changes.

[0261] As shown in the figure, although the standard U-net model outperforms the other two methods, all model types perform well. This is despite the cascaded model having twice the number of parameters.

[0262] Example 2

[0263] In another embodiment, natural language processing is used to automatically scan 2 million radiology reports to select 5,000 of the most prominent pathologies from brain MRI studies: infarction (hyperacute and acute) and tumors.

[0264] Under the supervision and quality control of radiologists, infarct and tumor pathology were annotated pixel-wise by trained annotators. Two sets of MRI brain protocols were established for clinically normal patients, patients with tumors, and patients with infarctions (and both). These included 1) a control using a standard clinical protocol and 2) a smart protocol according to the invention using a 4-base sequence and up to 2 additional pathology-specific sequences.

[0265] Example 2 - Results

[0266] On an independent dataset of 88 scans, the turnaround time from scan sequence to reporting results back to the hospital system was less than 60 seconds. The specificity and sensitivity of the detection were 95% (88-99%) and 78% (52-94%) for tumors, and 75% (63-85%) and 100% (83-100%) for infarctions, respectively.

[0267] On average, with the method according to the invention, 1.25 fewer sequences are obtained per patient, and for patients with pathology, a total of 0.23 specific sequences are lost. This demonstrates the advantages of the method of the invention.

[0268] Embodiments of the invention have been described with particular reference to the examples shown. However, it should be understood that variations and modifications can be made to the embodiments described within the scope of the invention.

[0269] For example, in one implementation, training of the U-Net can be further improved by artificially generating more training data using advanced data augmentation. For example, by using the optional PADDIT algorithm [3].

[0270] Additionally, for some findings (e.g., anatomical) or in the case of abundant training data, neural network algorithms / U-networks can be directly trained for the overall findings by taking medical scan tuples as input, and are trained directly on the clinical findings in the tuples rather than on voxel annotations.

[0271] In addition, bias field correction can be applied to reduce noise caused by bias fields in medical scans. For example, differential bias correction methods oriented toward the template [4, 5] can be used, or bias correction can be performed by training a U-net. The latter requires appropriate training data and can provide a very fast bias correction method. The training set can be the original MRI scan as input and the N4 (pre-existing bias field correction method) bias-corrected scan as the target. Furthermore, 3D information can be effectively utilized by using a multiplanar U-net [6] and / or random projection extension [7].

[0272] Finally, the voxel-level results can be post-processed by applying skull dissection to the original medical scan to obtain a map of brain tissue and cerebrospinal fluid voxels, which is used to filter out any incorrectly segmented voxels outside the brain. Fast skull dissection can be obtained by training a U-Net to skull dissection based on the basic facts of skull dissection, for example from the Robex algorithm [8] or by manual segmentation.

[0273] In various respects, the embodiments described herein relate to a method for extracting information from digital images. However, the embodiments described herein can also be applied as an instruction set for a computer performing the method or as a suitably programmed computer.

[0274] In use, the methods described herein are performed on a suitable computer system or device that runs one or more computer programs, formed of software and / or hardware, and is operable to perform the methods described above. A suitable computer system typically includes hardware and an operating system.

[0275] The term “computer program” means (but is not limited to) any application, middleware, operating system, firmware, or device driver or any other medium that supports executable program code.

[0276] The term "hardware" can be considered as any one or more of the physical components that make up a computer system / device, such as, but not limited to, processors, memory devices, communication ports, and input / output devices. The term "firmware" can be considered as any permanent storage and the program code / data stored therein, such as, but not limited to, embedded systems. The term "operating system" can be considered as one or more components, typically a collection, that manage computer hardware and provide common service software for computer programs.

[0277] This comparison step can also be performed using previous measurements of the dataset, for which numerical values ​​or averages are stored in a dataset or memory location in such a computer. The computer can be programmed to display the comparison results as readouts.

[0278] The methods described herein can be embodied in one or more parts of software and / or hardware. The software is preferably held on or otherwise encoded on a memory device, such as, but not limited to, hard disk drives, RAM, ROM, solid-state storage, or any one or more of other suitable memory devices or components configured as software. The methods can be implemented by executing / running the software. Additionally or alternatively, the methods can be hardware-coded.

[0279] Preferably, one or more processors are used to execute the method coded in software or hardware. Memory and / or hardware and / or processors are preferably included as at least part of one or more servers and / or other suitable computing systems.

[0280] References

[0281] 1. Ronneberger O., Fischer P., Brox T. (2015) “U-Net: Convolutional Networks for Biomedical Image Segmentation” In: Navab N., Hornegger J., In Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. Lecture Notes in Computer Science, vol 9351. Springer, Cham

[0282] 1.b Myronenko, A (2018) “3D MRI brain tumor segmentation using autoencoder regularization”, arXiv:abs / 1810.11654

[0283] 1.c. Hongying Liu, Xiongjie Shen, Fanhua Shang, Feihang Ge, and Fei Wang, “CU-net: Cascaded U-net with loss weighted sampling for brain tumor segmentation, Multimodal Brain Image Analysis and Mathematical Foundations of Computational Anatomy”, Pages 102-111, Cham, 2019. Springer International Publishing. ISBN 978-3-030-33226-6

[0284] 1.d. Fabian Isensee, Jens Petersen, Andre Klein, David Zimmerer, Paul F. Jaeger, Simon Kohl, Jakob Wasserthal, Gregor Koehler, Tobias Norajitra, Sebastian Wirkert, and Klaus H. Maier-Hein. “Nnu-net: A Self-Adapting Framework for U-Net-Based Medical Image Segmentation” ArXiv, abs / 1809.10486, 2018

[0285] 2. Patrice Y. Simard, Dave Steinkraus, John C. Platt (2003) Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis, In the Seventh International Conference on Document Analysis and Recognition

[0286] 3. Mauricio Orbes-Ortega, Lauge Jorge Cardoso, Marc Modat, Sebastien Ourselin, Stefan Sommer, Mads Nielsen, Christian Igel, Akshay Pai (2019) PADDIT: Probabilistic Augmentation of Data using Diffeomorphic Image Transformation, In Medical Imaging 2019: Image Processing, SPIE Medical Imaging 2019.

[0287] 4. Bin Zou, Akshay Pai, Lauge Mads Nielsen, Simultaneous registration and bilateral differential bias correction in brain MRI, in MICCAI 2014 Workshop on Intelligent Imaging: Linking MR acquisition and processing, 2014

[0288] 5. Bin Zou, Akshay Pai, Lauge Mads Nielsen,Bias Correction inImages,USpatent application US20170243336A1

[0289] 6. Kaisar Kushibar, Sergi Valverde, Sandra González-Villà, Jose Bernal, Mariano Cabezas, Arnau Oliver, Xavier Lladó, Automated sub-cortical brain structure segmentation combining spatial and deep convolutional features, Medical Image Analysis 48, 2018, 177-186.

[0290] 7. Perslev, Mathias; Pai, Akshay Sadananda Uppinakudru; Igel, Christian; Dam, Erik (2018), Knee Segmentation by Multiplanar Deep Learning Network – with data from OAI, In 12th International Workshop on Osteoarthritis Imaging.

[0291] 8. Iglesias JE, Liu CY, Thompson P, Tu Z: "Robust Brain Extraction Across Datasets and Comparison with Publicly Available Methods", IEEE Transactions on Medical Imaging, 30(9), 2011, 1617-1634.

Claims

1. A method for performing a medical scan using a medical imaging system, the method comprising: a) Initiate a medical scan session for the subject in the medical imaging system; b) A first image acquisition sequence is performed using a medical imaging system during a medical scanning session to obtain first image scan data, wherein the first image acquisition sequence includes a plurality of different scan types performed sequentially, and the first image scan data includes image scan data from each scan type of the first image acquisition sequence; c) Classifying specific features in the first image scan data using a classifier containing a trained machine learning algorithm, and performing image analysis on a computer system on the first image scan data acquired during the first image acquisition sequence to identify one or more quantitative indicators of pathology and / or image quality, wherein the one or more quantitative indicators of pathology include classifications of specific features in the first image scan data that indicate possible pathology. d) Based on the identification of one or more quantitative indicators in step c), a computer system is used to determine whether any additional image acquisition sequence is required during the medical scanning session; and, if so: e) Based on one or more quantitative indicators identified in step c), the computer system is used to select a second image acquisition sequence from a plurality of potential second image acquisition sequences; and f) During a medical scanning session, while the subject remains in the medical imaging system, the medical imaging system recommends performing a selected second image acquisition sequence to obtain second image scan data.

2. The method according to claim 1, wherein, The first image scan data and / or the second image scan data include two-dimensional image scan data.

3. The method according to claim 1, wherein, The first image scan data and / or the second image scan data include three-dimensional image scan data.

4. The method according to claim 2 or 3, wherein, The classifier is used to perform step c), and the one or more quantitative indicators include voxel classification.

5. The method according to claim 4, wherein, Step c) also includes: g) Generate a voxel classification map from the first image scan data.

6. The method according to claim 5, wherein, Step g) also includes: h) Aggregate the voxel classification maps into a total classification of the first image scan data.

7. The method according to claim 5, further comprising the following step: i) Identify any spots in the voxel classification map that belong to a predetermined voxel classification; and j) Compare the identified spots with a predetermined threshold; and k) If the predetermined threshold is exceeded, a positive indication of potential pathology is identified.

8. The method according to claim 6, further comprising the following step: i) Identify any spots in the voxel classification map that belong to a predetermined voxel classification; and j) Compare the identified spots with a predetermined threshold; and k) If the predetermined threshold is exceeded, a positive indication of potential pathology is identified.

9. The method according to any one of claims 1-3, wherein, The machine learning algorithm includes neural network algorithms.

10. The method according to claim 9, wherein, The neural network algorithm includes the convolutional neural network algorithm.

11. The method according to claim 9, wherein, The neural network algorithm includes a U-network.

12. The method according to claim 10, wherein, The neural network algorithm includes a U-network.

13. The method according to claim 9, further comprising, before step a): l) A neural network algorithm is trained using a training set containing a set of medical scan tuples to identify the quantitative indicators.

14. The method of claim 10, further comprising, before step a): l) A neural network algorithm is trained using a training set containing a set of medical scan tuples to identify the quantitative indicators.

15. The method of claim 11, further comprising, before step a): l) A neural network algorithm is trained using a training set containing a set of medical scan tuples to identify the quantitative indicators.

16. The method of claim 12, further comprising, before step a): l) A neural network algorithm is trained using a training set containing a set of medical scan tuples to identify the quantitative indicators.

17. The method according to claim 13, wherein, Step m) includes using relevant basic fact notes.

18. The method according to any one of claims 1-3, wherein, Step c) can be used to operate the quantitative indicators for blind prediction of pathology and / or image quality.

19. The method according to any one of claims 1-3, wherein, Step c) does not require prior image scan data for a specific subject.

20. The method according to any one of claims 1-3, wherein, Step f) includes: during a medical scanning session, performing a second image acquisition sequence using the medical imaging system to obtain second image scan data.

21. The method according to any one of claims 1-3, wherein, The medical imaging system includes an MRI scanner, and the various scan types are selected from the group consisting of T2 FLAIR; DWI; 3D SWI; T2* gradient echo and T1 TSE.

22. The method according to any one of claims 1-3, wherein, Step b) includes performing a first image scan of the first type, followed by a second image scan of the second type; while performing the second image scan, step c) is performed on the image scan data from the first image scan.

23. The method according to any one of claims 1-3, wherein, Quantitative indicators provide clues to one or more of the following potential pathologies: tumors; gliomas; granulomas; abscesses; hemorrhages; microbleeds; infarctions; localized ischemic changes; demyelination; vegetations; neurodegenerative diseases; and hydrocephalus.

24. The method according to any one of claims 1-3, wherein, Step d) also includes: m) Based on the identification of one or more of the quantitative indicators of pathology in step c), a computer system determines whether a notification should be generated; if so, the method further includes: n) During a medical scan session, a notification is generated that relates to a possible pathology, which is associated with one or more identified quantitative indicators.

25. The method according to claim 24, wherein, During the medical scanning session, the notification is provided to the operator and / or medical professional of the medical imaging system.

26. The method according to claim 24, wherein, The notification indicates the potential need for emergency treatment of the subject.

27. The method according to claim 25, wherein, The notification indicates the potential need for emergency treatment of the subject.

28. The method according to claim 26, wherein, The one or more quantitative indicators identified in step c) provide an indication of one or more of the following: bleeding; microbleeds; or infarction; and emergency treatment related to stroke.

29. The method according to claim 28, wherein, The notification may indicate that: thrombolytic therapy; blood-thinning medication; or treatment to avoid blood-thinning medication.

30. The method according to any one of claims 1-3, wherein, The medical imaging system includes an MRI scanner, and the image scan data is generated by magnetic resonance imaging (MRI).

31. The method according to any one of claims 1-3, wherein, The medical imaging system includes a CT scanner, and the image scan data includes computed tomography (CT) data.

32. The method according to any one of claims 1-3, wherein, The medical imaging system includes a PET scanner, and the image scan data includes PET data.

33. The method according to any one of claims 1-3, wherein, The quantitative metric was derived from a previous training process using unrelated image scan data.

34. A computer system comprising a processing device configured to perform the method according to any one of claims 1-33.

35. A computer-readable medium comprising instructions configured to perform the method according to any one of claims 1-33 when executed by a processor.

36. A computer system, comprising: Processing apparatus, storage apparatus, and computer-readable medium according to claim 35.

Images