Analyzing medical images

Abstract

According to one aspect, there is provided an apparatus for analyzing a medical image, the apparatus comprising: a memory comprising instruction data representing an instruction set; and a processor configured to communicate with the memory and execute the instruction set. The instruction set, when executed by the processor, causes the processor to: i) predict a tag for the image using a first ML model; ii) obtaining a saliency map indicating a correlation of different regions of the image with the first ML model when the first ML model predicts the tag for the image; iii) obtaining a segmentation of the image; and iv) using the segmentation and the saliency map in combination to determine an indication of a feature in the image causing the first ML model to predict the tag.

Description

Technical Field

[0001] The disclosure herein relates to analyzing medical images. Specifically, but not exclusively, embodiments herein relate to explaining predictions made by an AI model when analyzing medical images. Background Art

[0002] Deep learning models and artificial intelligence are becoming a reality in clinical decision support and quantification in many fields, including cardiac ultrasound. Classification models are able to distinguish between normal and abnormal cardiac function and, for example, detect certain pathologies. Regression models can also be employed, for example, to determine quantities such as cardiac ejection fraction.

[0003] The accuracy of many models approaches (or even exceeds) that of individual human experts. However, uptake in clinical settings is limited by the fact that machine learning models can feel like a “black box” to clinical users and do not produce interpretable results that can be easily validated or challenged.

[0004] Decision support systems based on machine learning or deep learning typically output only a probability score for a disease without any or very limited local resolution. As a result, the interpretation of the model output can be challenging and therefore have limited value in practice.

[0005] Methods such as the use of saliency maps, which indicate how relevant different regions in the input image are to the output label, have been proposed previously. See the following papers: Selvaraju, R, Cogswel, M, Das, A, Vedantam, R, Parikh, D, and Batra, D (2019), entitled "Grad-CAM: Visual Explatations from Deep Networks via Gradient-Based Localization" (International Journal of Computer Vision, 128(2), pp. 336-359); and the following papers: Zeiler, MD and Fergus, R. (2014) entitled "Visualizing and understanding convolutional networks" (Conference on Computer Vision, pp. 818-833).

[0006] A conference paper titled "COVID-view: Diagnosis of COVID-19 using Chest CT" (2021) by Jadhav, S, Deng, G, Zawin, M, and Kaufman, A (IEEE Transactions on Visualization & Computer Graphics, September 29, 2021, Vol. 28, No. 1, pp. 227-237) discloses a visualization application customized for radiologists to diagnose COVID-19 from chest CT data. The system includes a complete pipeline of the following items: automatic lung segmentation, localization / isolation of lung abnormalities, followed by visualization, visual and deep learning DL analysis and measurement / quantification tools. The present application includes a DL model to classify patients as positive / negative COVID-19 cases and provides attention heatmaps as interpretable DL outputs of the model. Summary of the invention

[0007] As mentioned above, saliency maps (whereby portions of an input image are highlighted in order to indicate which parts of the image contribute most strongly to the output score) have been used in an attempt to reduce the black-box nature of machine learning models when performing classification or regression tasks on medical images.

[0008] Although in theory, saliency maps can help visualize the image regions that contribute most strongly to the model, in practice, they are often fuzzy and difficult to interpret. Furthermore, they may be imprecise, for example, indicating a large scan of the image (e.g., at low levels), or they may indicate regions from a variety of different features. Therefore, they can be non-trivial for interpretation, and this has hampered the adoption of such models in clinical practice.

[0009] Therefore, it is an object of this document to improve the interpretability of machine learning models used for classification or regression of medical images in a clinical setting.It is another object of embodiments of this document to provide an improved medical classification and regression system for improving uptake among clinicians.

[0010] According to a first aspect, there is provided a computer-implemented method of analyzing a medical image according to claim 1. The method comprises: i) using a first machine learning (ML) model to predict a label for an image; ii) obtaining a saliency map indicating the relevance of different regions of the image to the first ML model when the first ML model predicts a label for the image; iii) obtaining a segmentation of the image; and iv) using the segmentation and the saliency map in combination to determine an indication of features in the image that caused the first ML model to predict a label.

[0011] According to a second aspect, there is an apparatus for analyzing a medical image as claimed in claim 15, the apparatus comprising: a memory including instruction data representing an instruction set; and a processor configured to communicate with the memory and execute the instruction set. The instruction set, when executed by the processor, causes the processor to: i) use a first ML model to predict a label for an image; ii) obtain a saliency map indicating the relevance of different regions of the image to the model when the first ML model predicts a label for the image; iii) obtain a segmentation of the image; and iv) use the segmentation and the saliency map in combination to determine an indication of a feature in the image that caused the first ML model to predict a label.

[0012] According to a third aspect, there is a computer program product according to claim 17, comprising a computer readable medium having computer readable code embodied therein, the computer readable code being configured such that when executed by a suitable computer or processor, the computer or processor is caused to perform the method according to the first aspect.

[0013] In particular, determining an indication of a feature in the image may advantageously comprise determining a region of interest ROI in the saliency map having a saliency value above a first saliency threshold, and matching the ROI with the feature in the segmentation.

[0014] Thus, an improved system and method for analyzing medical images using a machine learning model is provided, whereby the output of the machine learning model is enhanced using a combination of segmentation of the input image and a saliency map of the input image to provide insight into which anatomical features in the image caused the model to predict a particular label for that image. This allows the saliency to be given anatomical context, and provides anatomical explanations for why the ML model reached the conclusions it did, thereby further enhancing trust and adoption of the machine learning model in clinical practice.

[0015] These and other aspects will be apparent from and elucidated with reference to the embodiment(s) described hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Example embodiments will now be described, by way of example only, with reference to the following drawings, in which:

[0017] Figure 1 shows an apparatus according to some embodiments herein;

[0018] Figure 2 An example method according to some embodiments herein is shown;

[0019] Figure 3 shows an example saliency map according to some embodiments herein;

[0020] Figure 4 An example process according to some embodiments herein is shown;

[0021] FIG5 shows a bull's eye diagram (prior art);

[0022] Figure 6 An example process according to some embodiments herein is shown;

[0023] Figure 7 illustrates example processes according to some embodiments herein; and

[0024] Figure 8 An example process according to some embodiments herein is shown. DETAILED DESCRIPTION

[0025] Now go to Figure 1 In some embodiments, there is an apparatus 100 for use in a computer-implemented method of analyzing a medical image according to some embodiments herein. Typically, the apparatus may form part of a computer apparatus or system, such as a laptop computer, desktop computer, or other computing device. In some embodiments, the apparatus 100 may be included in a patient monitor. In some embodiments, the apparatus 100 may form part of a distributed computing apparatus or cloud.

[0026] The apparatus includes a memory 104 including instruction data representing an instruction set 106 and a processor 102 (e.g., processing circuitry or logic), the memory 104 including instruction data representing an instruction set 106, and the processor 102 configured to communicate with the memory and execute the instruction set. Generally, the instruction set, when executed by the processor, can cause the processor to perform any embodiment of the method 200 described below.

[0027] More specifically, the set of instructions, when executed by a processor, causes the processor to: i) use a first ML model to predict a label for an image; ii) obtain a saliency map indicating the relevance of different regions of the image to the model when the first ML model predicts a label for the image;

[0028] iii) obtaining a segmentation of the image; and iv) using the segmentation and the saliency map to provide an indication of features in the image that caused the model to predict a label.

[0029] The processor 102 may include one or more processors, processing units, multi-core processors, or modules configured or programmed to control the device 100 in the manner described herein. In a particular embodiment, the processor 102 may include a plurality of software and / or hardware modules, each of which is configured to perform or be used to perform individual or multiple steps of the method described herein. The processor 102 may include one or more processors, processing units, multi-core processors, and / or modules, which are configured or programmed to control the device 100 in the manner described herein. In some embodiments, for example, the processor 102 may include multiple (e.g., interoperable) processors, processing units, multi-core processors, and / or modules configured for distributed processing. Those skilled in the art will appreciate that such processors, processing units, multi-core processors, and / or modules may be located in different locations and may perform different steps of the method described herein and / or different parts of a single step.

[0030] The memory 104 is configured to store program code that can be executed by the processor 102 to perform the methods described herein. Alternatively or additionally, one or more memories 104 may be external to the device 100 (e.g., separate or remote from the device 100). For example, one or more memories 104 may be part of another device. The memory 104 may be used to store machine learning models, saliency maps, segmentation, and / or any other information or data received, calculated, or determined by the processor 102 of the device 100 or from any interface, memory, or device external to the device 100. The processor 102 may be configured to control the memory 104 to store machine learning models, saliency maps, segmentation, and / or any other information or data received, calculated, or determined by the processor 102 of the device 100.

[0031] In some embodiments, the memory 104 may include multiple sub-memories, each of which may store one instruction data. For example, at least one sub-memory may store instruction data representing at least one instruction in an instruction set, and at least one other sub-memory may store instruction data representing at least one other instruction in the instruction set.

[0032] It should be understood that Figure 1 Only components necessary to illustrate this aspect of the present disclosure are shown, and in actual implementations, the apparatus 100 may include additional components to those shown. For example, the apparatus 100 may also include a display. The display may include, for example, a computer screen and / or a screen on a mobile phone or tablet computer. The display may be used to display images, saliency maps, segmentations, and / or indications output from the method 200 described herein.

[0033] In some embodiments, the device 100 may also include audio equipment. For example, a speaker may be used to provide the instructions mentioned herein auditorily. For example, the device may be configured to convert the text instructions described herein into speech.

[0034] The apparatus may also include a user input device, such as a keyboard, mouse or other input device, which enables a user to interact with the apparatus, for example to provide initial input parameters to be used in the methods described herein. The apparatus 100 may include a battery or other power source for powering the apparatus 100 or a module for connecting the apparatus 100 to a mains power source.

[0035] Go to Figure 2 , there is a computer implemented method 200 for use in analyzing a medical image. Embodiments of the method 200 may be performed, for example, by an apparatus such as the apparatus 100 described above.

[0036] In brief summary, in a first step 202, the method 200 comprises: i) predicting a label for an image using a first machine learning ML model. In a second step 204, the method comprises: ii) obtaining a saliency map indicating the relevance of different regions of the image to the first ML model when the first ML model predicts a label for the image. In a third step 206, the method comprises: iii) obtaining a segmentation of the image; and in a fourth step 208, the method comprises: iv) using the segmentation and the saliency map in combination to determine (208) an indication of features in the image that caused the first ML model to predict a label.

[0037] The method 200 thus uses the segmentation data to provide anatomical context to the saliency map.For example, typically, salient regions of a medical image can be matched or linked to anatomical features determined in the segmentation.

[0038] Medical images may include any type of image acquired using any imaging modality. Examples of medical images include, but are not limited to, computed tomography (CT) images (e.g., from a CT scan) (such as a C-arm CT image, a spectral CT image, or a phase contrast CT image), X-ray images (e.g., from an X-ray scan), magnetic resonance (MRI) images (e.g., from an MRI scan), ultrasound (US) images (e.g., from an ultrasound scan), fluoroscopic images, nuclear medicine images, or any other medical image.

[0039] The medical image may be a two-dimensional (2D) image or a three-dimensional (3D) image. In an example where the medical image is a 2D image, the medical image may include pixels; in an example where the medical image is a 3D image, the medical image may include voxels.

[0040] Analyzing a medical image may refer to processing the image in an automated or semi-automated manner in order to gain insight into the contents of the medical image, for example, to diagnose a condition, disease, or other illness.

[0041] In step 202 of method 200, the method includes using a first machine learning ML model to predict a label for the image. The first ML model may be further referred to herein as a "first model." Technicians will be familiar with ML and models that can be used to predict labels (also referred to as annotations) for medical images.

[0042] Technicians will also be familiar with machine learning (ML) and machine learning processes for use in training machine learning models. ML is a method that allows programmers to implement programs by finding patterns in data samples. The program or model obtained by machine learning is called a machine learning model. The model can be used for various analysis tasks, such as classification or regression. The data set of samples used to train the model is called a training set. The training data includes training examples (each training example includes an example input and a corresponding "correct" real situation output). The model is trained on the training data using a machine learning process.

[0043] The machine learning process includes a process that is run on training data to create a machine learning model. The machine learning process includes processes and / or instructions by which the training data can be processed or used in the training process to generate a machine learning model. The machine learning process learns from the training data. For example, the process can be used to determine how one set of parameters in the training data (the input parameters of the model) is related to another set of parameters in the training data (the output parameters of the model). The machine learning process can be used to fit the model to the training data.

[0044] There are many machine learning procedures, such as algorithms for classification (such as k-nearest neighbors), algorithms for regression (such as linear regression or logistic regression), and algorithms for clustering (such as k-means). Machine learning algorithms can be implemented in any of a range of programming languages.

[0045] A model or machine learning model may include both data and a process of how to use the data to make predictions such as those described herein. A model is the output from a machine learning (e.g., training) process, such as a set of rules or data processing steps that can be performed on input data to produce an output. As such, a model may include, for example, rules, numbers, and any other algorithm-specific data structures or architectures required, for example, to make predictions.

[0046] Some examples of machine learning processes and models that may be used herein include, but are not limited to: a linear regression process that produces a model comprising a vector of coefficients (data) whose values ​​are learned through training; a decision tree process that produces a model comprising a tree of if / then statements (e.g., rules) that include learned values; and a neural network model that comprises a graph structure of vectors or matrices of weights and biases with specific values, the values ​​of which are learned using machine learning processes such as backpropagation and gradient descent.

[0047] Those skilled in the art will appreciate that training of the first ML model may require hundreds or thousands of such training examples, and that the efficacy and accuracy of the resulting first ML model will depend on the type, quality and size of the training dataset used. The skilled person will also be familiar with ways of training machine learning models, for example, using open source libraries such as Scikit-learn: "Scikit-learn: Machine Learning in Python" (Pedregosa et al., JMLR12, pp. 2825-2830, 2011).

[0048] The first ML model can be any type of ML model that can take an image as input. For example, the first ML model can be an artificial neural network, such as a convolutional neural network (CNN). As another example, the first ML model can be an attention-based model (e.g., a transformer) as described in the paper "An image is worth 16x16word: Transformer for image recognition at scale" by Dosovitskiy, Alexey et al. (arXiv preprintarXiv:2010.11929(2020)).

[0049] In some examples, the first ML model can be any type of ML model that can obtain a saliency map. Typically, a saliency map can be generated for a fully differentiable model. As such, the first ML model can be fully differentiable (e.g., a neural network) such that saliency map methods such as guided backpropagation can be performed.

[0050] Although examples have been given above, it should be understood that the first ML model herein can generally be any type of ML model that can take an image as input and can obtain a saliency map (as described below).

[0051] The first ML model can be a clinical decision support model. For example, the first ML model can be used to diagnose a condition, a disease, etc. For example, the first ML model can be a model trained to perform a classification task to distinguish between anatomical and clinical conditions. As an example, the first ML model can have a binary output. As an example, the first ML model can be trained to output whether a medical image is "normal" or "abnormal" or "required to follow". For example, the first ML model can be trained to detect whether an image of the heart is normal or abnormal in order to detect a heart condition; whether an image of the liver is normal or abnormal, or whether an image of the brain is normal or abnormal. It should be understood that these are merely examples, and the first ML model can be trained to output many different types of classification labels for images containing many different types of anatomical features and organs.

[0052] In some examples, the first ML model can be a regression model. Examples of regression models include, but are not limited to, models for predicting continuous scores (such as health scores). As another example, the first ML model can output a probability (e.g., a score) that a subject of a medical image has a particular condition or disease. Note that a classification model can also output a probability score. For example, a normal / abnormal classification can be accompanied by, for example, get(0.1, 0.9), which indicates that there is a 90% probability that the image is normal / abnormal.

[0053] In some examples, as described above, the first ML model is a CNN or a transformer. The input is an image or a sequence of images (e.g., a video). If a sequence of images is provided as input, the sequences can all be of the same modality, or there can be different modalities (US+MR, CT+PET, ...). The output can be a continuous score.

[0054] As an example, an output may be the ejection fraction, eg, a metric indicating how much blood the heart is pumping.

[0055] As another example, the output may be a health score provided by a clinician for training. For example, when annotating a training dataset, a clinician may provide a score from 1 to 10 indicating severity. Thus, a first ML model may be trained to predict the score.

[0056] As another example, the first ML model can be used to detect regional wall motion abnormalities (RWMA) in a medical image of the heart. For example, the first ML model can take a medical image of the heart as input and output a probability that a RWMA exists somewhere in the patient's heart.

[0057] In some embodiments, the first ML model may have multiple outputs. For example, the first ML model may output an indication of the RWMA of (each of) the anterior wall, the anterior septal wall, the inferior lateral wall, the inferior wall, the inferior septal wall, and the anterior septal wall.

[0058] As described above, the second step of method 200 comprises: ii) obtaining 204 a saliency map indicating the relevance of different regions of the image to the first ML model when the first ML model predicts a label for the image.

[0059] The saliency map may indicate, on a pixel-by-pixel (2D) or voxel-by-voxel (3D) basis, how relevant one or more pixels / voxels are to the first ML model when the first ML model predicts a label for the image. The relevance may be described in comparison to other pixels / voxels in the medical image. For example, the most relevant pixels / voxels that contribute most strongly to the determination of a particular label may be at one end of the scale, and the least relevant pixels / voxels that contribute least to the determination of a particular label may be at the other end of the scale.

[0060] In examples where the first ML model is a neural network, the correlation may, for example, indicate how strongly the corresponding region of the image activates a neuron in the neural network (or the strength of the activation).

[0061] In other words, the saliency map may include a heat map that indicates the pixels / voxels or regions of pixels / voxels that contribute most strongly to (or are determining factors in) the prediction / output label made by the model.

[0062] The output of the saliency map generation (step 204) can be a spatially resolved saliency map corresponding to the size of the input medical image. Alternatively, the output of the saliency map generation (step 204) can be a spatially resolved saliency map corresponding to a portion (or region of interest) of the medical image. As described above, the intensity of the saliency map is an indicator of the degree of relevance of a particular region to the machine learning model used for generation. A high intensity can indicate a higher relevance of a region compared to a lower intensity region.

[0063] The saliency map may be in any format. For example, the saliency map may be in a format that is an image, such as DICOM, PNG, JPEG, or any other image format. More generally, the saliency map may be stored in any format that includes multiple image locations and saliency values, such as in the form of multiple vectors.

[0064] The saliency map may be generated in a variety of ways. For example, where the first ML model is a neural network, the saliency map may be generated, for example, by guided back-propagation. Thus, in some embodiments, the saliency map may be obtained (e.g., generated or output) by the first ML model itself.

[0065] Saliency map generation is described in more detail in the following paper: Selvaraju, R, Cogswel, M, Das, A, Vedantam, R, Parikh, D, and Batra, D (2019), titled "Grad-CAM: Visual Explatations from Deep Networks via Gradient-Based Localization" (International Journal of Computer Vision, 128(2), pp. 336-359). Another approach is described in the paper by Zeiler, MD and Fergus, R (2014), titled "Visualizing and understanding convolutional networks" (European Conference on Computer Vision, pp. 818-833).

[0066] However, these are merely examples, and those skilled in the art will appreciate that a saliency map as described herein may equally well be obtained in any other manner.

[0067] Figure 3 An example saliency map is shown in . In this example, a first ML model is used to classify an ultrasound image of a heart 300. The first ML model classifies the heart 300 as "abnormal". Figure 3 Two regions from a saliency map are shown, illustrating a relevant region 304 and a high correlation region 306, which are used by the first ML model to label the image as “anomalous.” In this example, the first ML model is a neural network and uses guided backpropagation to generate the saliency map.

[0068] Returning to method 200, in step 206, the method includes obtaining a segmentation of the image (e.g., all or part of the image). The skilled person will be familiar with segmentation (or image segmentation) which involves extracting shape / form information about objects or shapes captured in the image. Segmentation may be referred to as anatomical region labeling.

[0069] A variety of different segmentation methods can be used. For example, segmentation can involve converting an image into constituent blocks or "segments" representing different features in the image. As another example, segmentation can include outlining different structures in an image. As another example, binary segmentation can involve labeling each pixel as part of a particular anatomical feature (e.g., as part of the heart) or as part of the background (e.g., not part of the heart).

[0070] In some methods, image segmentation may include fitting a model to one or more features (e.g., features of interest) in an image. One image segmentation method is model-based segmentation (MBS), whereby a triangulated mesh of a target structure (e.g., heart, brain, lungs, etc.) is adapted to features in an image in an iterative manner. Segmentation models typically encode population-based appearance features and shape information. Such information describes the allowed shape variations based on the actual shape of the target structure in the population members. The shape variations may be encoded, for example, in the form of eigenmodes that describe how changes in one part of the model are constrained or depend on the shapes of other parts of the model. Model-based segmentation has been used in various applications to segment one or more target organs from medical images, see, for example, Ecabert, O. et al., 2008, entitled "Automatic Model-Based Segmentation of the Heart in CT Images" (IEEE Trans. Med. Imaging 27(9), 1189-1201).

[0071] Another segmentation approach uses a machine learning (ML) model to convert an image into multiple component shapes (e.g., block shapes or block volumes) based on similar pixel / voxel values ​​and image gradients. This is described, for example, in a paper titled “Fully Convolutional Networks for Semantic Segmentation” by Long et al.

[0072] Step 206 of method 200 may include performing any of the above-mentioned segmentation methods on the medical image, for example, performing model-based segmentation (MBS) or deep learning-based segmentation on the medical image.

[0073] In an example using an MBS-based method, the grid resulting from the segmentation may be converted into a segmentation map. Such a segmentation map may typically have the same shape as the input image. The ML model may directly output the segmentation map. Thus, in the case of segmentation using an ML model, the output of step 206 may be a segmentation map indicating which region of the image belongs to which anatomical structure. The segmentation map may be binary or contain multiple different values ​​for different anatomical regions (e.g., for different heart wall segments).

[0074] In step 208, the method includes: iv) using the segmentation and the saliency map in combination to determine (208) an indication of features in the image that caused the first ML model to predict the label. In other words, the segmentation (e.g., the output of the segmentation process or the segmentation data) and the saliency map are both used to provide an indication of features in the image that caused the first ML model to predict the label.

[0075] As used herein, a feature may be an anatomical feature, such as an organ, a part of an organ, a group of organs, a lesion, or any other feature in a medical image.

[0076] The indication can be a natural language indication, e.g., a textual description of the feature(s) that caused the first ML model to output the label it made. In other words, the indication can include a natural language interpretation of the features in the image that caused the first ML model to predict the label. In some examples, step 208 includes matching regions of high significance (e.g., above a threshold significance, such as a first significance threshold) to features in the segmentation, and providing the matched features as an indication in a text string. For example, "determine the label based on feature X."

[0077] Alternatively or additionally, the indication may include matching features / segments highlighted in the medical image, for example using shading or color.

[0078] In some examples, the saliency map and segmentation are converted to natural language descriptions of the most relevant feature(s) using, for example, a lookup table. As an example, high saliency regions (e.g., above a threshold saliency, such as a first saliency threshold), also referred to as regions of interest, can be matched to points in an MBS grid or segmentation map. A lookup table of grid points / saliency values ​​and associated natural language phrases can then be used to select an appropriate natural language interpretation. Such a lookup table can be pre-configured by a clinical expert.

[0079] In some embodiments, step 208 is performed using ML. For example, step 208 may include using a second ML model to predict indications from the segmentation and the saliency map. The second ML model may be any of the types of ML models described above with respect to the first ML model. The second ML model may take as input the segmentation (a product of the segmentation, such as segments; parameters from the MBS model; a mesh or any other output of step 206) and the saliency map. The second ML model may output the indication. The second ML model may have been trained using training data that includes example segmentations and saliency maps, such as medical images and corresponding ground truth or "correct" indications.

[0080] In some examples, the second ML model is a natural language ML model that outputs interpretable results in natural language. For example, the second ML model can be a neural network trained to take as input the saliency map, the label, and the segmentation and output a natural language description of the features in the image that resulted in the first ML model predicting the label. In such an example, the second ML model may have been trained using training data including training examples, each training example including an example segmentation of an example medical image, an example label of an example medical image, and an example saliency map of an example medical image, and an example ground-truth natural language explanation of an example feature in the example image.

[0081] Note that a subset of the input can be used as training data, e.g., the data can cover only saliency maps, segmentations, and corresponding ground-truth natural language interpretations. In this particular example, the trained natural language ML model will be modality-agnostic and can be used for, e.g., CT interpretation, even though the saliency maps and segmentations were obtained from US images. Furthermore, the natural language ML model can be improved and made more robust by including saliency maps produced by different algorithms (e.g., Guided Backpropagation and GradCAM) in the training set and using multiple different ground-truth natural language phrases.

[0082] The natural language ML model can be a (conventional) RNN / LSTM or a combination of CNN and RNN / LSTM. In such an example, the CNN part of the model will be responsible for the image / segmentation map / saliency map processing, and the RNN / LSTM part will dynamically generate natural language phrases. Alternatively, attention-based methods such as transformers or image transformers can be used. The image / segmentation map / saliency map can be converted into tokens and fed into a transformer model, as described in "An image is worth 16x16word: Transformer for image recognition at scale" by Dosovitskiy, Alexey et al. (arXiv preprintarXiv:2010.11929(2020)).

[0083] Those skilled in the art will appreciate that this is merely an example, and that the second ML model may be trained to have other inputs and / or outputs to those described above. For example, the second ML model may additionally take as input the medical image itself and / or the labels output by the first ML model.

[0084] Those skilled in the art will appreciate that training of the second ML model may require hundreds or thousands of such training examples, and that the efficacy and accuracy of the resulting second ML model will depend on the type, quality, and size of the training dataset used. The skilled person will also be familiar with ways to train machine learning models, for example, using open source libraries such as Scikit-learn: "Scikit-learn: Machine Learning in Python" (Pedregosa et al., JMLR 12, pp. 2825-2830, 2011).

[0085] In some embodiments, the method 200 may further include determining a region of interest ROI in the saliency map having a saliency value above a first saliency threshold. The ROI may then be mapped to the segmentation to obtain an anatomical label for the ROI. In some of these embodiments, the determined anatomical label may be provided as a further input to a second ML model.

[0086] In embodiments where the medical image comprises an image of a heart, an electrocardiogram (ECG) signal and / or a cardiac phase of the heart in the image may be used to derive additional context and also improve the indication. For example, the ECG signal and / or the cardiac phase of the heart may also be provided as an additional input to the second ML model, or as an additional parameter in a lookup table, etc.

[0087] Thus, in such embodiments, the method 200 may further include determining an electrocardiogram, ECG, signal and / or cardiac phase of the heart in the image, and providing the ECG signal and / or cardiac phase of the heart as a further input to the second ML model.

[0088] In this way, the natural language text output by the model can be more specific and include features of the cardiac phase and / or ECG. This also places significance in the time correspondence relative to the cardiac cycle, such as diastolic ED, systolic ES, etc. As an example, the output can be: "AI classified the image as abnormal based on the mitral valve during systole". In this example, the natural language string is based on: the output label of the first ML model + the high-significance matching anatomical points from the segmentation + the cardiac phase detected in the input image.

[0089] In other words, the second ML model can be used for natural language generation using the labels from step 202 , the saliency map from step 204 , and the anatomical labels / segmentations from step 206 .

[0090] The high-saliency regions are combined with the anatomical knowledge of the anatomical model, so an understandable representation of the first ML model can be derived, for example using predefined language building blocks (e.g., lookup tables) or advanced generative models (e.g., deep learning).

[0091] Figure 4 An example process according to some embodiments of the present invention is illustrated. In this example, the first ML model is a binary classification model that takes an image of a heart as input and outputs a label indicating whether the heart in the input image is classified as "normal" or "abnormal". In short, the embodiment includes: (1) inputting an image, (2) running the first ML model and MBS of the image individually, (3) saliency maps and segmentations output by the first ML model, and (4) combining the anatomical labels from the segmentations and the saliency maps to generate tangible results in visual and textual form.

[0092] In more detail, the following steps are performed:

[0093] 1- Run a first ML model 404 on a medical image 402 (e.g., according to step 202 above).

[0094] 1.a. The first ML model 404 generates its classification output “label”, e.g., normal or abnormal

[0095] 1.b. Second, the model generates a saliency map 406, for example using guided back-propagation (eg, according to step 204 above).

[0096] 2- Perform model-based segmentation (MBS) 408 on the input medical image 402 (eg, according to step 206).

[0097] 3- The salient regions are then matched to anatomical landmarks to produce an adjusted heart model 410 by:

[0098] a. Use local maximum filter / threshold on the saliency map to detect high saliency regions

[0099] b. Map the filtered pixels / voxels to the closest heart model grid point

[0100] c. Get (one or more) anatomical labels from the mapped grid points

[0101] 4- The matched anatomical landmarks are then converted / translated into natural language 412 (as per step 208).

[0102] In this embodiment, the detected cardiac phase or ECG signal completes the information so that a rich natural language translation of a specific result can be derived, for example, "the AI ​​detected abnormal cardiac function based on suspicious representations of the LV epicardium during systole."

[0103] Turning now to other embodiments, in some embodiments, the indication output in step 208 of method 200 comprises a bull's eye diagram of the organ in the image.

[0104] Technicians will be familiar with the bull's eye diagram used in cardiac ultrasound (US) to visualize the properties of anatomical regions of the heart, such as for strain measurement. This helps clinicians get an overview of different regions of the heart. FIG. 5 illustrates a bull's eye diagram 500 showing the strain of the heart. An example bull's eye diagram showing strain in the heart is illustrated in FIG. 5 from the paper "A Case of Myocaral Ischemia Detected by Global Longual Strain Using Intraoperative Speckle-Tracking Echocardiography in a High-Risk Patient Under Undering Abdominal Aortic Aneyarism Repair" (: April 2019; American Journal of Case Reports 20: 607-611 DOI: 10.12659 / AJCR.914858).

[0105] In the disclosure of this article, it is proposed to rank the quadrants in the bull's eye plot according to the saliency of the corresponding regions of the organs in the image to the first ML model when the first ML model predicts a label for the image.

[0106] In a 3D image of the heart, a bull's eye map of significance can be obtained by projecting the values ​​of the (3D) saliency map onto a plane that bisects the organ in the image and averaging the projection values. In this sense, averaging can include calculating the mean, median, maximum, minimum, mode, or any other statistical combination of the projection values ​​for each point on the plane. In some embodiments, a maximum intensity projection is used. This ensures that the most significant parts of the organ are always highlighted in the bull's eye map. In this way, the 3D saliency can be mapped onto a 2D bull's eye representation of the organ to highlight the imported anatomical regions where decisions are made in an interpretable manner. Therefore, the anatomical background is simplified and given to the interpretation of the 3D saliency map.

[0107] In some embodiments, a bull's eye map is obtained by segmenting the saliency map using a segmentation of the image. This is used to obtain a "significance" segmentation corresponding to the significance of different parts of the organ to the first ML model when predicting the corresponding label. In other words, the 3D saliency map can be segmented to produce a segmentation of the saliency map that shows a map of saliency across a specific organ. For example, the saliency segmentation can include a segmentation of the saliency map corresponding to the heart. In this way, portions of the saliency map that do not correspond to the organ of interest (which will be mapped to the bull's eye map) can be masked. The saliency segmentation can then be projected onto a projection plane, and the projected saliency segmentation can be superimposed on each quadrant of the bull's eye map.

[0108] Any projection plane or slice through an organ can be used as the plane onto which the bull's eye plot is projected. For example, the projection plane can be a central slice through the anatomical region. More advanced 3D to 2D projections can also be used. For example, an anatomically reasonable plane specific to the anatomical region can be selected as the projection plane. For the heart, this can be the apical 2-chamber plane commonly used in 2D US acquisitions. The intensity projection itself can also be displayed directly with continuous intensity (higher intensity indicates higher importance), or the display can be simplified by thresholding the significance values ​​and providing a constant score for each region.

[0109] The above methods can be used when the first ML model outputs a single label, such as a single classification or output value applied to the entire organ (e.g., a classification such as "normal" or "abnormal" or a score such as "0.7").

[0110] In some embodiments, the first ML model can have multiple outputs. For example, a first ML model for diagnosing RWMA of the heart may have outputs indicating the RWMA of the anterior wall, anterior septal wall, inferior lateral wall, inferior wall, inferior septal, and anterior septal wall. While this provides some local information, it is still unclear whether the abnormality exists in the basal, middle, or apex region. Note that as an alternative to a first ML model with multiple outputs, multiple first ML models (e.g., a collection of individual models) can be used, where each first ML model outputs a probability of a particular wall segment, so one model is for the anterior wall, one for the inferior wall, etc.

[0111] In such embodiments, method 200 may include repeating steps i) and ii) for each of a plurality of labels for the organ. Then, the step of superimposing the projected significance segmentation onto each quadrant of the bull's eye plot may include combining the values ​​of the projected significance segmentation for the plurality of labels. The combination may be performed, for example, using a weighted average. For example, the projected significance segmentation for the plurality of labels may be combined by applying a weighting to each significance segmentation according to the confidence (e.g., probability score) output by the first ML model for the corresponding segmentation in its corresponding label.

[0112] For example, if a first model is used to diagnose RWMA of the heart and provides outputs indicating RWMA of the anterior wall, anterior septal wall, inferior lateral wall, inferior wall, inferior septal, and anterior septal wall [abnormal, normal, normal, normal, abnormal] with confidences [0.9, 0.8, 0.7, 0.85, 0.6, 0.9], respectively, the significance values ​​can be weighted by the confidence values ​​when projected onto the plane. Weighting in this way normalizes the significance values ​​of the different outputs and ensures that the most important areas are highlighted for the user.

[0113] Note that for multi-class outputs (e.g., where there are more than two possible output labels), steps i)-iv of method 200 may be repeated for each class. Thus, embodiments herein are not limited to binary classification. In a multi-class scenario, a bull's eye plot may be generated for each class. For example, for RWMA, bull's heart plots may be generated for hypokinesia, dyskinesia, and dyskinesia.

[0114] Now go to Figure 6 , which illustrates a process for bull's eye map generation according to some embodiments of the present invention. In this embodiment, a first ML model 604 is used to diagnose RWMA in a medical image 602 of a heart. The first ML model 604 outputs a label indicating whether the heart is normal or abnormal for RWMA. In this embodiment, there is a saliency generator module 606, which is used to perform step 204 of method 200 and generate a saliency map 608 of the medical image as described above. The segmentation module 610 performs step 206 of method 200 and segments the image 602 to produce a segmentation 612 of the heart in the image. The segmentation 612 is used to generate saliency segmentations (e.g., select portions of the saliency map corresponding to the heart), and this is used by a bull's eye generation module 614 that projects the saliency segmentations onto a plane to produce a bull's eye map 616.

[0115] In more detail, in one embodiment, Figure 6 The module shown performs the following steps:

[0116] One or more machine learning models 604 are trained to predict a disease, such as RWMA, for example, with a single output or multiple outputs for different locations on the heart.

[0117] Set up a saliency generator 606 , which may be based on Grad-CAM, or guided back-propagation as cited above, or any other saliency map generator. The saliency map generator uses the model(s) 604 from step 1 .

[0118] Next, a module 610 is created to generate an anatomical label map from the 3D image. This can be a model-based segmentation method or a deep learning-based segmentation method.

[0119] Module 614 uses the saliency map and the anatomical labeling map to generate a bull's eye map. The possible process is as follows:

[0120] For each anatomical label:

[0121] Segment the saliency map into the current anatomical region

[0122] Determine the projection plane from the current anatomical region (e.g., central slice)

[0123] Project the segmented saliency map onto a projection plane (e.g., maximum intensity projection)

[0124] Overlay the projection data on each segment of the bull's eye plot.

[0125] Optionally, the projection data is weighted using the model probabilities for decision support (e.g., if there are multiple local scores corresponding to different outputs of the first model(s))

[0126] Optionally, threshold the values ​​of each segment

[0127] The display module 616 shows the final bull's heart diagram to the user.

[0128] One or more machine learning models 604 are applied to the 3D image 602, thereby generating one or more probability scores. A saliency generator 606 generates one or more saliency maps (depending on the number of outputs from the model 604, where each saliency map corresponds to a probability score. A segmentation algorithm 610 generates an anatomical label map 612 (e.g., segmentation) based on the current 3D image. The saliency map is fused with the anatomical label map. A bull's eye generation algorithm is performed for each anatomical region with its corresponding saliency map. Note that for a single label output, the saliency map for all regions is the same, and for a multi-label output model, the saliency map for each region is different.

[0129] Figure 7 The output bull's eye plot of the single label output from the first model 704 operating on the input image 702 is illustrated. The first model outputs a single RWMA probability score and the probability (or confidence level) in the score. The bull's eye plot 706 shows the output of the method 200 applied to this embodiment and shows how detailed spatial resolution of the most significant segments of the output label can be obtained.

[0130] The bull's eye plot 708 illustrates the baseline that would result if no saliency-based guidance was used at all. In other words, the first ML model prediction is intuitively or directly mapped onto the bull's eye plot (without spatial resolution as to which quadrant(s) contribute more strongly to the predicted label than other quadrants).

[0131] Bull's eye plot 710 shows an alternative display in which the significance values ​​are not thresholded, but are plotted as continuous values, as described above (see optional thresholding at the end), for example, the values ​​may vary within a region of the bull's eye plot.

[0132] Figure 8The output bull's eye plot of the multi-label output from the first model 804 operating on the input image 802 is illustrated. The multi-label output may include, for example, the output of the RWMA indicating the anterior wall, the near anterior septal wall, the lateral wall, the inferior wall, the inferior septal wall, and the anterior septal wall and the probability (or confidence level) of each score. The bull's eye plot 806 shows the output of the method 200 applied to this embodiment, whereby the saliency map of each output is projected onto the projection plane and the weighted average is used as the value of each segment of the bull's eye plot. Again, this method provides substantial value by providing more detailed local resolution.

[0133] The bull's eye plot 808 illustrates the baseline that would result if no significance-based guidance was used at all. For example, a direct mapping of the first ML model predictions to the bull's eye plot. Figure 8 As can be seen in FIG. 8 , using the method herein to obtain graph 806 allows for more detailed information to be displayed (more details in sub-regions, etc.).

[0134] Bull's eye plot 810 corresponds to 710 and shows significance values ​​without thresholding, ie, with varying importance values ​​within one bull's eye segment.

[0135] It should be understood that although some of the foregoing embodiments in which a bull's eye plot has been generated have been shown with respect to the RWMA of the heart, this is merely an example and the same principles apply to other diagnostic properties of the heart and / or other diagnostic properties of other organs. Examples of other organs include, but are not limited to, the brain, kidneys, and liver.

[0136] For example, the method of this paper can be applied to any localized disease diagnosis model output. In addition, other types of bull's eye diagrams, such as those including fewer or more segments, can be used instead of those shown in the accompanying drawings.

[0137] Turning now to other embodiments, there is provided a computer program product comprising a computer readable medium having computer readable code embodied therein, the computer readable code being configured to, when executed by a suitable computer or processor, cause the computer or processor to perform one or more of the methods described herein.

[0138] It will therefore be appreciated that the present disclosure also applies to computer programs, in particular computer programs on or in a carrier, suitable for practicing the embodiments. The program may be in the form of source code, object code, a code intermediate source and object code, such as in a partially compiled form, or in any other form suitable for use in the implementation of the method according to the embodiments described herein.

[0139] It should also be understood that such a program can have many different architectural designs. For example, the program code of the function of the implementation method or system can be subdivided into one or more subroutines. Many different ways of allocating functions between these subroutines will be apparent to those skilled in the art. The subroutines can be stored together in an executable file to form a self-contained program. Such an executable file may include computer executable instructions, such as processor instructions and / or interpreter instructions (e.g., Java interpreter instructions). Alternatively, one or more or all of the subroutines may be stored in at least one external library file and, for example, statically or dynamically linked to the main program at runtime. The main program includes at least one call to at least one subroutine. The subroutines may also include function calls to each other.

[0140] The carrier of a computer program may be any entity or device capable of carrying the program. For example, the carrier may include a data storage device, such as a ROM, such as a CD ROM or a semiconductor ROM, or a magnetic recording medium, such as a hard disk. In addition, the carrier may be a transmissible carrier such as an electrical signal or an optical signal, which may be transmitted via an electrical or optical cable or by radio or other means. When the program is embodied in such a signal, the carrier may be constituted by such a cable or other device or module. Alternatively, the carrier may be an integrated circuit in which the program is embedded, the integrated circuit being suitable for executing or for executing the relevant method.

[0141] By studying the drawings, the disclosure and the appended claims, those skilled in the art can understand and implement changes to the disclosed embodiments when practicing the principles and techniques described herein. In the claims, the word "comprising" does not exclude other elements or steps, and the word "one" or "an" does not exclude multiple. A single processor or other unit can implement the functions of several items recorded in the claims. Although specific measures are recorded in mutually different dependent claims, this does not indicate that a combination of these measures cannot be used advantageously. The computer program can be stored or distributed on a suitable medium, such as an optical storage medium or solid-state medium provided together with other hardware or as part of other hardware, but the computer program can also be distributed in other forms, such as distributed via the Internet or other wired or wireless telecommunications systems. Any figure mark in the claims should not be interpreted as limiting the scope.

Claims

1. A computer-implemented method (200) for analyzing a medical image, the method comprising: i) using (202) a first machine learning ML model to predict a label for the image; ii) obtaining (204) a saliency map, the saliency map indicating the relevance of different regions of the image to the first ML model when the first ML model predicts the label for the image; iii) obtaining (206) a segmentation of said image; as well as iv) using the segmentation and the saliency map in combination to determine (208) an indication of a feature in the image that caused the first ML model to predict the label, comprising: Determining a region of interest (ROI) in the saliency map having a saliency value higher than a first saliency threshold; as well as The ROI is matched to features in the segmentation.

2. The method according to claim 1, wherein: Matching the ROI with features in the segmentation includes matching the ROI with points in a model-based segmentation grid or segmentation map.

3. The method according to claim 1 or 2, wherein: Matching the ROI to features in the segmentation includes mapping the ROI to the segmentation to obtain an anatomical label for the ROI.

4. The method according to any one of claims 1 to 3, wherein: The step of determining (208) an indication of a feature in the image comprises: A second ML model is used to predict the indication based on the segmentation and the saliency map.

5. The method according to claims 3 and 4, wherein: The method also includes providing the anatomical labels as a further input to the second ML model.

6. The method according to claim 4 or 5, wherein: The image comprises an image of a heart, and wherein the method further comprises: determining an electrocardiogram (ECG) signal and / or a cardiac phase of the heart in the image; and The ECG signal of the heart and / or the cardiac phase are provided as further input to the second ML model.

7. A method according to any one of the preceding claims, wherein: The indication includes a natural language explanation of the feature in the image that caused the first ML model to predict the label.

8. The method according to claim 7 when dependent on any one of claims 4 to 6, wherein: The second ML model is a natural language ML model that has been trained to take as input the segmentation, the label, and the saliency map, and to output the natural language interpretation of the feature in the image that causes the first ML model to predict the label.

9. The method according to claim 1, wherein: The indication comprises a bull's eye plot of the organ in the image; quadrants in the bull's eye plot are ranked according to the significance of corresponding regions of the organ in the image to the first ML model when the first ML model predicts the label for the image.

10. The method according to claim 9, wherein: The bull's eye map is obtained by projecting the values ​​of the saliency map onto a plane bisecting the organ in the image and averaging the projected values.

11. The method according to claim 9, wherein: The bull's eye diagram is obtained by the following operations: segmenting the saliency map using the segmentation of the image to obtain saliency segments corresponding to saliency of different parts of the organ to the first ML model when predicting corresponding labels; projecting the saliency segment onto a projection plane; and The projected saliency segments are superimposed on each quadrant of the bull's eye plot.

12. The method according to claim 11, comprising: Repeating steps i) and ii) for each tag in a plurality of tags of the organ; And wherein the step of superimposing the projected significance segments onto each quadrant of the bull's eye diagram comprises: The values ​​of the projected saliency segments of the plurality of labels are combined.

13. The method according to claim 12, wherein: The projected saliency segments of the plurality of labels are combined by applying a weighting to each saliency segment according to a confidence output by the first ML model in a corresponding label of the first ML model for a corresponding segment.

14. A method according to any one of the preceding claims, wherein: The first ML model outputs a multi-class label, and the method comprises repeating steps i)-iv) for each class.

15. An apparatus (100) for analyzing a medical image, the apparatus comprising: a memory (104) comprising instruction data representing an instruction set (106); as well as A processor (102) configured to communicate with the memory and execute the set of instructions, wherein the set of instructions, when executed by the processor, causes the processor to: predicting a label for the image using a first ML model; obtaining a saliency map indicating relevance of different regions of the image to the first ML model when the first ML model predicts the label for the image; obtaining a segmentation of the image; and Using the segmentation and the saliency map in combination to determine an indication of a feature in the image that caused the first ML model to predict the label includes: Determining a region of interest (ROI) in the saliency map having a saliency value higher than a first saliency threshold; and The ROI is matched to features in the segmentation.

16. The apparatus of claim 15, further comprising a display; and in, The processor is further caused to send instructions to the display to cause the display to display the image and the indication.

17. A computer program product comprising a computer readable medium having computer readable code embodied therein, the computer readable code being configured such that when executed by a suitable computer or processor, the computer or processor is caused to perform the method according to any one of claims 1 to 14.

Images