Apparatus for analysing medical images

Abstract

Figure 1 shows generally at (100) an apparatus for analysing ultrasound images of a patient. The apparatus comprises a processor (110) having a first input (120). The first input (120) is arranged to receive ultrasound image data from an ultrasound scan of a patient's abdomen. A second input (130) is arranged to receive one or more items of metadata relating to the patient (not shown). The processor (110) uses a trained, machine learning model (140) to predict the probability that the ultrasound images depict one or more conditions present in the liver of the patient, and the calculated probabilities are output at (150).

Description

[0001] Apparatus for Analysing Medical Images

[0002] The present invention relates to apparatus for analysing medical images , and is concerned particularly, though not exclusively, with an apparatus for identi fying a probability of the existence of a medical condition from an analysis of ultrasound images of a patient ' s abdomen .

[0003] Metabolic Associated Fatty Liver Disease (MAFLD) is a very common disease in which fat accumulates in the liver in people who do not have an exces sive intake of alcohol . Around 30% of the population worldwide are thought to have MAFLD . Most have simple fatty infiltration ( steatosis ) . Nonalcoholic steatohepatitis , or metabolic-associated steatohepatitis ( known as MASH) is a less common ( 2 . 5% of population) but aggres sive form of MALFD in which the liver cells become inflamed and damaged . It may eventually lead to scarring ( f ibrosis / cirrhosis ) or liver cancer .

[0004] Currently, the most reliable way of distinguishing MASH from MAFLD is to perform a liver biopsy on the patient . However, this can be painful and can have signi ficant side ef fects , including a risk of mortality of around 0 . 2 % . While non- invasive techniques for measuring some components of MASH have been developed, such as Magnetic Resonance Imaging Proton Density Fat Fraction (MRI-PDFF) for steatosis , and transient elastography [ Fibroscan] or shear wave propagation for fibrosis , these imaging modalities are slow, expensive and cannot currently distinguish MASH from MAFLD .

[0005] There is currently no inexpensive , fast , and reliable method to screen a large proportion of the population to determine who is likely to have MASH . One potential imaging medium that could be deployed at scale is ultrasound . Unfortunately, ultrasound is not commonly used in diagnosis or monitoring fatty liver conditions at present as it is challenging to accurately assess fatty infiltration, inflammation (MASH) and scarring by human interpretation of ultrasound images .

[0006] Embodiments of the present invention aim to provide a system to distinguish MASH from MAFLD using ultrasound and deep learning .

[0007] The present invention is defined in the attached independent claims , to which reference should now be made . Further, preferred features may be found in the sub-claims appended thereto .

[0008] According to a first aspect of the present invention there is provided apparatus for analysing ultrasound images of a patient ' s abdomen, the apparatus comprising : a first input arranged to receive at least one ultrasound image of the abdomen of a patient ; and a processor arranged to process the data from the first input using a trained machine learning model ; wherein the processor is arranged to determine a probability that the , or each, ultrasound image of the patient exhibits one or more medical conditions .

[0009] In a preferred arrangement , the apparatus comprises a second input arranged to receive at least one item of metadata related to the patient . The apparatus may be arranged to process the data from the second input . The first input may be arranged to receive data of a plurality of ultrasound images of the patient ' s abdomen . Preferably, the second input is arranged to receive a plurality of items of metadata related to the patient .

[0010] In a preferred arrangement , the trained model comprises a machine learning model trained using both supervised and unsupervised machine learning techniques , respectively in an unsupervised phase and a supervised phase .

[0011] By a supervised learning phase , we mean the process of training a machine learning by supplying to it a quantity of example inputs along with their classi fications ( often referred to as "ground-truths" ) and then using optimisation methods to encourage the model to emit predictions that correspond as closely as possible to those classi fications .

[0012] By an unsupervised learning phase , we mean a process of training a machine learning model in which no ground-truth data is present in the training data, and instead the optimisation task is crafted in such a way that the resulting model has some desired properties . Methods for doing this include contrastive learning, de-noising autoencoders and sel f-distillation, for example .

[0013] While the descriptions above were framed in terms of a classi fication task, the skilled practitioner will recognise that they any be easily adapted to equivalently describe other prediction types , including, for example , regression tasks . Unsupervised techniques may be used to generate generic features of ultrasound images from ultrasound data, more preferably from unannotated ultrasound date , and still more preferably from a large amount o f such data . The supervised learning phase may take features found during the unsupervised phase and may use them together with the metadata and appropriate ground-truth to produce models that predict various clinical indications . This two-phase process allows an ef fective model to developed from the very limited quantity of labelled data that is available .

[0014] Preferably, the unsupervised techniques are used to generate features of a large quantity of unannotated ultrasound data .

[0015] In a preferred arrangement the supervised learning techniques are arranged to take features found during the unsupervised phase and use them, optionally together with patient metadata, to make a model that predicts a cl inical indication .

[0016] The apparatus is preferably arranged to produce a probability density function for one or more of the following :

[0017] • the degree of steatosis ,

[0018] • the probability that the non-alcoholic fatty liver activity score [NAS ] is greater than or equal to 4 [whether the patient has MASH or not ] ,

[0019] • the degree of liver fibrosis , as a probabi lity distribution over the following bands : o F0 / F1 : No fibrosis or Zone 3 mild / moderate perisinusoidal or portal fibrosis , o F2 : Zone 3 and portal / periportal fibrosis , o F3 : Bridging fibrosis , o F4 : Cirrhosis .

[0020] According to another aspect of the present invention, there is provided a method of analysing ultrasound images of a patient , the method comprising : receiving at least one ultrasound image of the abdomen of a patient ; processing the data using a trained machine learning model ; and determining a probability that the , or each, ultrasound image of the patient ' s abdomen exhibits one or more medical conditions .

[0021] In a preferred arrangement , the method includes receiving at least one item of metadata related to the patient . The method may include processing the meta data .

[0022] The method may comprise training the machine learning model using both supervised and unsupervised learning techniques . The unsupervised techniques may be used to generate generic features of ultrasound images from ultrasound data, more preferably from unannotated data, still more preferably from a large amount of such data . The supervised learning phase may take features found during the unsupervised phase and may use them together with the metadata and appropriate ground-truth to produce models that predict the various clinical indications listed above . This two-phase process allows an ef fective model to developed from the very limited quantity of labelled data that is available .

[0023] The method may include incorporating proxy ground-truth data from several associated indicators . These may include MRI- PDFF results which may be used to measure the degree of steatosis ; shear wave or other elastography results which may be used to measure the degree of fibrosis ; or blood test results for biomarkers speci fic to fibrosis , steatosis and / or MASH .

[0024] In a further aspect , the invention provides a computer programme product on a computer readable medium, comprising instructions that , when executed by a computer, cause the computer to perform a method of analysing ultrasound images of a patient , the method comprising : receiving at least one ultrasound image of the abdomen of a patient ; and processing the data from the first input using a trained machine learning model ; determine a probability that the , or each, ultrasound image of the patient exhibits one or more medical conditions .

[0025] The method may include receiving at least one item of metadata related to the patient . The method may include processing the metadata from the second input . The invention also comprises a program for causing a device to perform a method of analysing ultrasound images of a patient , the method comprising : receiving at least one ultrasound image of the abdomen of a patient ; and processing the data from the first input using a trained machine learning model ; determine a probability that the , or each, ultrasound image of the patient exhibits one or more medical conditions .

[0026] The method may include receiving at least one item of metadata related to the patient . The method may include processing the metadata from the second input .

[0027] The invention may include any combination of the features or limitations referred to herein, except such a combination of features as are mutually exclus ive , or mutually inconsistent .

[0028] A preferred embodiment of the present invention will now be described, by way of example only, with reference to the accompanying diagrammatic drawings , in which :

[0029] Figure 1 shows schematically an apparatus for analysing ultrasound images of a patient , according to an embodiment of the present invention; and

[0030] Figure 2 shows schematically a method for analysing ultrasound images of a patient according to an embodiment of the present invention . Some recent studies have suggested that machine learning techniques applied to ultrasound images of the liver can reliably predict the degree of steatosis and fibrosis . I f ultrasound could also be used to distinguish MASH from MALFD, this could enable large-scale screening of the population for MASH .

[0031] In some implementations , embodiments of the present invention include a system to distinguish MASH from MAFLD using ultrasound and deep learning . The inputs to the system are ultrasound images of the abdomen and easily obtained metadata about the patient ( for example , but not limited to : BMI , age , liver function test results , alcohol intake etc . ) .

[0032] The system uses machine learning to analyse the images and metadata to produce the following outputs :

[0033] • The degree of steatosis ( the percentage of fat in the liver )

[0034] • The probability that the non-alcoholic fatty liver activity score (NAS ) is greater than or equal to 4 ( does the patient have MASH or not )

[0035] • The degree of liver fibrosis , as a probabi lity distribution over the following bands : o F0 / F1 : no fibrosis or Zone 3 mild / moderate perisinusoidal or portal fibrosis o F2 : Zone 3 and portal / periportal fibrosis o F3 : Bridging fibrosis o F4 : Cirrhosis

[0036] These outputs cover the key clinical markers that describe the health of the liver .

[0037] Supervised model training requires suitable ground-truth corresponding to the desired clinical indications . The "gold-standard" for measuring steatosis , fibrosis and the presence of MASH is liver biopsy, so the ideal ground truth data would consist of abdominal ultrasound images with contemporaneous liver biopsy results . However, because liver biopsies carry a degree of risk, obtaining large numbers of biopsy results for model training may be di f ficult .

[0038] To obtain adequate model performance , it is preferable to incorporate proxy ground-truth data from several associated indicators . These may include MRI-PDFF results which may be used to measure the degree of steatosis ; shear wave results which may be used to measure the degree of fibrosis ; or blood test results for biomarkers speci fic to fibrosis , steatosis and / or MASH . Data for these is more readily available . While this may only act as a weak signal of the outcome of interest (MASH) , it is nevertheless helpful in boosting the ef fectiveness of the model for a given quantity of labelled data for that outcome .

[0039] Additionally, the system is trained using both supervised and unsupervised learning techniques . The unsupervised techniques are used to generate generic features of ultrasound images from a large amount of unannotated ultrasound data . In a typical example (but not exclusively) , the quantity of available unannotated data will exceed that of annotated data by at least an order of magnitude , and sometimes by several orders of magnitude . The supervised learning phase takes features found during the unsupervised phase and uses them together with the metadata and appropriate ground-truth to produce models that predict the various clinical indications listed above . This two-phase process allows an ef fective model to be developed from the very limited quantity of labelled data that is available .

[0040] Turning to Figure 1 , this shows generally at 100 an apparatus for analysing ultrasound images of a patient . The apparatus comprises a processor 110 having a first input 120 . The first input 120 is arranged to receive ultrasound image data from an ultrasound scan of a patient ' s abdomen . A second input 130 is arranged to receive one or more items of metadata relating to the patient (not shown) . The processor 110 uses a trained, machine learning model 140 to predict the probability that the ultrasound images depict one or more conditions present in the liver of the patient , and the calculated probabilities are output at 150 .

[0041] Figure 2 shows schematically a method of analysing the image data of a patient . At step S I the ultrasound image data of the patient ' s abdomen is acquired . At step S2 one or more items of metadata relating to the patient are acquired . At Step S3 a processor uses a trained machine learning model to predict the probability that the image ( s ) acquired at step S I depicts one or more medical conditions , and at step S4 the or each probability is output .

[0042] Whilst endeavouring in the foregoing speci fication to draw attention to those features of the invention believed to be of particular importance , it should be understood that the applicant claims protection in respect of any patentable feature or combination of features referred to herein, and / or shown in the drawings , whether or not particular emphasis has been placed thereon .

Claims

CLAIMS1 . Apparatus for analysing ultrasound images of a patient, the apparatus comprising : a first input arranged to receive data of at least one ultrasound image of a patient ; and a processor arranged to process the data from the first input using a trained machine learning model ; wherein the processor is arranged to determine a probability that the or each ultrasound image of the patient exhibits one or more medical conditions , wherein the apparatus includes a second input arranged to receive at least one item of metadata related to the patient, and the apparatus is arranged to process data from the second input and wherein the apparatus is arranged to produce a probability density function for one or more of the following :• the degree of steatosis ,• the probabi lity that the non-alcoholic fatty liver activity score [NAS ] is greater than or equal to 4 [whether the patient has MASH or not ] ,• the degree of liver fibrosis , as a probability distribution over the following bands : o F0 / F1 : No fibrosis or Zone 3 mild / moderate perisinusoidal or portal fibrosis , o F2 : Zone 3 and portal / periportal fibrosis , o F3 : Bridging fibrosis , o F4 : Cirrhosis .2 . Apparatus according to Claim 1 , wherein the machine learning model is trained using both supervised and unsupervised machine learning techniques , respectively in an unsupervised phase and a supervised phase .3 . Apparatus according to Claim 2 , wherein the unsupervised techniques are used to generate features of a large quantity of unannotated ultrasound data .4 . Apparatus according to Claim 2 or 3 , wherein the apparatus is arranged to incorporate proxy ground-truth data from a number of associated indicators into the training of the model .5 . Apparatus according to Claim 2 , wherein the supervi sed learning techniques are arranged to take features found during the unsupervised phase and use them, together with the or each item of patient metadata, to make a model that predicts a clinical indication .6 . A method of analysing ultrasound images of a patient , the method comprising : receiving data of at least one ultrasound image of a patient ; processing the data using a trained machine learning model ; and determining a probability that the or each ultrasound image of the patient exhibits one or more medical conditions , wherein the method includes receiving at least one item of metadata related to the patient and processing the metadata , and wherein the method comprises producinga probability density function for one or more of the following :• the degree of steatosis ,• the probabi lity that the non-alcoholic fatty liver activity score [NAS ] is greater than or equal to 4 [whether the patient has MASH or not ] ,• the degree of liver fibrosis , as a probability distribution over the following bands : o F0 / F1 : No fibrosis or Zone 3 mild / moderate perisinusoidal or portal fibrosis , o F2 : Zone 3 and portal / periportal fibrosis , o F3 : Bridging fibrosis , o F4 : Cirrhosis .7 . A method according to Claim 6 , compris ing training the machine learning model using both supervised and unsupervised learning techniques .8 . A method according to Claim 7 wherein the unsupervi sed techniques are used to generate generic features of ultrasound images from unannotated ultrasound data .9 . A method according to Claim 7 or 8 , wherein the supervised learning phase comprises taking features found during the unsupervised phase and using them together with the metadata and appropriate ground-truth to produce models that predict the various clinical indications listed above .10 . A method according to any of Claims 7 - 9 , wherein the method includes incorporating proxy ground-truth datafrom a number of associated indicators into the training of the model.

11. A method according to Claim 10, wherein the proxy ground-truth data include MRI-PDFF results which may be used to measure the degree of steatosis.

12. A method according to Claim 10 or 11, wherein the proxy ground-truth data includes shear wave or other elastography results to measure the degree of fibrosis.

13. A method according to any of Claims 10-12, wherein the proxy ground-truth data includes one or more of: MRI- PDFF results which may be used to measure the degree of steatosis; shear wave results which may be used to measure the degree of fibrosis; or blood test results for biomarkers specific to fibrosis, steatosis and / or MASH.

14. A computer programme product on a computer readable medium, comprising instructions that, when executed by a computer, cause the computer to perform a method of analysing ultrasound images of a patient's liver, the method being in accordance with any of Claims 6-13.

15. A program for causing a device to perform a method of analysing ultrasound images of a patient, the method being in accordance with any of Claims 6-13.

Images