Method for the automatic analysis of digitized images of kidney biopsies using artificial intelligence

Abstract

Object of the invention is a computer-implemented method for automatically analysing digitised images of kidney biopsies, wherein said method comprises the following steps: - (a) automatic tissue identification on WSI (Whole Slide Image) of the kidney biopsy and automatic extraction of a plurality of tissue sub-regions at different spatial resolution on each WSI; - (b) automatic segmentation using a supervised machine learning method of the renal structures on the WSI sub-regions extracted in step (a) to derive structures of interest; - (c) automatic classification using a supervised machine learning method of potential lesions in the structures of interest segmented in step (b); - (d) automatic generation of a synoptic report containing quantitative indicators derived from the results of steps (b) and (c).

Description

[0001] “METHOD FOR THE AUTOMATIC ANALYSIS OF DIGITIZED IMAGES OF KIDNEY BIOPSIES USING ARTIFICIAL INTELLIGENCE”

[0002] FIELD OF THE INVENTION

[0003] The present invention relates to a method for automatically analysing digitised images of kidney biopsies using Artificial Intelligence.

[0004] In particular, the invention is based on the application of artificial intelligence (Al) algorithms on histological preparations of digitised kidney biopsy (virtual slide or whole slide images - WSI) for the purpose of:

[0005] 1. automatically recognising the structures that constitute the kidney tissue (segmentation);

[0006] 2. automatically characterising pathological changes in these structures (classification);

[0007] 3. applying objective ranking criteria based on the assessments made in 1 and 2.

[0008] KNOWN PRIOR ART

[0009] Kidney biopsy is useful for intercepting kidney diseases that may result in end-stage renal failure over time, requiring dialysis or transplantation.

[0010] To date, kidney biopsy is still the main tool for the diagnosis of glomerulonephritis counted among the rare diseases and responsible for high clinical prognostic impact with significant repercussions for patients, with high risk of dialysis and transplantation.

[0011] However, current classification criteria based on microscopic analysis by the pathologist still suffer from subjectivity and high inter-observer variability.

[0012] Interpretation is currently based on histological analysis with a microscope by a pathologist.

[0013] The increasing complexity of the classifications of these diseases, the rarity of some entities and the irregularity of biopsy practice make it difficult to develop adequate expertise in this area.

[0014] This is particularly important, given the high inter-observer variability among non-expert practitioners in renal disease1 2.

[0015] The need to initiate targeted therapy in the shortest possible time to reduce subsequent consequences has an impact on biopsy turnaround time estimated to be about 5 working days to ensure timely treatment especially in the most urgent cases3.

[0016] All of these needs have led to the centralisation of large biopsy volumes in specialized centers, thanks also to the aid of digital pathology and the conversion of physical slides to computer counterparts (whole slide images, WSI)4,5, with the requirement to ensure adequate levels of diagnostic quality and timeliness.

[0017] Currently, the process of kidney biopsy interpretation by the pathologist is a pathway characterized by successive steps (Figure 1) adapted to:

[0018] (1) recognise elementary renal structures (e.g., glomerulus, tubule, interstitium, vessels) and enumerate some of them (e.g., glomeruli and vessels),

[0019] (2) identify patterns and pathological changes in the context of some of these compartments (e.g., glomeruli),

[0020] (3) synthesise this information into a useful report for the clinician / nephrologist to have an overview of the kidney health condition, a diagnosis of severity and possibly a prognostic score based on the information extracted in steps (1) and (2).

[0021] Document Diagnosis of diabetic kidney disease in whole slide images via Al-driven quantification of pathological indicators, by Liu Xuwyu et al. introduces an Artificial Intelligence (Al)-based framework intended for the quantification of pathological indicators for the diagnosis and prediction of Diabetic Nephropathy (DKD), specifically designed to measure and quantify the volume or area of glomeruli and the degree of nodular mesangial sclerosis (Kimmelstiel-Wilson or K-W modules) in high-resolution digital histological images (Whole Slide Images - WSIs). The aim is to provide an auxiliary diagnostic tool that transforms qualitative pathological descriptions into quantitative indicators, thus improving the accuracy and clinical effectiveness of DKD diagnosis.

[0022] Document "The impact of pre- and post-image processing techniques on deep learning frameworks: A comprehensive review for digital pathology image analysis" by Salvi Massimo et Al. examines the influence and specific functions of techniques pre- and post-image processing on deep learning frameworks applied to image analysis in digital pathology. The article provides an overview of methodologies used to optimise performance in key tasks of computer vision, such as classification, detection, segmentation and quantification, in the context of digital pathology.

[0023] Object of the present invention is to develop an Artificial Intelligence (Al)-based tool capable of processing the digitised image (WSI) related to kidney biopsy so as to:

[0024] (1) automatically recognize the elemental structures of the kidney, to allow more objective quantification / enumeration than that of the pathologist / human eye (segmentation),

[0025] (2) automatically classify the pathological changes affecting each of the identified compartments to assist in quantification / enumeration useful in defining the patient's disease (classification),

[0026] (3) summarise the information obtained in (1) and (2) in a final synoptic report, possibly by processing a score / classification with prognostic purpose depending on the disease in question. BRIEF SUMMARY OF THE INVENTION

[0027] These and other objects are achieved by a computer-implemented method for automatically analysing digitised images of kidney biopsies, wherein the aforesaid method comprises the following steps:

[0028] - (a) automatic tissue identification on WSI (Whole Slide Image) of the kidney biopsy and automatic extraction of a plurality of tissue sub-regions at different spatial resolution on each WSI;

[0029] - (b) automatic segmentation using a supervised machine learning method of the renal structures on the WSI sub-regions extracted in step (a) to derive structures of interest;

[0030] - (c) automatic classification using a supervised machine learning method of potential lesions in the structures of interest segmented in step (b);

[0031] - (d) automatic generation of a synoptic report containing quantitative indicators derived from the results of steps (b) and (c).

[0032] Further characteristics and advantages of the invention can be deduced from the dependent claims.

[0033] BRIEF DESCRIPTION OF THE FIGURES

[0034] Further characteristics and advantages of the invention will become apparent from the following description provided by way of non-limiting example, with the aid of the figures depicted in the attached drawings, in which:

[0035] - Figure 1 shows an example of the process of sequential evaluation of kidney biopsy by the pathologist to obtain a final report, according to the known art; and

[0036] - Figure 2 shows an example of the automated process of kidney biopsy analysis by Al, according to an embodiment of the present invention, to obtain a final report.

[0037] DETAILED DESCRIPTION OF SOME EMBODIMENTS OF THE PRESENT INVENTION

[0038] Referring first to Figure 1, the process of sequential evaluation of kidney biopsy by the pathologist to obtain a final report, according to the known art, is shown.

[0039] In Figure 2 an example of the automatic process of analysing the kidney biopsy by Al is shown, according to an embodiment of the present invention, to obtain a final report.

[0040] The structuring of this report by the invention consists, in summary, of the following 4 steps (which are shown in Figure 2):

[0041] (a) automatic tissue identification on WSI of the kidney biopsy and automatic extraction of a series of tissue sub-regions at different spatial resolution (for example: 0.5 pm / pixel);

[0042] (b) automatic segmentation by a supervised machine learning method of renal structures (including but not limited to glomeruli, distal tubules, proximal tubules, atrophic tubules, interstitial fibrosis, peritubular capillaries, arteries) on the WSI sub-regions extracted in step (a);

[0043] (c) automatic classification using a supervised machine learning method of potential lesions in the structures of interest segmented in step (b);

[0044] (d) automatic generation of a report containing quantitative indicators generated in steps (b) and (c).

[0045] Thus, in general, supervised learning methods are described for steps (b) and (c), which, in an embodiment of the invention, are deep neural networks.

[0046] Furthermore, the supervised learning methods used in steps (b) and (c) exploit the entire WSI or areas of the WSI itself, which are obtained by tessellation techniques including, but not limited to, tiling of square areas.

[0047] Therefore, the approach taken in the invention introduces a multi-step pathway with the aim of extracting objective metrics expressed in both absolute and percentage values.

[0048] These metrics can be combined in different ways to improve diagnostic characterization and allow more accurate stratification of the prognosis of patients having glomerulonephritis, based on kidney biopsies.

[0049] The process starts with a first step (a) wherein automatic tissue identification on WSI of the kidney biopsy and automatic extraction of a series of tissue sub-regions at different spatial resolution are performed (for example: 0.25 and 0.5 pm / pixel).

[0050] In a second step (b), carried out by using a supervised machine learning method, the digital slide, which is, as it is known, a digital representation of a biological or other material sample obtained by high-resolution scanning of a physical microscope slide, is analysed.

[0051] This supervised machine learning method is trained on a large retrospective cohort of kidney biopsies, each of which has been noted in detail.

[0052] In more detail, the models were trained on a retrospective cohort of WSIs (Whole Slide Images) in which experienced nephropathologists noted each individual renal structure present and pointed out the lesions present of major renal structures (for example: glomeruli). The purpose of this second step (b) is the preliminary identification of renal structures of interest.

[0053] Once these structures are identified, they will be subjected to further analysis in the next step.

[0054] In a third step (c), a supervised machine learning method is applied on each identified structure to classify the different pathological lesions that are present.

[0055] This supervised machine learning method is trained on a retrospective set of pathological cases that includes a variety of pathological entities and lesions.

[0056] The training of this supervised machine learning method enables automatic and precise recognition of lesions at the structures identified in step (b).

[0057] Finally, in a fourth step (d), the information obtained from the two supervised machine learning methods is integrated to generate a synoptic report.

[0058] This report objectively summarizes the histopathological results of kidney biopsy, by using different classification approaches depending on the pathology under investigation.

[0059] For example, the Oxford classification is used for IgA nephropathy, while the activity and chronicity indices are used for lupus nephritis.

[0060] Thus, this report provides a clear and systematic representation of the observed pathological conditions, thus facilitating both diagnosis and prognosis for patients.

[0061] Model predictions are used to generate a structured and adaptive report with respect to the disease under investigation.

[0062] For step (a), an algorithm is used that makes use of morphological operators, which are, as it is known, mathematical tools used for image processing, particularly to analyze and modify the shapes included in a binary or gray-scale image.

[0063] These operators are often used in segmentation techniques, extraction of characteristics and structural analysis of images.

[0064] The supervised machine learning methods are used in steps (b) and (c).

[0065] Step (d) contains fields related to pixel-level predictions of the machine learning method used in step (b) and fields related to predictions at the level of interest structures of the machine learning method used in step (c) for the disease under investigation.

[0066] In an embodiment of the invention, the supervised learning methods used for steps (b) and (c) are deep neural networks of convolutional type. These have been developed by minimising the number of total parameters so as to improve both their efficiency and portability in the inference phase, while maximising their goodness-of-fit metrics.

[0067] The fields filled by the predictions of the machine learning method used in step (b) are the number of glomeruli, the percentage of tissue characterized by tubulo-interstitial fibroatrophy, the number of arterioles and medium / large caliber arteries, and the percentage of lumen obliteration; the fields obtained by the machine learning method used in step (c) are the number of globally sclerotic glomeruli and relative percentage to the total, the number and percentage of pathological changes in glomeruli such as crescent, mesangial hypercellularity and endocapillary hypercellularity.

[0068] RESULTS At present, how good the network predictions are, has been evaluated in terms of Intersection Over Union, Fl-Score, precision, recall and accuracy.

[0069] Modifications or improvements dictated by contingent or particular reasons may be made to the invention as described, without thereby departing from the scope of the invention as claimed below.

[0070] REFERENCES

[0071] 1. Furness, P. N. & Taub, N. Interobserver reproducibility and application of the ISN / RPS classification of lupus nephritis-a UK-wide study. Am. J. Surg. Pathol. 30, 1030-1035 (2006).

[0072] 2. Restrepo-Escobar, M., Granda-Carvajal, P. A. & Jaimes, F. Systematic review of the literature on reproducibility of the interpretation of renal biopsy in lupus nephritis. Lupus 26, 1502-1512 (2017).

[0073] 3. Requisti per la biopsia renale: diagnostica nefropatologica ed esecuzione clinica - Nephromeet. http: / / www.nephromeet.com / web / procedure / protocollo. cfm?List=WsIdEvento,WsPageName Caller, W sldRisposta, W sRelease&c 1 =00226&c2=%2Fweb%2F eventi%2FNEPHROMEET% 2Findex%2Ecfm&c3=l&c4=l .

[0074] 4. LTmperio, V. et al. Improvements in digital pathology equipment for renal biopsies: updating the standard model. J. Nephrol. 37, 221-229 (2024).

[0075] 5. LTmperio, V. et al. Digital pathology for the routine diagnosis of renal diseases: a standard model. J. Nephrol. 34, 681-688 (2021).

Claims

CLAIMS1. A computer-implemented method for automatically analysing digitised images of kidney biopsies, wherein said method comprises the following steps:- (a) automatic tissue identification on WSI (Whole Slide Image) of the kidney biopsy and automatic extraction of a plurality of tissue sub-regions at different spatial resolution on each WSI;- (b) automatic segmentation using a supervised machine learning method of the renal structures on the WSI sub-regions extracted in step (a) to derive structures of interest;- (c) automatic classification using a supervised machine learning method of potential lesions in the structures of interest segmented in step (b);- (d) automatic generation of a synoptic report containing quantitative indicators derived from the results of steps (b) and (c).

2. Method according to claim 1, wherein the supervised learning methods used in steps (b) and (c) exploit the entire WSI or areas of the WSI itself obtained by tessellation techniques including, but not limited to, tiling of square areas.

3. Method according to claim 1, wherein the step (a) of automatically identifying tissue on the WSI of the renal biopsy and automatically extracting a series of tissue sub-regions is performed at different spatial resolution between 0.25 and 0.5 pm / pixel.

4. Method according to claim 1, wherein the step (b) of segmenting the elemental structures of the kidney is performed using various histological indicators including, but not limited to, morphological and colorimetry operators.

5. Method according to any one of the preceding claims, wherein the predictions of the supervised machine learning method for step (b) of segmentation are evaluated using predictive capability evaluation metrics including, but not limited to, Intersection Over Union, Fl -Score, precision, recall and accuracy.

6. Method according to claim 1, wherein the supervised machine learning method used in step (c) of classifying potential lesions present in the structures of interest is trained on a cohort of annotated WSI images of the kidney biopsy in order to recognise pathological changes in the renal structures.

7. Method according to claim 1, wherein the quantitative indicators of step (d) are calculated both based on the prediction of the structures of interest present in the WSI of the kidney biopsy performed by the supervised type learning method used in step (b), and based on the prediction of the potential lesions present in the WSI of the kidney biopsy performed by the supervised type learning method used in step (c).

8. Method according to claim 1, wherein the synoptic report generated in step (d) discloses the values of the quantitative indicators derived from the results of steps (b) and (c) and the specific diagnostic classifications for renal diseases including, but not limited to, the Oxford classification for IgA nephropathy and the activity and chronicity indices for lupus nephritis.

9. Method according to claim 1, wherein the predictions of the supervised machine learning methods used for step (c) of classification are evaluated using predictive capability metrics including, but not limited to, Intersection Over Union, Fl -Score, precision, recall and accuracy parameters.

10. Method according to any one of the preceding claims, wherein the supervised learning methods of steps (b) and (c) are deep neural networks.

11. Method according to claim 8, wherein the supervised type learning methods used in steps (b) and (c) are optimised to reduce the total number of parameters, improving the efficiency and portability of the system.

Images