Method and system of histological characterisation of bowel disease

Abstract

A method and system of identification and characterisation of histological features of bowel disease are disclosed. The method and system quantify inflammatory indicators and changes in tissue architecture. Spatial identification and measurement of attributes in IBD tissue biopsies are used for insight into disease activity.

Description

[0001] Method and System of Histological Characterisation of Bowel Disease

[0002] Field of the Invention

[0003] The present invention relates to a method and system for the identification and characterisation of histological features of bowel disease.

[0004] Background

[0005] Inflammatory Bowel Disease (IBD) is a chronic inflammatory disease with effective therapies, such as anti-TNFa, but no definitive cure. The two main disorders of IBD are Crohn’s Disease (CD) and Ulcerative Colitis (UC) which lead to debilitating symptoms of abdominal pain, weight loss, diarrhoea, and rectal bleeding and carry an increased risk of colorectal cancer.

[0006] Bowel disease was originally considered a disease of early industrialised regions (Hracs, L., Windsor, J.W., Gorospe, J. et al. Global evolution of inflammatory bowel disease across epidemiologic stages. Nature 642, 458-466 (2025). https: / / doi.org / 10.1038 / s41586-025-08940-0) and according to recent estimations >0.7% of US population (J.D. Lewis, L.E. Parlett, M.L.J. Funk, C. Brensinger, V. Pate, Q. Wu, G.K. Dawwas, A. Weiss, B.D. Constant, M. McCauley, Incidence, prevalence, and racial and ethnic distribution of inflammatory bowel disease in the United States, Gastroenterology, 165 (2023) 1197-1205. e1192) and >0.8% of UK population (G.-R. Jones, M. Lyons, N. Plevris, P.W. Jenkinson, C. Bisset, C. Burgess, S. Din, J. Fulforth, P. Henderson, G.-T. Ho, IBD prevalence in Lothian, Scotland, derived by capturerecapture methodology, Gut, 68 (2019) 1953-1960, B. Hamilton, H. Green, N. Heerasing, P. Hendy, L. Moore, N. Chanchlani, G. Walker, C. Bewshea, N.A. Kennedy, T. Ahmad, Incidence and prevalence of inflammatory bowel disease in Devon, UK, Frontline Gastroenterology, 12 (2021) 461-470) are affected. However, it is now a global problem (Zhou J-L, Bao J-C, Liao X-Y, et al. Trends and projections of inflammatory bowel disease at the global, regional, and national levels, 1990-2050: a bayesian age-period-cohort modeling study. BMC Public Health. 2023;23(1):2507) that will continue to worsen (Caron B, Honap S, Peyrin-Biroulet L. Epidemiology of Inflammatory Bowel Disease across the Ages in the Era of Advanced Therapies. J Crohns Colitis. 2024 Oct 30;18(Supplement_2):ii3-ii15. doi: 10.1093 / ecco-jcc / jjae082. PMID: 39475082; PMCID: PMC11522978). Endoscopy is the gold standard in IBD diagnoses, with endoscopic remission displaying improved long-term outcomes and becoming the focus of most treatment targets (M.F. Neurath, S.P. Travis, Mucosal healing in inflammatory bowel diseases: a systematic review, Gut, 61 (2012) 1619-1635).

[0007] However, histological assessment is often required as up to one third of patients with endoscopic healing can exhibit microscopic disease (R.K. Pai, V. Jairath, N.V. Casteele, F. Rieder, C.E. Parker, G.Y. Lauwers, The emerging role of histologic disease activity assessment in ulcerative colitis, Gastrointestinal endoscopy, 88 (2018) 887-898). It is therefore proposed that histological assessment is a deeper and more accurate measurement of disease activity. This is evident in clinical trial endpoints for UC patients where histological healing is a major therapeutic goal (C. Langner, F. Magro, A. Driessen, A. Ensari, G.J. Mantzaris, V. Villanacci, G. Becheanu, P. Borralho Nunes, G. Cathomas, W. Fries, The histopathological approach to inflammatory bowel disease: a practice guide, Virchows Archiv, 464 (2014) 511-527, B. Lemmens, I. Arijs, G. Van Assche, X. Sagaert, K. Geboes, M. Ferrante, P. Rutgeerts, S. Vermeire, G. De Hertogh, Correlation between the endoscopic and histologic score in assessing the activity of ulcerative colitis, Inflammatory bowel diseases, 19 (2013) 1194-1201 , L. Peyrin-Biroulet, A. Bressenot, W. Kampman, Histologic remission: the ultimate therapeutic goal in ulcerative colitis?, Clinical Gastroenterology and Hepatology, 12 (2014) 929-934. e922).

[0008] In healthy bowel biopsies, crypts, which are a significant biological structure of interest, have an architecture that displays structural uniformity and little or no pathological features such as abscesses, ulceration, epithelial attenuation, and inflammatory cell infiltration. However, in IBD disease, a plethora of landmark features, relating to both structure and inflammation, can indicate IBD-related histological activity. Structural changes comprise erosion and ulceration, changes in the crypt architecture and mucin depletion.

[0009] In respect of erosion and ulceration, an important histological feature is the presence of epithelial damage (R.K. Pai, G.Y. Lauwers, R.K. Pai, Measuring histologic activity in inflammatory bowel disease: why and how, Advances in anatomic pathology, 29 (2022) 37-47). This is coupled with signs of acute inflammation and displayed as loss of normal epithelium (mucosal breaks) which is replaced with granulated or fibrinous tissue. Differentiating between erosions and ulceration is challenging for pathologists, as ulcers extend deeper than the muscularis mucosae while erosions do not.

[0010] Although a rare architectural change in a normal colon, crypt distortion can include branching, irregularity, dilation and changes in size and shape. Crypts can undergo atrophy (crypt shortening) and result in large spaces between the muscularis mucosae and crypt base (F. Magro, G. Doherty, L. Peyrin-Biroulet, M. Svrcek, P. Borralho, A. Walsh, F. Carneiro, F. Rosini, G. de Hertogh, L. Biedermann, ECCO position paper: harmonization of the approach to ulcerative colitis histopathology, Journal of Crohn's and Colitis, 14 (2020) 1503-1511). In addition, crypt distortion is associated with higher rates of clinical relapse in patients exhibiting endoscopic remission (A. Gupta, A. Yu, L. Peyrin-Biroulet, A.N. Ananthakrishnan, Treat to target: the role of histologic healing in inflammatory bowel diseases: a systematic review and meta-analysis, Clinical Gastroenterology and Hepatology, 19 (2021) 1800-1813. e1804). In addition, relapse in patients with endoscopic remission can exhibit a significant loss of goblet cells which is indicative of mucin depletion (R. Ozaki, T. Kobayashi, S. Okabayashi, M. Nakano, S. Morinaga, A. Hara, M. Ohbu, K. Matsuoka, T. Toyonaga, E. Saito, Histological risk factors to predict clinical relapse in ulcerative colitis with endoscopically normal mucosa, Journal of Crohn's and Colitis, 12 (2018) 1288-1294).

[0011] Inflammation-related changes include mucosal neutrophilic inflammation. This is observed within the lamina propria, surface and crypt epithelium (cryptitis) as well as the lumens of crypts (crypt abscesses) and is a core histological feature that defines IBD. Neutrophil infiltration in crypts is indicative of higher disease severity and the absence of neutrophils strongly correlates with histological remission (F. Magro, G. Doherty, L. Peyrin-Biroulet, M. Svrcek, P. Borralho, A. Walsh, F. Carneiro, F. Rosini, G. de Hertogh, L. Biedermann, ECCO position paper: harmonization of the approach to ulcerative colitis histopathology, Journal of Crohn's and Colitis, 14 (2020) ISOSIS 11 ). While the significance of the presence of eosinophils on IBD histological activity is unclear, the presence of such mucosal eosinophilic infiltrates is associated with disease activity and a worse response to treatment (E.M. Kim, C. Randall, R. Betancourt, S. Keene, A. Lilly, M. Fowler, E.S. Dellon, H.H. Herfarth, Mucosal eosinophilia is an independent predictor of vedolizumab efficacy in inflammatory bowel diseases, Inflammatory Bowel Diseases, 26 (2020) 1232-1238).

[0012] The presence of plasma cells between colonic crypts and muscularis mucosae is referred to as basal plasmacytosis. Despite often being missed in biopsies, in early stages of IBD it is a highly specific feature of IBD diagnosis (Gupta, A. Yu, L. Peyrin- Biroulet, A.N. Ananthakrishnan, Treat to target: the role of histologic healing in inflammatory bowel diseases: a systematic review and meta-analysis, Clinical Gastroenterology and Hepatology, 19 (2021) 1800-1813. e1804).

[0013] Increased presence of both lymphocytes and plasma cells within the lamina propria is often associated with lamina propria chronic inflammatory and a key feature in IBD diagnosis (E. Vespa, F. D’Amico, M. Sollai, M. Allocca, F. Furfaro, A. Zilli, A. Dal Buono, R. Gabbiadini, S. Danese, G. Fiorino, Histological scores in patients with inflammatory bowel diseases: the state of the art, Journal of Clinical Medicine, 11 (2022) 939).

[0014] As indicated by recent evidence, evaluation or scoring systems should ideally evaluate and therefore apply to both UC and CD (C. Ma, R. Sedano, A. Almradi, N.V. Casteele, C.E. Parker, L. Guizzetti, D.F. Schaeffer, R.H. Riddell, R.K. Pai, R. Battat, An international consensus to standardize integration of histopathology in ulcerative colitis clinical trials, Gastroenterology, 160 (2021) 2291-2302, A. Almradi, C. Ma, G.R. D'Haens, W.J. Sandborn, C.E. Parker, L. Guizzetti, P. Borralho Nunes, G. De Hertogh, R.M. Feakins, R. Khanna, An expert consensus to standardise the assessment of histological disease activity in Crohn's disease clinical trials, Alimentary Pharmacology & Therapeutics, 53 (2021) 784-793). Currently over 30 scoring systems exist across UC and CD. The most widely used systems in clinical trials are the Geboes score (GS), the Nancy Index (Nl) and the Robarts Histopathological Index (RHI).

[0015] The GS system, developed by Geboes et al., although not formally validated, is widely used in clinical trials (K. Geboes, R. Riddell, A. Ost, B. Jensfelt, T. Persson, R. Ldfberg, A reproducible grading scale for histological assessment of inflammation in ulcerative colitis, Gut, 47 (2000) 404-409). However, due to its complexity, the GS system is yet to be routinely adopted in practical use.

[0016] The GS system uses a 0-5 scale and multiple sub-scores, with the higher values indicating severe inflammation. The key features evaluated and combined to produce the final score are as follows: crypt architecture (GS 0), lamina propria chronic inflammatory infiltrate (GS 1), neutrophils in lamina propria (GS 2A), eosinophils present in lamina propria (GS 2B), intraepithelial neutrophils (GS 3), crypt destruction (GS 4) and epithelial injury (GS 5). Simplified versions of the GS score have been adopted in addition to a continuous score, yet all versions correlate with a valid histological outcome measure (B.E. Sands, L. Peyrin-Biroulet, E.V. Loftus Jr, S. Danese, J.-F. Colombel, M. Tdruner, L. Jonaitis, B. Abhyankar, J. Chen, R. Rogers, Vedolizumab versus adalimumab for moderate-to-severe ulcerative colitis, New England Journal of Medicine, 381 (2019) 1215-1226).

[0017] The Nl correlates with the GS system, but is simpler and investigates lamina propria chronic inflammation, neutrophilic inflammation and presence of ulcers / erosions to create a continuous score of 0-4 (A. Marchal-Bressenot, J. Salleron, C. Boulagnon- Rombi, C. Bastien, V. Cahn, G. Cadiot, M.-D. Diebold, S. Danese, W. Reinisch, S. Schreiber, Development and validation of the Nancy histological index for UC, Gut, 66 (2017) 43-49). Depending on the presence of individual features, the metric indicating the most severe disease marker determines the overall score (surface damage being an Nl of 4 and only a mild level of lymphocytes / plasma cells and eosinophils being an Nl of 0).

[0018] The RHI is derived from GS, and evaluates the lamina propria chronic inflammation, neutrophils in lamina propria, intraepithelial neutrophils and epithelial injury. Each feature is weighted differently but receives a score of 1-4 to achieve a scale from 0-33. Similarly to GS, there is strong evidence of RHI indicating histological response in clinical trials (F. Magro, J. Lopes, P. Borralho, S. Lopes, R. Coelho, J. Cotter, F. Dias de Castro, H. Tavares de Sousa, M. Salgado, P. Andrade, Comparing the continuous Geboes score with the Robarts histopathology index: definitions of histological remission and response and their relation to faecal calprotectin levels, Journal of Crohn's and Colitis, 14 (2020) 169-175).

[0019] Acute inflammatory indicators are associated with a two to threefold increased risk of colitis relapse at 12 months follow-up (M.H. Mosli, B.G. Feagan, G. Zou, W.J. Sandborn, G. D'Haens, R. Khanna, C. Behling, K. Kaplan, D.K. Driman, L.M. Shackelton, Reproducibility of histological assessments of disease activity in UC, Gut, 64 (2015) 1765-1773) and basal plasmacytosis (present of basal cells) predicts UC clinical relapse in patients with complete mucosal healing (S. Riley, V. Mani, M. Goodman, S. Dutt, M. Herd, Microscopic activity in ulcerative colitis: what does it mean?, Gut, 32 (1991) 174-178). A deep learning algorithm designed to identifying eosinophils for UC diagnoses correlated with manual counts by pathologists but did not correlate well with histological activity (N. Vande Casteele, J. A. Leighton, S.F. Pasha, F. Cusimano, A. Mookhoek, C.E. Hagen, C. Rosty, R.K. Pai, R.K. Pai, Utilizing deep learning to analyze whole slide images of colonic biopsies for associations between eosinophil density and clinicopathologic features in active ulcerative colitis, Inflammatory bowel diseases, 28 (2022) 539-546).

[0020] In CD, a model was able to identify adipocyte shrinkage and mast cell infiltration as histological features correlating with disease recurrence (H. Kiyokawa, M. Abe, T. Matsui, M. Kurashige, K. Ohshima, S. Tahara, S. Nojima, T. Ogino, Y. Sekido, T. Mizushima, Deep learning analysis of histologic images from intestinal specimen reveals adipocyte shrinkage and mast cell infiltration to predict postoperative Crohn disease, The American journal of pathology, 192 (2022) 904-916).

[0021] Another model developed a Global Histology Activity Score proved capable of differentiating biopsies for UC or CD with moderate accuracy compared to histological scores (D. Rymarczyk, W. Schultz, A. Borowa, J.R. Friedman, T. Danel, P. Branigan, M. Chalupczak, A. Bracha, T. Krawiec, M. Warchol, Deep learning models capture histological disease activity in Crohn’s disease and ulcerative colitis with high fidelity, Journal of Crohn's and Colitis, 18 (2024) 604-614).

[0022] Several automated methods of tissue segmentation or inflammatory cell detection are also known and have been applied to IBD in various capacities.

[0023] The Paddington International virtual ChromoendoScopy ScOre Histologic Remission Index (PICaSSO - PHRI) was developed for UC with an artificial intelligence (Al) based system to accurately predict histological remission. This score detects neutrophils across whole slide images and determines whether there is histological remission / non- remission based on the presence of neutrophils. PHRI is highly correlated with endoscopic scores (Mayo Endoscopic Score and UC Endoscopic Index of Severity) as well as clinical outcomes such as hospitalisation and colectomy (X. Gui, A. Bazarova, R. Del Amor, M. Vieth, G. De Hertogh, V. Villanacci, D. Zardo, T.L. Parigi, E.S. Royset, U.N. Shivaji, PICaSSO Histologic Remission Index (PHRI) in ulcerative colitis: development of a novel simplified histological score for monitoring mucosal healing and predicting clinical outcomes and its applicability in an artificial intelligence system, Gut, 71 (2022) 889-898).

[0024] Al-powered histopathology platforms are available that identify inflammatory microenvironments across IBD whole slide images. A platform from PathAI® quantifies Geboes scoring for histology assessment for UC.

[0025] Further, a recent study described an Al model which correlated with UC disease activity and the Nancy index (F. Najdawi, K. Sucipto, P. Mistry, S. Hennek, C.K. Jayson, M. Lin, D. Fahy, S. Kinsey, I. Wapinski, A.H. Beck, Artificial intelligence enables quantitative assessment of ulcerative colitis histology, Modern Pathology, 36 (2023) 100124).

[0026] However, there remains a distinct requirement for accurate quantification of inflammatory indicators and changes in tissue architecture (e.g. inflammatory cell quantification or detection of mucin depletion). This is to not only improve the accuracy and objectivity of IBD assessment, but also to reduce inter-rater reliability of pathologist scorings (M. lacucci, G. Santacroce, I. Zammarchi, Y. Maeda, R. Del Amor, P. Meseguer, B.B. Kolawole, U. Chaudhari, A. Di Sabatino, S. Danese, Artificial intelligence and endo-histo-omics: new dimensions of precision endoscopy and histology in inflammatory bowel disease, The Lancet Gastroenterology & Hepatology, (2024)). Quantifying the microscopic normalisation of mucosal biopsies requires extremely sensitive tools (sensitive meaning accurate sensitivity and specificity), for example, the level of sensitivity Al provides.

[0027] The term ‘crypt’ is used herein to mean the mucus secreting glands of the colon and rectum with a test-tube like appearance in healthy tissue (but which distort in unhealthy tissue) and are responsible for renewal of the intestinal lining, which are a characteristic feature of bowel histology images.

[0028] The term ‘tissue foreground’ is used herein to describe the area that consists of only tissue on a digitised histology slide.

[0029] The term ‘muscularis mucosa’ is used herein to mean both the muscularis mucosa and submucosal muscle layers that are a feature of bowel histological samples.

[0030] The term ‘crypt structural abnormality’ is used herein to describe differences in the appearance of crypts from their appearance in healthy bowel, including branching, serrated architecture, presence of abscesses and infiltration of cells in the crypt interior and epithelial cell lining, epithelial attenuation (thinning of the crypt epithelial layer), mucin depletion (lessening and loss of integrity of the mucinous interior of crypts), irregularity of size and shape, crypt shortfall (distancing of the basal portion of crypts from the lamina propria), surface ulceration and cryptitis.

[0031] The term ‘lamina propria’ is used herein to describe the subepithelial layer of loose connective tissue that is a characteristic feature of colorectal biopsies and corresponds roughly to the interstitial regions between epithelium, crypts, and muscle regions.

[0032] The term ‘surface epithelium’ is used herein to mean the surface layer of cells comprising the lining of the small and large intestine and rectum, which comprises a characteristic feature of bowel histology images.

[0033] Summary of the Invention

[0034] According to a first aspect of the invention there is a method of histological characterisation of bowel disease as disclosed by claim 1 . There is further disclosure of the method in the dependent claims.

[0035] According to an aspect of the invention there is a method of histological characterization of inflammatory bowel disease IBD, comprising: analysing segmented objects from at least one pixelated image of a biopsy tissue sample from a bowel, wherein the segmented objects comprise cellular types and tissue sections, wherein the cellular types comprise neutrophils, eosinophils, plasma cells, epithelial cells, lymphocytes, macrophages, fibroblasts, goblet cells, connective cells or other cells of the bowel, and the tissue sections comprise tissue foreground, muscularis mucosa, lamina propria, mucin-containing regions, crypt(s), surface epithelium, crypt interior, blood, or other bowel-associated tissues or regions; associating each of the cellular types with one or more of the tissue sections, and determining metrics that score the histological characterization of IBD.

[0036] According to an aspect of the invention there is a system to execute the method of histological characterization of IBD, the system comprising a computer, an input receiving device by which the computer receives at least one pixelated image, a display by which the computer shows the metrics, and a computer program directly loadable into a memory of the computer for controlling the method when the program is run on the computer.

[0037] The method and system identify and characterise histological features of bowel disease. The method and system quantify inflammatory indicators and changes in tissue architecture. Spatial identification and measurement of hallmark features in IBD tissue biopsies are used for insight into disease activity. Interrelated steps identify both non-pathological and pathological features.

[0038] Characterisation of tissue features may comprise semantic segmentation of tissue features, subsets of tissue features and the quantification or assessment of each feature spatially with respect to other tissue features and tissue feature subsets.

[0039] Semantic segregation may distinguish between various anatomical structures, such as between normal anatomical structures and abnormalities. Each pixel of an image may be labelled. They could be labelled according to the anatomical structures or abnormalities. Semantic segmentation may comprise pixel-level classification for anatomical structure and abnormality differentiation.

[0040] Characterisation of tissue features may comprise instance segmentation of tissue features, subsets of tissue features and the quantification or assessment of each feature spatially with respect to other tissue features and tissue feature subsets.

[0041] Instance segmentation may utilize the semantic segmentation. The pixels may be sorted into groups according to the label of each pixel. Each group of pixels may have one or more characteristics such as shape or thickness or chemical composition. The normal anatomical structures or abnormalities may be individually recognizable according to the characteristics. A specific or preselected one of the normal anatomical structures or abnormalities may be identified among the groups.

[0042] Characterisation of tissue features may comprise semantic segmentation and instance segmentation, subsets of tissue features and the quantification or assessment of each feature spatially with respect to other tissue features and tissue feature subsets.

[0043] Segmentation of component features may be performed using a neural network, convolutional neural network, transformer model (including vision transformer), or a combination of multiple such networks trained for the purpose. Alternatively or in conjunction, segmentation of component features may be performed using a neural network not specifically trained for the purpose but capable of performing the task, for example a segmentation foundation model, or an autoencoder or transformer capable of learning and producing embeddings of arbitrary image features.

[0044] Alternatively or in conjunction, segmentation of component features may be performed using non-neural network-based image processing, for example thresholding, pixel classification, filtering, feature detection, in isolation or in a combined series of processing steps.

[0045] Alternatively or in conjunction, segmentation of component features may be effected manually; and approximately with the use of a tile-based classification, whereby smaller tiles of a larger image (e.g. of dimensions (128,128) pixels, (256,256) pixels, (512,512) pixels, or any other size) are extracted and classified according to whether the tile contains each component feature using a neural network or other means.

[0046] Component features may be identified as bounding boxes, centroids or other summary sets of coordinates where the summary descriptors are spatially associated.

[0047] Tissue features, subsets of tissue features, and the quantification or assessment of each feature may be displayed to a viewer, user, or other operator, including a pathologist via visual aids which display measured properties of tissue features and tissue feature subsets. For example, a tissue feature may be displayed as a polygon geometry, bounding box, centroid or perimeter coordinates, convex or concave hull coordinates, or set of tile coordinates. The features may appear as colour, texture, transparency or other means that represents a facet of the underlying feature that is intrinsic to it but derived by the system, e.g. its area, density of inflammatory cells or other metric.

[0048] Areas of the bowel tissue may be delineated via a combination of foreground features and other tissue features through logical predicates, or morphological operations, for example by taking the union, intersection, or difference of select features.

[0049] Interface regions may be delineated between identified features through constructive and destructive geometric operations of features, for example buffering, dilation or erosion. Tissue fragmentation is computed from the tissue foreground and used to adjust or filter predictions of other tissue components.

[0050] Cells or other features identified may be associated with geometric, morphological, logical predicates to tissue regions, e.g. spatial regions identified as the lamina propria, which contain them. For example, through the use of spatial joining, overlay operators, or ‘in’, ‘within’, or ‘contains’ operations or other.

[0051] The location and abundance of communities of cooperating cells defined by statistical frequency, nearest neighbour, ‘bag of words’ and / or other techniques are established relative to lamina propria, crypts, crypt epithelium, muscularis mucosa and other regions.

[0052] The disease feature ‘cryptitis’ may be identified through spatial association of neutrophil locations to crypt regions by Euclidean or other distance measurement, spatial clustering of neutrophil and crypt coordinates, association of neutrophil locations to dilated and / or eroded crypt geometries.

[0053] Inflammatory cell foci (e.g. as seen in Crohn’s disease) and clusters of inflammatory cell foci (e.g. tertiary lymphoid structures and B cell follicles) may be detected and their physical extents characterised through direct segmentation with a neural network or other segmentation process, or processing of cellular coordinate data including spatial clustering methods, nearest neighbour, topological data analysis, kernel density estimation, peak finding, local area counting including quadrat counting.

[0054] Diversity of a discrete entity (e.g. populations of cell types; categories of crypt) may be quantified within a region of the sample with statistical descriptors of diversity e.g. Simpson’s diversity index, Shannon’s H index, or Gini-Simpson coefficient.

[0055] Crypt structural abnormality may be quantified, for example, with a process whereby the overall geometry of a crypt is transformed into its morphological skeleton. The morphological skeleton is used to create an undirected graph representation where each segment of the skeleton is an edge in the graph. Branching crypts are detected through analysis of the graph representation, e.g. through node degree, connectedness, centrality or combinations of descriptors. Crypt structural abnormality may be assessed through analysis of the coordinates of the perimeter of the crypt describing a mathematical closed curve. Crypt structural abnormality may be quantified by: elliptic Fourier analysis of the closed curve describing the perimeter of the crypt; principal component analysis (PCA) of the closed curve describing the perimeter of the crypt; morphological descriptors of the geometry of the crypts (e.g. area, solidity, ellipticity, roughness); or assessment of fractal dimension; or determination of fragmentation through mathematical descriptors of fragmentation; or any of the above applied to fragments of crypts determined likely to form part of a parent structure.

[0056] The difference between crypt geometries and crypt interior geometries is measured, for example by measuring the average distance between a radial point on the crypt interior and its equivalent point on the crypt geometry.

[0057] Crypt architecture involves measurement of properties of cells situated within the crypt, including determining the types of cells in the crypt.

[0058] The spatial distribution and structural arrangement of crypts within a sample, including the regularity of the arrangement of crypts is determined with measurements of points on each crypt to neighbouring crypts and performing statistical tests.

[0059] Preferably, measuring crypt abnormality is an integral feature of the model and a set of crypts are represented as vectors of these features which are then grouped via an unsupervised clustering method into families or clades or clusters or groups of similarly featured crypts.

[0060] Mucin content of crypts may be assessed by associating crypt geometries derived from the crypt segmentation model with corresponding geometries derived from the crypt interior and / or mucin pocket and / or goblet segmentation models. For example, the area ratio of the crypt interior to the crypt geometry, overall area of the crypt interior, number of mucin pockets, density of mucin pockets within the crypt, number density of mucin pockets within the crypt.

[0061] Mucin may be assessed through textural analysis, analysis of encoded representations of the segmented crypt interior acquired via e.g. deep learning. Mucin and / or crypt architecture may be assessed by representing the crypt as a point cloud of goblet and epithelial and other cell locations and using topological data analysis, point cloud analysis, deep learning-based deep point cloud analysis and / or spatial statistics. Distances of the crypt base from the muscularis mucosa may be measured to derive metrics of crypt shortfall.

[0062] Spatial analysis of surface epithelium, crypts, lamina propria and muscularis mucosa or other tissue features enables the detection and quantification of inflammatory cells (e.g. plasma cells) in spatially localised niches and facilitates the determines the cell population gradients across whole tissue sections (e.g. loss of plasma cell gradient).

[0063] The geometry of the surface epithelium may be iteratively dilated and the difference taken between the dilated geometry and the original geometry, and the dilated geometries restricted to the lamina propria geometry resulting in concentric bands at equal intervals into the lamina propria in which cell densities or other properties can be measured over space to measure disease characteristics.

[0064] The geometry of the muscularis mucosa may be iteratively dilated and the difference taken between the dilated geometry and the original geometry, and the dilated geometries restricted to the lamina propria geometry resulting in concentric bands at equal intervals into the lamina propria in which cell densities or other properties can be measured over space to measure disease characteristics.

[0065] The geometry of any tissue feature or tissue section may be iteratively dilated and the difference taken between the dilated geometry and the original geometry, and the dilated geometries restricted to the lamina propria geometry resulting in concentric bands at equal intervals into the lamina propria in which cell densities or other properties can be measured over space to measure disease characteristics.

[0066] The orientation of a crypt with respect to the slicing of the histological sample may be estimated with a classifier, neural network or other means, based on geometric and characteristic properties, or by extracting an image of the crypt from the originating slide images and passing this image through a classifier trained to discriminate crypt orientation.

[0067] The orientation of a crypt and the basal section of a crypt may be calculated through geometric, morphological or cartographic analysis of crypt geometries relative to surface epithelial and / or muscle geometries.

[0068] Numerical measurements relating to cell presence in delineated regions and / or crypt architectural abnormality may form components of a summary scoring system for assessing the stage and / or severity of disease.

[0069] The method may be applied to longitudinally sampled histological images to assess change to bowel samples, including the lamina propria, crypt, surface epithelium, and muscularis mucosa or other regions over time, including for the assessment of reduction or increase in inflammation, crypt structural abnormality.

[0070] The method may be applied to biopsy samples that correspond to multiple regions of the bowel of a single patient and quantification and scoring is performed for all samples, and disease features and severity are assessed in either an integrated or an unintegrated fashion. Scores of multiple spatially separated samples of a bowel may be turned into a summary score or set of scores for the whole bowel. Spatially and temporally sampled images may be integrated into a resultant scoring.

[0071] Detected properties or measurements made of crypt geometries, cell densities or other properties may be used as overlays to inform a user about the properties of tissue regions. Region geometries representing identified tissue features may be overlaid on the originating images with visual differentiation of a value measured in that region: for example, density of inflammatory cells is represented by colour, colour intensity, shading style, or transparency.

[0072] Colorectal crypts (also known as glands) have histological features which may be characterized. Colorectal crypts are delineated by their external epithelial cell layer. The interior of colorectal crypts comprise the crypt lumen and mucin pockets; crypt abscesses; the surface epithelium; the lamina propria; the muscularis mucosa and sub-mucosal muscle layers. The colorectal crypts are delineated from the overall tissue foreground and / or specifically bowel tissue in samples with mixed tissue; blood aggregates, individual red blood cells, areas of haemorrhage; features of the bowel microenvironment such as veins and arteries; individual cells (including epithelial cells, lymphocytes, eosinophils, plasma cells, neutrophils, macrophages, fibroblasts and connective tissue cells), Normal, atypical and abnormal colorectal crypts may be characterized and delineated by features of architecture including by analysis of cell densities, shape, size, geometry, and contents of the crypt.

[0073] Characterisation of tissue features may involve subdividing a tissue feature into smaller constituents e.g. through tiling or otherwise splitting the geometry of the feature. Component features may be represented as location-based probabilities that the component feature exists at that location (a probability map), and spatial association of component features may be performed for geometries derived from thresholding the probability map at a value.

[0074] Component features may be identified through segmentation and association of their component cell types with a cell segmentation or detection model trained to predict cell types. For example, crypt features are identified as epithelial or goblet cells in proximity with one another. Likewise, for muscularis mucosa, features are identified as connective tissue cells or fibroblasts in proximity with one another.

[0075] The method may identify ulceration through spatial, morphological and / or cartographic analysis of the surface epithelium; quantifies cells (e.g. density, fraction) in the lamina propria, crypt, surface epithelium, muscle and any other relevant region with geometric, cartographic or morphological analysis. The method may detect crypt abscesses through spatial, morphological or cartographic analysis of cell coordinates with reference to geometries of crypts (e.g. neutrophils, eosinophils and lymphocytes).

[0076] A synthesis scoring of the above metrics may be produced by the method which corresponds to a measure of disease severity.

[0077] The invention will now be described, by way of example only, with reference to the accompanying figures in which:

[0078] Brief Description of the Figures

[0079] Figure 1 shows an overview system overview and flowchart of the method;

[0080] Figure 2 shows tissue sections in a biopsy tissue sample from a bowel;

[0081] Figure 3A shows a first example of spatially differentiated tissue region measurement and measurement overlays;

[0082] Figure 3B shows a second example of spatially differentiated tissue region measurement and measurement overlays;

[0083] Figure 4A shows a biopsy tissue sample from a bowel from which to determine abstract geometries of segmented objects; Figure 4B shows crypt and epithelium geometries in the sample of Figure 4A are visualised by characteristic features of roughness and solidity;

[0084] Figure 4C shows crypt and epithelium geometries in the sample of Figure 4A are visualised by characteristic features of extent and epithelial density;

[0085] Figure 5A shows crypt geometries;

[0086] Figure 5B shows extracted features;

[0087] Figure 5C shows a graph to perform clustering of crypt feature vectors;

[0088] Figure 5D shows archetypal clusters of crypts in biopsy tissue samples;

[0089] Figure 6A shows a crypt geometry;

[0090] Figure 6B shows a skeleton geometry of the crypt in Figure 6A;

[0091] Figure 6C shows a skeleton graph comprising a first type of marking dots at nodes where the branches of the skeleton geometry change direction;

[0092] Figure 6D shows branching crypts and non-branching crypts;

[0093] Figure 7A shows bowel tissue displaying a characteristic gradient of cells;

[0094] Figure 7B shows parallel interior regions that are parallel to the exterior edges of the biopsy tissue sample of the bowel;

[0095] Figure 7C shows a measure of properties of inflammatory cells in each region;

[0096] Figure 8 shows segmented muscularis mucosa and highlighted buffer zone; and

[0097] Figures 9A and 9B show identified lymphoid structures.

[0098] Detailed Description

[0099] A flowchart is shown in Figure 1 of a method of histological characterization of inflammatory bowel disease IBD 100. At least one pixelated image of a biopsy tissue sample from a bowel is received 102. There is a system to execute the method, and the system comprises a computer and an input receiving device by which the computer receives the at least one pixelated image.

[0100] As shown by the flowchart in Figure 1 , the method applies segmentation models 104 to search for and find objects with abstract geometries in each pixelated image and then segment the objects.

[0101] Once the segmented objects are produced 106 they are analysed. The segmented objects are analysed because they comprise cellular types and tissue sections which are associated with each other. The method makes these associations 128.

[0102] As shown by the flowchart in Figure 1 , the cellular types are identified 108 and the tissue sections are identified 109. Each identified cellular type is associated with one or more of identified tissue section 128 as appropriate.

[0103] Different segmented objects are analysed to determine which cellular types the segmented object comprises 108. The cellular types in each segmented object comprise one or more of: neutrophils 142, eosinophils 144, plasma cells 146, epithelial cells 148, lymphocytes 150, macrophages 152, fibroblasts 154, connective cells 155 or other cells 156 of the bowel.

[0104] The different segmented objects are analysed to determine which tissue sections or features the segmented object comprises 110. The tissues sections or features in each segmented object comprise one or more of: tissue foreground 112, muscularis mucosal 14, lamina propria 116, goblets 118, crypt(s) 120, surface epithelium 122, crypt interior 124, blood 126, or other tissue section of the bowel.

[0105] As shown in the flowchart in Figure 1 , all the tissue sections are analysed to quantify them according to their associated cellular type and segmented object 134. The overall crypt 120 and the crypt interior 124 are subject to geometric and morphological analysis 130. The quantification of cellular type and segmented object 134 of all tissue sections is combined with the geometric and morphological analysis 130 of the overall crypt and the crypt interior to determine quantitative features 132.

[0106] Integrated scoring 136 is determined from the quantification of cellular type and segmented object 134, and from the quantitative features 132, and from the geometric and morphological analysis 130. The integrated scoring includes domain specific cell metrics 142 determined from the quantification of cellular type and segmented object 134. The integrated scoring 136 includes integrated metrics 140 determined from the quantitative measures 132. The integrated scoring 136 includes architectural metrics 138 determined from the geometric and morphological analysis 130.

[0107] In Figure 2 are shown pixelated images of tissue sections 202 in abstract geometries of segmented objects in the biopsy tissue sample from a bowel. Tissue sections shown include: crypt 204, surface epithelium 206, crypt interior 208, lamina propria 210, and muscularis mucosa 212.

[0108] In Figure 3A and Figure 3B are shown spatially differentiated tissue region measurement and measurement overlays. In the left side column is shown pixelated images of the biopsy tissue sample from a bowel. In the right side column there are overlays on specific tissue sections to highlight them. There is a first overlay indicating the surface epithelium in red and a second overlay indicating the muscularis mucosa.

[0109] Figure 3A shows in the left column biopsy tissue samples 301 , 303, 305 from the bowel, and a first example of these overlays and Figure 3B shows a second example. In the first example in Figure 3A the muscularis mucosa 302, 304, 306 is overlaid in yellow. The surface epithelium 308, 310, 312 is overlaid in red. Lamina propria lymphocytes are shown with a colour scale 316 and crypts mucin content is also shown with a colour scale 318. In the second example in Figure 3B the muscularis mucosa 362, 364 is overlaid in yellow onto the biopsy tissue samples 351 , 353, 355. The surface epithelium 352, 354, 356 is overlaid in red. Lamina propria lymphocytes are shown with a colour scale 372 and crypts mucin content is also shown with a colour scale 374.

[0110] Figures 4A, 4B, and 4C show an overview of how crypt and epithelium geometries in a sample are visualised by characteristic features. Figure 4A shows a pixelated image of a biopsy tissue sample from a bowel 402. On the left side of Figure 4B is shown roughness 404 of the crypt and epithelium geometries according to a colour scale 406 from about 1.5 to about 5.0. On the right side of Figure 4B is shown solidity 408 of the crypt and epithelium geometries according to a colour scale 410 from about 0.4 to about 0.9.

[0111] On the left side of Figure 4C is shown extent 412 of the crypt and epithelium geometries according to a colour scale 414 from about 0.2 to about 0.8. On the right side of Figure 4B is shown epithelial density 416 of the crypt and epithelium geometries according to a colour scale 418 from zero to about 0.0008.

[0112] Figures 5A, 5B, 5C, and 5D show an overview of how a crypt abnormality is classified with a system utilising unsupervised hierarchical clustering of features. Figure 5A shows crypt geometries 502. Figure 5B shows extracted features 508, 516, especially shape and cell densities. Shapes that surround the extracted features, such as circles 504, 510, or rectangles 506, 512, or more conforming polygons 514 are used to progressively more precisely determine the abstract geometries of the segmented features or segmented objects.

[0113] Figure 5C shows an example of a graph 518 used to perform clustering of crypt feature vectors. This is done by hierarchical or k-means. Figure 5D shows visible results on the tissue samples of assigning crypts to archetypal clusters 520, 522, 524, 526, 528, 530 and quantifying the crypts assigned to each cluster.

[0114] Figure 6A, 6B, 6C, 6D, and 6E show an overview of an embodiment of the method of histological characterization’s graphical technique to identify crypt branching.

[0115] Figure 6A shows a crypt geometry 602. Figure 6B shows a skeleton geometry 604 determined by analysis of the crypt geometry 602 in Figure 6A. The skeleton geometry comprises branches 606, 608, 610, 612. Figure 6C shows a skeleton graph comprising a first type of marking dots 614 at nodes where the branches meet. The skeleton graph also comprises a second type of marking dots 616 at nodes where the branches change direction. The skeleton graph also comprises a third type of marking dots 618 at nodes where the branches have a free end. This provides both node classification and branch enumeration. Figure 6D shows a branching crypt 620 and non-branching crypt 622 are both able to be distinguished and quantified by this technique using skeleton graphs to analyse the skeleton geometry.

[0116] Figures 7A, 7B, and 7C and Figure 8 show an overview of spatial analyses performed by the method of histological characterization of IBD. The spatial analysis is performed with respect to cell locations, surface epithelium, lamina propria and muscularis mucosa. The spatial analysis is performed in order to quantify the disease feature basal lymphoplasmacytosis and / or basal plasmacytosis.

[0117] Bowel tissue displaying a characteristic gradient of cells is shown in the left most view of the tissue sample in Figure 7 A. The arrows 702, 704, 706 point in the direction from highest concentration of the cells to lowest concentration. The highest concentration is by the epithelium region 712 at the edge.

[0118] In the centre view of the tissue sample shown in Figure 7A concentration buffering 708 is indicated. The epithelium region 712 is highlighted by an overlay of yellow along the edge. In the rightmost view of the tissue sample in Figure 7A is shown intersection of the epithelium region 712 along the with a first band 714 of the lamina propria region 710 in the interior. The first band 714 is about 100 urn (micrometers) wide and as long as the tissue sample. The first band 714 is shown in the region 0 urn to 100 urn in the interior from the epithelium region 712. The cell concentration is higher in the first band 714 than in other bands further from the epithelium region 712.

[0119] Figure 7B shows three more views of the tissue sample. In the left most view a second band 716 in the lamina propria is adjacent to the first band 714 of lamina propria. The second band is shown in the region 100 urn to 200 urn in the interior from the epithelium region 712. In the centre most view a third band 718 in the lamina propria is adjacent to the second band 716 of lamina propria. The third band 718 is shown in the region 200 urn to 300 urn in the interior from the epithelium region 712. In the rightmost view a fourth band 720 in the lamina propria is adjacent to the third band 718 of lamina propria. The fourth band 718 is shown in the region 300 urn to 400 urn in the interior from the epithelium region 712. The epithelium region 712, first band 714, second band 716, third band 718, and fourth band 720 are all parallel to each other.

[0120] Figure 7C shows three more views of the tissue sample. A fifth band 722 of lamina propria 722 that is 400 urn to 500 urn from the epithelium region 712 and a sixth band 724 that is 600 urn to 700 urn from the epithelium region 712. The sixth band 724 is disposed along the opposite edge of the tissue sample from the epithelium region.

[0121] Observing Figures 7A, 7B, and 7C the cell concentration decreases from band to band the further each band is from the epithelium region 712. Properties are measured in each region 726 such as the bands 714, 716, 718, 720, 722, 724, for example of the inflammatory cells as noted above.

[0122] Figure 8 shows segmented muscularis mucosa 802 and a buffer zone 804. Figure 9A illustrates that spatial clustering of cell coordinates plus cluster border determination 902 by which a follicle 904 is identified as shown in Figure 9B. The invention has been described by way of examples only. Therefore, the foregoing is considered as illustrative only of the principles of the invention. Further, since numerous modifications and changes will readily occur to those skilled in the art, it is not desired to limit the invention to the exact construction and operation shown and described, and accordingly, all suitable modifications and equivalents may be resorted to, falling within the scope of the claims.

Claims

Claims:

1. A method of histological characterization of inflammatory bowel disease IBD, comprising: analysing segmented objects from at least one pixelated image of a biopsy tissue sample from a bowel, wherein the segmented objects comprise cellular types and tissue sections, wherein the cellular types comprise neutrophils, eosinophils, plasma cells, epithelial cells, lymphocytes, macrophages, fibroblasts, goblet cells, connective cells or other cells of the bowel, and the tissue sections comprise tissue foreground, muscularis mucosa, lamina propria, mucin-containing regions, crypt(s), surface epithelium, crypt interior, blood, or other bowel-associated tissues or regions; associating each of the cellular types with one or more of the tissue sections, and determining metrics that score the histological characterization of IBD.

2. The method of histological characterization of IBD of claim 1 comprising forming tiles of pixels in the pixelated image, wherein each tile comprises all of the pixels within a perimeter of the tile.

3. The method of histological characterization of IBD of claim 2 comprising classifying each tile according to the cellular types or tissue sections in which the tile is disposed.

4. The method of histological characterization of IBD of any preceding claim comprising associating a geometrical property, morphological property, or logical predicate with the cellular types or tissue sections.

5. The method of histological characterization of IBD of claim 4 delineating or identifying the cellular types or tissue sections using the geometrical property, morphological property, or logical predicate.

6. The method of histological characterization of IBD of claim 5 locating an interface where adjacent ones of the cellular types or tissue sections are delineated using constructive or destructive geometric operations comprising buffering, dilation, or erosion.

7. The method of histological characterization of IBD of any preceding claim quantifying an amount of tissue fragmentation in the cellular types or tissue sections from a tissue foreground.

8. The method of histological characterization of IBD of claim 7 quantifying the amount of tissue fragmentation through mathematical descriptors of fragmentation.

9. The method of histological characterization of IBD of any preceding claim wherein a location or abundance of communities of the cellular types that are cooperating is determined by statistical frequency, nearest neighbour, or bag of words established relative to the lamina propria, crypt(s), crypt epithelium, muscularis mucosa, inflammatory cells, other of the tissue sections.

10. The method of histological characterization of IBD of any preceding claim wherein spatial association of the regions of neutrophil(s) to crypt(s) is determined by distance measurement, spatial clustering of neutrophil and crypt coordinates, association of neutrophil locations to dilated or eroded crypt geometries to identify a cryptitus.11 . The method of histological characterization of IBD of any preceding claim wherein Inflammatory cell foci or clusters of inflammatory cells are detected and their physical extents characterised by the segmenting or by processing of cellular coordinate data.

12. The method of histological characterization of IBD of claim 11 wherein processing cellular coordinate data comprises spatial clustering methods, nearest neighbour, topological data analysis, kernel density estimation, peak finding, or local area counting including quadrat counting.

13. The method of histological characterization of IBD of any preceding claims wherein to quantify a structural abnormality of the crypt, a geometry of the crypt is transformed into a morphological skeleton that is used to create an undirected graph representation where each segment of the skeleton is an edge in the graph.

14. The method of histological characterization of IBD of claim 13 wherein branching ofthe crypt is detected through analysis of the undirected graph representation through node degree, connectedness, centrality, or combinations of descriptors.

15. The method of histological characterization of IBD of any preceding claim wherein a difference between crypt exterior geometries and crypt interior geometries is measured as an average distance between a radial point on the crypt interior geometry and its equivalent point on the crypt interior geometry.

16. The method of histological characterization of IBD of any preceding claim wherein a spatial distribution and structural arrangement of crypts within the biopsy tissue sample is determined with measurements from points on each crypt to neighbouring crypts and performing statistical tests.

17. The method of histological characterization of IBD of any preceding claim wherein a mucin content of the crypt(s) is assessed by associating or linking a crypt geometry derived from a crypt segmentation model with a geometry of the crypt interior or mucin containing pocket, goblet cells, or any combination thereof.

18. The method of histological characterization of IBD of any preceding claim wherein the architecture of the mucin content or architecture of the crypt(s) is assessed by representing the crypt as a point cloud of goblet and epithelial cells and other cell locations using topological data analysis, point cloud analysis, deep learning-based deep point cloud analysis or spatial statistics.

19. The method of histological characterization of IBD of any preceding claim wherein distances from a base of the crypt to the muscularis mucosa is measured to derive metrics of crypt shortfall.

20. The method of histological characterization of IBD of any preceding claim using spatial analysis of the surface epithelium, crypt(s), lamina propria, or muscularis mucosa to detect and quantify inflammatory cells in spatially localised niches.

21. The method of histological characterization of IBD of any preceding claim using spatial analysis of the surface epithelium, crypt(s), lamina propria, neutrophil, or muscularis mucosa to determine plasma cell population gradient across a whole cellular type or tissue section.

22. The method of histological characterization of IBD of any preceding claim wherein a geometry of one of the tissue sections is iteratively dilated and a difference taken between the dilated geometry and the original geometry, and the dilated geometry is restricted to the lamina propria geometry resulting in concentric bands at equal intervals into the lamina propria in which cell densities or other properties are measured to measure disease characteristics.

23. The method of histological characterization of IBD of claim 22 wherein the tissue section that is iteratively dilated is the surface epithelium or the muscularis mucosa.

24. The method of histological characterization of IBD of any preceding claim wherein the biopsy tissue sample is a longitudinal sample.

25. The method of histological characterization of IBD of any preceding claim including characterization of normal, atypical, and abnormal colorectal crypts architecture by analysis of the shape, size, geometry, and contents of the crypt.

26. The method of histological characterization of IBD of any preceding claim wherein component features are represented as location-based probabilities that the component feature exists at that location in a probability map, and spatial association of the component features is performed for geometries derived from thresholding the probability map at a value.

27. The method of histological characterization of IBD of claim 26 wherein component features of crypt comprise epithelial and goblet cells, and the crypt is identified through segmentation and association of epithelial or goblet cells in proximity with one another with a cell segmentation or detection model trained to predict cell types.

28. The method of histological characterization of IBD of claim 26 wherein component features of muscularis mucosa comprise connective tissue cells or fibroblasts, and the muscularis mucosa is identified through segmentation and association of the connective tissue cells or fibroblasts in proximity with one another with a cell segmentation or detection model trained to predict cell types.

29. The method of histological characterization of IBD of any preceding claim wherein diversity of a discrete entity is quantified within at least one of the segmented objects with statistical descriptors of diversity comprising Simpson’s diversity index, Shannon’s H index, or Gini-Simpson coefficient.

30. A system to execute the method of histological characterization of IBD of any preceding claim, the system comprising a computer, an input receiving device by which the computer receives at least one pixelated image, a display by which the computer shows the metrics, and a computer program directly loadable into a memory of the computer for controlling the method when the program is run on the computer.

Images