Decision support for toxicity assessment

Abstract

Systems and methods are provided for decision support in toxicology assessment. The system includes a processor and a non-transitory computer readable medium storing instructions executable by the processor to provide an imager interface that receives an image of organ tissue of an animal and a vision encoder trained on a plurality of training images that receives the image of organ tissue and provides a representation of the image of organ tissue. Each training image includes an image of organ tissue of an animal. A toxicology front end generates one of a value representing lesions within the image of organ tissue or an image having a similar representation from the representation of the image of organ tissue.

Description

DECISION SUPPORT FOR TOXICITY ASSESSMENTRELATED APPLICATIONS

[0001] This application claims priority to U.S. Provisional Application Serial No. 63 / 674,163, titled “Novel Computational Models for Drug Toxicity Assessment” and filed on July 22, 2024, which is hereby incorporated by reference.TECHNICAL FIELD

[0002] This invention relates to clinical decision support systems, and more particularly, to a system for providing decision support for toxicity assessment.BACKGROUND

[0003] The transition from preclinical research to early-stage clinical trials in drug development is a challenging phase with a high attrition rate of drug candidates. A key challenge in this transition is assessing toxicity, which accounts for eighty-two percent of drug development discontinuation at the preclinical stage. Drug safety is evaluated in animal models through non-clinical laboratory studies, which involve a comprehensive histopathological examination of standardized tissue sections to identify and document drug-induced injury in critical organs. Among these organs, the liver is of particular concern due to its central role in drug metabolism, especially for orally administered compounds. The current approach to toxicity assessment relies heavily on manual inspection of tissue sections by pathologists, which is timeintensive and prone to high inter-observer variability. This variability is particularly pronounced when diagnosing focal or ill-defined abnormalities and lesions that may be overlooked, such as focal hepatocellular necrosis, which can account for less than one percent of the tissue section. Moreover, semi-quantitative approaches frequently used in histological assessment, which involve scoring lesions on a scale (e.g., minimal, mild, moderate, severe) instead of fully quantifying them (e.g. percentage of affected tissue), further contribute to this variability. Efficient quantification requires automated tools for lesion detection, and the identification of small individual cell lesions (e.g., mitosis, single-cell necrosis) necessitates extensive review of different tissue regions at high magnification, which can quickly become prohibitive in large screening studies. These challenges contribute to the “Valley ofDeath” in drug development, resulting in substantial costs and unnecessary animal sacrifices.SUMMARY

[0004] In accordance with one example, a system is provided for decision support in toxicology assessment. The system includes a processor and a non-transitory computer readable medium storing instructions executable by the processor to provide an imager interface that receives an image of organ tissue of an animal and a vision encoder trained on a plurality of training images that receives the image of organ tissue and provides a representation of the image of organ tissue. Each training image includes an image of organ tissue of an animal. A toxicology front end generates one of a value representing lesions within the image of organ tissue or an image having a similar representation from the representation of the image of organ tissue.

[0005] In accordance with another example, a method is provided for toxicology assessment of a therapeutic. A plurality of animals are provided with a dose of the therapeutic, and an image of tissue from each of the plurality of animals is obtained to provide a plurality of sample images. Each of the plurality of sample images is provided to a vision encoder to provide a plurality of image representations, corresponding to the plurality of images, in an embedding space associated with the vision encoder. A predicted gene expression associated with each of the plurality of sample images is determined from the corresponding image representation at a gene expression model to provide a plurality of pseudogene expression profiles. A predicted toxicity of the therapeutic in human beings is determined based on the plurality of pseudogene expression profiles.

[0006] In accordance with a further example, a system provides decision support for toxicology assessment. The system includes a processor and a non-transitory computer readable medium storing instructions executable by the processor to provide an imager interface that receives an image of organ tissue of an animal and a vision encoder trained on a plurality of training images that receives the image of organ tissue and provides a representation of the image of organ tissue. Each training image includes an image of kidney or liver tissue of an animal. A gene expression model, trained on a set of training images and corresponding geneexpression profiles, produces a gene expression profile from the representation of the image of organ tissue.BRIEF DESCRIPTION OF THE DRAWINGS

[0007] FIG. 1 illustrates one example of a system for analysis of drug toxicity images;

[0008] FIG. 2 illustrates an example of a system for evaluating the toxicity of compounds;

[0009] FIG. 3 illustrates one example of a method for toxicology assessment of a therapeutic; and

[0010] FIG. 4 is a schematic block diagram illustrating an exemplary system of hardware components capable of implementing examples of the systems and methods disclosed herein.DEFINITIONS

[0011] As used herein, a “Log2 fold change” is a measurement used to quantify the relative change between two experimental conditions that is calculated by taking the base two logarithm of the ratio between the percentage of a certain lesion under some conditions (such as high dose, sacrifice time of a specified number of days, etc.) and the percentage of that same lesion in the control group.DETAILED DESCRIPTION

[0012] Considering the diversity of lesions induced by compound administration and the range of protocols used by contract research organizations, the current paradigm of developing task-specific model poses challenges to widespread adoption. The limited availability of cases for rare lesions further exacerbates this limitation. Furthermore, developing, deploying, and maintaining large ensembles of task-specific models quickly becomes impractical. The systems and methods disclosed herein provide a general-purpose and transferable model, specifically designed for toxicologic pathology. The model is trained ondrug safety data to universally encode histology images, making it versatile for various tasks without the need to retrain for each task. This approach facilitates wide-ranging applicability across various studies and compounds, streamlining the diagnostic process. The disclosed systems and methods provide a computational framework that canaddress a set of diagnostic tasks for hepatotoxicity assessment across multiple biological scales. This framework can be seamlessly adapted to a comprehensive set of tasks spanning histopathological response prediction, lesion severity scoring, morphology retrieval, and automatic dose-response characterization and consistently outperforms state-of-the-art baselines and clinical benchmarks from an international group of ten board-certified veterinary pathologists.

[0013] FIG. 1 illustrates one example of a system 100 for analysis of drug toxicity images. The system 100 includes a processor 102 and a non-transitory computer readable medium 110 storing executable instructions, executed by the processor 102. It will be appreciated that the executable instructions can be spread across multiple non-transitory computer readable media that are operatively connected via an appropriate data connection, such that the executable instructions can be executed by multiple processors. In particular, the system 100 can be trained across multiple graphics processing units, with a number of graphics processing units and storage at the non-transitory computer readable medium 110 used for a given application be scalable with the application and the amount of available training data.

[0014] The executable instructions stored on the non-transitory computer readable medium 110 can be executed by the processor 102 to provide decision support for drug toxicity assessment, including an imager interface 112 that receives an image of organ tissue of an animal. Along with managing image intake across a connection to an imager (not shown) or remote or local storage medium, the imager interface 112 can provide image conditioning, such as filtering and scaling, as well as converting the images to a common format.

[0015] A vision encoder 114 receives the image of organ tissue and provides a representation of the image of organ tissue for use at a toxicology front end 116 that generates either a value representing lesions within the image of organ tissue or an image having a similar representation from the representation of the image of organ tissue. This value can be provided to a user at an output device (not shown) via a user interface 118, for example, in a format compliant with the Standard for Exchange of Nonclinical Data. In one example, the vision encoder 114 is implemented as a vision transformer that uses self-attention to capture spatial relationships among small regions, or tokens, of the input. In this implementation, the vision transformer is trained to compress 256x256 pixel image patches into 768-dimensional representations that can further be used at the toxicology front end 116 for downstream tasks, such as image and slide classification or retrieval. In this example. The vision transformer is pretrained using two objectives, a self-distillation objective that attempts to align different representations, or views, of the input image and a masked image modeling objective that attempts to reconstruct small regions from the neighboring regions. These objectives enable capturing high-level context and semantics information of the image, for instance, to build stain-invariant or rotation-invariant representations, as well as the image’s internal structure, in a manner similar to masked language modeling in large language models.

[0016] The vision encoder 114 is trained on a plurality of training images, each comprising an image of organ tissue of an animal. It will be appreciated that the training images can include multiple species of animals, including various species of rats, rabbits, dogs, and monkeys, to allow for the output of the vision encoder to generalize across multiple species. Similarly, the training images can include tissue from multiple organs to allow the results to generalize for various toxicity studies, for example, liver tissue, kidney tissue, and central nervous system tissue. In one implementation, the vision encoder 114 can be trained on fifteen million image patches extracted from around forty-seven thousand whole slide images representing kidney and liver tissue.

[0017] Continuing the above example, two representations, or views, of an input image are generated using augmentation techniques, such as color jittering, mirroring, and zooming. Each view is then randomly masked using blockwise masking, where a series of blocks, defined as rectangles of sixteen patches with a random aspect ratio, are generated until a pre-defined percentage of the input is masked. The two masked views are further tokenized into a set of small tokens of 16x16 pixels and are passed as a sequence through a teacher network, implemented as a vision transformer. Likewise, the tokenized sets for the two masked views are passed through a student network, also implemented as a vison transformer. The self-distillation objective is then defined by computing the crossentropy loss between the classification token from the teacher network and the classification token from the student network. This can be understood as aligning high-level representations of masked and unmasked views. The masked image modeling objective uses learnable projections of the masked tokens from the studentnetwork to predict projections of the patch tokens from the teacher network. The projected patch tokens from the teacher network can be understood as a visual tokenizer for each masked token. The teacher network is then updated using an exponential moving average from the student network to provide the vision encoder 114. The vision encoder 114 can, in some implementations, be fine-tuned on annotated patches, including both patches representing lesions and normal patches.

[0018] The toxicology front end 116 can perform any of a number of functions. In one example, the toxicology front end 116 performs classification of whole slide learning. In one example, a weakly-supervised multiple instance learning paradigm is applied to identify the presence of five different classes of lesions, selected to include common liver lesions, such as necrosis, hypertrophy, increased mitosis, fatty change, and bile duct and oval cell proliferation. In this example, the toxicology front end 116 is trained on around fifteen thousand whole-slide images. To this end, the slides are tessellated into small non-overlapping patches, extracting patch embeddings using the vision encoder 1 14 and an aggregation function is learned to derive a slide embedding, subsequently used for slide classification. In one example, the model implemented in the toxicology front end 1 16 uses an attention mechanism to enable derivation of both a slide classification prediction and patchlevel predictions using slide-level supervision only. Since the attention-based, multiple instance learning model can derive patch-level predictions at no additional training cost, patch attribution can be provided for each individual lesion to understand the model’s internal behavior. This property is enabled by the aggregation function of the model, which derives the final slide-level logits as a sum of patch-level logits (interpreted as patch predictions). By following this approach, it is possible to provide class-wise predictions as heatmaps, which can be interpreted as segmentation predictions.

[0019] Some compounds may induce rare morphological lesions for which only a small collection of training examples is available, thereby rendering weakly supervised classification methods inappropriate. For instance, while some lesions such as cellular necrosis, are common in the liver, other ones are almost never encountered, such as giant cell hepatitis or mineralization, and thus are underrepresented in the training set. Other lesions can be compound-specific, for instance, puromycin aminonucleoside is unique within the compounds evaluated inthe training set for the vision encoder in inducing hepato-cellular basophilic changes (basophilia of hepatocellular cytoplasm). Therefore, the vision encoder 114 and toxicology front end 116 can be configured to allow for strong few-shot learning, that is, the ability to learn with a limited number of training examples. In one example, a slide-level embedding is generated for the vision encoder by taking a mean patch embedding, resulting in a fixed-dimensional and compressed representation of each slide. This representation can be fed to a logistic regression classifier for training using varying numbers of examples per class, a setting called linear probing.

[0020] In addition to few-shot learning, the toxicology front end 1 16 can be used for morphological retrieval, the ability to retrieve morphologies similar to an exemplar. This is accomplished via visual prompting, which uses prototypes, a form of visual morphological descriptor representing each distinctive category, to retrieve morphological regions similar to the prompt. Visual prompting is analogous to text prompting, which aims to retrieve the most similar regions corresponding to a textual description. The toxicology front end can be trained to perform either or both of single-slide digital prompting or single-patch prompting. In single-slide prompting, a slide prototype is formed by taking an average patch embedding across a class. In single-patch prompting, an image patch representing a canonical exemplar of a lesion of interest is selected according to a pathologist’s annotations. For each type of prompting, a cosine similarity metric is generated between the prompt and all patch embeddings in a test slide and averaged to get a similarity score. Visual prompting obviates the need to train a complex dedicated architecture by simply relying on similarity metrics in the embedding space between the prompt and test slides. As prototyping requires a single training example, it can be considered a “one-shot” learning strategy. Visual prompting via patch or slide prototyping offers easy integration for quick and efficient lesion detection, classification, and quantification. Pathologist workload can be reduced by simplifying diagnosis to a small selection of positive slides from a study, which can be used as a prompt to diagnose the remaining slides. Slide prompting only requires slide-level labels for prototype construction, whereas patch prompting requires a small tissue region that contains the lesion of interest. When the lesion comprises only a small portion of the slide (as can be the case in necrosis or fibrosis), patch prompting enables moreprecise visual prompting and ensures that the prompt solely encodes the morphology of interest.

[0021] Despite the promising performance of weakly supervised learning, it is less accurate at detecting small lesions such as extramedullary hematopoiesis (formation and activation of blood cells outside the bone marrow), which may only cover a small area of a slide and often appear alongside various other morphological features. This makes it difficult to distinguish these lesions using the slide-level supervision trained with multiple instance learning described above. Accordingly, in another implementation, the vision encoder 114 can be fine-tuned with patch-level annotations to enable explicit lesion predictions at the patch level, as opposed to the attention-based, multiple instance learning approach described above, in which patch-level predictions are implicitly generated from slide-level predictions. Training patches containing lesions were spread across twelve lesions frequently encountered in drug safety studies and selected to display a broad spectrum of morphological variations in terms of cell type, size, and cytoplasmic and nuclear alteration. Annotations were generated to include morphological features defined at the cellular level, such as mitosis and single-cell necrosis, as well as lesions affecting groups of cells, such as cellular infiltration and bile duct proliferation, and extensive lesions like hypertrophy, which can cover a large portions of the slide. This diverse range allows for a thorough characterization and quantification of morphological abnormalities detected in the slide.

[0022] Continuing this example, the toxicology front end 116 can be implemented as a set of binary classifiers, such as logistical regression models, with each classifier predicting the likelihood that a specific type of lesion is present in the patch embedding. Further, individual patch predictions can be used to create a detailed segmentation map that reveals every lesion identified within a slide. To enhance the resolution of these predictions, patches can be overlapped by eighty percent and average probabilities for each patch can be calculated, disregarding any probabilities below a lesion-specific threshold. This approach is particularly useful for identifying smaller lesions, like focal cellular infiltration, extramedullary hematopoiesis, or hydropic degeneration, that may constitute only a small portion of the slide.

[0023] Beyond single-slide diagnosis, preclinical safety assessments aim to understand and characterize the dose-response relationship of the compound. Tothis end, the toxicology front end 116 can be configured to monitor the onset of morphological abnormalities and lesions at consistent time intervals across various dosages. For each identified lesion, a severity score is assigned to describe the extent of the finding, to distinguish small lesions from severe drug injury. However, the reported scores generalize poorly within and across studies due to high interobserver variability, the complexity of measuring the extent of changes, and the lack of unified standards. This complicates the precise and quantifiable characterization of morphological changes, which, along with the burden of manual examination, renders drug-response assessment nontrivial even on the scale of a single study, which typically involves several hundred slides.

[0024] Using the toxicology front end 116 for patch classification, as described above, objective quantification scores can be generated for each lesion, which we express as a percentage of the lesion area in tissue compared to its normal counterpart. Quantification scores of each lesion can be averaged over each sample group to enable monitoring of the drug dynamics across all doses and sacrifice time points. As some lesions occur spontaneously, these values can be normalized according to background lesions found in treated samples by a set of control slides. For instance, extramedullary hematopoiesis is a naturally occurring lesion in rodents, regardless of the administered compound. To this end, we compute the Iog2 fold change for each sample. By adopting this approach, the onset of lesions as the dose and sacrifice time increase can be determined, allowing for new automation possibilities for more objective and precise lesion identification and more robust drug safety assessment.

[0025] The toxicology front end 116 can further be configured to perform additional retrieval and predictive functions based on data from the vision encoder 114. In one example, the toxicology front end is configured to retrieve a plurality of images sharing one of a lesion type or a metadata parameter with the image of organ tissue from the training data or from historical toxicity studies. The toxicology front end 116 can also be configured to predict one of lesion severity or a progression curve for lesion severity from a specified compound, dose, and time.

[0026] FIG. 2 illustrates an example of a system 200 for evaluating the toxicity of compounds. Identifying and characterizing the potential toxicity of a drug early in its development is a major challenge for the pharmaceutical industry. At the preclinicalphase, toxicity is assessed in animal models through histological examination of tissue sections, in which pathologists report drug-induced lesions and abnormalities to determine the dose-response relationship of the compound. Despite advancements of in vitro assays for early toxicity detection, safety concerns remain the leading cause of drug attrition at the preclinical stage. The system 200 of FIG. 2 identifies and imputes morphomolecular signatures in toxicology data by connecting specific morphologies to expression changes by predicting bulk expression levels of selected gene targets from images of organ tissue, which can be whole-slide images.

[0027] The system 200 includes a processor 202 and a non-transitory computer readable medium 210 storing executable instructions, executed by the processor 202. It will be appreciated that the executable instructions can be spread across multiple non-transitory computer readable media that are operatively connected via an appropriate data connection, such that the executable instructions can be executed by multiple processors. In particular, the system 200 can be trained across multiple graphics processing units, with a number of graphics processing units and storage at the non-transitory computer readable medium 210 used for a given application be scalable with the application and the amount of available training data.

[0028] The executable instructions stored on the non-transitory computer readable medium 210 can be executed by the processor 202 to evaluate the toxicity of a compound during animal studies, including an imager interface 212 that receives an image of organ tissue of an animal. Along with managing image intake across a connection to an imager (not shown) or remote or local storage medium, the imager interface 212 can provide image conditioning, such as filtering and scaling, as well as converting the images to a common format. A vision encoder 214 receives the image of organ tissue and provides a representation of the image of organ tissue for use at a gene expression model 216 that generates a predicted gene expression, referred to as a pseudogene expression, for the compound based on representations from a plurality of tissue images. The predicted gene expression can be provided to a user at an output device (not shown) via a user interface 218.

[0029] In the illustrated example, the system 200 employs a weakly supervised training approach to predict slide-level labels (i.e. , the gene expression) without requiring patch-level annotations and enabling scalable training on large datasets. Inthis example, the vision encoder 214 and gene expression model 216 are trained on whole liver tissue section samples, each paired with a gene expression profile measured with DNA microarrays. Each sample represents morphological changes caused by administering a specific dose of a compound at a particular time post administration. In addition, each slide is annotated with morphological lesions identified by toxicologic pathologists, such as reporting the presence of hepatocellular hypertrophy, with lesions that might be drug-induced or spontaneous.

[0030] Bulk gene expression profiles encode the molecular landscapes from the tissue section, which the action of the compound could have altered. Gene expression is measured with mRNA microarrays, which provide a bulk whole- transcriptome description post-drug administration for a set of genes selected to cover a large biological space relevant to toxicology. Specifically, through complementary a priori knowledge-driven and data-driven approaches, the full set of genes was filtered to a subset of genes that are either biologically relevant to toxicology (e.g., linked to liver metabolism and liver response to injury), or whose measured expression levels were highly correlated with the presence of drug- induced lesions. In one example, about fifteen hundred genes are included in the expression profile.

[0031] The gene expression model 216 is implemented as a deep learning model based on multiple instance learning that can predict the expression profile associated with an input whole-slide image. During training, each slide is tessellated into non-overlapping patches of 256x256 pixels. The vision encoder 214 is pretrained to extract patch embeddings from each patch. The vision encoder, for example, can be implemented as a vision transformer trained on a set of image patches using a self-supervised learning model. In one example, around fifteen million patches extracted from around forty-six thousand slides are used in pretraining. The gene expression model 216 can be implemented as a multilayer perceptron (MLP) that maps each patch embedding to an expression profile. Patchlevel predictions are averages, for example, as an arithmetic mean, to derive a slidelevel expression, where the resulting slide-level prediction represents the predicted Iog2 fold change expression for the slide. During training, a mean squared error between the predicted and measured expression can be used to train the system 200 in an end-to-end fashion with respect to the patch embeddings. The multipleinstance learning formulation naturally yields patch-level expression predictions, thereby enabling a pseudo-spatially resolved expression map, where the resolution is given by patch size. This differs from widely employed attention-based methods or gradient attribution methods, which rely on surrogate attribution.

[0032] As discussed above with respect to the system of FIG. 1 , the vision encoder 214 can be used in combination with a classifier to identify specific types of lesions in whole slide images and image patches. In one example, a classifier 220 trained to identify increased mitosis, necrosis, cellular infiltration, bile duct proliferation, fatty change, and hypertrophy using multilabel binary cross-entropy objective on the six lesions of interest can be used to obtain high-quality patch-level predictions, where each prediction describes the likelihood that a given lesion is present in the patch. A lesion correlation component 222 can utilize the outputs of the classifier 220 and the gene expression model 216, to establish relationships between the presence of each of the six lesions and the upregulation or downregulation of specific genes.

[0033] The classifier can also be trained to predict the severity of the lesion as well as its general type. The lesion correlation component 222 can measure a correlation between the measured slide-level expression and the predicted lesion size at the classifier 222 to observe a subset of genes shows a high correlation to establish precise and quantitative relationships between up and downregulation of genes and the extent of lesions. This allows the system 200 to automatically identify, quantify, and characterize the relationships between morphological lesions and gene expression changes. For example, the patch-level expression for patches associated with each lesion type can be used to extract the most upregulated and the most downregulated genes out of the gene targets. The average patch expression can be determined for each selected compound and then further averaged across all selected compounds. Genes with a mean expression above 1 Iog2 fold change (upregulated genes), and with a mean expression below -1 (downregulated genes) can be selected for each lesion type with the exception of increased mitosis, fatty change, cellular infiltration, and hypertrophy, for which a threshold of 0.5 and -0.5 were used. Varying this threshold controls the number of genes used for additional investigation.

[0034] The gene selection is then refined as a final step to focus on genes that are only differentially expressed for a single lesion. A gene is selected as specific to a lesion type if no other lesion is more differentially expressed than that type, its absolute average expression across the five other lesions is at least two times lower, and the absolute Pearson correlation between measured and predicted slide-level gene expressions, for slides identified as containing that lesion type, is above 0.3. Intuitively, the first condition screens an initial subset of genes associated with the lesion type, the second condition serves to refine this subset, where the constant c is selected as to identify genes with the largest correlation to the lesion type while removing those primarily indicative of generic toxic exposure effects. The third condition ensures that only the genes that can be satisfactorily predicted from tissue morphology are retained. Exceptions to these conditions include cellular infiltration, where the first condition is relaxed such that no other lesion is more differentially expressed than seventy-five percent of its value, as cellular infiltration often coappears with necrotic patches and bile duct proliferation. For the same reason, in the analysis of bile duct proliferation and necrosis, cellular infiltration is excluded from the first two conditions. The selected genes can be reported to a user via the user interface 218 for further investigation.

[0035] In view of the foregoing structural and functional features described above, methods in accordance with various aspects of the present invention will be better appreciated with reference to FIG. 3. While, for purposes of simplicity of explanation, the methods of FIG. 3 are shown and described as executing serially, it is to be understood and appreciated that the present invention is not limited by the illustrated order, as some aspects could, in accordance with the present invention, occur in different orders and / or concurrently with other aspects from that shown and described herein. Moreover, not all illustrated features may be required to implement a method in accordance with an aspect of the present invention.

[0036] FIG. 3 illustrates one example of a method 300 for toxicology assessment of a therapeutic. At 302, a plurality of animals each are provided with a dose of the therapeutic. In one example, a single dose is provided to each animal, although it will be appreciated that the animals can be divided into various groups that receive varying doses of the therapeutic, as well as a control group. In one implementation, three dose levels are tested for each compound, with a dose ratio between the low,middle, and high levels of 1 :3:10. In another example, one or more groups of the animals periodically receive additional doses at defined intervals (e.g., each day). At 304, an image of tissue is obtained from each of the plurality of animals to provide a plurality of sample images. It will be appreciated that the tissue for the images can be extracted at different times across the plurality of animals (e.g., three, six, nine, and twenty-four hours after administration of the therapeutic) to observe the progression of any lesions caused by the therapeutic.

[0037] At 306, each of the plurality of sample images to a vision encoder to provide a plurality of image representations, corresponding to the plurality of images, in an embedding space associated with the vision encoder. It will be appreciated that the vision encoder can be pretrained on a plurality of images of organ tissue and fine-tuned on annotated samples. At 308, a predicted gene expression associated with each of the plurality of sample images is generated from the corresponding image representation at a gene expression model to provide a plurality of pseudogene expression profiles. The gene expression model can be trained on paired samples of image data represented in the embedding space of the vision encoder and an actual gene expression for the tissue represented by the image data, as determined via a DNA microarray. In one example, a plurality of patches from each of the plurality of images are to the vision encoder to provide a plurality of patch embeddings associated with each image, and the gene expression associated with each of the plurality of images is determined as an average predicted gene expression across the plurality of patch embeddings.

[0038] At 310, an extent of the lesion in the image can be determined. For example, a classifier can be used to identify and segment specific types of lesions in the image and image patches, and a Iog2 fold change between the image and a control image can be determined. In one example, the classifier is trained to identify increased mitosis, necrosis, cellular infiltration, bile duct proliferation, fatty change, and hypertrophy using multilabel binary cross-entropy objective on the six lesions of interest can be used to obtain high-quality patch-level predictions, where each prediction describes the likelihood that a given lesion is present in a patch from the image. At 312, a predicted toxicity of the therapeutic in human beings is determined based on the plurality of pseudogene expression profiles. Specifically, a plurality of genes upregulated or downregulated by the therapeutic can be determined from thepredicted gene expression and assessed to see if the effects on the gene are likely to translate to human beings. Further, the known dosage and timing of associated with the various images can be used to determine a progression curve for lesion severity based on dosage that can be used to estimate the effects of a corresponding dose in human beings.

[0039] FIG. 4 is a schematic block diagram illustrating an exemplary system 400 of hardware components capable of implementing examples of the systems and methods disclosed herein. The system 400 can include various systems and subsystems. The system 400 can be a personal computer, a laptop computer, a workstation, a computer system, an appliance, an application-specific integrated circuit (ASIC), a server, a server BladeCenter, a server farm, etc.

[0040] The system 400 can include a system bus 402, a processing unit 404, a system memory 406, memory devices 408 and 410, a communication interface 412 (e.g., a network interface), a communication link 414, a display 416 (e.g., a video screen), and an input device 418 (e.g., a keyboard, touch screen, and / or a mouse). The system bus 402 can be in communication with the processing unit 404 and the system memory 406. The additional memory devices 408 and 410, such as a hard disk drive, server, standalone database, or other non-volatile memory, can also be in communication with the system bus 402. The system bus 402 interconnects the processing unit 404, the memory devices 406-410, the communication interface 412, the display 416, and the input device 418. In some examples, the system bus 402 also interconnects an additional port (not shown), such as a universal serial bus (USB) port.

[0041] The processing unit 404 can be a computing device and can include an application-specific integrated circuit (ASIC). The processing unit 404 executes a set of instructions to implement the operations of examples disclosed herein. The processing unit can include a processing core.

[0042] The additional memory devices 406, 408, and 410 can store data, programs, instructions, database queries in text or compiled form, and any other information that may be needed to operate a computer. The memories 406, 408 and 410 can be implemented as computer-readable media (integrated or removable), such as a memory card, disk drive, compact disk (CD), or server accessible over a network. In certain examples, the memories 406, 408 and 410 can include text,images, video, and / or audio, portions of which can be available in formats comprehensible to human beings.

[0043] Additionally or alternatively, the system 400 can access an external data source or query source through the communication interface 412, which can communicate with the system bus 402 and the communication link 414.

[0044] In operation, the system 400 can be used to implement one or more parts of a natural language system for analysis of pathology images. Computer executable logic for implementing the diagnostic system resides on one or more of the system memory 406, and the memory devices 408 and 410 in accordance with certain examples. The processing unit 404 executes one or more computer executable instructions originating from the system memory 406 and the memory devices 408 and 410. The term "computer readable medium" as used herein refers to a medium that participates in providing instructions to the processing unit 404 for execution. This medium may be distributed across multiple discrete assemblies all operatively connected to a common processor or set of related processors.

[0045] Implementation of the techniques, blocks, steps, and means described above can be done in various ways. For example, these techniques, blocks, steps, and means can be implemented in hardware, software, or a combination thereof. For a hardware implementation, the processing units can be implemented within one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, other electronic units designed to perform the functions described above, and / or a combination thereof.

[0046] Also, it is noted that the embodiments can be described as a process which is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although a flowchart can describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations can be re-arranged. A process is terminated when its operations are completed but could have additional steps not included in the figure. A process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to afunction, its termination corresponds to a return of the function to the calling function or the main function.

[0047] Furthermore, embodiments can be implemented by hardware, software, scripting languages, firmware, middleware, microcode, hardware description languages, and / or any combination thereof. When implemented in software, firmware, middleware, scripting language, and / or microcode, the program code or code segments to perform the necessary tasks can be stored in a machine-readable medium such as a storage medium. A code segment or machine-executable instruction can represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a script, a class, or any combination of instructions, data structures, and / or program statements. A code segment can be coupled to another code segment or a hardware circuit by passing and / or receiving information, data, arguments, parameters, and / or memory contents. Information, arguments, parameters, data, etc. can be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, ticket passing, network transmission, etc.

[0048] For a firmware and / or software implementation, the methodologies can be implemented with modules (e.g., procedures, functions, and so on) that perform the functions described herein. Any machine-readable medium tangibly embodying instructions can be used in implementing the methodologies described herein. For example, software codes can be stored in a memory. Memory can be implemented within the processor or external to the processor. As used herein the term "memory" refers to any type of long term, short term, volatile, nonvolatile, or other storage medium and is not to be limited to any particular type of memory or number of memories, or type of media upon which memory is stored.

[0049] Moreover, as disclosed herein, the term "storage medium" can represent one or more memories for storing data, including read only memory (ROM), random access memory (RAM), magnetic RAM, core memory, magnetic disk storage mediums, optical storage mediums, flash memory devices and / or other machine- readable mediums for storing information. The term "machine-readable medium" includes, but is not limited to, portable or fixed storage devices, optical storage devices, wireless channels, and / or various other storage mediums capable of storing that contain or carry instruction(s) and / or data.

[0050] In the preceding description, specific details have been set forth in order to provide a thorough understanding of example implementations of the invention described in the disclosure. However, it will be apparent that various implementations may be practiced without these specific details. For example, circuits, systems, networks, processes, and other components may be shown as components in block diagram form in order not to obscure the example implementations in unnecessary detail. In other instances, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail in order to avoid obscuring the examples. The description of the example implementations will provide those skilled in the art with an enabling description for implementing an example of the invention, but it should be understood that various changes may be made in the function and arrangement of elements without departing from the spirit and scope of the invention. Accordingly, the present invention is intended to embrace all such alterations, modifications, and variations that fall within the scope of the appended claims.

Claims

What is claimed is:1 . A system for providing decision support for toxicology assessment, the system comprising: a processor; and a non-transitory computer readable medium storing instructions executable by the processor, the machine-executable instructions comprising: an imager interface that receives an image of organ tissue of an animal; a vision encoder trained on a plurality of training images, each comprising an image of organ tissue of an animal, that receives the image of organ tissue and provides a representation of the image of organ tissue; and a toxicology front end that generates one of a value representing lesions within the image of organ tissue or an image having a similar representation from the representation of the image of organ tissue.

2. The system of claim 1 , wherein the image of organ tissue is a whole slide image and the toxicology front end comprises a classifier that provides at least one value representing the presence of one of a plurality of types of lesions in the organ tissue.

3. The system of claim 2, wherein the classifier is trained using an attention-based deep learning instance learning process, the classifier determining a value for each of a plurality of patch embeddings generated from the image of organ tissue and generates a weighted linear combination of the values for the plurality of patch embeddings to provide the value representing lesions within the image of organ tissue, the weights of the weighted linear combination being derived from respective attention scores associated with the plurality of patches.

4. The system of claim 1 , wherein the plurality of training images include a first set of images representing tissue from a first organ and a second set of images representing tissue from a second organ.

5. The system of claim 4, wherein each of the first organ and the second organ are selected to e relevant for drug-induced toxicity assessment.

6. The system of claim 1 , wherein the plurality of training images include a first set of images representing tissue from a first species of animal and a second set of images representing tissue from a second species of animal.

7. The system of claim 1 , wherein the vision encoder is pre-trained on the plurality of training images and then fine-tuned on a set of annotated patches, the set of annotated patches including both patches representing lesions and normal patches.

8. The system of claim 1 , wherein the image of organ tissue is a region of interest from a whole slide image and the toxicology front end comprises a classifier, trained on a set of images of organ tissue from animals and associated values representing a percentage of lesion area in the organ tissue, that provides at least one value representing a severity of lesioning within the region of interest.

9. The system of claim 8, wherein the classifier determines the value representing the severity of lesioning within the region of interest as a relative morphological change between the region of interest and an image representing a control group.

10. The system of claim 9, wherein the severity of lesioning is determined as the morphological Iog2 fold change in the region of interest.11 . The system of claim 1 , wherein the image of organ tissue is a region of interest within a whole slide image and the toxicology front end comprises a classifier that provides at least one value representing the presence of one of a plurality of types of lesions in the region of interest.

12. The system of claim 1 , wherein the image of organ tissue is a whole slide image and the toxicology front end comprises a retrieval component thatgenerates a representation of the image of organ tissue and generates a similarity score between the representation of the image of organ tissue and at least a subset of the plurality of training images to identity a training image that is most similar to the image of organ tissue.

13. The system of claim 12, wherein the representation of the image of organ tissue is generated by creating a plurality of representations, each representing one of a plurality of patches within the image of organ tissue in an embedding space associated with the vision encoder, and averaging the plurality of representations.

14. The system of claim 1 , wherein the toxicology front end is configured to retrieve a plurality of images sharing one of a lesion type or a metadata parameter with the image of organ tissue.

15. The system of claim 1 , wherein the toxicology front end is configured to retrieve a plurality of images from historical toxicity studies that are similar to the image of organ tissue.

16. The system of claim 1 , wherein the toxicology front end is configured to predict one of lesion severity or a progression curve for lesion severity from a specified compound, dose, and time.

17. The system of claim 1 , wherein an output of the toxicology front end is configured to be compliant with the Standard for Exchange of Nonclinical Data.

18. The system of claim 1 , wherein the image of organ tissue is a region of interest within a whole slide image and the toxicology front end comprises a retrieval component that generates a representation of the image of organ tissue and generates a similarity score between the representation of the image of organ tissue and at least a subset of the plurality of training images to identity a training image that is most similar to the image of organ tissue.19 The system of claim 1 , wherein the toxicology front end comprises a gene expression model, trained on a set of images and corresponding gene expression profiles, that produces a gene expression profile from the representation of the image of organ tissue.

20. A method for toxicology assessment of a therapeutic, the method comprising: providing a plurality of animals with a dose of the therapeutic; obtaining an image of tissue from each of the plurality of animals to provide a plurality of sample images; providing each of the plurality of sample images to a vision encoder to provide a plurality of image representations, corresponding to the plurality of images, in an embedding space associated with the vision encoder; determining a predicted gene expression associated with each of the plurality of sample images from the corresponding image representation at a gene expression model to provide a plurality of pseudogene expression profiles; and determining a predicted toxicity of the therapeutic in human beings based on the plurality of pseudogene expression profiles.21 . The method of claim 20, wherein providing each of the plurality of sample images to the vision encoder to provide the plurality of image representations comprises providing a plurality of patches from each of the plurality of images to the vision encoder to provide a plurality of patch embeddings associated with each image, and determining the predicted gene expression associated with each of the plurality of sample images comprises determining a predicted gene expression for each of the plurality of patch embeddings associated with each image and averaging the gene expression across the plurality of patch embeddings.

22. The method of claim 20, wherein the plurality of animals is a first plurality of animals, the dose of the therapeutic is a first dose, the plurality of sample images is a first plurality of sample images, the plurality of pseudogene expression profiles is a first plurality of psuedogene expression profiles, and providing the plurality of animals with the dose of the therapeutic comprises providing each of thefirst plurality of animals and a second plurality of animals with the first dose of the therapeutic at a first time, the method further comprising: providing each of the second plurality of animals with a second dose of the therapeutic at a second time after the first time; obtaining an image of tissue from each of the second plurality of animals to provide a second plurality of sample images; providing each of the second plurality of sample images to the vision encoder to provide a second plurality of image representations, corresponding to the second plurality of images, in an embedding space associated with the vision encoder; and determining a predicted gene expression associated with each of the second plurality of sample images from the corresponding image representation at a gene expression model to provide a second plurality of pseudogene expression profiles; wherein determining the predicted toxicity of the therapeutic in human beings comprises determining the predicted toxicity based on the first plurality of pseudogene expression profiles and the second plurality of pseudogene expression profiles.

23. A system for providing decision support for toxicology assessment, the system comprising: a processor; and a non-transitory computer readable medium storing instructions executable by the processor, the machine-executable instructions comprising: an imager interface that receives an image of organ tissue of an animal; a vision encoder trained on a plurality of training images, each comprising an image of kidney or liver tissue of an animal, that receives the image of organ tissue and provides a representation of the image of organ tissue; and a gene expression model, trained on a set of training images and corresponding gene expression profiles, that produces a gene expression profile from the representation of the image of organ tissue.

24. The system of claim 23, further comprising a classifier that provides at least one value representing the presence of one of a plurality of types of lesions in the image of organ tissue.

25. The system of claim 24, wherein the gene expression profile associated with each of the set of training images represents a difference between a representation of the training image in the embedding space of the vision encoder and a representation of an image representing a control group in the embedding space of the vision encoder.

26. The system of claim 25, wherein the difference between the representation of the training image in the embedding space of the vision encoder and the representation of the image representing the control group in the embedding space of the vision encoder are Iog2 normalized across the set of training images.

27. The system of claim 23, wherein the vision encoder encodes a plurality of patches from the image of organ tissue to provide a plurality of patch embeddings, and the gene expression model determines a predicted gene expression for each of the plurality of patch embeddings and averages the predicted gene expression across the plurality of patch embeddings to provide the gene expression profile.

Images