Deep learning model to determine a fixation status of a morphologically stained biological specimen

Abstract

The present disclosure describes systems and methods for estimating a fixation status of a morphologically stained test biological sample or any portion thereof treated for an unknown duration with one or more fixatives. For instance, systems and methods of the present disclosure may estimate that the fixation status of a morphologically stained test biological sample is one of "underfixed," "fixed," or "over fixed."

Description

DEEP LEARNING MODEL TO DETERMINE A FIXATION STATUS OF A MORPHOLOGICALLY STAINED BIOLOGICAL SPECIMENCROSS REFERENCE TO RELATED APPLICATIONS

[0001] The present disclosure claims the benefit of the filing date of United States Provisional Patent Application No. 63 / 533,079 filed on August 16, 2023, the disclosure of which is hereby incorporated by reference herein in its entirety.BACKGROUND OF THE DISCLOSURE

[0002] Thin tissue sections are used in histology in order to obtain representative information about a tissue sample. Formalin has been used in the histology field for over half a century. Formalin diffuses into a tissue section and reacts to form cross-links between proteins, nucleic acids, and other molecules, thereby halting metabolism, preserving biomolecules, and readying the tissue for paraffin wax infiltration. In practice, formalin fixation is done at room temperature or higher. Some groups perform fixation at slightly elevated temperatures to increase the cross-linking rate. Just as heat increases cross-linking rate, cold formalin significantly decreases cross-linking rate. For this reason, histologists typically perform tissue fixation at room temperature or higher.

[0003] Several effects are observed in tissues that are either under exposed or over exposed to formalin. If a tissue sample is not treated with formalin for a sufficiently long period of time (i.e., under exposed to formalin), tissue morphology is typically poor when the tissues are subjected to standard tissue processing. For example, in inadequately fixed tissue, subsequent exposure to ethanol shrinks the cellular structures and condenses nuclei since the tissues will not have the chance to form a proper cross-linked lattice. When under fixed tissue is stained, such as with hematoxylin and eosin (H&E), many white spaces are observed in between the cells and tissue structures. In addition, the nuclei are condensed, there is a loss of cytoplasm, and samples appear pink and unbalanced with the hematoxylin stain. Tissues that have been exposed to formalin too long (i.e., over exposed to formalin), typically do not stain well in downstream histochemical processes, presumably because of nucleic acid and / or protein denaturation and degradation. As a result, the tissue samples may appear under stained.

[0004] Proper medical diagnosis and patient safety rely upon properly fixing tissue samples prior to staining. Accordingly, guidelines have been established by oncologists and pathologists for proper fixation of tissue samples. For example, according to the American Societyof Clinical Oncology (ASCO), the current guideline for fixation duration in neutral buffered formalin solution for HER2 immunohistochemistry analysis is at least 6 hours, and up to 72 hours.BRIEF SUMMARY OF THE DISCLOSURE

[0005] It would be advantageous to develop a process for fixing tissue samples both to better preserve biological molecules and tissue morphology before significant degradation occurs, and to provide accurate test results to medical professionals and patients as quickly as possible. It would also be advantageous to have a method or system that permits an assessment of whether fixation of a sample was done properly, so that precious reagents used for immunohistochemical or in situ hybridization analyses are not wasted in attempting to stain tissue samples that are unlikely to yield acceptable results.

[0006] Variations in the fixation of a biological specimen may impact downstream processes, e.g., biomarker labeling processes, which may result in inconclusive results and / or a misdiagnosis. Advantageously, the disclosed systems and methods facilitate the estimation of a fixation status of a morphologically stained test biological sample or any portion thereof.

[0007] Applicant has surprisingly discovered that the systems and methods of the present disclosure provide for an accurate estimation of the fixation status of a morphologically stained test biological specimen or any portion thereof which was subjected to a fixation process for an unknown amount of time prior to morphological staining. Moreover, Applicant submits that the presently disclosed systems and methods facilitate a quantitative estimation of a fixation duration of a morphologically stained test biological specimen which was subjected to one or more fixation processes for unknown amounts of time. Applicant further submits that the presently disclosed trained fixation estimation engine provides for a highly accurate estimation of fixation status even when the trained fixation estimation engine is applied to different tissue types, sizes, or thicknesses in which the fixation estimation engine was not previously trained. These and other features are described further herein.

[0008] A first aspect of the present disclosure is a system for determining a fixation status of an at least partially fixed test biological specimen, the system comprising: (i) one or more processors, and (ii) one or more memories coupled to the one or more processors, the one or more memories to store computer-executable instructions that, when executed by the one or more processors, cause the system to perform operations comprising: obtaining test image data from theat least partially fixed test biological specimen, wherein the test specimen is stained with a morphological stain, and wherein the test image data includes image data derived from at least a portion of the at least partially fixed test biological specimen; deriving fixation features from the obtained test image data using a trained fixation estimation engine trained for the morphological stain; and determining the fixation status of the at least partially fixed test biological specimen based on the derived fixation features. In some embodiments, the derived fixation features include one or more of nuclear size, cellular distortion, inferior nuclear and cytoplasmic morphology, effects of tissue shrinkage (such as cellular distances and / or macro level morphology), cells with poor cellular delineation, altered nuclear detail, and / or distorted cellular organelles. In some embodiments, the estimated fixation status is a qualitative estimate of fixation quality. In some embodiments, the estimated fixation status is a quantitative estimate of fixation duration.

[0009] In some embodiments, the fixation estimation engine is trained using one or more training image data sets, such as one or more training image data sets acquired from a plurality of differentially fixed training biological specimens. In some embodiments, each training image in each of the one or more training image data sets is stained with the same morphological stain as the test biological specimen. In some embodiments, the training biological specimens are the same type of tissue as the test biological specimen; and stained with the same morphological stain. In some embodiments, the training biological specimens comprise a different type of tissue as the test biological specimen but stained with the same morphological stain.

[0010] In some embodiments, each training image in the training image data sets includes class labels of known fixation duration (e.g., 2 hours, 4, hours, 6 hours, 12 hours, 24 hours, etc.). In some embodiments, each training image in the one or more training image data sets includes class labels of known fixation duration (e.g., 2 hours, 4, hours, 6 hours, 12 hours, 24 hours, etc.) and fixation temperature (e.g., 4°C, 6°C, 10°C, 15°C, 20°C, 25°C, 30°C, 35°C, 40°C, 45°C, 50°C, etc.). In some embodiments, each training image in the one or more training image data sets comprises 2 different class labels of known fixation duration. In some embodiments, each training image in the one or more training image data sets comprises 3 different class labels of known fixation duration. In some embodiments, each training image in the one or more training image data sets comprises 4 different class labels of known fixation duration. In some embodiments, each training image in the one or more training image data sets comprises 5 different class labels of known fixation duration. In some embodiments, each training image in the one or more trainingimage data sets comprises 6 different class labels of known fixation duration. In some embodiments, each training image in the one or more training image data sets comprises 7 different class labels of known fixation duration. In some embodiments, each training image in the one or more training image data sets comprises 8 different class labels of known fixation duration. In some embodiments, each training image in the one or more training image data sets comprises 9 different class labels of known fixation duration. In some embodiments, each training image in the one or more training image data sets comprises 10 different class labels of known fixation duration. In some embodiments, the class labels of known fixation duration are a time period ranging from Oh to about 24h. In some embodiments, the class labels of known fixation duration are Oh, 2h, 6h, and 12h. In some embodiments, the class labels of known fixation duration are 12h. Oh, 2h, 6h, 12h, and 24h. In some embodiments, the class labels of known fixation duration are Oh, Ih, 2h, 6h, 12h, and 24h. In some embodiments, the class labels of known fixation duration are Oh, 2h, 4h, 6h, 12h, and 24h. In some embodiments, the class labels of known fixation duration are Oh, Ih, 2h, 4h, 6h, 12h, and 24h. In some embodiments, the class labels of known fixation duration are Oh, Ih, 2h, 4h, 6h, 12h, 16h, and 24h. In some embodiments, the class labels of known fixation duration are Oh, Ih, 2h, 4h, 6h, 12h, 16h, 20h, and 24h. In some embodiments, the class labels of known fixation duration are Oh, Ih, 2h, 4h, 6h, 8h, 12h, 16h, 20h, and 24h.

[0011] In some embodiments, each training image in the training image data sets comprise a class label of known fixation quality (e.g., underfixed, adequately fixed, overfixed, etc.). In some embodiments, each training image in the one or more training image data sets comprises images having one of at least 2 different class labels of known fixation quality. In some embodiments, each training image in the one or more training image data sets comprises images having one of at least 3 different class labels of known fixation quality. In some embodiments, each training image in the one or more image training data sets comprises 4 different class labels of known fixation quality. In some embodiments, the class labels of known fixation quality are selected from underfixed and adequately fixed. In some embodiments, the class labels of known fixation quality are selected from underfixed, adequately fixed, and overfixed. In some embodiments, the class labels of known fixation quality are selected from unfixed, underfixed, and adequately fixed. In some embodiments, the class labels of known fixation quality are selected from unfixed, underfixed, adequately fixed, and overfixed. In some embodiments, the class labels of knownfixation quality are selected from unfixed and overfixed. In some embodiments, the class labels of known fixation quality are selected from adequately fixed and overfixed.

[0012] In some embodiments, each training image in the one or more training image data sets includes class labels of known fixation duration and known fixation quality. In some embodiments, the class labels of known fixation quality are "unfixed," "underfixed," and "adequately fixed." In some embodiments, the class labels of known fixation quality are "unfixed," "underfixed," "adequately fixed," and "over fixed." In some embodiments, each training image in the one or more image training data sets includes class labels of known fixation duration, known fixation temperature, and known fixation quality. In some embodiments, each training image in the one or more image training data sets includes class labels of known fixation duration, known fixation quality, and known fixation temperature.

[0013] In some embodiments, each training image set data set is derived by: (i) obtaining a training biological specimen; (ii) dividing the obtained training biological specimen into a plurality of training tissue samples; and (iii) fixing each training tissue sample of the plurality of training tissue samples for a different pre-determined amount of time and / or at a pre-determined temperature. In some embodiments, the different pre-determined amounts of time range from between about 0 hours to about 24 hours. In some embodiments, the different pre-determined amounts of time range from between about 0 hours to about 18 hours. In some embodiments, the different pre-determined amounts of time range from between about 0 hours to about 16 hours. In some embodiments, the different pre-determined amounts of time range from between about 0 hours to about 12 hours. In some embodiments, the different pre-determined amounts of time range from between about 0 hours to about 9 hours.

[0014] In some embodiments, the trained fixation status estimation engine includes a neural network and / or a convolutional neural network. In some embodiments, the neural network includes DenseNet or a variation thereof. In some embodiments, the neural network includes MiniNet or a variation thereof. In some embodiments, the neural network operates on crossentropy or "log loss." In some embodiments, the training biological specimens comprise the same tissue type as the test biological specimen. In some embodiments, the training biological specimens comprise a different tissue type than the test biological specimen. In some embodiments, the training biological specimens comprise the same tissue shape and / or tissue sizeas the test biological specimen. In some embodiments, the training biological specimens comprise a different tissue shape and / or tissue size than the test biological specimen.

[0015] In some embodiments, the morphological stain is hematoxylin. In some embodiments, the morphological stain is eosin. In some embodiments, the morphological stain includes hematoxylin and eosin. In some embodiments, the morphological stain includes a "special stain," as that term is defined herein.

[0016] A second aspect of the present disclosure is a method for predicting a fixation duration of an at least partially fixed a morphologically stained test biological specimen comprising: obtaining test image data from the at least partially fixed test biological specimen, wherein the test specimen is stained with a morphological stain, and wherein the test image data includes image data derived from at least a portion of the at least partially fixed test biological specimen; deriving fixation features from the obtained test image data using a trained fixation estimation engine trained for a morphological stain; and determining the estimated fixation duration of the at least partially fixed test biological specimen based on the derived fixation features.

[0017] In some embodiments, the estimated fixation status is a qualitative estimate of fixation quality. In some embodiments, the estimated fixation status is a quantitative estimate of fixation duration.

[0018] In some embodiments, the fixation estimation engine is trained using one or more image training data sets acquired from a plurality of differentially fixed training biological specimens, and where each training image derived from the one or more image training data sets is stained with the same morphological stain as the test biological specimen. In some embodiments, each training image in the one or more image training data sets includes class labels of known fixation duration. In some embodiments, each training image in the one or more image training data sets includes class labels of known fixation quality. In some embodiments, the class labels of known fixation quality are unfixed, underfixed, and adequately fixed. In some embodiments, the class labels of known fixation quality are unfixed, underfixed, adequately fixed, and overfixed. In some embodiments, each training image in the one or more image training data sets includes class labels of known fixation duration and known fixation quality.

[0019] In some embodiments, each training image set data set is derived by: (i) obtaining a training biological specimen; (ii) dividing the obtained training biological specimen into aplurality of training tissue samples; and (iii) fixing each training tissue sample of the plurality of training tissue samples for a different pre-determined amount of time. In some embodiments, the different pre-determined amounts of time range from between about 0 hours to about 24 hours. In some embodiments, the different pre-determined amounts of time range from between about 0 hours to about 18 hours. In some embodiments, the different pre-determined amounts of time range from between about 0 hours to about 12 hours.

[0020] In some embodiments, the training biological specimen includes the same tissue type as the test biological specimen. In some embodiments, the training biological specimen includes a different same tissue type than the test biological specimen. In some embodiments, the training biological specimen includes the same tissue type as the test biological specimen, but where the training biological specimen and the test biological specimen have different sizes and / or different shapes.

[0021] In some embodiments, the method further includes assessing whether the test biological specimen includes a fixation level suitable for labeling with one or more specific binding entities, e.g., two or more specific binding entities, three or more specific binding entities, etc. In some embodiments, the method further includes assessing staining the test biological specimen for the presence of one or more biomarkers, e.g., two or more biomarkers, three or more biomarkers, etc. In some embodiments, the one or more biomarkers comprise cancer biomarkers, proliferation markers, cell type markers, etc.

[0022] In some embodiments, the trained fixation status estimation engine includes a neural network. In some embodiments, the neural network includes DenseNet or a variation thereof. In some embodiments, the neural network includes MiniNet or a variation thereof.

[0023] In some embodiments, the method further includes classifying connective tissue within the test biological specimen prior to the deriving of the fixation features from the obtained test image data. In some embodiments, the classifying of the connective tissue is performed using a trained connective tissue classifier. In some embodiments, the trained connective tissue classifier includes a neural network. In some embodiments, the neural network is DenseNet or MiniNet.

[0024] In some embodiments, the method further includes identifying connective tissue within training biological specimens using a connective tissue classifier prior to training the fixation estimation engine. In some embodiments, the connective tissue classifier comprises a neural network, such as a DenseNet or MiniNet.

[0025] In some embodiments, the morphological stain is hematoxylin. In some embodiments, the morphological stain is eosin. In some embodiments, the morphological stain includes hematoxylin and eosin. In some embodiments, the morphological stain includes a "special stain," as that term is defined herein.

[0026] A third aspect of the present disclosure is a non-transitory computer-readable medium storing instructions for determining an estimated fixation quality of an at least partially fixed test biological specimen, comprising: obtaining test image data from the at least partially fixed test biological specimen, wherein the test specimen is stained with a morphological stain, and wherein the test image data includes image data derived from at least a portion of the at least partially fixed test biological specimen; deriving fixation features from the obtained test image data using a trained fixation estimation engine trained for the morphological stain, wherein the fixation estimation engine is trained using one or more image training data sets acquired from a plurality of differentially fixed training biological specimens; and determining the estimated fixation quality of the at least partially fixed test biological specimen based on the derived fixation features.

[0027] In some embodiments, the morphological stain includes hematoxylin. In some embodiments, the trained fixation status estimation engine includes a neural network. In some embodiments, the neural network includes DenseNet or a variation thereof. In some embodiments, the neural network includes MiniNet or a variation thereof.

[0028] In some embodiments, the estimated fixation status is a qualitative estimate of fixation quality. In some embodiments, the estimated fixation status is a quantitative estimate of fixation duration.

[0029] In some embodiments, the fixation estimation engine is trained using one or more image training data sets acquired from a plurality of differentially fixed training biological specimens, and where each training image derived from the one or more image training data sets is stained with the same morphological stain as the test biological specimen. In some embodiments, each training image in the one or more image training data sets includes class labels of known fixation duration. In some embodiments, the class labels of known fixation duration are Oh, 2h, 6h, and 12h. In some embodiments, the class labels of known fixation duration are 12h. Oh, 2h, 6h, 12h, and 24h. In some embodiments, the class labels of known fixation duration are Oh, Ih, 2h, 6h, 12h, and 24h. In some embodiments, the class labels of known fixation durationare Oh, 2h, 4h, 6h, 12h, and 24h. In some embodiments, the class labels of known fixation duration are Oh, Ih, 2h, 4h, 6h, 12h, and 24h. In some embodiments, the class labels of known fixation duration are Oh, Ih, 2h, 4h, 6h, 12h, 16h, and 24h. In some embodiments, the class labels of known fixation duration are Oh, Ih, 2h, 4h, 6h, 12h, 16h, 20h, and 24h. In some embodiments, the class labels of known fixation duration are Oh, Ih, 2h, 4h, 6h, 8h, 12h, 16h, 20h, and 24h.

[0030] In some embodiments, each training image in the one or more image training data sets includes class labels of known fixation quality. In some embodiments, the class labels of known fixation quality are selected from: underfixed and adequately fixed; underfixed, adequately fixed, and overfixed; unfixed, underfixed, and adequately fixed; unfixed, underfixed, adequately fixed, and overfixed; unfixed and overfixed; or adequately fixed and overfixed.

[0031] In some embodiments, the non-transitory computer-readable medium further includes instructions for classifying connective tissue within the test biological specimen prior to the deriving of the fixation features from the obtained test image data.

[0032] In some embodiments, the non-transitory computer-readable medium further includes for classifying a glass slide upon which the test biological specimen is disposed prior to the deriving of the fixation features from the obtained test image data.

[0033] In some embodiments, the non-transitory computer-readable medium further includes for identifying one or more fields of view within the obtained test image data and / or for identifying one or more regions of interest within the obtained test image data.BRIEF DESCRIPTION OF THE FIGURES

[0034] For a general understanding of the features of the disclosure, reference is made to the drawings. In the drawings, like reference numerals have been used throughout to identify identical elements.

[0035] FIG. 1 illustrates a representative digital pathology system including an image acquisition device and a computer system in accordance with one embodiment of the present disclosure.

[0036] FIG. 2 sets forth various modules that can be utilized in a system or within a digital pathology workflow to estimate a fixation duration of a test biological sample in accordance with one embodiment of the present disclosure.

[0037] FIG. 3 sets forth a flowchart illustrating the various steps of estimate a fixation duration of a test biological specimen using a trained fixation estimation engine in accordance with one embodiment of the present disclosure.

[0038] FIG. 4A illustrates the division of individual training biological specimens into three training biological samples. Each of the training biological samples may be differentially fixed for pre-determined amounts of time in a fixative solution, such as an aldehyde-based fixative solution.

[0039] FIG. 4B illustrates the division of individual training biological specimens into three training biological samples. FIG. 4B further illustrates that each training biological sample may be differentially fixed for pre-determined amounts of time in a fixative solution, such as an aldehyde-based fixative solution. Following morphological staining (e.g., staining with hematoxylin), each differentially fixed training biological sample may then be used in the preparation of different training image data sets, such as for training a fixation estimation engine.

[0040] FIG. 4C illustrates the preparation of training image data sets derived from differentially fixed and morphologically stained training biological samples, such as training biological samples derived from training biological specimens.

[0041] FIG. 5 illustrates the acquisition of image data for test biological specimens according to one embodiment of the present disclosure.

[0042] FIG. 6 illustrates overlay masks applied to images of biological specimens, where the overlay masks show tissue that has been underfixed or adequately fixed. Also labeled are areas of the biological specimen including connective tissue.

[0043] FIG. 7 depicts overlay masks applied to images of biological specimens, where the overlay masks show tissue that has been underfixed or adequately fixed. Also labeled are areas of the biological specimen including connective tissue.

[0044] FIG. 8 illustrates overlay masks applied to images of biological specimens, where the overlay masks show tissue that is unfixed; tissue that has been underfixed; and tissue that has been adequately fixed. Also labeled are areas of the biological specimen including connective tissue.

[0045] FIG. 9 represents a flowchart illustrating the steps in training a fixation estimation engine.

[0046] FIG. 10 illustrates the classification of connective tissue and a glass slide using one or more classifiers, e.g., connective tissue classifiers, glass slide classifiers, etc.

[0047] FIG. 11 illustrates the annotation of two samples prior to the two sample being used to training a fixation estimate engine.DETAILED DESCRIPTION

[0048] It should also be understood that, unless clearly indicated to the contrary, in any methods claimed herein that include more than one step or act, the order of the steps or acts of the method is not necessarily limited to the order in which the steps or acts of the method are recited.

[0049] References in the specification to "one embodiment," "an embodiment," "an illustrative embodiment," etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may or may not necessarily include that particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.

[0050] As used herein, the singular terms "a," "an," and "the" include plural referents unless context clearly indicates otherwise. Similarly, the word "or" is intended to include "and" unless the context clearly indicates otherwise. The term "includes" is defined inclusively, such that "includes A or B" means including A, B, or A and B.

[0051] As used herein in the specification and in the claims, "or" should be understood to have the same meaning as "and / or" as defined above. For example, when separating items in a list, "or" or "and / or" shall be interpreted as being inclusive, for example, the inclusion of at least one, but also including more than one, of a number or list of elements, and, optionally, additional unlisted items. Only terms clearly indicated to the contrary, such as "only one of' or "exactly one of," or, when used in the claims, "consisting of," will refer to the inclusion of exactly one element of a number or list of elements. In general, the term "or" as used herein shall only be interpreted as indicating exclusive alternatives (for example "one or the other but not both") when preceded by terms of exclusivity, such as "either," "one of," "only one of or "exactly one of." "Consistingessentially of," when used in the claims, shall have its ordinary meaning as used in the field of patent law.

[0052] The terms "comprising," "including," "having," and the like are used interchangeably and have the same meaning. Similarly, "comprises," "includes," "has," and the like are used interchangeably and have the same meaning. Specifically, each of the terms is defined consistent with the common United States patent law definition of "comprising" and is therefore interpreted to be an open term meaning "at least the following," and is also interpreted not to exclude additional features, limitations, aspects, etc. Thus, for example, "a device having components a, b, and c" means that the device includes at least components a, b, and c. Similarly, the phrase: "a method involving steps a, b, and c" means that the method includes at least steps a, b, and c. Moreover, while the steps and processes may be outlined herein in a particular order, the skilled artisan will recognize that the ordering steps and processes may vary.

[0053] As used herein in the specification and in the claims, the phrase "at least one," in reference to a list of one or more elements, should be understood to mean at least one element selected from any one or more of the elements in the list of elements, but not necessarily including at least one of each and every element specifically listed within the list of elements and not excluding any combinations of elements in the list of elements. This definition also allows that elements may optionally be present other than the elements specifically identified within the list of elements to which the phrase "at least one" refers, whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, "at least one of A and B" (or, equivalently, "atleast one of A orB," or, equivalently "at least one of A and / or B") can refer, in one embodiment, to at least one, optionally including more than one, A, with no B present (and optionally including elements other than B); in another embodiment, to at least one, optionally including more than one, B, with no A present (and optionally including elements other than A); in yet another embodiment, to at least one, optionally including more than one, A, and at least one, optionally including more than one, B (and optionally including other elements); etc.

[0054] As used herein, the term "about" means + / - 1%, + / - 2%, + / - 3%, + / -4, + / - 5%, 6%, + / - 7%, + / - 8%, + / - 9%, + / - 10%, + / - 15%, + / - 20%, etc. of the revised value or attribute.

[0055] As used herein, the terms "biological specimen," "biological sample," "sample," "tissue specimen," or "tissue sample" refer to any sample including a biomolecule (such as a protein, a peptide, a nucleic acid, a lipid, a carbohydrate, or a combination thereof) that is obtainedfrom any organism including viruses. Other examples of organisms include mammals (such as humans; veterinary animals like cats, dogs, horses, cattle, and swine; and laboratory animals like mice, rats, and primates), insects, annelids, arachnids, marsupials, reptiles, amphibians, bacteria, and fungi. Biological specimens include tissue samples (such as tissue sections and needle biopsies of tissue), cell samples (such as cytological smears such as Pap smears or blood smears or samples of cells obtained by microdissection), or cell fractions, fragments, or organelles (such as obtained by lysing cells and separating their components by centrifugation or otherwise). Other examples of biological specimens include blood, serum, urine, semen, fecal matter, cerebrospinal fluid, interstitial fluid, mucous, tears, sweat, pus, biopsied tissue (for example, obtained by a surgical biopsy or a needle biopsy), nipple aspirates, cerumen, milk, vaginal fluid, saliva, swabs (such as buccal swabs), or any material containing biomolecules that is derived from a first biological specimen. In certain embodiments, the term "biological specimen" as used herein refers to a sample (such as a homogenized or liquefied sample) prepared from a tumor or a portion thereof obtained from a subject.

[0056] As used herein, the terms "biomarker" or "marker" refer to a measurable indicator of some biological state or condition. In particular, a biomarker may be a protein or peptide, e.g., a surface protein, which can be specifically stained, and which is indicative of a biological feature of the cell, e.g., the cell type or the physiological state of the cell. An immune cell marker is a biomarker that is selectively indicative of a feature that relates to an immune response of a mammal. A biomarker may be used to determine how well the body responds to a treatment for a disease or condition or if the subject is predisposed to a disease or condition. In the context of cancer, a biomarker refers to a biological substance that is indicative of the presence of cancer in the body. A biomarker may be a molecule secreted by a tumor or a specific response of the body to the presence of cancer. Genetic, epigenetic, proteomic, glycomic, and imaging biomarkers can be used for cancer diagnosis, prognosis, and epidemiology. Such biomarkers can be assayed in non-invasively collected biofluids like blood or serum. Several gene and protein based biomarkers have already been used in patient care including but, not limited to, AFP (Liver Cancer), BCR- ABL (Chronic Myeloid Leukemia), BRCA1 / BRCA2 (Breast / Ovarian Cancer), BRAF V600E (Melanoma / Colorectal Cancer), CA-125 (Ovarian Cancer), CA19.9 (Pancreatic Cancer), CEA (Colorectal Cancer), EGFR (Non-small-cell lung carcinoma), HER-2 (Breast Cancer), KIT (Gastrointestinal stromal tumor), PSA (Prostate Specific Antigen), SI 00 (Melanoma), and manyothers. Biomarkers may be useful as diagnostics (to identify early-stage cancers) and / or prognostics (to forecast how aggressive a cancer is and / or predict how a subject will respond to a particular treatment and / or how likely a cancer is to recur).

[0057] As used herein, the term "cell" refers to a prokaryotic cell or a eukaryotic cell. The cell may be an adherent or a non-adherent cell, such as an adherent prokaryotic cell, adherent eukaryotic cell, non-adherent prokaryotic cell, or non-adherent eukaryotic cell. A cell may be a yeast cell, a bacterial cell, an algae cell, a fungal cell, or any combination thereof. A cell may be a mammalian cell. A cell may be a primary cell obtained from a subject. A cell may be a cell line or an immortalized cell. A cell may be obtained from a mammal, such as a human or a rodent. A cell may be a cancer or tumor cell. A cell may be an epithelial cell. A cell may be a red blood cell or a white blood cell. A cell may be an immune cell such as a T cell, a B cell, a natural killer (NK) cell, a macrophage, a dendritic cell, or others. A cell may be a neuronal cell, a glial cell, an astrocyte, a neuronal support cell, a Schwann cell, or others. A cell may be an endothelial cell. A cell may be a fibroblast or a keratinocyte. A cell may be a pericyte, hepatocyte, a stem cell, a progenitor cell, or others. A cell may be a circulating cancer or tumor cell or a metastatic cell. A cell may be a marker specific cell such as a CD8+ T cell or a CD4+ T cell. A cell may be a neuron. A neuron may be a central neuron, a peripheral neuron, a sensory neuron, an interneuron, an intraneuronal, a motor neuron, a multipolar neuron, a bipolar neuron, or a pseudo-unipolar neuron. A cell may be a neuron supporting cell, such as a Schwann cell. A cell may be one of the cells of a blood-brain barrier system. A cell may be a cell line, such as a neuronal cell line. A cell may be a primary cell, such as cells obtained from a brain of a subject. A cell may be a population of cells that may be isolated from a subject, such as a tissue biopsy, a cytology specimen, a blood sample, a fine needle aspirate (FNA) sample, or any combination thereof. A cell may be obtained from a bodily fluid such as urine, milk, sweat, lymph, blood, sputum, amniotic fluid, aqueous humor, vitreous humor, bile, cerebrospinal fluid, chyle, chyme, exudates, endolymph, perilymph, gastric acid, mucus, pericardial fluid, peritoneal fluid, pleural fluid, pus, rheum, saliva, sebum, serous fluid, smegma, sputum, tears, vomit, or other bodily fluid. A cell may comprise cancerous cells, non-cancerous cells, tumor cells, non-tumor cells, healthy cells, or any combination thereof.

[0058] As used herein, the term "cytological sample" refers to a cellular sample in which the cells of the sample have been partially or completely disaggregated, such that the sample no longer reflects the spatial relationship of the cells as they existed in the subject from which thecellular sample was obtained. Examples of cytological samples include tissue scrapings (such as a cervical scraping), fine needle aspirates, samples obtained by lavage of a subject, et cetera.

[0059] As used herein, the term "fixation" refers to a process by which molecular and / or morphological details of a cellular sample are preserved. There are generally three kinds of fixation processes: (1) heat fixation, (2) perfusion; and (3) immersion. With heat fixation, samples are exposed to a heat source for a sufficient period of time to heat kill and adhere the sample to the slide. Perfusion involves use of the vascular system to distribute a chemical fixative throughout a whole organ or a whole organism. Immersion involves immersing a sample in a volume of a chemical fixative and allowing the fixative to diffuse throughout the sample. Chemical fixation involves diffusion or perfusion of a chemical throughout the cellular samples, where the fixative reagent causes a reaction that preserves structures (both chemically and structurally) as close to that of living cellular sample as possible. Chemical fixatives can be classified into two broad classes based on mode of action: cross-linking fixatives and non-cross-linking fixatives. Crosslinking fixatives - typically aldehydes - create covalent chemical bonds between endogenous biological molecules, such as proteins and nucleic acids, present in the tissue sample. Formaldehyde is the most commonly used cross-linking fixative in histology. Formaldehyde may be used in various concentrations for fixation, but it primarily is used as 10% neutral buffered formalin (NBF), which is about 3.7% formaldehyde in an aqueous phosphate buffered saline solution. Paraformaldehyde is a polymerized form of formaldehyde, which depolymerizes to provide formalin when heated. Glutaraldehyde operates in similar manner as formaldehyde but is a larger molecule having a slower rate of diffusion across membranes. Glutaraldehyde fixation provides a more rigid or tightly linked fixed product, causes rapid and irreversible changes, fixes quickly and well at 4 °C, provides good overall cytoplasmic and nuclear detail, but is not ideal for immunohistochemistry staining. Some fixation protocols use a combination of formaldehyde and glutaraldehyde. Glyoxal and acrolein are less commonly used aldehydes. Denaturation fixatives - typically alcohols or acetone - act by displacing water in the cellular sample, which destabilizes hydrophobic and hydrogen bonding within proteins. This causes otherwise water-soluble proteins to become water insoluble and precipitate, which is largely irreversible.

[0060] As used herein, the term "immunohistochemistry" refers to a method of determining the presence or distribution of an antigen in a sample by detecting interaction of the antigen with a specific binding agent, such as an antibody. A sample is contacted with an antibodyunder conditions permitting antibody-antigen binding. Antibody-antigen binding can be detected by means of a detectable label conjugated to the antibody (direct detection) or by means of a detectable label conjugated to a secondary antibody, which binds specifically to the primary antibody (indirect detection). In some instances, indirect detection can include tertiary or higher antibodies that serve to further enhance the detectability of the antigen. Examples of detectable labels include enzymes, fluorophores and haptens, which in the case of enzymes, can be employed along with chromogenic or fluorogenic substrates.

[0061] As used herein, the term "machine learning" refers to a type of learning in which the machine (e.g., computer program) can learn on its own without being programmed.

[0062] As used herein, the term "slide" refers to any substrate (e.g., substrates made, in whole or in part, glass, quartz, plastic, silicon, etc.) of any suitable dimensions on which a biological specimen is placed for analysis, and more particularly to a "microscope slide" such as a standard 3 inch by 1 inch microscope slide or a standard 75 mm by 25 mm microscope slide. Examples of biological specimens that can be placed on a slide include, without limitation, a cytological smear, a thin tissue section (such as from a biopsy), and an array of biological specimens, for example a tissue array, a cellular array, aDNA array, an RNA array, a protein array, or any combination thereof. Thus, in one embodiment, tissue sections, DNA samples, RNA samples, and / or proteins are placed on a slide at particular locations. In some embodiments, the term slide may refer to SELDI and MALDI chips, and silicon wafers.

[0063] As used herein the term "specific binding entity" refers to a member of a specificbinding pair. Specific binding pairs are pairs of molecules that are characterized in that they bind each other to the substantial exclusion of binding to other molecules (for example, specific binding pairs can have a binding constant that is at least 103M'1greater, 104M'1greater or 105M'1greater than a binding constant for either of the two members of the binding pair with other molecules in a tissue sample). Particular examples of specific binding moieties include specific binding proteins (for example, antibodies, lectins, avidins such as streptavidins, and protein A). Specific binding moieties can also include the molecules (or portions thereof) that are specifically bound by such specific binding proteins.

[0064] As used herein, the term "substantially" means the qualitative condition of exhibiting total or near-total extent or degree of a characteristic or property of interest. In some embodiments, "substantially" means within 1%. In some embodiments, "substantially" meanswithin 2%. In some embodiments, "substantially" means within 3%. In some embodiments, "substantially" means within 4%. In some embodiments, "substantially" means within 5%. In some embodiments, "substantially" means within 10%. In some embodiments, "substantially" means within 15%. In some embodiments, "substantially" means within 20%.

[0065] OVERVIEW

[0066] It is believed that fixation quality and / or fixation duration, may have an impact on downstream analytical methods (e.g., histochemical or cytochemical staining methods and / or analysis). For example, under current clinical practice it is important to control the tissue fixation duration to achieve a compromise between the preservation of tissue morphology and the loss of antigenicity. Indeed, a fixation duration that is either too short or too long may negatively impact downstream sample processing, either of the entire sample or one or more portions thereof. Thus, there remains a need for an accurate prediction of fixation quality and / or fixation duration of a biological sample prior to downstream processing, e.g., prior to contacting the biological sample with one or more specific binding entities and / or staining the biological sample for the presence of one or more biomarkers (e.g., ER, PR, PD-L1, CD markers, etc ).

[0067] In some embodiments, the present disclosure describes systems and methods for estimating a fixation status of a morphologically stained test biological sample or any portion thereof treated for an unknown duration (and / or at an unknown temperature) with one or more fixatives. For instance, in some embodiments the systems and methods of the present disclosure may estimate that the fixation status of a morphologically stained test biological sample is one of "underfixed," "adequately fixed," or "fixed." In other embodiments, the systems and methods of the present disclosure may estimate that the fixation status of a morphologically stained test biological sample is one of "underfixed," "adequately fixed," "fixed," or "over fixed." In yet other embodiments, the systems and methods of the present disclosure may estimate that the fixation status of a morphologically stained test biological sample is one of "unfixed," "underfixed," "fixed," or "over fixed." In some embodiments, a "fixed" specimen may be one that substantially meets ASCO standards.

[0068] In some embodiments, the systems and methods of the present disclosure may provide a quantitative estimate of a fixation duration, i.e., the amount of time a morphologically stained test biological specimen has been subjected to a fixation process. By way of example, the quantitative estimate of fixation duration may be one of about 0 hours, about 1 hour, about 2 hours,about 3 hours, about 4 hours, about 6 hours, about 8 hours, about 12 hours, about 16 hours, about 20 hours, about 24 hours, etc.

[0069] In some embodiments, the systems and methods of the present disclosure may provide both a qualitative estimate of fixation quality and a quantitative estimate of a fixation duration.

[0070] The present disclosure also provides systems and methods for training a fixation estimation engine whereby the resulting trained fixation estimation engine may be utilized to determine a fixation status (i.e., quality and / or fixation duration) of a morphologically stained test biological specimen. As will be described herein, the fixation estimation engine may be trained using training image data sets derived from morphologically stained training biological samples, where the morphologically stained training biological samples have been differentially fixed for predetermined amounts of time.

[0071] At least some embodiments of the present disclosure relate to computer systems and methods for analyzing image data acquired from morphologically stained biological specimens which have been subjected to a fixation process for either an estimated period of time or an unknown period of time. For instance, a trained fixation estimation engine trained on a particular morphological stain (e.g., hematoxylin, eosin, a "special stain," etc.) may be used to provide an estimation of a fixation status of a morphologically stained test biological specimen where the duration of fixation of the test biological specimen is either unknown or can only be roughly estimated (e.g., estimated to be fixed for a timer period of under 6 hours). Additionally, the trained fixation estimation engine may be used to verify a fixation status or a duration of fixation of a morphologically stained test biological specimen that was subjected to one or more fixation processes for unknown amounts of time. For example, if a received morphologically stained test biological specimen includes a notation that the specimen was fixed for 10 hours, the systems and methods of the present disclosure may be used to verify the fixation duration noted and / or to provide a further qualitative or quantitative assessment of the fixation status of the test biological sample. In this manner, an assessment of a biological sample or any portion thereof may be made so as to determine whether the sample is ready for downstream processing and / or analysis, e.g., whether the test biological specimen is in a state of fixation suitable for labeling with one or more specific binding entities.

[0072] The present disclosure also relates to computer systems and methods for analyzing image data from morphologically stained specimens which have been subjected to a fixation process for a known duration. For instance, a fixation estimation engine may be trained using a plurality of training samples that have been fixed for known durations of time.

[0073] A system 200 for acquiring image data and estimating a fixation status of a test biological specimen or for training a fixation estimation engine is illustrated in FIGS. 1 and 2. The system may include an image acquisition device 12 and a computer 14, whereby the image acquisition device 12 and computer may be communicatively coupled together (e.g., directly, or indirectly over a network 20). The computer system 14 can include a desktop computer, a laptop computer, a tablet, or the like, digital electronic circuitry, firmware, hardware, memory 201, a computer storage medium (240), a computer program or set of instructions (e.g., where the program is stored within the memory or storage medium), one or more processors (209) (including a programmed processor), and any other hardware, software, or firmware modules or combinations thereof (such as described further herein). For example, the system 14 illustrated in FIG. 1 may include a computer with a display device 16 and an enclosure 18. The computer system can store acquired image data locally, such as in a memory, on a server, or another network connected device.

[0074] In some embodiments, images captured from the image acquisition device 12 may be stored in binary form, such as locally or on a server. The digital images can also be divided into a matrix of pixels. In some embodiments, the pixels can include a digital value of one or more bits, defined by the bit depth. In general, the imaging apparatus (or other image source including pre-scanned images stored in a memory) can include, without limitation, one or more image capture devices. Image capture devices can include, without limitation, a camera (e.g., an analog camera, a digital camera, etc ), optics (e.g., one or more lenses, sensor focus lens groups, microscope objectives, etc.), imaging sensors (e.g., a charge-coupled device (CCD), a complimentary metal-oxide semiconductor (CMOS) image sensor, or the like), photographic film, or the like. In digital embodiments, the image capture device can include a plurality of lenses that cooperate to prove on-the-fly focusing. An image sensor, for example, a CCD sensor can capture a digital image of the specimen. In some embodiments, the imaging apparatus is a brightfield imaging system, a multispectral imaging (MSI) system or a fluorescent microscopy system. The digitized tissue data may be generated, for example, by an image scanning system, such as aVentana DP 200® slide scanner by Ventana Medical Systems, Inc. (Tucson, Arizona) or other suitable imaging equipment. Additional imaging devices and systems are described further herein. The skilled artisan will appreciate that the digital color image acquired by the imaging apparatus is conventionally composed of elementary color pixels. In some embodiments, each colored pixel can be coded over three digital components, each comprising the same number of bits, each component corresponding to a primary color, generally red, green, or blue, also denoted by the term "RGB" components.

[0075] The skilled artisan will appreciate that other computer devices or systems may be utilized and that the computer systems described herein may be communicatively coupled to additional components, e.g., microscopes, automated slide preparation equipment, specimen milling / dissection devices, automated staining devices, analytical devices, sequencing devices, etc. Some of these additional components and the various computers, networks, etc. that may be utilized are described further herein.

[0076] FIG. 2 provides an overview of the system 200 of the present disclosure and the various modules utilized within the system. In some embodiments, the system 200 employs a computer device or computer-implemented method having one or more processors 209 and one or more memories 201, the one or more memories 201 storing non-transitory computer-readable instructions for execution by the one or more processors to cause the one or more processors to execute certain instructions as described herein.

[0077] In some embodiments, and as noted above, the system includes an image acquisition module 202 for acquiring image data (e.g., RGB image data) from an obtained morphologically stained biological specimen (see, e.g., step 310 of FIG. 3) or any portion thereof (see, e.g., step 320 of FIG. 3). In some embodiments, the obtained morphologically stained biological specimen is a training biological specimen, e.g., a biological specimen which has been morphologically stained to train a fixation estimation engine. In other embodiments, morphologically stained biological specimen is a test biological specimen, e.g., a biological specimen where it is desirable to quantitatively or qualitatively understand the fixation status of the specimen.

[0078] In some embodiments, the system 200 further includes an image processing module 212 adapted to process acquired image data. In some embodiments, the image processing module 212 is configured to pre-process image data, to identify regions of the image that correspond tothe substrate (e.g., a microscope slide) on which the sample is disposed, to identify regions of different tissue types (e.g., connective tissue), or to interpret one or more annotations. In some embodiments, the image processing module 212 includes one or more submodules, such as tissue classification modules, glass recognition modules, etc. The one or more submodules may implement support vector machines and / or neural networks. Examples of overlay generation modules, tissue classification modules, glass / slide recognition modules are described in U.S. Publication Nos. 2020 / 0105413, 2021 / 0027462, 2021 / 0216746, 2021 / 0285056 and in U.S. Patent Nos. 11,010,892 and 10,628,658, the disclosures of which are each hereby incorporated by reference herein in their entireties.

[0079] In some embodiments, the system 200 further includes a training module 211 adapted to receive training image data and to use the received training image data derived from morphologically stained trained biological specimens to train a fixation estimation engine 210. In some embodiments, the system 200 includes a fixation estimation engine 210 which is trained to detect fixation features within test image data derived from morphologically stained test biological specimens (see, e.g., step 340 of FIG. 3) and provide an estimate of a fixation status (qualitative) and / or fixation duration (quantitative) based on the detected fixation features (see, e.g., step 350 of FIG. 3). In some embodiments, fixation features that may be detected include nuclear detail, cellular morphology, and tissue distortion. In other embodiments, the fixation features include nuclear size, cellular distortion, inferior nuclear and cytoplasmic morphology, effects of tissue shrinkage (such as cellular distances and macro level morphology), cells with poor cellular delineation, altered nuclear detail, and / or distorted cellular organelles.

[0080] In some embodiments, the trained fixation estimation engine 210 includes one or more machine-learning algorithms. In some embodiments, one or more machine-learning algorithms is based on convolutional neural networks, such as ResNet, DenseNet, or MiniNet (or is derived from one of these exemplary neural networks).

[0081] The skilled artisan will also appreciate that additional modules may be incorporated into the workflow or into system 200. In some embodiments, one or more an automated algorithms may be run such that cells may be detected, classified, and / or scored (see, e.g., United States Patent Publication No. 2017 / 0372117, the disclosure of which is hereby incorporated by reference herein in its entirety).

[0082] IMAGE ACQUISITION MODULE AND ACQUIRED IMAGE DATA

[0083] With reference to FIG. 2, in some embodiments, the system 200 runs an image acquisition module 202 to capture image data of at least a portion of an at least partially fixed biological specimen stained with a morphological stain. In some embodiments, the at least partially fixed biological specimen stained with a morphological stain is a test biological specimen. In other embodiments, the at least at least partially fixed biological specimen stained with a morphological stain is a training biological specimen. Once the image data from the at least partially fixed and morphologically stained test or training biological specimen is acquired, the acquired image data may be provided to one or more memories 201 or may be stored in a storage module 240 (e.g., a local storage module or a networked storage module). The image data stored in the one or more memories 201 or within the storage module 240 may be retrieved for further downstream processing.

[0084] In some embodiments, image data may be acquired for an entire test or training biological specimen. For instance, image data may be acquired for the entirety of a test or training biological specimen disposed on a substrate, e.g., a microscope slide. In other embodiments, the image data may be acquired from a portion of a test or training biological specimen. This can be useful where only specific regions of interest of the test or training biological specimen are relevant for analysis. For instance, certain regions of interest may include a specific type of tissue or a comparatively higher population of a specific type of cell as compared with another region of interest. By way of example, a region of interest may be selected in a training biological specimen that includes tissue of interest (e.g., tonsil tissue) but excludes connective tissue. In these embodiments, the image acquisition module 202 (or any submodules thereof) may be programmed to acquire the image data from one or more predefined portions of the sample; or may acquire image data through random sampling or by sampling at regular intervals across a grid covering the entire sample. In some embodiments, a connective tissue classifier is run such that connective tissue is identified; and the identified areas of connective tissue may be excluded from subsequent processing and / or training routines.

[0085] Training Biological Specimens

[0086] In some embodiments, the acquired image data is derived from training biological specimens or training biological samples derived therefrom. The obtained training biological specimens may be obtained from any source. For instance, the obtained training biological specimens may be obtained from a tumor, including, for example, tumor biopsies samples,resection samples, cell smears, fine needle aspirates (FNA), liquid-based cytology samples, and the like.

[0087] In some embodiments, the acquired training image data is used to train a fixation estimation engine 210, such as by using the training module 211 described herein. In some embodiments, the acquired training image data includes, is associated with, or is combined with one or more class labels, such as fixation quality labels and / or fixation duration labels. In some embodiments, the acquired training image data includes, is associated with, or is combined with two different class labels. In some embodiments, the acquired training image data includes, is associated with, or is combined with three different class labels. In some embodiments, the acquired training image data includes, is associated with, or is combined with four different class labels. In some embodiments, the acquired training image data includes, is associated with, or is combined with five different class labels. In some embodiments, the acquired training image data includes, is associated with, or is combined with six different class labels. In some embodiments, the acquired training image data includes, is associated with, or is combined with seven different class labels. In some embodiments, the acquired training image data includes, is associated with, or is combined with eight different class labels. In some embodiments, the acquired training image data includes, is associated with, or is combined with nine different class labels. In some embodiments, the acquired training image data includes, is associated with, or is combined with ten different class labels. In some embodiments, the acquired training image data includes, is associated with, or is combined with eleven different class labels. In some embodiments, the acquired training image data includes, is associated with, or is combined with twelve or more different class labels.

[0088] In some embodiments, the acquired training image data includes, is associated with, or is combined with one or more class labels of known fixation duration and / or one or more class labels of known fixation temperature. In some embodiments, the training image data sets comprise 2 different class labels of known fixation duration. In some embodiments, the training image data sets comprise 3 different class labels of known fixation duration. In some embodiments, the training image data sets comprise 4 different class labels of known fixation duration. In some embodiments, the training image data sets comprise 5 different class labels of known fixation duration. In some embodiments, the training image data sets comprise 6 different class labels of known fixation duration. In some embodiments, the training image data sets comprise 7 differentclass labels of known fixation duration. In some embodiments, the training image data sets comprise 8 different class labels of known fixation duration. In some embodiments, the training image data sets comprise 9 different class labels of known fixation duration. In some embodiments, the training image data sets comprise 10 different class labels of known fixation duration.

[0089] In some embodiments, the class labels of known fixation duration include a known time period (known fixation duration) for fixation ranging from Oh to 32h, such as Oh to 24h. In some embodiments, the class labels are selected from: Ih, 2h, 3h, 4h, 5h, 6h, 7h, 8h, 9h, lOh, 1 Ih, 12h, 13h, 14h, 15h, 16h, 17h, 18h, 19h, 20h, 21h, 22h, 23h, 24h, etc. By way of example, the class labels of known fixation duration are Oh, 2h, 6h, and 12h. In some embodiments, the class labels of known fixation duration are 12h. Oh, 2h, 6h, 12h, and 24h. In some embodiments, the class labels of known fixation duration are Oh, Ih, 2h, 6h, 12h, and 24h. In some embodiments, the class labels of known fixation duration are Oh, 2h, 4h, 6h, 12h, and 24h. In some embodiments, the class labels of known fixation duration are Oh, Ih, 2h, 4h, 6h, 12h, and 24h. In some embodiments, the class labels of known fixation duration are Oh, Ih, 2h, 4h, 6h, 12h, 16h, and 24h. In some embodiments, the class labels of known fixation duration are Oh, Ih, 2h, 4h, 6h, 12h, 16h, 20h, and 24h. In some embodiments, the class labels of known fixation duration are Oh, Ih, 2h, 4h, 6h, 8h, 12h, 16h, 20h, and 24h.

[0090] In some embodiments, the acquired training image data includes, is associated with, or is combined with one or more class labels of known fixation quality. In some embodiments, the training image data sets comprise 2 different class labels of known fixation quality. In some embodiments, the training image data sets comprise 3 different class labels of known fixation quality. In some embodiments, the training image data sets comprise 4 different class labels of known fixation quality. In some embodiments, the class labels of known fixation quality are selected from underfixed and adequately fixed; underfixed, adequately fixed, and overfixed. In some embodiments, the class labels of known fixation quality are selected from unfixed, underfixed, and adequately fixed. In some embodiments, the class labels of known fixation quality are selected from unfixed, underfixed, adequately fixed, and overfixed. In some embodiments, the class labels of known fixation quality are selected from unfixed and overfixed. In some embodiments, the class labels of known fixation quality are selected from adequately fixed andoverfixed. Yet other labels may be attributed to any acquired training image data sets that describe the fixation quality of the training sample to the person of ordinary skill in the art.

[0091] In some embodiments, each training image in the training image data sets includes one or more class labels of known fixation duration and / or one or more class labels of known fixation quality. In some embodiments, the class labels of known fixation quality are unfixed, underfixed, and adequately fixed. In some embodiments, the class labels of known fixation quality are unfixed, underfixed, adequately fixed, and overfixed. In some embodiments, the class labels of known fixation duration are Oh, Ih, 2h, 6h, 12h, and 24h. In some embodiments, the class labels of known fixation duration are Oh, 2h, 4h, 6h, 12h, and 24h. In some embodiments, each training image in the training image data sets includes one or more class labels of known fixation temperature (e.g., 4°C, 6°C, 10°C, 15°C, 20°C, 25°C, 30°C, 35°C, 40°C, 45°C, 50°C, etc.).

[0092] In some embodiments, at least two training biological specimens are obtained, and each of the at least two obtained training biological specimens are differentially fixed. Differential fixation is a process whereby each sample of a set of obtained samples are each subjected to a different fixation process, e.g., a fixation processes having different durations (e.g., between Oh and 32h) and / or at different temperatures (e.g., 4°C, 6°C, 10°C, 15°C, 20°C, 25°C, 30°C, 35°C, 40°C, 45°C, 50°C, etc.) but using the same fixative solution. In some embodiments, at least four morphologically stained training biological specimens are obtained, and each are differentially fixed. In some embodiments, at least six morphologically stained training biological specimens are obtained, and each are differentially fixed. In some embodiments, at least eight morphologically stained training biological specimens are obtained, and each are differentially fixed. In some embodiments, at least ten morphologically stained training biological specimens are obtained, and each are differentially fixed.

[0093] In some embodiments, the obtained training biological specimens are of the same tissue type, size, and / or shape. In other embodiments, the obtained training biological specimens are of the same tissue type, but of substantially the same shape and / or size. In other embodiments, the obtained training biological specimens are of the same tissue type but having different shapes and / or sizes. In yet other embodiments, the obtained training biological specimens are of different tissue types, shapes, and / or sizes.

[0094] Differential Fixation of One or More Training Biological Specimens

[0095] In other embodiments, one or more obtained training biological specimens are divided into a plurality of training samples (e.g., a first training tissue sample, a second training tissue sample, and nthtraining tissue sample) (see, e.g., FIG. 4A), and where each training sample of the plurality of training samples is subjected to a different fixation process (see, e.g., FIG. 4B).

[0096] Fixatives can be classified as cross-linking agents (such as aldehydes, e.g., formaldehyde, paraformaldehyde, and glutaraldehyde, as well as non-aldehyde cross-linking agents), oxidizing agents (e.g., metallic ions and complexes, such as osmium tetroxide and chromic acid), protein-denaturing agents (e.g., acetic acid, methanol, and ethanol), fixatives of unknown mechanism (e.g., mercuric chloride, acetone, and picric acid), combination reagents (e.g., Camoy's fixative, methacarn, Bouin's fluid, B5 fixative, Rossman's fluid, and Gendre's fluid), microwaves, and miscellaneous fixatives (e.g., excluded volume fixation and vapor fixation). Additives may also be included in the fixative, such as buffers, detergents, tannic acid, phenol, metal salts (such as zinc chloride, zinc sulfate, and lithium salts), and lanthanum. The most commonly used fixative in preparing samples is formaldehyde, generally in the form of a formalin solution (formaldehyde in an aqueous (and typically buffered) solution). In an embodiment, the samples used in the present methods are fixed by a method comprising fixation in a formalin-based fixative. In one example, the fixative is 10% neutral buffered formalin. Notwithstanding these examples, the tissues can be fixed by process using any fixation medium that is compatible with the biomarker-specific reagents and specific detection reagents utilized.

[0097] In some embodiments, a training biological specimen may be sectioned into 3 or more parts, and each part may be fixed for a different amount of time (e.g., 1 hour, 4 hours, and 6 hours), thus providing three differentially fixed training samples. In other embodiments, a training biological specimen may be sectioned into 5 or more parts, and each part may be fixed for a different amount of time (e.g., 0 hours, 1 hour, 2 hours, 4 hours, and 6 hours), thus providing five differentially fixed training samples. In yet other embodiments, a training biological specimen may be sectioned into 7 or more parts, and each part may be fixed for a different amount of time (e.g., 0 hours, 1 hour, 2 hours, 4 hours, 6 hours, 12 hours, and 24 hours), thus providing seven differentially fixed training samples. In further embodiments, a training biological specimen may be sectioned into 9 or more parts, and each part may be fixed for a different amount of time (e.g., 0 hours, 1 hour, 2 hours, 4 hours, 6 hours, 10 hours, 16 hours, 20 hours, and 24 hours), thus providing nine differentially fixed training samples. By way of example, a single tonsil tissuebiological specimen may be divided into 7 parts, and each part may be differentially fixed for a pre-determined amount of time, e.g., 0 hours, about 1 hour, about 2 hours, about 4 hours, about 6 hours, about 12 hours, about 24 hours. By way of example, a single tonsil tissue biological specimen may be divided into 7 parts, and each part may be differentially fixed for a predetermined amount of time, e.g., 0 hours, about 1 hour, about 2 hours, about 4 hours, about 8 hours, about 12 hours, about 16 hours. By way of yet another example, a single tonsil tissue biological specimen may be divided into 9 parts, and each part may be differentially fixed for a predetermined amount of time, e.g., 0 hours, about 1 hour, about 2 hours, about 4 hours, about 8 hours, about 12 hours, about 16 hours, about 20 hours, and about 24 hours.

[0098] In some embodiments, any training biological specimen or any training sample derived therefrom may be fixed for any pre-determined amount of time, e.g., about 1 hour, about2 hours, about 4 hours, about 6 hours, about 12 hours, etc., and the training image data acquired from the training biological specimens and / or training samples derived therefrom (following morphological staining) may serve as ground truth in training a fixation estimation engine 210. In some embodiments, training samples are differentially fixed so as to provide representative samples that are "underfixed" and "adequately fixed." In other embodiments, the training samples are differentially fixed so as to provide representative samples that are "unfixed," "underfixed," and "adequately fixed." In yet other embodiments, the training samples are differentially fixed so as to provide representative samples that are "unfixed," "underfixed," "adequately fixed," and "overfixed." In further embodiments, training samples are differentially fixed so as to provide representative samples that are "unfixed," and samples that have been fixed for a plurality of specific time periods, e.g., for about 2h, for about 3h, for about 4h, for about 5h, for about 6h, for about 7h, for about 8h, for about 9h, for about 12h, for about 15h, for about 18h, for about 24h, etc. In further embodiments, training samples are differentially fixed so as to provide representative samples that have been fixed for about 0 hours, about 1 hour, about 2 hours, about3 hours, about 4 hours, about 5 hours, about 6 hours, about 7 hours, about 8 hours, about 9 hours, about 10 hours, about 11 hours, about 12 hours, about 13 hours, about 14 hours, about 15 hours, about 16 hours, about 17 hours, about 18 hours, about 19 hours, about 20 hours, about 21 hours, about 22 hours, about 23 hours, about 24, hours, etc. or any interval therebetween. In some embodiments, the known differential fixation times and / or qualities are associated the trainingsamples as class labels as noted herein. In some embodiments, both a fixation quality and fixation duration class label are associated with a training sample.

[0099] In other embodiments, a training biological specimen may be sectioned into 5 or more parts, and each part may be fixed at a different temperature (e.g., 4°C, 6°C, 10°C, 15°C, 20°C, 25°C, 30°C, 35°C, 40°C, 45°C, 50°C, etc.).

[0100] In some embodiments, different training image data sets may be derived based on training samples differentially fixed in different fixatives (or different concentrations of fixatives) such that different trained fixation estimation engines particular to different fixatives may be generated and stored in a memory.

[0101] Morphological Staining of Differentially Fixed Training Biological Specimens

[0102] Following differential fixation, each of the training biological samples are morphologically stained. Regardless of the number of training biological specimens obtained or the number of training biological samples derived therefrom, and also regardless of the duration of time in which those training biological specimens / samples were subject to fixation, each of the obtained training biological specimens are stained with the same one or more morphological stains when training any individual fixation estimation engine. The skilled artisan will appreciate that different training image data sets may be generated for different morphological stains, thereby facilitating the training of a plurality of fixation estimation engines, where each trained fixation estimation engine is trained on a different morphological stain. In some embodiments, the training biological samples are associated with a class label of the morphological stain applied to the training biological sample.

[0103] In some embodiments, the morphological stain is hematoxylin which stains the nuclei blue. In other embodiments, the morphological stain is eosin which stains the cytoplasm pink. In yet other embodiments, the obtained biological specimen is stained with both hematoxylin and / or eosin (H&E).

[0104] In other embodiments, the morphological stain is a "special stain." A "special stain" refers to any chemically based stain useful for histological analysis that is not an immunohistochemical stain, an in-situ hybridization stain, or H&E. In some embodiments, the special stain includes one or more reagents selected from Acid fuchsin (C.I. 42685; absorbance maximum 546 nm), Alcian blue 8 GX (C.I. 74240; absorbance maximum 615 nm), Alizarin red S (C.I. 58005; absorbance maximum 556 and 596 nm), Auramine O (C.I. 41000; absorbancemaximum 370 and 432 nm), Azocarmine B (C.I. 50090; absorbance maximum 516 nm), Azocarmine G (C.I. 50085; absorbance maximum 511 nm), Azure A (C.I. 52005; similar absorbance to Azure B), Azure B (C.I. 52010; absorbance maximum 639 nm), Basic fuchsine (C.I. 42510; absorbance maximum 547-552 nm), Bismarck brown Y (C.I. 21000; absorbance maximum 643 nm), Brilliant cresyl blue (C.I. 51010; absorbance maximum 622 nm), Carmine (C.I. 75470; absorbance maximum protonated 490-495, increasing in base and when combined with metal salts), Chlorazol black E (C.I. 30235; absorbance maximum 500-504 nm and 574-602 nm), Congo red (C.I. 22120; absorbance maximum 497 nm), Cresyl violet (absorbance maximum 596-601 nm), Crystal violet (C.I. 42555; absorbance maximum 590 nm), Darrow red (absorbance maximum 502 nm), Ethyl green (C.I. 42590; absorbance maximum 635 nm 420 nm), Fast green F C F (C.I. 42053; absorbance maximum 624 nm, pH dependent), Giemsa Stain (mixture of impure azure B, methylene blue and eosin Y), Indigo carmine (C.I. 73015; absorbance maximum 608 nm), Janus green B (C.I. 11050; absorbance maximum 630 nm), Jenner stain 1899, Light green SF (C.I. 42095; absorbance maximum 422 and 630 nm), Malachite green (C.I. 42000; absorbance maximum 614 and 425 nm), Martius yellow (C.I. 10315; absorbance maximum 420-432 nm), Methyl orange (C.I. 13025; absorbance maximum 507 nm), Methyl violet 2B (C.I. 42535; absorbance maximum 583-587 nm), Methylene blue (C.I. 52015; absorbance maximum 656-661 nm), Methylene violet (Bemthsen), (C.I. 52041; absorbance maximum 580-601 nm), Neutral red (C.I. 50040; absorbance maximum 454, 529, 541 nm depending upon pH and solvent), Nigrosin (C.I. 50420; absorbance maximum 570-580 nm), Nile blue A (C.I. 51180; absorbance maximum 633-660 nm), Nuclear fast red (C.I. 60760; absorbance maximum 535 and 505 nm), Oil Red O (C.I. 26125; absorbance maximum 518 and 359 nm), Orange G (C. I. 16230; absorbance maximum 475 nm), Orange II (C.I. 15510; absorbance maximum 483 nm), Orcein (absorbance maximum 575-590 pH dependent, Pararosaniline (C.I. 42500; absorbance maximum 545 nm), Phloxin B (C.I. 45410; absorbance maximum 548 and 510 nm), , Pyronine B (C.I. 45010; closely related to Pyronine Y), Pyronine Y (C.I. 45005; absorbance maximum 546-549 nm), Resazurin (absorbance maximum 598 nm in water, 478 in methanol), Rose Bengal (C.I. 45435; absorbance maximum 546 nm), Safranine O (C.I. 50240; absorbance maximum 530 nm), Sudan black B (C.I. 26150; absorbance maximum 598 and 415 nm nm), Sudan III (C.I. 26100; absorbance maximum 503-507 and 503 nm), Sudan IV (C.I. 26105; absorbance maximum 520 nm), Tetrachrome stain (MacNeal), Thionine (C.I. 52000; absorbance maximum 598-602 nm), Toluidine blue (C.I. 52040; absorbancemaximum 626-630 nm), Weigert’s resorcin fuchsine (absorbance maximum 508 nm), Wright stain, and any combination thereof. In each of these examples, "C.I." refers to Color IndexTM. The Color IndexTM describes a commercial product by its recognized usage class, its hue, and a serial number (which simply reflects the chronological order in which related colorant types have been registered with the Color Index). This definition enables a particular product to be classified along with other products whose essential colorant is of the same chemical constitution and in which that essential colorant results from a single chemical reaction or a series of reactions.

[0105] In some embodiments, samples are morphologically stained according to the processes described in PCT Application Nos. PCT / EP2021 / 073738 or PCT / EP2021 / 073733, the disclosures of which are hereby incorporated by reference herein in their entireties.

[0106] In some embodiments, an H&E staining protocol may be performed, including applying the tissue section with hematoxylin stain mixed with a metallic salt, or mordant. The tissue section can then be rinsed in a weak acid solution to remove excess staining (differentiation), followed by bluing in mildly alkaline water. In some embodiments, after the application of hematoxylin, the tissue can be counterstained with eosin. It will be appreciated that other H&E staining techniques can be implemented.

[0107] The process of differential fixation, morphological staining, and of acquiring image data from the differentially fixed samples is further illustrated in FIG. 4C. As noted above, one or more training biological specimens are first obtained (step 410). In some embodiments, each of the one or more training biological specimens are then divided into at least two parts (step 420). In this way, each of the one or more training biological specimens provide at least two "training biological samples." Each of these training biological samples may be differentially fixed, for example each may be fixed for a different pre-determined amount of time (step 430).

[0108] Following differential fixation, each of the at least two training biological samples are morphologically stained (step 440). Subsequently, one or more of regions in each of the at least two training samples are identified (step 450). Next, image data is acquired for each of the one or more identified regions (step 460) from each of the training biological samples. In some embodiments, the acquired image data from all training samples from all training biological specimens (referred to as "training image data") are stored (step 470), such as in storage module 240. In this way, the training image data may be retrieved from the storage module 240 by the training module 211 for training of a fixation estimation engine 210. In addition to storing theimage data acquired from all training samples, the storage module 240 is also adapted to store any class labels associated with the acquired image data, e.g., known fixation durations, qualitative fixation estimates, etc.

[0109] In some embodiments, different training image data sets may be derived based on training samples stained with different morphological stains such that different trained fixation estimation engines particular to different morphological stains may be generated and stored in a memory.

[0110] Test Biological Specimens

[0111] In some embodiments, the acquired image data stored in the storage module 240 includes "test image data" derived from morphologically stained test biological specimens. With reference to FIG. 5, a test biological specimen may be obtained (step 510), and then one or more regions within the test biological specimen may be identified (step 520). Image data may then be acquired for each identified one or more regions (step 530). As described further herein, the test image data may be supplied to a trained fixation estimation engine 210 which has been trained for the particular morphological stain in which the test biological specimen was stained with such that a fixation status (quality and / or duration) of the test biological specimen may be estimated. The estimated fixation status (quality and / or duration) may then be used in downstream processes or downstream decision making, e.g., to determine whether a specimen is adequately fixed or not, to determine whether the specimen needs further fixation, or to determine whether the extent of fixation was or is sufficient for a particular assay, such as a IHC or ISH assay.

[0112] FIXATION ESTIMATION ENGINE

[0113] The systems and methods of the present disclosure employs one or more machine learning techniques to mine acquired image data, such as acquired image data from test biological specimens and / or from training biological specimens. In some embodiments, at least the fixation estimation engine 210 employs one or more machine learning techniques including, but not limited to, neural networks or convolutional neural networks.

[0114] The fixation estimation engine 210 may run in a test mode or a training mode. In the case of a fixation estimation engine 210 running in a training mode, the fixation estimation engine 210 may learn features (e.g., fixation features) from a plurality of acquired and processed training image data sets (e.g., acquired from one or more training biological specimens that have been differentially fixed and stained with one or more morphological stains, such as describedherein) and correlate those learned features with class labels associated with the training image data (e.g., known values of fixation duration, such as Oh, Ih, 2h, 4h, 6h, 10g, 16h, 24h, etc.; qualitative labels such as "unfixed," "underfixed," "fixed," "overfixed," etc.). In the case of a trained fixation estimation engine 210 (e.g., a fixation estimation engine that has already been trained using training image data and associated class labels), the trained fixation engine 210 for a particular morphological stain may derive features (e.g., fixation features) from a morphologically stained test biological specimen and, based on the learned datasets, predict a fixation status of the test biological specimen based on the derived fixation features.

[0115] Machine learning can be generally defined as a type of artificial intelligence (Al) that provides computers with the ability to learn without being explicitly programmed. Machine learning focuses on the development of computer programs that can teach themselves to grow and change when exposed to new data. In other words, machine learning can be defined as the subfield of computer science that gives computers the ability to learn without being explicitly programmed. Machine learning explores the study and construction of algorithms that can learn from and make predictions on data — such algorithms overcome following strictly static program instructions by making data driven predictions or decisions, through building a model from sample inputs. The machine learning described herein may be further performed as described in "Introduction to Statistical Machine Learning," by Sugiyama, Morgan Kaufmann, 2016, 534 pages; "Discriminative, Generative, and Imitative Learning," lebara, MIT Thesis, 2002, 212 pages; and "Principles of Data Mining (Adaptive Computation and Machine Learning)," Hand et al., MIT Press, 2001, 578 pages; which are incorporated by reference as if fully set forth herein. The embodiments described herein may be further configured as described in these references.

[0116] In some embodiments, the fixation estimation engine 210 employs "supervised learning" for the task of predicting a fixation state of a test image derived from a test biological specimen. Supervised learning is the machine learning task of learning a function that maps an input to an output based on example input-output pairs. It infers a function from labeled training data (here, the fixation time is the label associated with training image data) consisting of a set of training examples (here training image data). In supervised learning, each example is a pair consisting of an input object (typically a vector) and a desired output value (also called the supervisory signal). A supervised learning algorithm analyzes the training data and produces aninferred function, which can be used for mapping new examples. An optimal scenario will allow for the algorithm to correctly determine the class labels for unseen instances.

[0117] The fixation estimation engine 210 may include any type of machine learning algorithm known to those of ordinary skill in the art. Suitable machine learning algorithms include regression algorithms, similarity-based algorithms, feature selection algorithms, regularization method-based algorithms, decision tree algorithms, Bayesian models, kernel-based algorithms (e.g., support vector machines), Random Forest, K nearest neighbor, decision tree, clusteringbased methods, artificial neural networks, deep learning networks, ensemble methods, genetic algorithms, and dimensionality reduction methods. Examples of suitable dimensionality reduction methods include principal component analysis (such as principal component analysis plus discriminant analysis), projection onto latent structure regression, and t-Distributed Stochastic Neighbor Embedding (t-SNE), and Uniform Manifold Approximation and Projection (UMAP).

[0118] In some embodiments, the fixation estimation engine 210 utilizes principal component analysis. The main idea of principal component analysis (PCA) is to reduce the dimensionality of a data set consisting of many variables correlated with each other while retaining the variation present in the dataset, up to the maximum extent. The same is done by transforming the variables to a new set of variables, which are known as the principal components (or simply, the PCs) and are orthogonally ordered such that the retention of variation present in the original variables decreases as they move down in the order. In this way, the first principal component retains maximum variation that was present in the original components. The principal components are the eigenvectors of a covariance matrix, and hence they are orthogonal. Principal component analysis and methods of employing the same are described in U.S. Patent Publication No. 2005 / 0123202 and in U.S. Patent Nos. 6,894,639 and 8,565,488, the disclosures of which are hereby incorporated by reference herein in their entireties. PCA and Linear Discriminant Analysis are further described by Khan et. al., "Principal Component Analysis-Linear Discriminant Analysis Feature Extractor for Pattern Recognition," "UCSI International Journal of Computer Sciences Issues, Vol. 8, Issue 6, No. 2, Nov. 2011, the disclosure of which is hereby incorporated by reference herein in its entirety.

[0119] The t-SNE algorithm is a non-linear dimensionality reduction technique well-suited for embedding high-dimensional data for visualization in a low-dimensional space of two or three dimensions. Specifically, it models each high-dimensional object by a two- or three-dimensionalpoint in such a way that similar objects are modeled by nearby points and dissimilar objects are modeled by distant points with high probability. The t-SNE algorithm comprises two main stages. First, t-SNE constructs a probability distribution over pairs of high-dimensional objects in such a way that similar objects have a high probability of being picked while dissimilar points have an extremely small probability of being picked. Second, t-SNE defines a similar probability distribution over the points in the low-dimensional map, and it minimizes the Kullback-Leibler divergence between the two distributions with respect to the locations of the points in the map. The t-SNE algorithm is further described in United States Patent Publication Nos. 2018 / 0046755, 2014 / 0336942, and 2018 / 0166077, the disclosures of which are hereby incorporated by reference herein in their entireties.

[0120] In some embodiments, the fixation estimation engine 210 utilizes reinforcement learning. Reinforcement Learning (RL) refers to a type of Machine Learning method in which the agent receives a delayed reward in the next time step to evaluate its previous action. Said another way, RL is model-free machine learning paradigm concerned with how software agents ought to take actions in an environment so as to maximize some notion of cumulative reward. Typically, a RL setup is composed of two components, an agent, and an environment. The environment refers to the object that the agent is acting on, while the agent represents the RL algorithm. The environment starts by sending a state to the agent, which then based on its knowledge to take an action in response to that state. After that, the environment sends a pair of next state and reward back to the agent. The agent will update its knowledge with the reward returned by the environment to evaluate its last action. The loop keeps going on until the environment sends a terminal state, which ends to episode. Reinforcement learning algorithms are further described in U.S. Patent Nos. 10,279,474 and 7,395,252, the disclosures of which are hereby incorporated by reference herein in their entireties.

[0121] In some embodiments, the fixation estimation engine 210 includes a Support Vector Machine ("SVM"). In general, an SVM is a classification technique, which is based on statistical learning theory where a nonlinear input data set is converted into a high dimensional linear feature space via kernels for the non-linear case. A support vector machines project a set of training data, E, which represents two different classes into a high-dimensional space by means of a kernel function, K. In this transformed data space, nonlinear data are transformed so that a flat line can be generated (a discriminating hyperplane) to separate the classes so as to maximize the classseparation. Testing data are then projected into the high-dimensional space via K, and the test data (such as the features or metrics enumerated below) are classified on the basis of where they fall with respect to the hyperplane. The kernel function K defines the method in which data are projected into the high-dimensional space.

[0122] In some embodiments, the fixation estimation engine 210 includes a neural network. As used herein, the term "neural network" refers to one or more computer-implemented networks capable of being trained to achieve a goal. Unless otherwise indicated, references herein to a neural network include one neural network or multiple interrelated neural networks that are trained together. Examples of neural networks include, without limitation, convolutional neural networks (CNNs), recurrent neural networks (RNNs), fully connected neural networks, encoder neural networks (e.g., "encoders"), decoder neural networks (e.g., "decoders"), dense- connection neural networks, and other types of neural networks. In some embodiments, a neural network can be implemented using special hardware (e.g., GPU, tensor processing units (TPUs), systolic arrays, single instruction multiple data (SIMD) processor, etc.), using software code and a general-purpose processor, or a combination of special hardware and software code.

[0123] In some embodiments, the neural network is configured as a deep learning network. Generally speaking, "deep learning" is a branch of machine learning based on a set of algorithms that attempt to model high level abstractions in data. Deep learning is part of a broader family of machine learning methods based on learning representations of data. An observation can be represented in many ways such as a vector of intensity values per pixel, or in a more abstract way as a set of edges, regions of particular shape, etc. Some representations are better than others at simplifying the learning task. One of the promises of deep learning is replacing handcrafted features with efficient algorithms for unsupervised or semi-supervised feature learning and hierarchical feature extraction.

[0124] In some embodiments, the neural network is a generative network. A "generative" network can be generally defined as a model that is probabilistic in nature. In other words, a "generative" network is not one that performs forward simulation or rule-based approaches. Instead, the generative network can be learned (in that its parameters can be learned) based on a suitable set of training data (e.g., a plurality of training image data sets). In some embodiments, the neural network is configured as a deep generative network. For example, the network may be configured to have a deep learning architecture in that the network may include multiple layers,which perform a number of algorithms or transformations. As used herein, the term "layer" or "network layer" refers to an analysis stage in a neural network. Layers perform different types of analysis related to the type of the neural network. For example, layers in an encoder may perform different types of analysis on an input image to encode the input image. In some cases, a particular layer provides features based on the particular analysis performed by that layer. In some cases, a particular layer down-samples a received image. An additional layer performs additional downsampling. In some cases, each round of down-sampling reduces the visual quality of the output image, but provides features based on the related analysis performed by that layer.

[0125] In some embodiments, the neural network includes an autoencoder. An autoencoder neural network is an unsupervised learning algorithm that applies backpropagation (see further description herein), setting the target values to be equal to the inputs. The aim of an autoencoder is to learn a representation (encoding) for a set of data, typically for dimensionality reduction, by training the network to ignore signal "noise." Along with the reduction side, a reconstructing side is learnt, where the autoencoder tries to generate from the reduced encoding a representation as close as possible to its original input. Additional information regarding autoencoders can be found at http: / / ufldl.stanford.edu / tutorial / unsupervised / Autoencoders / , the disclosure of which is hereby incorporated by reference herein in its entirety.

[0126] In some embodiments, the neural network may be a deep neural network with a set of weights that model the world according to the data that it has been fed to train it. Neural networks typically consist of multiple layers, and the signal path traverses from front to back between the layers. Any neural network may be implemented for this purpose. Suitable neural networks include LeNet, AlexNet, ZFnet, GoogLeNet, VGGNet, VGG16, DenseNet (also known as a Dense Convolutional Network or DenseNet-121), MiniNet, and the ResNet. In some embodiments, a fully convolutional neural network is utilized, such as described by Long et al., "Fully Convolutional Networks for Semantic Segmentation," Computer Vision and Pattern Recognition (CVPR), 2015 IEEE Conference, June 20015 (INSPEC Accession Number: 15524435), the disclosure of which is hereby incorporated by reference. Yet other suitable neural networks include a Convolutional Neural Network (CNN), a Recurrent Neural Network, a Long Short-Term Memory Neural Network (LSTM), a Compound Scaled Efficient Neural Network (EfficientNet), a Normalizer Free Neural Network (NFNet), a Densely Connected Convolutional Neural Network (DenseNet), an Aggregated Residual Transformation Neural Network(ResNeXT), a Channel Boosted Convolutional Neural Network (CB-CNN), a Wide Residual Network (WRN), or a Residual Neural Network (RNN). In some embodiments, the neural network is one that operates on cross-entropy, e.g., one that may recognize per pixel cross-entropy.

[0127] In some embodiments, the neural network utilizes cross-entropy. Cross-entropy is commonly used in machine learning as a loss function. Cross-entropy is a measure from the field of information theory, building upon entropy and generally calculating the difference between two probability distributions. It is closely related to but is different from KL divergence that calculates the relative entropy between two probability distributions, whereas cross-entropy can be thought to calculate the total entropy between the distributions. Cross-entropy is also related to and often confused with logistic loss, called log loss. Although the two measures are derived from a different source, when used as loss functions for classification models, both measures calculate the same quantity and can be used interchangeably.

[0128] In some embodiments, the neural network is configured as an DenseNet. In a DenseNet, each layer may connect to all preceding layers in a feed-forward fashion, where the output feature maps of all preceding layers and the original input feature maps may be concatenated to form the input feature maps for the current layer. DenseNets may use fewer parameters than an equivalent traditional CNN because there is no need to learn redundant feature maps. DenseNets may use less memory and computation to achieve a high performance. In a standard convolutional network, an input image goes through multiple convolutions to obtain high- level features. In ResNet, identity mapping is proposed to promote the gradient propagation. Element-wise addition is used. It can be viewed as algorithms with a state passed from one ResNet module to another one. In DenseNet, each layer obtains additional inputs from all preceding layers and passes on its own feature-maps to all subsequent layers. Concatenation is used. Each layer is receiving a "collective knowledge" from all preceding layers. Since each layer in DenseNet receives feature maps from all preceding layers, the network can be thinner and comparatively more compact, i.e., the number of channels can be fewer. The growth rate k is the additional number of channels for each layer, so, it has higher computational efficiency and memory efficiency.

[0129] Basic DenseNet Composition Layer: For each composition layer, Pre- Activation Batch Norm (BN) and ReLU, then 3x3 Conv are done with output feature maps of k channels, say for example, to transform xO, xl, x2, x3 to x4. This is the idea from Pre-Activation ResNet.

[0130] DenseNet-B (Bottleneck Layers): To reduce the model complexity and size, BN- ReLU-lxl Conv is done before BN-ReLU-3x3 Conv.

[0131] Multiple Dense Blocks with Transition Layers: 1 >< 1 Conv followed by 2X2 average pooling is used as the transition layers between two contiguous dense blocks. Feature map sizes are the same within the dense block so that they can be concatenated together easily. At the end of the last dense block, a global average pooling is performed and then a softmax classifier is attached.

[0132] DenseNet-BC (Further Compression): If a dense block contains m feature-maps, The transition layer generates 9m output feature maps, where 9 < 9 < 1 is referred to as the compression factor. When 9 = 1, the number of feature-maps across transition layers remains unchanged. DenseNet with 9 < 1 is referred as DenseNet-C, and 9 = 0.5 in the experiment.

[0133] When both the bottleneck and transition layers with 9 < 1 are used, the model is referred as DenseNet-BC.

[0134] Finally, DenseNets with / without B / C and with different L layers and k growth rate are trained.

[0135] For the apparatuses, methods, and non-transitory computer readable media disclosed herein, even though the dense convolutional network may represent a particular type of convolutional neural network with extensive intra-layer connections, the dense convolutional network may be implemented in various forms by varying the number of such layers and the number of dense blocks in the network. Apart from this, hyperparameters such as learning rate, may further add to the various possibilities of dense convolutional network implementations.

[0136] In some embodiments, the neural network is configured as an AlexNet. For example, the classification network structure can be AlexNet. The term "classification network" is used herein to refer to a CNN, which includes one or more fully connected layers. In general, an AlexNet includes a number of convolutional layers (e.g., 5) followed by a number of fully connected layers (e.g., 3) that are, in combination, configured and trained to classify data.

[0137] In other embodiments, the neural network is configured as a Google Net. While the Google Net architecture may include a relatively high number of layers (especially compared to some other neural networks described herein), some of the layers may be operating in parallel, and groups of layers that function in parallel with each other are generally referred to as inception modules. Other of the layers may operate sequentially. Therefore, a Google Net is different fromother neural networks described herein in that not all of the layers are arranged in a sequential structure. Examples of neural networks configured as Google Nets are described in "Going Deeper with Convolutions," by Szegedy et al., CVPR 2015, which is incorporated by reference as if fully set forth herein.

[0138] In other embodiments, the neural network is configured as a VGG network. For example, the classification network structure can be VGG. VGG networks were created by increasing the number of convolutional layers while fixing other parameters of the architecture. Adding convolutional layers to increase depth is made possible by using substantially small convolutional filters in all of the layers.

[0139] In other embodiments, the neural network is configured as a deep residual network. For example, the classification network structure can be a Deep Residual Net or ResNet. Like some other networks described herein, a deep residual network may include convolutional layers followed by fully connected layers, which are, in combination, configured and trained for detection and / or classification. In a deep residual network, the layers are configured to learn residual functions with reference to the layer inputs, instead of learning unreferenced functions. In particular, instead of hoping each few stacked layers directly fit a desired underlying mapping, these layers are explicitly allowed to fit a residual mapping, which is realized by feedforward neural networks with shortcut connections. Shortcut connections are connections that skip one or more layers. A deep residual net may be created by taking a plain neural network structure that includes convolutional layers and inserting shortcut connections which thereby takes the plain neural network and turns it into its residual learning counterpart. Examples of deep residual nets are described in "Deep Residual Learning for Image Recognition" by He et al., NIPS 2015, which is incorporated by reference as if fully set forth herein. The neural networks described herein may be further configured as described in this reference.

[0140] TRAINING A FIXATION ESTIMATION ENGINE

[0141] In some embodiments, the fixation estimation engine 210 is adapted to operate in a training mode. In some embodiments, a training module 211 is in communication with the fixation estimation engine 210 and is configured to receive training image data and supply the training image data to the fixation estimation engine 210.

[0142] In some embodiments, the training module 211 may operate to provide training image data to the fixation estimation engine 210 and to operate the fixation estimation engine 210in its training mode in accordance with any suitable training algorithm, e.g., back propagation, few shot learning, etc. In some embodiments, the training algorithms utilize a known set of acquired training image data (such as described herein). In some embodiments, a training module 211 is in communication with the fixation estimation engine 210 and is configured to receive training image data (or a further processed variants of the training absorbance image data, e.g., a first or second derivative of the training image data, magnitudes of individual bands within the training image data, the integral of bands within the training image data, the ratio of two or more band intensities within the training image data, the ratios from second and third order derivatives of the training image data, etc.) and supply the training image data to the fixation estimation engine 210. In some embodiments, the training module 211 is also adapted to supply the class labels (e.g., known values of fixation duration, such as Oh, Ih, 2h, 4h, 6h, 8h, 10g, 12h, 16h, 20h, 24h, etc.; qualitative labels such as "unfixed," "underfixed," "fixed," "overfixed," etc.) associated with the training image data.

[0143] In some embodiments, the training algorithms utilize a known set of training image data (such as described herein) and a corresponding set of known output class labels (e.g., known values of fixation duration (including any of those described herein), such as Oh, Ih, 2h, 4h, 6h, 8h, 10g, 12h, 16h, 20h, 24h, etc.; qualitative labels such as "unfixed," "underfixed," "fixed," "overfixed," etc.), and are configured to vary internal connections within the fixation estimation engine 210 such that processing of input training image data provides the desired corresponding class labels.

[0144] The fixation estimation engine 210 may be trained in accordance with any methods known to those of ordinary skill in the art. For example, any of the training methods disclosed in U.S. Patent Publication Nos. 2018 / 0268255, 2019 / 0102675, 2015 / 0356461, 2016 / 0132786, 2018 / 0240010, and 2019 / 010834; and PCT Publication No. WO / 2022 / 046463, the disclosures of which are hereby incorporated by reference herein in their entireties.

[0145] In the training phase, the fixation estimation engine may be trained in multiple iterations based on few-shot learning. Each training iteration may be referred to as an episode. In each episode, selected support images and query images are sent to the network, the network loss (e.g., classification error) is determined, and the error gradients are backpropagated to the fully connected layer. In each episode, K image classes may be randomly, pseudorandomly, or otherwise selected from M image classes, where K < M. For each image class of the K imageclasses, N support images of the image class may be randomly, pseudorandomly, or otherwise selected, where N is a hyper-parameter (the number of shots) and may be selected based on the number of available samples. For example, N may be about 5 to 10. For each image class of the K image classes, Q query images may also be randomly, pseudorandomly, or otherwise selected. Embeddings (e.g., d-dimensional feature vectors, where d=1024 in one example) of the support images may be generated by passing the support images through the machine-learning model. Thus, for each image class, N embeddings may be generated. The mean, median, or center of the N embeddings for the N support images in each class may be calculated to determine a prototype representing the image class. To classify the Q query images, each query image in the Q query images may be encoded into a query embedding. A squared Euclidean distance metric may be calculated to determine the distance from each prototype to each query embedding. Each query image may be classified into the class prototype of which is the closest to the query embedding. The classification errors may be back-propagated to optimize the parameters of the machinelearning model using the negative log-likelihood loss and an Adam optimizer with a fixed learning rate of 0.001. In some embodiments, the machine-learning model is trained for 10 iterations.

[0146] In embodiments where the fixation estimation engine 210 includes a neural network, the back propagation algorithm for training the fixation estimation engine 210 is an iterative process in which the connections between network nodes are given some random initial values, and the network is operated to calculate corresponding output vectors for a set of input vectors (the training image data set). The output vectors are compared to the desired output of the training image data set and the error between the desired and actual output is calculated. The calculated error is propagated back from the output nodes to the input nodes and is used for modifying the values of the network connection weights in order to decrease the error. After each such iteration the training module 211 may calculate a total error for the entire training set and the training module 211 may then repeat the process with another iteration. The training of the fixation estimation engine 210 is complete when the total error reaches a minimum value. If a minimum value of the total error is not reached after a predetermined number of iterations and if the total error is not a constant the training module 211 may consider that the training process does not converge.

[0147] In the context of training the fixation estimation engine 210 with acquired image data derived from training biological specimens differentially fixed for pre-determined periods oftime (described above), each acquired training image is associated with a known fixation quality and / or duration. By way of example, training image data sets may be provided having known fixation qualities (e.g., "unfixed," "under fixed," "fixed," or "over fixed) and / or fixation durations (e.g., 0 hours, 1 hour, 2 hours, 4 hours, 6 hours, etc.). In this manner, tissue samples representing different fixation qualities and / or fixation durations are provided for training. In some embodiments, training image data sets may be provided which are associated with only fixation quality class labels. In other embodiments, training image data sets may be provided which are associated with only fixation duration class labels. In yet other embodiments, training image data sets may be provided which are associated with both fixation quality class labels and fixation duration class labels. In this way, the fixation estimation engine 210 may be trained not only to detect fixation features associated with a fixation quality, but also a duration of the fixation.

[0148] When the training of the fixation estimation engine 210 is complete, the system 200 is ready to operate to detect fixation features from test image data and then estimate a fixation status (quality and / or duration) of the test biological specimen based on the detected fixation features. In some embodiments, the fixation estimation engine 210 may be periodically retrained to adapt for variations in input data.

[0149] ESTIMATION OF FIXATION OUALTIY AND / OR DURATION USING A TRAINED FIXATION ESTIMATION ENGINE

[0150] Once the fixation estimation engine 210 has been trained, such as described above, it may be used to detect fixation features within test image data and, based on the detected fixation features, estimate at least one of a fixation quality or a fixation duration.

[0151] In some embodiments, and with reference to FIG. 3, a test biological specimen is obtained (step 310) (such as from a subject suspected of having a certain disease or known to have a certain disease) and then test image data is acquired from that test biological specimen or a portion thereof (step 320) (see also FIG. 5). Once test image data has been acquired and processed, fixation features may be detected within the test image data using a trained fixation estimation engine 210 trained on the particular morphological stain in which the test biological specimen was stained (step 340). In some embodiments, fixation features that may be detected include nuclear detail, cellular morphology, and tissue distortion. In other embodiments, the fixation features include nuclear size, cellular distortion, inferior nuclear and cytoplasmic morphology, effects oftissue shrinkage (such as cellular distances and macro level morphology), cells with poor cellular delineation, altered nuclear detail, and / or distorted cellular organelles.

[0152] Based on detected fixation features, estimates of fixation quality and / or duration may be computed (step 350). In some embodiments, if the fixation estimation engine 210 was trained using a two-class classifier, the test biological specimen may, by way of example, be classified as "underfixed" or "adequately fixed." For example, in some embodiments, the test biological specimen may be classified as "underfixed," e.g., only subjected to a fixative or a fixation process for Oh, Ih, or 2h. In other embodiments, the test biological specimen may be classified as "fixed" or "adequately fixed" or the like, e.g., fixed according to ASCO / CAP guidelines, or fixed for 6h, 12h, or 24h.

[0153] In other embodiments, if the fixation estimation engine 210 was trained using a three-class classifier, the test biological specimen may be classified as "unfixed," e.g., subjected to a fixative or a fixation process for Oh; "underfixed," e g., only subjected to a fixative or a fixation process for Ih, 2h, 4hr; or "adequately fixed," e.g., fixed according to ASCO / CAP guidelines, or fixed for 6h, 12h, or 24h.

[0154] In yet other embodiments, as an alternative to a qualitative fixation estimation, a quantitative estimate of fixation duration may be computed (e.g., Oh, 2h, 4h, 6h, 8h, lOh, 12h, 16h, 20h, 24h). In even further embodiments, both a qualitative fixation estimate is provided along with a quantitative estimate of fixation duration.

[0155] In some embodiments, one or more overlays may be superimposed over any acquired image of a test biological specimen (or any portion thereof) to show different areas of the test biological specimen having different fixation qualities or fixation durations (see, e.g., FIG. 6 herein). In one exemplary embodiment, the overlays were generated in HALO (Indicia Labs, Inc., Albuquerque, NM) using a one-to-one mapping between the trained fixation model engine the H&E image for each tissue. In some embodiments, the color overlays represent which fixation time / class each pixel in the image is predicted to be in. Because it is believed that fixation cannot vary on a pixel-by-pixel basis, the fixation estimation engine may, in some embodiments, be trained to incorporate a minimum size feature in which a small region of tissue is calculated to be unique from its surroundings.

[0156] In some embodiments, a visualization module prepares computer-generated overlays of the image analysis results and provides the user with a user-friendly visual renderingof the scoring results atop the originally selected whole-slide image, as a representation of density. In some embodiments, the overlays can represent one or more maps, such as a heat map, a location map, a correlation map, etc., and can show variations of information using color and / or lines depending on the application.

[0157] For instance, and as shown in FIGS. 6 - 8, overlays may be generated which show "underfixed" tissue in a first color (e.g., yellow) and adequately fixed tissue (which means ASCO / CAP guidelines) in a second color (e.g., green). In some embodiments, the substrate (e.g., glass microscope slide) may be shown in yet another color which differs from the first and second colors. In addition, connective tissue (or other types of tissue) may be shown in a fourth color, which differs from the first, second, and third colors.

[0158] By way of another example, overlays may be generated which show "unfixed" tissue in a first color (e g., red), "underfixed" tissue in a second color (e.g., yellow) and adequately fixed tissue (which means ASCO / CAP guidelines) in a third color (e g., green). Substrate material (e.g., glass) and specific tissue types (e.g., connective tissue) may also be represented in the overlays in yet additional colors (see, e.g., FIG. 8).

[0159] Other System Components

[0160] The system 200 of the present disclosure may be tied to a specimen processing apparatus that can perform one or more preparation processes on the tissue specimen. The preparation process can include, without limitation, deparaffinizing a specimen, conditioning a specimen (e.g., cell conditioning), staining a specimen, performing antigen retrieval, performing immunohistochemistry staining (including labeling) or other reactions, and / or performing in situ hybridization (e.g., SISH, FISH, etc.) staining (including labeling) or other reactions, as well as other processes for preparing specimens for microscopy, microanalyses, mass spectrometric methods, or other analytical methods.

[0161] The processing apparatus can apply fixatives to the specimen. Fixatives can include cross-linking agents (such as aldehydes, e.g., formaldehyde, paraformaldehyde, and glutaraldehyde, as well as non-aldehyde cross-linking agents), oxidizing agents (e.g., metallic ions and complexes, such as osmium tetroxide and chromic acid), protein-denaturing agents (e.g., acetic acid, methanol, and ethanol), fixatives of unknown mechanism (e.g., mercuric chloride, acetone, and picric acid), combination reagents (e.g., Camoy's fixative, methacarn, Bouin's fluid, B5fixative, Rossman's fluid, and Gendre's fluid), microwaves, and miscellaneous fixatives (e.g., excluded volume fixation and vapor fixation).

[0162] If the specimen is a sample embedded in paraffin, the sample can be deparaffmized using appropriate deparaffmizing fluid(s). After the paraffin is removed, any number of substances can be successively applied to the specimen. The substances can be for pretreatment (e.g., to reverse protein-crosslinking, expose cells acids, etc.), denaturation, hybridization, washing (e.g., stringency wash), detection (e.g., link a visual or marker molecule to a probe), amplifying (e.g., amplifying proteins, genes, etc.), counterstaining, coverslipping, or the like.

[0163] The specimen processing apparatus can apply a wide range of substances to the specimen. The substances include, without limitation, stains, probes, reagents, rinses, and / or conditioners. The substances can be fluids (e.g., gases, liquids, or gas / liquid mixtures), or the like. The fluids can be solvents (e.g., polar solvents, non-polar solvents, etc.), solutions (e.g., aqueous solutions or other types of solutions), or the like. Reagents can include, without limitation, stains, wetting agents, antibodies (e.g., monoclonal antibodies, polyclonal antibodies, etc.), antigen recovering fluids (e.g., aqueous- or non-aqueous-based antigen retrieval solutions, antigen recovering buffers, etc.), or the like. Probes can be an isolated cells acid or an isolated synthetic oligonucleotide, attached to a detectable label or reporter molecule. Labels can include radioactive isotopes, enzyme substrates, co-factors, ligands, chemiluminescent or fluorescent agents, haptens, and enzymes. As used herein, the term "fluid" refers to any liquid or liquid composition, including water, solvents, buffers, solutions (e.g., polar solvents, non-polar solvents), and / or mixtures. The fluid may be aqueous or non-aqueous. Non-limiting examples of fluids include washing solutions, rinsing solutions, acidic solutions, alkaline solutions, transfer solutions, and hydrocarbons (e.g., alkanes, isoalkanes and aromatic compounds such as xylene). In some embodiments, washing solutions include a surfactant to facilitate spreading of the washing liquids over the specimenbearing surfaces of the slides. In some embodiments, acid solutions include deionized water, an acid (e.g., acetic acid), and a solvent. In some embodiments, alkaline solutions include deionized water, a base, and a solvent. In some embodiments, transfer solutions include one or more glycol ethers, such as one or more propylene-based glycol ethers (e.g., propylene glycol ethers, di(propylene glycol) ethers, and tri(propylene glycol) ethers, ethylene-based glycol ethers (e.g., ethylene glycol ethers, di(ethylene glycol) ethers, and tri(ethylene glycol) ethers), and functional analogs thereof. Non-liming examples of buffers include citric acid, potassium dihydrogenphosphate, boric acid, diethyl barbituric acid, piperazine-N,N'-bis(2-ethanesulfonic acid), dimethylarsinic acid, 2-(N-morpholino)ethanesulfonic acid, tris(hydroxymethyl)methylamine (TRIS), 2-(N-morpholino)ethanesulfonic acid (TAPS), N,N-bis(2-hydroxyethyl)glycine(Bicine), N-tris(hydroxymethyl)methylglycine (Tricine), 4-2-hydroxy ethyl -1 -piperazineethanesulfonic acid (HEPES), 2-{[tris(hydroxymethyl)methyl]amino}ethanesulfonic acid (TES), and combinations thereof. In some embodiments, the unmasking agent is water. In other embodiments, the buffer may be comprised of tri s(hydroxymethyl)m ethylamine (TRIS), 2-(N- morpholino)ethanesulfonic acid (TAPS), N,N-bis(2-hydroxyethyl)glycine(Bicine), N - tris(hydroxymethyl)methylglycine (Tricine), 4-2-hydroxy ethyl -1 -piperazineethanesulfonic acid (HEPES), 2-{[tris(hydroxymethyl)methyl]amino}ethanesulfonic acid (TES), or a combination thereof. Additional wash solutions, transfer solutions, acid solutions, and alkaline solutions are described in United States Patent Application Publication No. 2016 / 0282374, the disclosure of which is hereby incorporated by reference herein in its entirety.

[0164] Staining may be performed with a histochemical staining module or separate platform, such as an automated IHC / ISH slide Stainer. Automated IHC / ISH slide Stainers typically include at least: reservoirs of the various reagents used in the staining protocols, a reagent dispense unit in fluid communication with the reservoirs for dispensing reagent to onto a slide, a waste removal system for removing used reagents and other waste from the slide, and a control system that coordinates the actions of the reagent dispense unit and waste removal system. In addition to performing staining steps, many automated slide Stainers can also perform steps ancillary to staining (or are compatible with separate systems that perform such ancillary steps), including slide baking (for adhering the sample to the slide), dewaxing (also referred to as deparaffinization), antigen retrieval, counterstaining, dehydration and clearing, and coverslipping. Prichard, Overview of Automated Immunohistochemistry, Arch Pathol Lab Med., Vol. 138, pp. 1578-1582 (2014), incorporated herein by reference in its entirety, describes several specific examples of automated IHC / ISH slide Stainers and their various features, including the intelliPATH (Biocare Medical), WAVE (Celerus Diagnostics), DAKO OMNIS and DAKO AUTO STAINER LINK 48 (Agilent Technologies), BENCHMARK (Ventana Medical Systems, Inc.), Leica BOND, and Lab Vision Autostainer (Thermo Scientific) automated slide stainers. Additionally, Ventana Medical Systems, Inc. is the assignee of a number of United States patents disclosing systems and methods for performing automated analyses, including U.S. Pat. Nos. 5,650,327, 5,654,200, 6,296,809,6,352,861, 6,827,901 and 6,943,029, and U.S. Published Patent Application Nos. 20030211630 and 20040052685, each of which is incorporated herein by reference in its entirety. As used herein, the term "reagent" refers to solutions or suspensions including one or more agents capable of covalently or non-covalently reacting with, coupling with, interacting with, or hybridizing to another entity. Non-limiting examples of such agents include specific-binding entities, antibodies (primary antibodies, secondary antibodies, or antibody conjugates), nucleic acid probes, oligonucleotide sequences, detection probes, chemical moieties bearing a reactive functional group or a protected functional group, enzymes, solutions or suspensions of dye or stain molecules.

[0165] Commercially-available staining units typically operate on one of the following principles: (1) open individual slide staining, in which slides are positioned horizontally and reagents are dispensed as a puddle on the surface of the slide containing a tissue sample (such as implemented on the DAKO AUTOSTAINER Link 48 (Agilent Technologies) and intelliPATH (Biocare Medical) Stainers); (2) liquid overlay technology, in which reagents are either covered with or dispensed through an inert fluid layer deposited over the sample (such as implemented on Ventana BenchMark and DISCOVERY stainers); (3) capillary gap staining, in which the slide surface is placed in proximity to another surface (which may be another slide or a coverplate) to create a narrow gap, through which capillary forces draw up and keep liquid reagents in contact with the samples (such as the staining principles used by DAKO TECHMATE, Leica BOND, and DAKO OMNIS stainers). Some iterations of capillary gap staining do not mix the fluids in the gap (such as on the DAKO TECHMATE and the Leica BOND). In variations of capillary gap staining termed dynamic gap staining, capillary forces are used to apply sample to the slide, and then the parallel surfaces are translated relative to one another to agitate the reagents during incubation to effect reagent mixing (such as the staining principles implemented on DAKO OMNIS slide stainers (Agilent)). In translating gap staining, a translatable head is positioned over the slide. A lower surface of the head is spaced apart from the slide by a first gap sufficiently small to allow a meniscus of liquid to form from liquid on the slide during translation of the slide. A mixing extension having a lateral dimension less than the width of a slide extends from the lower surface of the translatable head to define a second gap smaller than the first gap between the mixing extension and the slide. During translation of the head, the lateral dimension of the mixing extension is sufficient to generate lateral movement in the liquid on the slide in a direction generally extending from the second gap to the first gap. See WO 2011 / 139978 Al . It has recentlybeen proposed to use inkjet technology to deposit reagents on slides. See WO 2016 / 170008 Al . This list of staining technologies is not intended to be comprehensive, and any fully or semiautomated system for performing biomarker staining may be incorporated into the histochemical staining platform.

[0166] Where a morphologically stained sample is also desired, an automated H&E staining platform may be used. Automated systems for performing H&E staining typically operate on one of two staining principles: batch staining (also referred to as "dip ‘n dunk") or individual slide staining. Batch stainers generally use vats or baths of reagents in which many slides are immersed at the same time. Individual slide stainers, on the other hand, apply reagent directly to each slide, and no two slides share the same aliquot of reagent. Examples of commercially available H&E stainers include the VENTANA SYMPHONY (individual slide Stainer) and VENTANA HE 600 (individual slide stainer) series H&E stainers from Roche; the Dako CoverStainer (batch stainer) from Agilent Technologies; the Leica ST4020 Small Linear Stainer (batch stainer), Leica ST5020 Multistainer (batch stainer), and the Leica ST5010 Autostainer XL series (batch stainer) H&E stainers from Leica Biosystems Nussloch GmbH.

[0167] A staining system including one or more modules for determining a fixation status of an at least partially fixed test biological specimen, the one or more modules comprising: (i) one or more processors, and (ii) one or more memories coupled to the one or more processors, the one or more memories to store computer-executable instructions that, when executed by the one or more processors, cause the one or more modules to perform operations comprising: obtaining test image data from the at least partially fixed test biological specimen, wherein the test specimen is stained with a morphological stain, and wherein the test image data includes image data derived from at least a portion of the at least partially fixed test biological specimen; deriving fixation features from the obtained test image data using a trained fixation estimation engine trained for the morphological stain; and determining the fixation status of the at least partially fixed test biological specimen based on the derived fixation features. In some embodiments, the staining system further includes one or more dispensers and / or one or more reagent reservoirs. In some embodiments, the staining system further includes one or more elements for supporting or moving a slide.

[0168] After the specimens are stained, the stained samples can be manually analyzed on a microscope, and / or digital images of the stained samples can be acquired for archiving and / ordigital analysis. Digital images can be captured via a scanning platform such as a slide scanner that can scan the stained slides at 20x, 40x, or other magnifications to produce high resolution whole-slide digital images. At a basic level, the typical slide scanner includes at least: (1) a microscope with lens objectives, (2) a light source (such as halogen, light emitting diode, white light, and / or multispectral light sources, depending on the dye), (3) robotics to move glass slides around or to move the optics around the slide or both, (4) one or more digital cameras for image capture, (5) a computer and associated software to control the robotics and to manipulate, manage, and view digital slides. Digital data at a number of different X-Y locations (and in some cases, at multiple Z planes) on the slide are captured by the camera’s charge-coupled device (CCD), and the images are joined together to form a composite image of the entire scanned surface. Common methods to accomplish this include:

[0169] (1) Tile based scanning, in which the slide stage or the optics are moved in small increments to capture square image frames, which overlap adjacent squares to a slight degree. The captured squares are then automatically matched to one another to build the composite image; and

[0170] (2) Line-based scanning, in which the slide stage moves in a single axis during acquisition to capture a number of composite image "strips." The image strips can then be matched with one another to form the larger composite image.

[0171] A detailed overview of various scanners (both fluorescent and brightfield) can be found at Farahani et al., Whole slide imaging in pathology: advantages, limitations, and emerging perspectives , Pathology and Laboratory Medicine Int'l, Vol. 7, p. 23-33 (June 2015), the disclosure of which is incorporated by reference in its entirety. Examples of commercially available slide scanners include: 3DHistech PANNORAMIC SCAN II; DigiPath PATHSCOPE; Hamamatsu NAN0Z00MER RS, HT, and XR; Huron TISSUESCOPE 4000, 4000XT, and HS; Leica SCANSCOPE AT, AT2, CS, FL, and SCN400; Mikroscan D2; Olympus VS 120-SL; Omnyx VL4, and VL120; PerkinElmer LAMINA; Philips ULTRA-FAST SCANNER; Sakura Finetek VISIONTEK; Unic PRECICE 500, and PRECICE 600x; and Zeiss AXIO SCAN.Z1. In some embodiments, the scanning device is a digital pathology device as disclosed any of United States Patent No. 9,575,301; U.S. Patent Application Publication No. 2014 / 0178169; United States Patent No. 9,575,301; U.S. Patent Application Publication No. 2014 / 0178169; United StatesPatent Publication Nos. 2021 / 0092308; and / or U.S. Patent Application Publication No. 2021 / 0088769, the content of each of which is incorporated by reference in its entirety.

[0172] Exemplary commercially available image analysis software packages include VENTANA VIRTUOSO software suite (Ventana Medical Systems, Inc.); TISSUE STUDIO, DEVELOPER XD, and IMAGE MINER software suites (Definiens); BIOTOPIX, ONCOTOPIX, and STEREOTOPIX software suites (Visi opharm); and the HALO platform (Indica Labs, Inc ).

[0173] In some embodiments, any imaging may be accomplished using any of the systems disclosed in U.S. Patent Nos. 10,317,666 and 10,313,606, the disclosures of which are hereby incorporated by reference herein in their entireties. The imaging apparatus may be a brightfield imager such as the iScan Coreo™ brightfield scanner or the DP200 scanner sold by Ventana Medical Systems, Inc.

[0174] In some cases, the images may be analyzed on an image analysis system. Image analysis system may include one or more computing devices such as desktop computers, laptop computers, tablets, smartphones, servers, application-specific computing devices, or any other type(s) of electronic device(s) capable of performing the techniques and operations described herein. In some embodiments, image analysis system may be implemented as a single device. In other embodiments, image analysis system may be implemented as a combination of two or more devices together achieving the various functionalities discussed herein. For example, image analysis system may include one or more server computers and a one or more client computers communicatively coupled to each other via one or more local-area networks and / or wide-area networks such as the Internet. The image analysis system typically includes at least a memory, a processor, and a display. Memory may include any combination of any type of volatile or nonvolatile memories, such as random-access memories (RAMs), read-only memories such as an Electrically Erasable Programmable Read-Only Memory (EEPROM), flash memories, hard drives, solid state drives, optical discs, and the like. It is appreciated that memory can be included in a single device and can also be distributed across two or more devices. Processor may include one or more processors of any type, such as central processing units (CPUs), graphics processing units (GPUs), special-purpose signal or image processors, field-programmable gate arrays (FPGAs), tensor processing units (TPUs), and so forth. It is appreciated that processor can be included in a single device and can also be distributed across two or more devices. Display may be implemented using any suitable technology, such as LCD, LED, OLED, TFT, Plasma, etc. Insome implementations, display may be a touch-sensitive display (a touchscreen). Image analysis system also typically includes a software system stored on the memory comprising a set of instructions implementable on the processor, the instructions comprising various image analysis tasks, such as object identification, stain intensity quantification, and the like. Exemplary commercially available software packages useful in implementing modules as disclosed herein include VENTANA VIRTUOSO; Definiens TISSUE STUDIO, DEVELOPER XD, and IMAGE MINER; and Visopharm BIOTOPIX, ONCOTOPIX, and STEREOTOPIX software packages.

[0175] After the specimens are processed, a user can transport specimen-bearing slides to the imaging apparatus. In some embodiments, the imaging apparatus is a brightfield imager slide scanner. One brightfield imager is the iScan Coreo brightfield scanner sold by Ventana Medical Systems, Inc. In automated embodiments, the imaging apparatus is a digital pathology device as disclosed in International Patent Application No.: PCT / US2010 / 002772 (Patent Publication No.: WO / 2011 / 049608) entitled IMAGING SYSTEM AND TECHNIQUES or disclosed in U.S. Patent Application No. 61 / 533,114, filed on Sep. 9, 2011, entitled IMAGING SYSTEMS, CASSETTES, AND METHODS OF USING THE SAME. International Patent Application No. PCT / US2010 / 002772 and U.S. Patent Application No. 61 / 533,114 are incorporated by reference in their entities.

[0176] Embodiments of the subject matter and the operations described in this specification can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, for example, one or more modules of computer program instructions, encoded on computer storage medium for execution by, or to control the operation of, data processing apparatus. Any of the modules described herein may include logic that is executed by the processor(s). "Logic," as used herein, refers to any information having the form of instruction signals and / or data that may be applied to affect the operation of a processor. Software is an example of logic.

[0177] A computer storage medium can be, or can be included in, a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of them. Moreover, while a computer storage medium is not a propagated signal, a computer storage medium can be a source or destination of computerprogram instructions encoded in an artificially generated propagated signal. The computer storage medium can also be, or can be included in, one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices). The operations described in this specification can be implemented as operations performed by a data processing apparatus on data stored on one or more computer-readable storage devices or received from other sources.

[0178] The term "programmed processor" encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable microprocessor, a computer, a system on a chip, or multiple ones, or combinations, of the foregoing. The apparatus can include special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). The apparatus also can include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or a combination of one or more of them. The apparatus and execution environment can realize various different computing model infrastructures, such as web services, distributed computing, and grid computing infrastructures.

[0179] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, subprograms, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

[0180] The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform actions by operating on input data and generating output. The processes and logic flows can alsobe performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).

[0181] Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a readonly memory or a random-access memory or both. The essential elements of a computer are a processor for performing actions in accordance with instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device (e g., a universal serial bus (USB) flash drive), to name just a few. Devices suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0182] To provide for interaction with a user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device, e.g., an LCD (liquid crystal display), LED (light emitting diode) display, or OLED (organic light emitting diode) display, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. In some implementations, a touch screen can be used to display information and receive input from a user. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be in any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. In addition, a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a web browser on a user's client device in response to requests received from the web browser.

[0183] Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back-end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front-end component, e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network ("LAN") and a wide area network ("WAN"), an internetwork (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks). For example, the network 20 of FIG. 1 can include one or more local area networks.

[0184] The computing system can include any number of clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In some embodiments, a server transmits data (e.g., an HTML page) to a client device (e.g., for purposes of displaying data to and receiving user input from a user interacting with the client device). Data generated at the client device (e.g., a result of the user interaction) can be received from the client device at the server.

[0185] EXAMPLE - TRAINING A FIXATION ESTIMATION ENGINE

[0186] A total of 92 differentially fixed tissue samples were obtained, where each of the 92 differentially fixed tissue samples were fixed for varying lengths of time in room temperature formalin. All 92 differentially fixed tissue samples were then scanned on a DP200 slide imager (available from Ventana Medical Systems, Inc., Tucson, AZ). The imaged / scanned datasets were divided into two sets: 1) the training set that was used to train the model; and 2) a validation set that the trained model applied to evaluate its performance.

[0187] Next, the fixation estimation engine was trained using only the training tissue. To initiate training, the number of classes that the training sets would be classified into was determined (e.g., 2 classes, 3 classes, etc.; alternatively, the number of classes was the number of different hours in which the differentially fixed tissue samples were fixed, e.g., 6 classes, where a first classwas 0 hours, a second class was 1 hour, a third class was 2 hours, a fourth class was 6 hours, a fifth class was 12 hours, and a sixth class was 24 hours).

[0188] Initially, it was attempted to classify the tissue samples into buckets defined exclusively by their fixation time in hours. This resulted in two problems: 1) the connective tissue was consistently being poorly classified; and 2) the predictive accuracy of the model, even on training datasets, was poor. To overcome these issues, a label classifier was incorporated into the workflow to specifically identify connective tissue to improve system performance (see FIG. 10). The classifier exclusively sought out the stroma / connective areas of tissue because they appeared to have a significantly different fixation signature or no fixation signature at all contained within them. Additionally, several fixation times were grouped into one classification scheme (i.e., Ohr, Ihr and 2hr samples = underfixed and 6hr, 12hr, 24hr sample = ASCO / CAP compliant fixed).

[0189] The next step in the process was training a model (e.g., a fixation estimation engine) with a connective tissue classifier and a different scoring scheme. To achieve this, all images were loaded, and the connective tissue was manually labeled for all tissues into a common class ("connective"). The remainder of the tissue sections were then manually annotated larger fixation class (i.e., underfixed or ASCO / CAP compliant) (see, FIG. 11).

[0190] Next, the parameters for training the fixation estimation engine were set. For this work, Mininet was initially selected because it could learn signatures quickly and with low amounts of training data. That being said, the principle was demonstrated to work with other base models such as DenseNet. Subsequently, the minimum object size and resolution to train the model were set to initial values. Then, the fixation estimate engine was trained using all of the training images and the cross-entropy of the system was calculated after each training iteration. The model was allowed to train until the entropy reached a horizontal asymptote, which indicated that further training would not improve the performance of the fixation estimate engine.

[0191] Next, the trained fixation estimated engine was applied back to the training data to assess its initial performance. For instance, in FIG. 6, the middle of the 6-hour samples appeared to be underfixed whereas the samples that were fixed for longer period of times were predicted to have more uniform fixation quality (compare the 6h samples to the 12- and 24-hour samples of FIG. 6; similar results are shown in FIGS. 7 and 8). This observation was consistent with known expectations. If the trained fixation estimation engine was not performing appropriating, the model was adjusted (number of labels, model used, training time, minimum object size, resolution oftraining model, annotations accuracy, etc.) and its performance was reevaluated and eventually optimized.

[0192] Finally, once the trained fixation estimation engine was optimized, it is applied to the validation set to determine how well the trained fixation estimation engine was performing on "real life" test samples that it had never "seen" before. If the trained fixation estimation engine performed well in this situation, it was declared that the model had deciphered the true fixation signature from the H&E stained image and could be used in downstream fixation determination processes.

Claims

CLAIMS1. A system for determining a fixation status of an at least partially fixed test biological specimen, the system comprising: (i) one or more processors, and (ii) one or more memories coupled to the one or more processors, the one or more memories to store computer-executable instructions that, when executed by the one or more processors, cause the system to perform operations comprising: a. obtaining test image data from the at least partially fixed test biological specimen, wherein the test specimen is stained with a morphological stain, and wherein the test image data includes image data derived from at least a portion of the at least partially fixed test biological specimen; b. deriving fixation features from the obtained test image data using a trained fixation estimation engine trained for the morphological stain; and c. determining the fixation status of the at least partially fixed test biological specimen based on the derived fixation features.

2. The system of claim 1, wherein the estimated fixation status is a qualitative estimate of fixation quality.

3. The system of claim 1, wherein the estimated fixation status is a quantitative estimate of fixation duration.

4. The system of any one of the preceding claims, wherein the fixation estimation engine is trained using one or more training image data sets acquired from a plurality of differentially fixed training biological specimens, and where each training image in the one or more training image data sets is stained with the same morphological stain as the test biological specimen.

5. The system of claim 4, wherein each training image in the one or more training image data sets comprises class labels of known fixation duration.

6. The system of claim 5, wherein the training images in the one or more training image data sets data sets comprise 4 different class labels of known fixation duration.

7. The system of claim 5, wherein the training images in the one or more training image data sets data sets comprise 5 different class labels of known fixation duration.

8. The system of claim 5, wherein the training images in the one or more training image data sets data sets comprise 6 different class labels of known fixation duration.

9. The system of any one of claims claim 5 - 8, wherein the class labels of known fixation duration are selected from the group consisting of:Oh, 2h, 6h, and 12h;Oh, 2h, 6h, 12h, and 24h;Oh, Ih, 2h, 6h, 12h, and 24h;Oh, 2h, 4h, 6h, 12h, and 24h;Oh, Ih, 2h, 4h, 6h, 12h, and 24h;Oh, Ih, 2h, 4h, 6h, 12h, 16h, and 24h;Oh, Ih, 2h, 4h, 6h, 12h, 16h, 20h, and 24h; andOh, Ih, 2h, 4h, 6h, 8h, 12h, 16h, 20h, and 24h.

10. The system of claim 4, wherein the training images in the one or more training image data sets comprises class labels of known fixation quality.

11. The system of claim 10, wherein the training images in the one or more training image data sets comprise 2 different class labels of known fixation quality.

12. The system of claim 10, wherein the training images in the one or more training image data sets comprise 3 different class labels of known fixation quality.

13. The system of claim 10, wherein the training images in the one or more training image data sets comprise 4 different class labels of known fixation quality.

14. The system of any one of claims 10 - 13, wherein the class labels of known fixation quality are selected from the group consisting of underfixed and adequately fixed; underfixed, adequately fixed, and overfixed; unfixed, underfixed, and adequately fixed; unfixed, underfixed, adequately fixed, and overfixed; unfixed and overfixed; and adequately fixed and overfixed.

15. The system of claim 10, wherein the class labels of known fixation quality are unfixed, underfixed, and adequately fixed.

16. The system of claim 10, wherein the class labels of known fixation quality are unfixed, underfixed, adequately fixed, and overfixed.

17. The system of claim 4, wherein each training image in the one or more training image data sets comprises class labels of known fixation duration and known fixation quality.

18. The system of claim 17, wherein the class labels of known fixation quality are unfixed, underfixed, and adequately fixed.

19. The system of claim 17, wherein the class labels of known fixation quality are unfixed, underfixed, adequately fixed, and overfixed.

20. The system of claim 4, wherein each training image in the one or more training image data sets set is derived by: (i) obtaining a training biological specimen; (ii) dividing the obtained training biological specimen into a plurality of training tissue samples; and (iii) fixing each training tissue sample of the plurality of training tissue samples for a different predetermined amount of time.

21. The system of claim 20, wherein the different pre-determined amounts of time range from between about 0 hours to about 24 hours.

22. The system of claim 20, wherein the different pre-determined amounts of time range from between about 0 hours to about 12 hours.

23. The system of any one of the preceding claims, wherein the trained fixation status estimation engine comprises a neural network or a convolutional neural network.

24. The system of claim 23, wherein the neural network comprises DenseNet or a variation thereof.

25. The system of claim 23, wherein the neural network comprises MiniNet or a variation thereof.

26. The system of claim 4, wherein the plurality of differentially fixed training biological specimens comprises the same tissue type as the test biological specimen.

27. The system of claim 4, wherein the plurality of differentially fixed training biological specimens comprises a different tissue type than the test biological specimen.

28. The system of claim 4, wherein the plurality of differentially fixed training biological specimens comprises the same tissue shape and / or tissue size as the test biological specimen.

29. The system of claim 4, wherein the plurality of differentially fixed training biological specimens comprises a different tissue shape and / or tissue size than the test biological specimen.

30. The system any one of the claims 1 - 29, wherein the morphological stain is hematoxylin.

31. A method for predicting a fixation duration of an at least partially fixed a morphologically stained test biological specimen comprising:(a) obtaining test image data from the at least partially fixed test biological specimen, wherein the test specimen is stained with a morphological stain, and wherein the test image data includes image data derived from at least a portion of the at least partially fixed test biological specimen;(b) deriving fixation features from the obtained test image data using a trained fixation estimation engine trained for a morphological stain; and(c) determining the estimated fixation duration of the at least partially fixed test biological specimen based on the derived fixation features.

32. The method of claim 31, wherein the estimated fixation status is a qualitative estimate of fixation quality.

33. The method of claim 31, wherein the estimated fixation status is a quantitative estimate of fixation duration.

34. The method of any one of claims 31 - 33, wherein the fixation estimation engine is trained using one or more training image data sets acquired from a plurality of differentially fixed training biological specimens, and where each training image in the one or more training image data sets is stained with the same morphological stain as the test biological specimen.

35. The method of claim 34, wherein each training image in the one or more training image data sets comprises class labels of known fixation duration.

36. The method of claim 34, wherein each training image in the one or more training image data sets comprises class labels of known fixation quality.

37. The method of claim 36, wherein the class labels of known fixation quality are unfixed, underfixed, and adequately fixed.

38. The method of claim 36, wherein the class labels of known fixation quality are unfixed, underfixed, adequately fixed, and overfixed.

39. The method of claim 36, wherein each training image in the one or more training image data sets comprises class labels of known fixation duration and known fixation quality.

40. The method of claim 31, wherein each training image set data set is derived by: (i) obtaining a training biological specimen; (ii) dividing the obtained training biological specimen into a plurality of training tissue samples; and (iii) fixing each training tissue sample of the plurality of training tissue samples for a different pre-determined amount of time.

41. The method of claim 40, wherein the different pre-determined amounts of time range from between about 0 hours to about 24 hours.

42. The method of claim 34, wherein the plurality of differentially fixed training biological specimens comprise the same tissue type as the test biological specimen.

43. The method of claim 34, wherein the plurality of differentially fixed training biological specimens comprise a different same tissue type than the test biological specimen.

44. The method of any one of claims 31 - 43, further comprising assessing whether the test biological specimen comprises a fixation level suitable for labeling with one or more specific binding entities.

45. The method of claim 44, further comprising staining the test biological specimen for the presence of one or more biomarkers.

46. The method of claim 45, wherein the one or more biomarkers comprise one or more cancer biomarkers.

47. The method of any one of claims 31 - 46, wherein the trained fixation status estimation engine comprises a neural network or a convolutional neural network.

48. The method of claim 47, wherein the neural network comprises DenseNet or a variation thereof.

49. The method of claim 47, wherein the neural network comprises MiniNet or a variation thereof.

50. The method of any one of claims 31 - 45, further comprising classifying connective tissue within the test biological specimen prior to the deriving of the fixation features from the obtained test image data.

51. The method of claim 50, wherein the classifying of the connective tissue is performed using a trained connective tissue classifier.

52. The method of claim 51, wherein the trained connective tissue classifier comprises a neural network.

53. The method of claim 52, wherein the neural network is DenseNet or MiniNet.

54. The method of any one of claims 31 - 53, wherein the morphological stain comprises hematoxylin.

55. A non-transitory computer-readable medium storing instructions for determining an estimated fixation quality of an at least partially fixed test biological specimen, comprising:(a) obtaining test image data from the at least partially fixed test biological specimen, wherein the test specimen is stained with a morphological stain, and wherein the test image data includes image data derived from at least a portion of the at least partially fixed test biological specimen;(b) deriving fixation features from the obtained test image data using a trained fixation estimation engine trained for the morphological stain, wherein the fixation estimation engine is trained using training image data sets acquired from a plurality of differentially fixed training biological specimens; and(c) determining the estimated fixation quality of the at least partially fixed test biological specimen based on the derived fixation features.

56. The non-transitory computer-readable medium of claim 55, wherein the morphological stain comprises hematoxylin.

57. The non-transitory computer-readable medium of claim 55, wherein the trained fixation status estimation engine comprises a neural network.

58. The non-transitory computer-readable medium of claim 56, wherein the neural network comprises DenseNet or a variation thereof.

59. The non-transitory computer-readable medium of claim 56, wherein the neural network comprises MiniNet or a variation thereof.

60. The non-transitory computer-readable medium of any one of claims 55 - 59, wherein the estimated fixation status is a qualitative estimate of fixation quality.

61. The non-transitory computer-readable medium of any one of claims 55 - 59, wherein the estimated fixation status is a quantitative estimate of fixation duration.

62. The non-transitory computer-readable medium of any one of claims 55 - 61, wherein the fixation estimation engine is trained using one or more training image data sets acquired from a plurality of differentially fixed training biological specimens, and where eachtraining image in the one or more training image data sets is stained with the same morphological stain as the test biological specimen.

63. The non-transitory computer-readable medium of claim 62, wherein each training image in the one or more training image data sets comprises class labels of known fixation duration.

64. The non-transitory computer-readable medium of claim 63, wherein the class labels of known fixation duration are selected from the group consisting of:Oh, 2h, 6h, and 12h;Oh, 2h, 6h, 12h, and 24h;Oh, Ih, 2h, 6h, 12h, and 24h;Oh, 2h, 4h, 6h, 12h, and 24h;Oh, Ih, 2h, 4h, 6h, 12h, and 24h;Oh, Ih, 2h, 4h, 6h, 12h, 16h, and 24h;Oh, Ih, 2h, 4h, 6h, 12h, 16h, 20h, and 24h; andOh, Ih, 2h, 4h, 6h, 8h, 12h, 16h, 20h, and 24h.

65. The non-transitory computer-readable medium of claim 62, wherein each training image in the one or more training image data sets comprises class labels of known fixation quality.

66. The non-transitory computer-readable medium of claim 65, wherein the class labels of known fixation quality are selected from the group consisting of: underfixed and adequately fixed; underfixed, adequately fixed, and overfixed; unfixed, underfixed, and adequately fixed; unfixed, underfixed, adequately fixed, and overfixed; unfixed and overfixed; and adequately fixed and overfixed.

67. The non-transitory computer-readable medium of claim 55, further comprising instructions for classifying connective tissue within the test biological specimen prior to the deriving of the fixation features from the obtained test image data.

68. The non-transitory computer-readable medium of claim 55, further comprising instructions for classifying a glass slide upon which the test biological specimen is disposed prior to the deriving of the fixation features from the obtained test image data.

69. The non-transitory computer-readable medium of claim 55, further comprising instructions for identifying one or more fields of view within the obtained test image data.

70. A staining system including one or more modules for determining a fixation status of an at least partially fixed test biological specimen, the one or more modules comprising: (i) one or more processors, and (ii) one or more memories coupled to the one or more processors, the one or more memories to store computer-executable instructions that, when executed by the one or more processors, cause the one or more modules to perform operations comprising: a. obtaining test image data from the at least partially fixed test biological specimen, wherein the test specimen is stained with a morphological stain, and wherein the test image data includes image data derived from at least a portion of the at least partially fixed test biological specimen; b. deriving fixation features from the obtained test image data using a trained fixation estimation engine trained for the morphological stain; and c. determining the fixation status of the at least partially fixed test biological specimen based on the derived fixation features.

71. The staining system of claim 71, wherein the staining system further includes one or more dispensers and / or one or more reagent reservoirs.

72. The staining system of claim 71, wherein the staining system further includes one or more elements for supporting or moving a slide.

Images