Systems and methods for generating three-dimensional biomedical images of cell or tissue blocks

Abstract

Presented herein are systems and methods of generating biomedical images of cell blocks. A computing system may receive, for a subject at risk of or diagnosed with cancer, a plurality of first biomedical images of a volume with a biological sample having a plurality of cells from the subject in a first imaging modality. Each first biomedical image of the plurality of first biomedical images may correspond to a respective section of a plurality of sections of the volume. The computing system may generate, using the plurality of first biomedical images, a second biomedical image corresponding to the volume with the biological sample in the first imaging modality. The computing system may apply a machine learning model to the second biomedical image to convert from the first imaging modality to a second imaging modality to modify color corresponding to the plurality of cells.

Description

Atty. Dkt. No.: 115872-3415 SYSTEMS AND METHODS FOR GENERATING THREE-DIMENSIONAL BIOMEDICAL IMAGESOF CELL OR TISSUE BLOCKS CROSS REFERENCES TO RELATED APPLICATIONS

[0001] The present application claims the benefit of and priority to U.S. Provisional Patent Application No. 63 / 753,100, titled “Systems and Methods for Generating Three-Dimensional Biomedical Images of Cell or Tissue Blocks,” filed February 3, 2025, which is incorporated by reference in its entirety.STATEMENT OF GOVERNMENT SUPPORT

[0002] This invention was made with government support under P30CA008748 awarded by the National Institutes of Health. The government has certain rights in the invention.BACKGROUND

[0003] A computing device may use computer vision techniques to process a digital image to generate an output.SUMMARY

[0004] Aspects of the present disclosure are directed to systems, methods, and computer-readable media for generating biomedical images of cell or tissue blocks. One or more processors may receive, for a subject at risk of or diagnosed with a condition, a plurality of first biomedical images of a volume with a biological sample having a plurality of cells from the subject in a first imaging modality. Each first biomedical image of the plurality of first biomedical images may correspond to a respective section of a plurality of sections of the volume. The one or more processors may generate, using the plurality of first biomedical images, a second biomedical image corresponding to the volume with the biological sample in the first imaging modality. The one or more processors may apply a machine learning (ML) to the second biomedical image to convert from the first imaging modality to a second imaging modality. The ML model may be established using aAtty. Dkt. No.: 115872-3415 plurality of examples. Each of the plurality of examples may include (i) an example first image of an example biological sample having an example plurality of cells in the first imaging modality and (ii) an example second image of the example biological sample applied with a stain associated with the second imaging modality to modify example color corresponding to the example plurality of cells. The one or more processors may convert, based on applying the ML model, the second biomedical image from the first imaging modality to the second imaging modality to modify color corresponding to the plurality of cells. The one or more processors may store, using one or more data structures, an association between the subject and the second biomedical image in the second imaging modality.

[0005] In some embodiments, the one or more processors may provide a user interface to display the second biomedical image corresponding to the volume with the biological sample having the plurality of cells in at least one of the first imaging modality or the second imaging modality. In some embodiments, the one or more processors may receive, via the user interface, a selection of a section of the plurality of sections of the second biomedical image. The one or more processors may identify, from the second biomedical image, a two-dimensional slice corresponding to the section selected from the plurality of sections. The one or more processors may provide, for display via the user interface, the two-dimensional slice from the second biomedical image.

[0006] In some embodiments, the one or more processors may receive a plurality of third biomedical images in a third imaging modality. Each third biomedical image of the plurality of third biomedical images may correspond to a slice of a plurality of slices of the biological sample. The one or more processors may generate, using the plurality of third biomedical images, a fourth biomedical image corresponding to the volume with the biological sample in the third imaging modality. The one or more processors may provide a user interface to display at least one of the second biomedical image in the second imaging modality or the fourth biomedical image in the third imaging modality.

[0007] In some embodiments, the ML model may be established using a second plurality of examples. Each of the second plurality of examples may include (i) an example third image of a respective example biological sample and (ii) an annotation identifying atAtty. Dkt. No.: 115872-3415 least an example portion of the example third image as an example region of interest (ROI). The one or more processors may generate, based on applying the ML model to the second biomedical image, a third biomedical image identifying at least a portion of the second biomedical image as corresponding to an ROI.

[0008] In some embodiments, at least one of the second plurality of examples comprises the annotation may identify at least the example portion of the example third image as at least one of a plurality of classifications for the example ROI. The plurality of classification may include at least one of a blood vessel, a tissue, tumor, or a cell nucleus. The one or more processors may generate the third biomedical image identifying at least the portion of the second biomedical image as corresponding to a classification of the plurality of classification for the ROI.

[0009] In some embodiments, the one or more processors may generate a report using the ROI corresponding to at least the portion of the second biomedical image identified by the third biomedical image. The one or more processors may provide a user interface to display at least one of the third biomedical image or the report. In some embodiments, the report may include at least one of: (i) a first identifier for the first plurality of biomedical images, (ii) a second identifier for the second biomedical image, (iii) a third identifier for the ROI, or (iv) the classification for the ROI.

[0010] In some embodiments, the one or more processors may identify a section of interest (SOI) from a plurality of sections of the second biomedical image based on the ROI on the section. The one or more processors may provide a user interface to display an identification of the SOI in the second biomedical image. In some embodiments, the one or more processors may receive a plurality of third biomedical images in at least one of the second imaging modality or a third imaging modality. Each third biomedical image of the plurality of third biomedical images corresponding to a slice of a plurality of slices of the biological sample. The one or more processors may add at least a portion of the plurality of third biomedical images to generate the second biomedical image in the second imaging modality.[OOH J In some embodiments, the second biomedical image in the second imaging modality may identify at least one ROI corresponding to a feature in the biological sampleAtty. Dkt. No.: 115872-3415 that is not apparent in a fifth biomedical image in the third imaging modality. The fifth biomedical image may correspond to a section of the biological sample. In some embodiments, the second biomedical image in the second imaging modality may identify at least one condition not inferable from the fifth biomedical image in the third imaging modality.[0012} In some embodiments, the first imaging modality may include at least one of a micro computed tomography (micro-CT) imaging modality or a nano computed tomography (nano-CT) imaging modality. In some embodiments, the second imaging modality may include a microscopy imaging modality including at least one of a hematoxylin and eosin (H&E) stain, a Papanicolaou stain, or immunohistochemical (IHC) stain, to differentiate the color of the plurality of cells from a remainder of the second biomedical image. In some embodiments, the third imaging modality may include the microscopy imaging modality.[0013} In some embodiments, the one or more processors may receive the first plurality of biomedical images of the biological sample housed in a Formalin-Fixed Paraffin-Embedded (FFPE) cell block and scanned from within the FFPE cell block via an imaging device. In some embodiments, the condition may comprise at least one of cancer, an infection, an inflammation, or a cellular degeneration. In some embodiments, the cancer comprises at least one of carcinoma, sarcoma, hematopoietic cancer, adrenal cancer, bladder cancer, bone cancer, brain cancer, breast cancer, cervical cancer, colon cancer, colorectal cancer, corpus uterine cancer, ear, nose and throat (ENT) cancer, endometrial cancer, esophageal cancer, gastrointestinal cancer, head and neck cancer, Hodgkin's disease, intestinal cancer, kidney cancer, larynx cancer, leukemia, liver cancer, lymph node cancer, lymphoma, lung cancer, melanoma, mesothelioma, myeloma, nasopharynx cancer, neuroblastoma, non-Hodgkin’s lymphoma, oral cancer, ovarian cancer, pancreatic cancer, penile cancer, pharynx cancer, prostate cancer, rectal cancer, seminoma, skin cancer, stomach cancer, teratoma, testicular cancer, thyroid cancer, uterine cancer, vaginal cancer, vascular tumor, or metastases thereof.[0014 j In some embodiments, the infection may include at least one of a bacterial infection, a viral infection, a fungal infection, or a parasitic infection. In someAtty. Dkt. No.: 115872-3415 embodiments, the inflammation may include at least one of an acute inflammation or a chronic inflammation. In some embodiments, the cellular degeneration may include at least one of nuclear degeneration, cytoplasmic vacuolation, cytolysis, or cell atrophy. In some embodiments, the biological sample may be obtained from an organ associated with the condition within the subject. In some embodiments, a section of the biological sample may have a thickness of between 1 micrometers and 30 micrometers. In some embodiments the volume corresponding to the second biomedical image in the second imaging modality may have a thickness between 1 millimeter and 10 millimeters.BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The foregoing and other objects, aspects, features, and advantages of the disclosure will become more apparent and better understood by referring to the following description taken in conjunction with the accompanying drawings, in which:

[0016] FIG. 1 depicts a block diagram of a protocol for three-dimensional histology, in accordance with an illustrative embodiment.]0017J FIG. 2 depicts a block diagram of a system for reconstructing volumetric images from computed tomography data of cell blocks, in accordance with an illustrative embodiment.

[0018] FIG. 3 depicts a block diagram of a device for generating computed tomography (CT) images from cell blocks, in accordance with an illustrative embodiment.

[0019] FIG. 4 depicts a block diagram of a system for generating computed tomography (CT) images from cell blocks, in accordance with and illustrative embodiment.

[0020] FIGs. 5A-C depicts a flow diagram of a process for generating biomedical images of Formalin-Fixed Paraffin-Embedded (FFPE) specimens, in accordance with an illustrative embodiment.

[0021] FIG. 6 depicts a flow diagram of a process for applying machine learning on image data from computed tomography and histological slides, in accordance with an illustrative embodiment.Atty. Dkt. No.: 115872-3415

[0022] FIG. 7A depicts an example of a three-dimensional micro computed tomography (CT) image of lymph node with metastatic papillary thyroid carcinoma;

[0023] FIGs. 7B and 7C depict an example of an assessment of metastatic papillary thyroid carcinoma volume in lymph node;

[0024] FIG. 8A depicts a screenshot of an example user interface of three-dimensional views of whole slide images (WSI) and whole block images (WBI).

[0025] FIG. 8B depicts a screenshot of an example user interface of a juxtaposition of a whole slide image (WSI) and a whole block image (WBI) about a region of interest (ROI).

[0026] FIG. 9 depicts example images of adenocarcinoma from three-dimensional whole block images (WBI) generated from micro computed tomography (micro-CT).

[0027] FIG. 10 depicts an example of a protocol to perform serial sectioning of a cell block specimen for analyzing.

[0028] FIG. 11 depicts a block diagram of a system for generating biomedical images of cell blocks for analysis across image modalities, in accordance with an illustrative embodiment.

[0029] FIG. 12 depicts a block diagram for a process of obtaining image data in the system for generating biomedical images of cell blocks, in accordance with an illustrative embodiment.

[0030] FIG. 13 depicts a block diagram for a process of using machine learning to evaluate biomedical images in different imaging modalities in the system for generating biomedical images of cell blocks, in accordance with an illustrative embodiment.

[0031] FIG. 14 depicts a block diagram for a process of generating outputs based on the biomedical images of cell blocks in the system for generating biomedical images of cell blocks, in accordance with an illustrative embodiment.Aty. Dkt. No.: 115872-3415

[0032] FIG. 15 depicts a flow diagram of a method of generating biomedical images of cell blocks for analysis across image modalities, in accordance with an illustrative embodiment.[0033 J FIG. 16 shows directly Papanicolaou-stained images of a cell block section.

[0034] FIG. 17 depicts a schematic of a process for transforming color from hematoxylin and eosin (H&E) staining.

[0035] FIGs. 18A-C show (A) an image of a slide with cells having adenocarcinoma before color transformation, (B) the image with color transformation, and (C) the image formed into a three-dimensional cell block.

[0036] FIG. 19 shows a comparison of in three-dimensional (3D) reconstructed images of non-keratinizing squamous cell carcinoma and adenocarcinoma (top row), and a comparison of cell diagrams for non-keratinizing squamous cell carcinoma and adenocarcinoma (bottom row).

[0037] FIG. 20A and 20B show a comparison of 3D reconstructed images of (A) malignant mesothelioma and (B) adenocarcinoma cells.

[0038] FIG. 21 A and 21B show a comparison of H&E and Pap staining in 3D-reconstructed images.

[0039] FIGs. 22A and 22B show a comparison of H&E-stained cell block images and corresponding digitally generated Papanicolaou (Pap) images from the same specimen.

[0040] FIG. 23 depicts a comparison of conventional H&E (left) and digitally generated Papanicolaou (right) images from the same cell block region, illustrating how the digital Pap conversion improves distinction between nucleus and cytoplasm and reveals fine ultrastructural details.

[0041] FIG. 24A depicts a comparison of non-keratinizing squamous cell carcinoma (left) and adenocarcinoma (right) in H&E-stained cell slide images and corresponding cell diagrams.Aty. Dkt. No.: 115872-3415

[0042] FIG. 24B depicts a comparison of non-keratinizing squamous cell carcinoma (left) and adenocarcinoma (right) in Papanicolaou (Pap) stained slide images and corresponding cell diagrams.[0043J FIG. 24C depicts a comparison of adenocarcinoma (left) and malignant mesothelioma (right) in Papanicolaou (Pap) stained slide images and corresponding cell diagrams.

[0044] FIG. 24D depicts a comparison of adenocarcinoma (left) and malignant mesothelioma (right) in Papanicolaou (Pap) stained slide images and corresponding cell diagrams.

[0045] FIG. 25 depicts a schematic overview of the cell block concept and its optimization by the proposed digital staining method. Cells suspended in effusion fluids may be aggregated, fixed, and embedded to form a cell block, enabling direct observation of the cells themselves rather than background tissue.

[0046] FIG. 26 depicts a block diagram of a cell block being sectioned and then imaged to acquire H&E-stained cell slide images.

[0047] FIG. 27 depicts a digital conversion of a H&E stained slide image to a digital Papanicolaou (Pap) stained slide image via a reference image.

[0048] FIGs. 28A and 28B depict a comparison between H&E stained slide images to their corresponding digital Papanicolaou (Pap) stained slide images.

[0049] FIG. 29 depicts a comparison between a H&E stained slide image to a digital Papanicolaou (Pap) stained slide image

[0050] FIG. 30 depicts a block diagram of a server system and a client computer system in accordance with an illustrative embodiment.DETAILED DESCRIPTION

[0051] Following below are more detailed descriptions of various concepts related to, and embodiments of, systems and methods for generating biomedical images of cellAtty. Dkt. No.: 115872-3415 blocks. It should be appreciated that various concepts introduced above and discussed in greater detail below may be implemented in any of numerous ways, as the disclosed concepts are not limited to any particular manner of implementation. Examples of specific implementations and applications are provided primarily for illustrative purposes.

[0052] Section A describes systems and methods for generating biomedical images of cell blocks using machine learning and tomography

[0053] Section B methods for validating three-dimensional cellular arrangement visualization techniques for use in diagnostics.

[0054] Section C describes a network environment and computing environment which may be useful for practicing various computing related embodiments described herein.A. Systems and Methods for Generating Biomedical Images of Cell or Tissue Blocks Using Machine Learning and Tomography

[0055] Evaluating whole cell cluster in the cell (or tissue block) using Whole Block Scanner or using serial sections (100s slides) of the cell (or tissue block), the technology described herein can provide automated assessment (or manual assessment by pathologists) observing or analyzing the virtually created three-dimensional (3D) whole cells in the entire cell block. The 3D visualization may be use to evaluate malignancy or abnormality of the specimen more accurately. When a whole block scanner is used, it is nondestructive technology. Companion stain slides may be created to assess proteins, DNA, and RNA status which cytology slides by themselve cannot typically provide.[0056 J Cytology is an established branch of pathology, allowing disease diagnosis at a cellular level. Compared to tissue biopsies, the acquisition of cytological samples is a minimally invasive procedure with specimens obtained via fine needle aspiration or exfoliative procedures. A reduced patient burden, combined with the simplicity of the procedure and its cost effectiveness, makes cytology a valuable diagnostic tool for fast and early cancer detection, including screening.Atty. Dkt. No.: 115872-3415

[0057] Cytological specimens are either smeared on microscope slides and visualized with the addition of a stain (Papanicolaou stain), or a cell block is created.Standard Papanicolaou smears, being a 2D technique, only provide a coarse indication of the 3D cells structure, aiding the identification of cell and nuclei shape, clustering, and pattern. This is often essential information to determine the patient treatment pathway, in terms of ordering appropriate companion diagnosis. With this approach, limited staining is possible (typically Papanicolaou), which limits the extracted diagnostic information.[0058 J To address the technical challenges with cytology smears, cell blocks may be used as a diagnostic tool, replacing conventional cytology smears and liquid based biopsies in oncology. Cell blocks, formed by embedding cytological specimens in paraffin blocks, preserve tissue architecture (as opposed to standard cytology), and can be treated as a minibiopsy. Drawing information from the surrounding tissue architecture, combined with visualization of nuclear morphology, can lead to an increased diagnostic specificity.Paraffin-embedded cell blocks are compatible with a range of ancillary and molecular tests designed for Formalin-Fixed Paraffin-Embedded (FFPE) specimens, including immunochemistry, ultrastructure studies, and molecular testing, including cytogenetic and polymerase chain reaction (PCR)-based techniques. The stability of cell blocks allows for archival storage, leaving the cytological specimen available for further diagnostic tests or patient follow-up.

[0059] Cellular examination allows for the three-dimensional observation of individual cell structures in all organs. However, cytology slide cannot be performed with multiple staining types to decide treatment and evaluate treatment like histology slides can. Cell blocks enable staining across multiple slides, but they lack the capability to observe three-dimensional structures. Using a cell block combined with tomography (e.g., micro computed tomography (Micro-CT) or nano computed tomography (nano-CT)) allows for the confirmation of three-dimensional structures through CT imaging, and additionally enables thin sectioning and staining from the block. This may enable the determination of malignancy versus benignity, diagnosis of tissue type, and selection of therapeutic drugs.[0060J There may be a number of advantages from using cell blocks. First, fluid specimens such as cerebrospinal fluid, pleural effusions, and ascites provide findings thatAty. Dkt. No.: 115872-3415 cannot be obtained from tissue samples. While tissue blocks can be used for specimens like pleura and peritoneum, fluid samples like ascites or pleural effusions fall within the realm of cellular examination, offering information inaccessible through tissue examination. Second, specimen collection for cellular examination imposes far less physical burden on patients compared to tissue extraction.

[0061] Cell blocks are examined under a microscope or digitally using Whole Slide Imaging (WSI); however, the 3D information is largely lost, and the inherent 2D visualization can miss out on the key information provided by cell shapes and clustering that are often useful to discriminate between malignant and benign cancers and determine tumor sub-types in the malignant case. Cell blocks tend to be more fragile than regular blocks, making serial sectioning difficult and limiting the possibility of 3D recombination of serial slices, due to cutting artefacts.

[0062] Cells and nuclei are correctly visualized in the 3D CT scan, it would be possible to color-code all features in the volume (e.g., initially with the support of training from H&E slides). This would digitally stain the CT volumes for improved visualization by the pathologists or Al system, e.g., applying the equivalent of H&E and Papanicolaou staining. Diagnostic performance may be improved, resulting in better diagnosis including through an improved selection of companion diagnostic methods. This can ultimately benefit the patients and reduce the complexities of healthcare. A successful outcome would also open the use of cell block technology to various areas.

[0063] FIG. 1 depicts a block diagram of a protocol for three-dimensional histology. As depicted, a computing system can generate a whole cell or tissue block image from FFPE cell block or serial sections of whole slide images (WSIs). The computing system can support whole cell block images using computed tomography (CT) (e.g., a nondestructive imaging technique such as micro-CT or nano-CT) and hematoxylin and eosin (H&E) WSIs to create three-dimensional images of the cell block in a different modality (e.g., Papanicolaou stain). The cell block (e.g., in FFPE cell block) can be scanned using a high-resolution imaging scanner. The tomography can include, for example, 100-1000 serial sectioning, with a thickness between 0.5 pm-20 mm. The scanned image can beAtty. Dkt. No.: 115872-3415 reconstructed and can be stored (e.g., using an imaging format such as DICOM CT image format).

[0064] In addition, a serial section of the cell block can be produced and placed onto H&E slides. After the serial section, whole image images (WSIs) can be generated from the H&E slides. The WSI images can be, for example, 0.5 pm-2 pm per pixel. The computing system can create three-dimensional images from the WSIs or CT images and can convert the three-dimensional image into a Papanicolaou stain 3D model for assessment. A user of the system can examine the CT cell block, H&E image slides, and / or virtual three-dimensional cell block with the Papanicolaou stain to evaluate in a similar fashion as cytology images. The user can use the information from the images to make diagnoses and request additional slides for further assessment. To facilitate the evaluation, the computing system can use machine learning to detect features (e.g., using histological or cytology classification) and classify the features. The computing system can generate stacked WSI files with the classifications for the user to review.

[0065] FIG. 2 depicts a block diagram of a system for reconstructing volumetric images from computed tomography data of cell or tissue blocks. As depicted, the computing system can receive a computed tomography (CT) dataset acquired by an imaging device from scans of a FFPE cell block. Using the CT dataset, the computing system can perform a reconstruction to create a volumetric image corresponding to the cell block.

[0066] FIG. 3 depicts a block diagram of a device for generating computed tomography (CT) images from cell blocks. As depicted, an imaging device can acquire a scan of a FFPE cell block. The computing system can generate a digitally stained color model (e.g., H&E or PAP) of whole slide images (WSIs) or whole block images (WBIs).

[0067] FIG. 4 depicts a block diagram of a system for generating computed tomography (CT) images from cell or tissue blocks. As depicted, the cell block can be sliced and stained with H&E or PAP. The imaging device can generate WSI or PAP images from the slices. This can facilitate pathologists who evaluate slides by switching among different imaging modalities as represented by these stains. For example, the virtual Papanicolaou 3D image can be evaluated as if it were a cytology slide. Users can make diagnosis and also request additional stains such as IHC for further assessment. In addition,Aty. Dkt. No.: 115872-3415 artificial intelligence (Al) can be used to classify the case using histology classification and cytology classification to improve the accuracy of the assessment.f<»0681 FIGs. 5A-C depicts a flow diagram of a process for generating biomedical images of Formalin-Fixed Paraffin-Embedded (FFPE) specimens. Starting from FIG. 5A, a fresh tissue sample can be obtained from a subject. At least a portion of the fresh tissue sample can be stained or inked. From the tissue sample, N tissue blocks can be dissected from the stained portion. N can range anywhere from 5 to 100. Each of the N tissue blocks can be placed or housed in a respective cell block (e.g., an FFPE block). There may be a total of N cell blocks to house the N tissue blocks taken from the fresh tissue sample.

[0069] Moving onto FIG. 5B, each of the tissue blocks in the cell blocks can be scanned by a micro computed tomography (micro-CT) scanner to create micro-CT images. Machine learning (ML) can be used to convert the micro-CT images into a digitally stained whole block image (WBI). The WBI can include a set of serially stacked images, converted from the micro-CT images. The digital staining can include H&E staining in the depicted examples. In conjunction, the tissue blocks in the cell blocks can be sectioned, stained using H&E staining, and scanned using a slide scanner to generate whole slide images (WSIs). The WSIs can be used to supplement the WBI. The WSIs can be annotated to indicate regions of interest (RO I) corresponding to particular features on the depicted sample, such as cancerous cells.

[0070] With reference to FIG. 5C, ML can be applied to the WBI or WSI (or both) to generate various outputs about features depicted in the images. The outputs may include a set of slide identifiers, with findings about the WBI and corresponding WSI. The findings can include, for example, a presence or absence of tumors, a type of tumor (e.g., benign or malignant), and a type of cancerous tissue, among others. ML can be also used to determine correlations between WSIs and WBIs.

[0071] FIG. 6 depicts a flow diagram of a process for applying machine learning on image data from computed tomography and histological slides. As depicted, a WBI can be generated using micro-CT scan data acquired from scanning a tissue block. In conjunction, the tissue block can be sliced and stained (e.g., using H&E staining), and then scanned using a slide scanner to generate WSIs. The WSI can be annotated to identify portionsAtty. Dkt. No.: 115872-3415 corresponding to regions of interest (ROIs), such as tumors or lesions. Together with the annotations, ML can be used to detect ROIs within the WBI, leading to a higher quality and more useful WBI.[0072J FIG. 7A depicts an example of a three-dimensional micro computed tomography (micro-CT) image of lymph node with metastatic papillary thyroid carcinoma. As depicted, the micro-CT image may be three-dimensional, and generated by combining CT scans of different sections of the tissue block. FIGs. 7B and 7C depict an example of an assessment of metastatic papillary thyroid carcinoma volume in lymph node. In each of the depictions, on the left is a cell block with sample tissue taken from the lymph node. Second is the scan image of a section the cell block, digitally converted from micro-CT to H&E-stained image. Third is a three-dimensional image of the cell block. On the right are the results of ML applied to the image to provide a metastatic papillary thyroid carcinoma volume (e.g., 0.0079 cm3and 0.015 cm3).(0073) FIG. 8A depicts a screenshot of an example user interface of three-dimensional views of whole slide images (WSI) and whole block images (WBI). As depicted, the user interface can include various views of WSI images taken from scans of sections of a tissue sample. The WSI images may be generated by converting micro-CT images and digitally converting with H&E staining. FIG. 8B depicts a screenshot of an example user interface of a juxtaposition of a whole slide image (WSI) and a whole block image (WBI) about a region of interest (ROI). As depicted, the user interface can present a juxtaposition of a section from the WBI and the corresponding WSI. This can be used to assess for particular features (or regions of interest (ROI)) corresponding to a tumor or lesion in the tissue sample. FIG. 9 depicts example images of adenocarcinoma from three-dimensional whole block images (WBI) generated from micro computed tomography (micro-CT).100741 FIG. 10 depicts an example of a protocol to perform serial sectioning of a cell block specimen for analyzing. As depicted, a cell block may be serially sectioned, and each section may be stained and placed on a respective slide. The set of slides of the cell block may be imaged to generate a corresponding set of whole slide images (WSIs). The set of whole slide images (WSIs) may be used to generate a whole block image (WBI). AAty. Dkt. No.: 115872-3415 machine learning model may be used to detect the presence of regions of interest (ROIs) on each section of the WBI (or a corresponding WSI). The ROIs may correspond to suspicious areas in the cell block specimen, such as potential malignant tumors. From the WBI, the machine learning model can be used to find a section of the WBI (e.g., depicted generally along the bottom) with the largest amount or area of ROIs for further analysis. As seen, this section of the WBI may be detected as having adenocarcinoma cells.[0075 J Referring now to FIG. 11, depicted is a block diagram of a system 100 for generating biomedical images of cell or tissue blocks for analysis across image modalities. In brief overview, the system 100 may include at least one image processing system 105, at least one computed tomography (CT) scanner 110, at least one slide scanner 115, and at least one administrative device 120, among others, communicatively coupled with one another via at least one network 125. The image processing system 105 may include at least one image indexer 130, at least one block constructor 135, at least one model trainer 140, at least one model applier 145, at least one output evaluator 150, at least one interaction handler 155, at least one machine learning (ML) model 160, and at least one database 165, among others. The system 100 may be used to implement the functionalities as described in Section B. Each of the components in the system 100 as detailed herein may be implemented using hardware (e.g., one or more processors coupled with memory) or a combination of hardware and software as detailed herein in Section C.[00761 In further detail, the image processing system 105 (sometimes herein generally referred to as a computing system or a server) may be any computing device, including one or more processors coupled with memory and software and capable of performing the various processes and tasks described herein. The image processing system 105 may be associated with an entity assisting clinicians examining subjects for various cancers, or providing a platform for the analysis of biomedical images. The image processing system 105 may be in communication with the CT scanner 110, the slide scanner 115, and the administrative device 120, among others, via the network 125. The image processing system 105 may be situated, located, or otherwise associated with at least one server group. The server group may correspond to a data center, a branch office, or a site at which one or more servers corresponding to the image processing system 105 is situated.Atty. Dkt. No.: 115872-3415

[0077] The imaging processing system 105 can include or execute any number of modules, processes, components, or subcomponents to performing the various processes and tasks described herein. On the image processing system 105, the image indexer 130 may receive biomedical images from the CT scanner 110 or the slide scanner 115. The block constructor 135 can generate a volumetric biomedical image using the image data from the CT scanner 110. The model trainer 140 can initialize, train, and establish the ML model 160. The model applier 145 can use the ML model 160 to perform a number of functions using the biomedical images. The output evaluator 150 can generate information based on the application of the ML model 160 on the biomedical images. The interaction handler 155 can provide a user interface to facilitate invocations of various functions provided by the image processing system 105.

[0078] The ML model 160 may be any type of artificial intelligence algorithm or model architecture configured to perform any number of functions on the biomedical images. The functions can include, for example, digital conversion of the biomedical image from an initial imaging modality to a target imaging modality (e.g., introducing staining); image segmentation to detect various features within the biomedical images; and classification of the features in the image, among others. The architecture may include, for example, a clustering algorithm (e.g., ^means clustering), an artificial neural network (e.g., an encoder with a convolutional neural network architecture, a residual neural network (ResNet), or transformer-based network), a support vector machine (SVM), a decision tree, a Bayesian model, a regression model, or a transformer model, among others. In general, the ML model 160 may have at least one input and at least one output. The output and the input may be related via a set of weights. The input may be at least one biomedical image (e.g., from the CT scanner 110 or the slide scanner 115). The output may correspond to the functions using the input image in accordance with the set of weights.

[0079] The CT scanner 110 (sometimes herein generally referred to as a tomograph, imaging device, or an image acquirer) may be any device to acquire images of cell blocks (or tissue blocks). The CT scanner 110 may execute, carry out, or otherwise perform X-ray scan of a cell block (or tissue block) to generate a set of CT images. Each CT image may correspond to a two-dimensional slice of the cell block. The set of CT images may be generated by rotating or moving the scanner about the cell block. In some embodiments,Atty. Dkt. No.: 115872-3415 the CT scanner 110 may perform micro-computed tomography (micro-CT), with a resolution ranging between 1-100 pm, or nano-computed tomography (nano-CT) with resolution ranging between 20 nm to 400 nm. The CT scanner 110 may be in communication with the image processing system 105 and the administrative device 120, among others, via the network 125.

[0080] The slide scanner 115 (sometimes herein generally referred to as a whole slide scanner, a digital slide, scanner, an imaging device, or an image acquirer) may be any device to acquire images of slides. The slide scanner 115 may execute, carry out, or otherwise perform a scan of a slide with a tissue sample. The tissue sample on the slide may have been dyed using a histological or cytological stain (e.g., Hematoxylin and Eosin (H&E) stain) to differentiate colors of certain features (e.g., cells) within the tissue sample. The scanning of the tissue sample on the slide may be in accordance with microscopy (e.g., light microscopy, brightfield microscopy, fluorescence microscopy, confocal microscopy, or multi -photon microscopy). The slide scanner 115 may be in communication with the image processing system 105 and the administrative device 120, among others, via the network 125.

[0081] The administrative device 120 (sometimes herein referred to as a client device, a client, or an end user computing device) may be any computing device comprising one or more processors coupled with memory and software and capable of performing the various processes and tasks described herein. The administrative device 120 may be in communication with the image processing system 105, the CT scanner 110, the slide scanner 115, via the network 125. The administrative device 120 may have at least one display. The administrative device 120 may be associated with an entity (e.g., a clinician) examining a tissue sample from the subject or the biomedical images from the tissue sample. The display may present information about the subject provided by the image processing system 105.

[0082] The database 165 may store and maintain various resources and data associated with the data processing system 105 and the administrative device 120, among others. The database 165 may include a database management system (DBMS) to arrange and organize the data maintained thereon. The database 165 may be in communication with the data processing system 105 and the administrative device 120, via the network 125.Atty. Dkt. No.: 115872-3415 While running various operations, the data processing system 105 and the administrative device 120 may access the database 165 to retrieve identified data therefrom. The data processing system 105 and the administrative device 120 may also write data onto the database 165 from running such operations.

[0083] Referring now to FIG. 12, depicted is a block diagram for a process 200 of obtaining image data in the system 100 for generating biomedical images of cell blocks. Under the process 200, the image indexer 130 on the image processing system 105 may retrieve, identify, or otherwise receive a set of biomedical images of at least one biological sample 210 from at least one subject 205 from the CT scanner 110 or the slide scanner 115. The subject 205 may be a human or an animal at risk of or diagnosed with a condition (e.g., a medical condition or indication). In some embodiments, the subject 205 may be evaluated for the condition for diagnosis and treatment purposes.|00841 The condition may include at least one of a cancer, an infection, an inflammation, or a degeneration, among others. The cancer may be, for example, skin cancer (e.g., melanoma, Basal cell carcinoma (BCC) and squamous cell carcinoma (SCC)), lung cancer (e.g., non-small cell lung cancer (NSCLC) or small cell lung cancer (SCLC)), brain cancer (e.g., glioblastoma multiforme (GBM), astrocytoma, oligodendroglioma, meningioma, and medulloblastoma), head and neck cancer (e.g., squamous cell carcinoma of the larynx, nasopharyngeal carcinoma, oropharyngeal carcinoma, salivary gland tumors, and thyroid carcinoma), colon cancer (e.g., adenocarcinoma, mucinous carcinoma rectal cancer, and gastrointestinal stromal tumors (GISTs)), uterine cancer (e.g., endometrial cancer and uterine sarcoma), stomach cancer (e.g., adenocarcinoma, gastrointestinal stromal tumor (GIST), neuroendocrine tumors, and lymphoma), ovarian cancer (e.g., epithelial, germ cell, and stromal), cervical cancer (e.g., squamous cell carcinoma and adenocarcinoma), bladder cancer (e.g., urothelial carcinoma, squamous cell carcinoma, and adenocarcinoma), prostate cancer (e.g., adenocarcinoma or transitional cell carcinoma), or breast cancer (e.g., invasive ductal carcinoma (IDC), invasive lobular carcinoma (ILC), triple-negative breast cancer (TNBC), HER2 -positive breast cancer, and inflammatory breast cancer), among others.Atty. Dkt. No.: 115872-3415

[0085] The infection may include at least one of a bacterial infection (e.g., disease caused by bacteria infecting cells), a viral infection (e.g., disease caused by virus invading cell), a fungal infection (e.g., diseases caused by fungi growing on or in tissues), or a parasitic infection (e.g., disease caused by parasites living on or inside), among others. The inflammation may include, for example, at least one of an acute inflammation (e.g., involving neutrophils) or a chronic inflammation (e.g., involving lymphocytes, plasma cells, or macrophages), among others. The cellular degeneration may include, for example, at least one of a nuclear degeneration (e.g., structural breakdown or damage to cell nucleus), cytoplasmic vacuolation (e.g., formation of vacuoles or round spaces within cell cytoplasm), cytolysis (e.g., destruction or dissolution of cell, with loss of cytoplasm), or cell atrophy (e.g., decrease in cell size or function relative to healthy, normal cell), among others.

[0086] The biological sample 210 may be taken, collected, or otherwise obtained from at least one organ or anatomical site associated with the condition from the subject 205. The biological sample 210 may include, for example, a tissue sample, a bone marrow, or a blood sample, among others, or any combination thereof. The organ or anatomical site of the biological sample 210 may include for example, at least a portion of lung, breast, colon, stomach, liver, pancreas, prostate, cervix, ovary, skin, esophagus, kidney, urinary bladder, head and neck mucosa, femur, tibia, pelvis, spine, brain, uterine cervix, rectum, uterine corpus, endometrium, myometrium, ear, nasal cavity, paranasal sinuses, pharynx, larynx, small intestine, anus, oral cavity, salivary glands, spleen, lymphatic system, lymph nodes, pleura, peritoneum, pericardium, tunica vaginalis, bone marrow, eye, uvea, adrenal medulla, sympathetic nervous system, paraspinal ganglia, lips, tongue, floor of mouth, buccal mucosa, hard palate, gingiva, penis, nasopharynx, oropharynx, hypopharynx, testes, mediastinum, sacrococcygeal region, thyroid gland, vagina, blood vessels, lymphatic vessels, or adrenal gland, among others, of the subject 205.

[0087] The biological sample 210 may have been from the organ associated with the condition in the subject 205 via a biopsy (e.g., needle, incisional, excisional, and endoscopic biopsy). The biological sample 210 may include any number of cells. For a tissue sample, the cells may include, for example, epithelial cells, mesenchymal cells (e.g., fibroblasts or adipocytes), endothelial cells, immune cells (e.g., lymphocytes, macrophages, and neutrophils), and tumorous cells (e.g., benign or malignant), among others. The biologicalAtty. Dkt. No.: 115872-3415 sample 210 taken from the subject 205 may have any set of dimensions, for example, a length of between 0.1 cm to 10 cm, a width between 0.1 cm to 10 cm, and a thickness of between 1 mm to 10 mm (e.g., between 3 to 7 millimeters), among others.[0088 J A sample block 215 (sometimes herein referred to as a cell block) may contain, secure, or otherwise house the biological sample 210 from the subject 205. The sample block 215 may be a structure having an interior volume within which the biological sample 210 is to be placed, secured, or otherwise housed. The biological sample 210 may be placed, situated, or otherwise disposed within an interior volume of the sample block 215. The biological sample 210 may be housed within the interior volume of the sample block 215 for purposes of scanning by the CT scanner 110. In some embodiments, the sample block 215 may include or use any technique to fix the biological sample 210 for analysis. For example, the cell block may include a Formalin-Fixed, Paraffin-Embedded (FFPE) block, Ethanol-Fixed Paraffin-Embedded (EFPE) block, dry tissue preservation, or frozen tissue preservation, among others. For instance, to prepare for the FFPE block, the biological sample 210 may be immersed, secured, or fixed within using a solution. The solution may be, for example, formalin or formaldehyde solution to cross-link proteins and nucleic acids of the tissue sample in the biological sample 210. With the fixation, the biological sample 210 may undergo dehydration using a solution (e.g., an ethanol solution) to remove water from the biological sample 210. The biological sample 210 may also be included or embedded in a wax medium (e.g., paraffin). Upon cooling, the biological sample 210 may be arranged, placed, or otherwise disposed in the interior of the sample block 215 for additional analysis.(0089) In some embodiments, the biological sample 210 may be cut, divided, or otherwise sliced into one or more sample sections 220A-N (hereinafter generally referred to as sample sections 220). The biological samples 210 may be divided into the sample sections 220 prior to or subsequent to placement within the sample block 215. In some embodiments, the biological sample 210 may be divided into the sample sections 220 prior to or subsequent to the application of fixation (e.g., FFPE). The biological sample 210 may be sliced for the purposes of scanning by the CT scanner 110 or the slide scanner 115 (e.g., for whole slide imaging (WSI)). Each sample section 220 may correspond to a slice, layer, or portion of the biological sample 210. Each sample section 220 may have any dimensionAty. Dkt. No.: 115872-3415 or any set of dimensions, for example, a length of between 0.1 cm to 5 cm, a width between 0.1 cm to 5 cm, and a thickness of between 1 pm to 50 pm, among others. Each sample section 220 may be arranged, placed, or otherwise situated on a corresponding slide (e.g., a glass slide).

[0090] Each sample section 220 may be stained, inked, or otherwise modified using a stain to facilitate the imaging of thereof. In some embodiments, each sample section 220 may be stained using a histological stain to enhance visualization of cellular and tissue structures. The histological stain may include, for example, hematoxylin and eosin (H&E), hemosiderin stain, a Sudan stain, a Schiff stain, a Congo red stain, a Gram stain, a Ziehl-Neelsen stain, a Auramine-rhodamine stain, a trichrome stain, a silver stain, and Wright’s Stain, among others. In some embodiments, each sample section 220 may be stained using an immunohistochemical (IHC) stain to detect specific proteins within the cells of the biological sample 210. The IHC stain may include, for example, cytokeratin stain, epithelial membrane antigen (EMA) stain, Ki-67 stain, CD markers (e.g., CD3, CD4, CD8, CD20, CD34, CD56, and CD117), mesenchymal marker, and neural markers, among others. In some embodiments, each sample section 220 may be stained using a cytology stain to detect and evaluate cells from body fluids and tissue scrapings. The cytological stain may include, for example, Papanicolaou (PAP) stain, Shorr stain, and H&E stain, among others.

[0091] The CT scanner 110 may carry out, execute, or otherwise perform computed tomography (CT) (e.g., micro-computed tomography (micro-CT) or nano-CT) on the sample block 215 to generate a set of CT images 230A-N (hereinafter generally referred to as CT images 230 or biomedical images). The set of CT images 230 may be associated with or correspond to a set of scanning sections 225 A-N (hereinafter generally referred to as scanning sections 225) through the volume of the sample block 215. Each CT image 230 may be associated with or may correspond to a respective scanning section 225. Each scanning section 225 may correspond to a two-dimensional section of the biological sample 210. The set of scanning sections 225 may be at an interval of 1-10pm. The set of CT images 230 may be in accordance with an image modality 235 (e.g., the CT, micro-CT, or nano-CT imaging modality). The number of CT images 230 may range between 50 to 10,000, among others.Aty. Dkt. No.: 115872-3415

[0092] In performing CT, the CT scanner 110 may radiate, transmit, or emit X-rays to penetrate the biological sample 210 within the interior volume of the sample block 215. The X-rays may be about a corresponding scanning section 225 through the sample block 215. With the emission of the X-rays, the CT scanner 110 may detect, measure, or otherwise acquire the X-ray projections through the biological sample 210 via one or more sensors. Each acquisition may correspond to a respective scanning section 225 and by extension, a corresponding CT image 230. The corresponding CT image 230 may include a set of pixels, with each pixel indicating an amount of absorption or passing through of the X-ray through the biological sample 210. In some embodiments, the CT scanner 110 may acquire the CT images 230 in accordance with set scan parameters. The scan parameters may include, for example, a source-to-sample distance (affecting resolution and scan area), tube energy (affecting sample penetration and noise), tube power (affecting sample penetration and resolution), exposure time (affecting sample penetration, noise, and scan time), a number of projections (affecting projections line artefact), frames per projection (affecting random noise reduction), a source-to-detector distance (affecting sample penetration and noise), and gain of sensor panel (affecting sample penetration, noise, and scan time), among others. The CT scanner 110 may translate, rotate, or otherwise move the sensors relative to the biological sample 210 within the interior volume of the sample block 215 to acquire additional X-ray projections. By scanning across the volume of the sample block 215, the CT scanner 110 may generate the set of CT images 230 corresponding to the set of scanning sections 225. With the generation of the set of CT images 230 of the biological sample 210, the CT scanner 110 may send, transmit, or otherwise provide the set of CT images 230 to the image processing system 105.

[0093] In conjunction, the slide scanner 115 may carry out, execute, or otherwise perform imaging on the set of sample sections 220 from the biological sample 210 to generate a set of slide images 240A-N (hereinafter generally referred to as slide images 240). The set of slide images 240 may be associated with or correspond to the set of sample sections 220 from the biological sample 210. The set of slide images 240 may be of an image modality 235’. The image modality 235’ may be of a microscopy imaging modality for a histological stain, an H4C stain, or a cytological stain, among others. For each sample section 220, the slide scanner 115 may capture, scan, or otherwise acquire a corresponding1Atty. Dkt. No.: 115872-3415 slide image 240. By repeating over the set of sample sections 220, the slide scanner 115 may produce or generate the corresponding set of slide images 240. The number of slide images 240 may range from 10 to 500, among other numbers. In some embodiments, the slide scanner 115 may acquire scans of the biological sample 210 or the set of sample sections 220 at different focal planes or depths (also referred herein as Z-positions). The distance between adjacent focal planes or depths may range between 0.1 to 2.5 pm. The acquisition of the scans may be in accordance with Z-stacking to generate a Z-stacked images corresponding to set of slide images 240. Each slide image 240 may correspond to a different focal plane or depth. With the generation of the set of slide images 240, the slide scanner 115 may send, transmit, or otherwise provide the set of slide images 240 to the image processing system 105.[0094) The image indexer 130 may retrieve, identify, or otherwise receive the set of CT images 230 from the CT scanner 110. With receipt, the image indexer 130 may store and maintain the set of CT images 230 on the database 165, using one or more files (e.g., digital imaging and communications in medicine (DICOM) file, portable network graphics (PNG), or tagged image file format (TIFF)). In conjunction, the image indexer 130 may retrieve, identify, or otherwise receive the set of slide images 240 from the slide scanner 115. With receipt, the image indexer 130 may store and maintain the set of slide images 240 on the database 165, using one or more files (e.g., digital imaging and communications in medicine (DICOM) file, portable network graphics (PNG), or tagged image file format (TIFF)). In some embodiments, the image indexer 130 may store and maintain an association between the set of CT images 230 and the set of slide images 240 on the database 165. The association may be based on an identification of the subject 205, the biological sample 210, or the sample block 215, among others.100951 The block constructor 135 on the image processing system 105 may create, produce, or otherwise generate at least one whole block image (WBI) 245 using the set of CT images 230. The WBI 245 may correspond to the volume (e.g., a three-dimensional volume) including the biological sample 210 in the sample block 215. The WBI 245 may be of the image modality 235 (e.g., CT, micro-CT, or nano-CT). The block constructor 135 may apply any number of reconstruction algorithms to generate the WBI 245 using the two-dimensional scans corresponding to the CT images 230. The reconstruction algorithm mayAtty. Dkt. No.: 115872-3415 include, for example, iterative reconstruction, interpolation (e.g., linear or cubic interpolation), and three-dimensional volume rendering, among others. For example, in carrying out the reconstruction algorithm, the block constructor 135 may combine or stack the set of CT images 230 (e.g., using image registration) to generate an initial WBI 245 including a set of voxels. The block constructor 135 may apply interpolation to produce or output values for voxels corresponding to portions of the WBI 245 between two-dimensional slices corresponding to adjacent CT images 230.[0096 J In some embodiments, the block constructor 135 may generate the WBI 245 using the set of slide images 240. The generation of the WBI 245 using the set of slide images 240 may be similar to the generation using the set of CT images 230. The WBI 245 may correspond to the volume (e.g., a three-dimensional volume) including the biological sample 210 in the sample block 215. The WBI 245 generated using the set of slide images 240 may be of the image modality 235’ (e.g., a microscopy imaging modality). The block constructor 135 may apply any number of reconstruction algorithms to generate the WBI 245 using the two-dimensional scans corresponding to the slide images 240. The reconstruction algorithm may include, for example, iterative reconstruction, interpolation (e.g., linear or cubic interpolation), and three-dimensional volume rendering, among others. For example, in carrying out the reconstruction algorithm, the block constructor 135 may combine or stack the set of CT images 230 (e.g., using image registration) to generate an initial WBI 245 including a set of voxels. The block constructor 135 may apply interpolation to produce or output values for voxels corresponding to portions of the WBI 245 between two-dimensional slices corresponding to slide images 240.[0097) Referring now to FIG. 13, depicted is a block diagram for a process 300 of using machine learning to evaluate biomedical images in different imaging modalities in the system 1000 for generating biomedical images of cell blocks. Under the process 300, the model trainer 140 on the image processing system 105 may initialize, train, and establish the ML model 160 using training data 305. The model trainer 140 may identify, obtain, or otherwise retrieve the training data 305 from the database 165. The training data 305 may include or identify a set of training examples 310 for training the ML model 160 to perform various functions, such as digital conversion of the biomedical image from an initial imaging modality to a target imaging modality (e.g., changing or introducing staining);Atty. Dkt. No.: 115872-3415 image segmentation to detect various features within the biomedical images; and classification of the features in the image, among others.[0098 j Each training example 310 may identify or include at least one input sample image 315 of the image modality 235 and at least one expected sample image 320 of a target image modality 235”. The pair of the sample image 315 and the expected sample image 320 may be used to train the ML model 160 to digitally convert biomedical images from an initial imaging modality (e.g., the image modality 235) to the target image modality 235”. The sample images 315 and 320 may be for the same biological sample having any number of cells (e.g., similar to the biological sample 210). The sample image 315 may be of the example biological sample in the same image modality 235 as the set of CT images 230. The target image modality 235” may be the same as or differ from the image modality 235’ of the set of slide images 240. The expected sample image 320 may be of the example biological sample applied with a stain (e.g., a histological stain, an IHC stain, or a cytological stain) associated with the target image modality 235”. The stain associated with the target image modality 235” may be to alter, change, or otherwise modify colors correspond to the cells in the example biological sample. For instance, the sample image 315 may be a CT image of the example biological sample, without any application of a stain, and the expected sample image 320 may be a whole slide image of the same example biological sample applied with an H&E stain or Papanicolaou stain.100991 In some embodiments, at least one of the training examples 310 may identify or include at least one annotation 325. The annotation 325 may be used to train the ML model 160 to perform image segmentation of the input biomedical images. The annotation 325 may define or identify at least a portion of the expected sample image 320 (e.g., as depicted, or the sample image 315) as corresponding to at least one region of interest (ROI) 330. For instance, the annotation 325 may define pixel coordinates corresponding to the ROI 330 in the expected sample image 320. The ROI 330 may correspond to a feature associated with the condition (e.g., cancer or disease) in the example biological sample. The feature may include, for example, a tumor or lesion, among others. The ROI 330 may correspond to a feature, independent of the condition in the example biological sample, such as a blood vessel or tissue, among others. The annotation 325 may include any number of ROIs 330 corresponding to different types of features.Atty. Dkt. No.: 115872-3415

[0100] In some embodiments, the annotation 325 may be used to train the ML model 160 to perform image classification of the ROI 330 in the input image. The annotation 325 may define or identify at least one of a set of feature types for the ROI 330 or by extension the feature in the example biological sample associated with the ROI 330. The classification may include, for example, a blood vessel, a tissue (e.g., submucosa or mucosa), muscle (e.g., muscularis mucosa), tumor, lesion, cell nucleus, or any lack thereof, among others. The annotation 325 may identify the classification for each ROI 330 for the expected sample image 320 (or the sample image 315). In some embodiments, the annotation 325 may be for the overall expected sample image 320. The annotation 325 may identify a presence or an absence of the condition (e.g., tumor associated with the cancer). The annotation 325 may have been manually created or generated a clinician examining the expected sample image 320 (e.g., as depicted or the sample image 315). For instance, the clinician may view the expected sample image 320 through a user interface to demarcate a location of the ROI 330 within the expected sample image 320. The clinician may also input the classification for the feature associated with the ROI 330.

[0101] With the identification, the model trainer 140 may feed, provide, or otherwise apply the sample image 315 from each training example 310 of the training data 305 to the ML model 160. In applying, the model trainer 140 may process the input sample image 315 in accordance with the set of weights of the ML model 160. From processing, the model trainer 140 may produce, determine, or otherwise generate one or more outputs from the ML model 160. The outputs of the ML model 160 may depend on the functions. In some embodiments, the model trainer 140 may generate at least one output image 320’ in the target image modality 235”. The output image 320’ may correspond to a conversion of the input sample image 315 from the initial image modality 235 to the target image modality 235”. The conversion to the target image modality 235” may serve to differentiate the colors of certain cells (e.g., associated with the condition) from a remainder of the background in the image.

[0102] In some embodiments, based on applying the ML model 160, the model trainer 140 may generate the output image 320’ to include at least one ROI 330’. The ROI 330’ may define or identify a portion of the output image 320’ corresponding to a feature associated with the condition (e.g., cancer or disease) in the example biological sample fromAtty. Dkt. No.: 115872-3415 which the input sample image 315 is derived. For example, the ROI 330’ may define the pixel coordinates within the output image 320’ corresponding to the feature associated with the condition. In some embodiments, the output image 320’ may be a segmented image with the ROI 330’ in a defined color. In some embodiments, the ROI 330’ may be generated as an overlay for the output image 320’.101031 In some embodiments, based on applying the ML model 160, the model trainer 140 may determine or generate at least one classification 335. The classification 335 may identify at least one of a set of feature type for the ROI 330’ or by extension the feature in the example biological sample associated with the ROI 330’. The classification 335 may include any type of features, for example, such as a blood vessel, a tissue, muscle, tumor, lesion, cell nuclei, or any lack thereof, among others. The classification 335 may identify a presence or an absence of the condition, among others. The classification 335 may include a set of alphanumeric values or at least one enumerated value corresponding to the feature type of the ROI 330’.

[0104] The model trainer 140 may calculate, generate, or otherwise determine at least one loss metric 340. The loss metric 340 may be calculated, generated, or otherwise determined based on a comparison between the one or more outputs from the ML model 160 with the training example 310 of the training data 305. The loss metric 340 may be calculated in accordance with any number of loss functions, such as a norm loss (e.g., LI or L2), mean squared error (MSE), a quadratic loss, a cross-entropy loss, and a Huber loss, among others. In some embodiments, the model trainer 140 may determine the loss metric 340 based on a comparison between the output image 320’ and the expected sample image 320 associated with the input sample image 315. The loss metric 340 may indicate a degree of deviation between the output image 320’ and the expected sample image 320 with respect to the conversion from the initial image modality 235 to the target image modality 235’.

[0105] In some embodiments, the model trainer 140 may determine the loss metric 340 based on a comparison between the ROI 330’ in the output image 320’ and the ROI 330 of the expected sample image 320 (e.g., in terms of location, shape, or area) as identified in the annotation 325. The loss metric 340 may indicate a degree of deviation in theAtty. Dkt. No.: 115872-3415 determination of the output ROI 330’ and the expected ROI 330. In some embodiments, the model trainer 140 may determine the loss metric 340 based on a comparison between the classification 335 from the ML model 160 and the classification as indicated in the annotation 325. The loss metric 340 may indicate a degree of deviation in the output classification 335 and the expected classification as identified in the annotation 325. In some embodiments, the model trainer 140 may determine the loss metric 340 as a combination of loss metrics calculated based on the comparisons between the outputs and the respective component of the training example 310 of the training data 305.

[0106] Using the loss metric 340, the model trainer 140 may modify, change, or otherwise update one or more weights of the ML model 160. The updating of the weights of the ML model 160 may be in accordance with an optimization function (or an objective function). The optimization function may define one or more rates or parameters at which the weights of the ML model 160 are to be updated. The updating of the weights may be repeated until convergence. The model trainer 140 can update the ML model 160 in accordance with the loss metric 340. For example, the weights of the ML model 160 may be modified or updated using the loss metric 340 in accordance with an objective function (e.g., stochastic gradient descent (SGD)). The ML model 160 may be iteratively updated by the model trainer 140 using the training data 305, until convergence to complete the training and establish the ML model 160.

[0107] With the establishment of the ML model 160, the model applier 145 on the image processing system 105 may carry out, perform, or otherwise carry out various functions. The model applier 145 may feed, provide, or otherwise apply the WBI 245 to the ML model 160. The application of the WBI 245 may be to perform a digital conversion of the initial imaging modality 235 to the target imaging modality 235”, image segmentation, or classification of features of the image, among others. In applying the ML model 160 to the WBI 245, the model applier 145 may select or identify a set of slices (e.g., corresponding to the CT images 230) from the WBI 245. Each slice may correspond to a two-dimensional section from the WBI 245. With the identification of each slice from the WBI 245, the model applier 145 may feed the image slice to the ML model 160. In feeding, the model applier 145 may process the input image slice in accordance with the set of weights of the ML model 160. In some embodiments, the model applier 145 may apply oneAtty. Dkt. No.: 115872-3415 or more of the sets of slide image 240 to the ML model 160. The application of the slide images 240 may be to perform image segmentation or classification of features of the image, among others. From processing, the model applier 145 may produce, determine, or otherwise generate one or more outputs from the ML model 160.

[0108] Based on applying the ML model 160 (e.g., to the WBI 245), the model applier 145 may convert the input image of the WBI 245 from the initial image modality 235 to the target image modality 235”. The target image modality 235” may be to differentiate the colors of certain cells (e.g., associated with the condition) from a remainder of the background in the input image. In some embodiments, the model applier 145 may generate at least one output image 320’ in the target image modality 235”. The output image 320’ may correspond to a conversion of the input slice image from the initial image modality 235 to the target image modality 235”. The output image 320’ may correspond to a conversion of the corresponding slice image of the WBI 245 from the initial image modality 235 to the target image modality 235” to differentiate colors of certain cells of the biological sample 210.101.091 Using the output image 320’ for each input slice from the WBI 245, the model applier 145 may create, produce, or otherwise generate at least one whole block image (WBI) 245’. The model applier 145 may apply any number of reconstruction algorithms to generate the WBI 245’ using the two-dimensional output images 320’. The reconstruction algorithm may include, for example, iterative reconstruction, interpolation (e.g., linear or cubic interpolation), and three-dimensional volume rendering, among others. The WBI 245’ may correspond to the volume (e.g., a three-dimensional volume) including the biological sample 210 in the sample block 215. The WBI 245’ may correspond to a conversion of the WBI 245 from the initial image modality 235 to the target image modality 235”. In some embodiments, the model applier 145 may convert the WBI 245 from the initial image modality 235 (e.g., the CT imaging modality) to the target image modality 235” (e.g., a microscopy modality for H&E stain, PAP stain, or IHC stain) to generate the WBI 245’. For example, the WBI 245’ may correspond to the WBI 245 applied with a stain (e.g., a histological stain, an IHC stain, or a cytological stain) to accentuate, highlight, or otherwise differentiate colors of certain cells of the biological sample 210 depicted withinAtty. Dkt. No.: 115872-3415 the WBI 245’ . The WBI 245’ may be generated as a Z-stack image file, with the set of sections forming the WBI 245’ as the set of different depths or plane within the image.|011 OJ In some embodiments, the model applier 145 may use at least a portion of the slide images 240 together with the output images 320’ in generating the WBI 245’. The output images 320’ may be generated from the WBI 245 in the image modality 235 (e.g., CT, micro-CT, or nano-CT). The image modality 235’ (e.g., stain) of the slide images 240 may be the same as the image modality 235” in which the output image 320’ is in. In converting to the target image modality 235”, the model applier 145 may add at least a portion of the slide images 240 to the output images 320’ in generating the WBI 245’. To generate the WBI 245’, the model applier 145 may detect or identify slide images 240 and output images 320’ that correspond to adjacent or neighboring two-dimension slices within the volume of the sample block 215 or the biological sample 210. The identification of adjacent two-dimensional slices may be based on metadata (e.g., identifying depth) or a similarity metric (e.g., cross-correlation metric).

[0111] Using the identification of adjacent images, the model applier 145 may determine a spatial arrangement of the output images 320’ and the slide images 240. The model applier 145 may apply any number of reconstruction algorithms to generate the WBI 245’ using the slide images 240 and the output images 320’. The reconstruction algorithm may include, for example, iterative reconstruction, interpolation (e.g., linear or cubic interpolation), and three-dimensional volume rendering, among others. For example, in carrying out the reconstruction algorithm, the model applier 145 may combine or stack the set of slide images 240 and output images 320’ (e.g., using image registration) to generate the initial WBI 245’ including a set of voxels. The model applier 145 may apply interpolation to produce or output values for voxels corresponding to portions of the WBI 245’ between two-dimensional slices corresponding to slide images 240 or the output images 320’.

[0112] In some embodiments, based on applying the ML model 160 (e.g., to the WBI 245 or the slide images 240), the model applier 145 may generate the output image 320’ to include at least one ROI 330’. The ROI 330’ may define or identify a portion of the output image 320’ corresponding to a feature associated with the condition (e.g., cancer orAty. Dkt. No.: 115872-3415 disease) in the biological sample 210 from which the WBI 245 (or the slide images 240) is derived. For example, the ROI 330’ may define the pixel coordinates within the output image 320’ corresponding to the feature associated with the condition. In some embodiments, the output image 320’ may be a segmented image with the ROI 330’ in a defined color. In some embodiments, the ROI 330’ may be generated as an overlay for the output image 320’. In some embodiments, the model applier 145 may generate a set of ROIs 330’ for the set of slide images 240 or input slice images from the WBI 245. In some embodiments, the model applier 145 may generate the output WBI 245’ to include the ROI 330’ (e.g., when the input is taken from the WBI 245). The model applier 145 may apply any number of reconstruction algorithms to generate the WBI 245’ using the two-dimensional output images 320’ that include the ROI 330’. The reconstruction algorithm may include, for example, iterative reconstruction, interpolation (e.g., linear or cubic interpolation), and three-dimensional volume rendering, among others.

[0113] In some embodiments, based on applying the ML model 160 (e.g., to the WBI 245 or the slide images 240), the model applier 145 may determine or generate at least one classification 335. The classification 335 may identify at least one of a set of feature types for the ROI 330’ or by extension the feature in the example biological sample associated with the ROI 330’. The classification 335 may include any type of features, for example, such as a blood vessel, a tissue, muscle, tumor (e.g., benign or malignant), lesion, cell nuclei, or any lack thereof, among others. The classification 335 may identify a presence or an absence of the condition, among others. The classification 335 may include a set of alphanumeric values or at least one enumerated value corresponding to the feature type of the ROI 330’. In some embodiments, the model applier 145 may generate a set of classifications 335 corresponding to the set of slide images 240 or the set of image slices from the WBI 245. The classifications 335 derived from the slide images 240 may be the same or differ from the classifications 335 derived from the images slice of the WBI 245.

[0114] In some embodiments, the model applier 145 may store and maintain an association between the subject 205 and the outputs (e.g., the output images 320’, the WBI 245’, the ROI 330’, or the classification 335) from the ML model 160 on the database 165. In some embodiments, the model applier 145 may store the association between the subject 205 with one or more of: the set of CT images 230, the set of slide images 240, and the WBIAtty. Dkt. No.: 115872-3415 245, among others. The association may be stored and maintained on the database 165 using one or more data structures, such as an array, a linked list, a stack, a tree, a hash table, or a class object, among others.[Oil 5 J Referring now to FIG. 14, depicted is a block diagram for a process 400 of generating outputs based on the biomedical images of cell blocks in the system for generating biomedical images of cell blocks. Under the process 400, the output evaluator 150 on the image processing system 105 may produce, create, or otherwise generate at least one interface output 405. The interface output 405 may identify or include any one or more of: the WBI 245, the WBI 245’, the ROI 330’, the classification 335, the set of CT images 230, or the set of slide images 240, among others. In some embodiments, the output evaluator 150 may generate the interface output 405 to include computer-readable instructions defining the presentation or rendering of the interface output 405.101161 In some embodiments, the output evaluator 150 may generate at least one report 410 based on any one or more of: the WBI 245’, the ROI 330’, the classification 335, or the set of slide images 240, among others. The report 410 may include various information derived at least in part from the output of the ML model 160. For instance, the report 410 may include a listing of identifiers corresponding to slide images 240, CT images 230, or cross-sections of the WBI 245. For each identifier in the listing, the report 410 may include one or more of: the ROI 330’ (e.g., pixel coordinates or area within the respective image) or the classifications 335 (e.g., feature types), among others. The report 410 may also include various metadata, such as date of image acquisition, sample identifier, or subject identifier (e.g., anonymized identifier), among others. The report 410 may be stored and maintained in one or more files, such as an extensible markup language (XML) file, comma-separated value (CSV), JavaScript Object Notation (JSON), or structured query language (SQL) file, among others. The report 410 may be of the form as shown in FIG. 5C. With the generation of the report 410, the output evaluator 150 may insert, add, or otherwise include the report 410 in the interface output 405.[0117 j In some embodiments, the output evaluator 150 may select or identify at least one section of interest (SOI) 420 (also referred herein generally as a section) from the WBI 245’ based on the ROI 330’ on the SOI 420. The SOI 420 may be identified from aAtty. Dkt. No.: 115872-3415 set of sections (e.g., the image slices) forming the WBI 245’ (or the WBI 245). The SOI 420 may correspond to a two-dimensional slice (e.g., the set of CT images 230 or the slide images 240) of the WBI 245’. The SOI 420 may be identified based on a metric associated with the ROI 330’ on each slice, such as an area of the ROI 330’ on the slice or a number of cells associated with the ROIs 330’ on the slice. As each slice corresponds to a different section or layer on the biological sample 210, the ROIs 330’ may differ among the slices forming the WBI 245’. The ROI 330’ may be of a particular classification 335, such as a malignant tumor or a lesion classification. To identify, the output evaluator 150 may determine the metric associated with the ROI 330’ on each slice of the WBI 245’. Using the metrics of the slices, the output evaluator 150 may identify the slice as the SOI 420. For instance, the output evaluator 150 may select the slice with the largest area of the ROI 330’ or the greatest number of cells associated with the ROIs 330’ as the SOI 420. With the identification of the SOI 420, the output evaluator 150 may add or include an identifier (e.g., depth, z-coordinates, or slice number) for the SOI 420 to the interface output 405.

[0118] With the generation of the interface output 405, the output evaluator 150 may provide at least one user interface 415 for presentation, rendering, or display of the interface output 405 on the administrative device 120. In some embodiments, the output evaluator 150 may send the interface output 405 including the computer-readable instructions for rendering the contents to the administrative device 120. The user interface 415 may correspond to a graphical user interface (GUI) to be presented via the administrative device 120. For instance, the output evaluator 150 may provide a web application with the user interface 415 to the administrative device 120 to present various information and content from the interface output 405. The user interface 415 may include one or more user interface elements to navigate, manipulate, or otherwise select to retrieve additional information about the contents of the interface output 405.

[0119] Upon receipt, the administrative device 120 may display, render, or otherwise present the interface output 405 via the user interface 415. The administrative device 120 (e.g., an application executing thereon) may process or parse the interface output 405 to extract or retrieve the contents. With the retrieval, the administrative device 120 may generate or present the user interface 415 to include the contents of the interface output 405. For instance, the user interface 415 may display a juxtaposition of the set of slice imagesAtty. Dkt. No.: 115872-3415 240 (e.g., in H&E stain) with the WBI 245’ (e.g., with conversion to Papanicolaou stain). The WBI 245’ may correspond to the biological sample 210 in the volume of the sample block 215, with ROIs 330’ corresponding to features in the biological sample 210. The user interface 415 may present the ROIs 330’ as overlays over the pixel coordinates within the WBI 245’. The user interface 415 may also present the report 410, including the listing of slice images 240 or sections of the WBI 245’ along with the classifications 335 for each of the images. In some embodiments, the user interface 415 may include a marker (e.g., a highlight or a pin) on the SOI 420 within the WBI 245’. Examples of the user interface 415 are depicted herein on FIGs. 7A-8B.|0120| In some embodiments, the administrative device 120 may retrieve, identify, or otherwise receive at least one selection 425 via the user interface 415. The selection 425 may identify at least one section 430 within a set of sections (e.g., the image slices) from the WBI 245’ (or the WBI 245). The selected section 430 may correspond to a two-dimensional slice of the WBI 245’. For instance, the user of the administrative device 120 may interact with the user interface elements of the user interface 415 to navigate to a particular section within the WBI 245’ . The interactions with the user interface elements may correspond to the selection 425 of at least one section within the WBI 245’ . The selection 425 may define or identify a set of voxel (or pixel) coordinates for the plane within the WBI 245’ corresponding to the section 430. With the receipt, the administrative device 120 may provide, transmit, or otherwise send the selection 425 to the image processing system 105.101211 The interaction handler 155 on the image processing system 105 may retrieve, identify, or otherwise receive the selection 425 from the administrative device 120. The interaction handler 155 may identify the section 430 in the selection 425 from the WBI 245’ (or the WBI 245). In some embodiments, the interaction handler 155 may select the image slice used to form the WBI 245’ corresponding to the section 430. In some embodiments, the interaction handler 155 may apply a reconstruction technique (e.g., iterative reconstruction or interpolation) to derive the section 430 using the coordinates defined in the selection 425 and the WBI 245’. With the identification of the section 430, the interaction handler 155 may create, produce, or otherwise generate at least one supplemental output 435 to include the section 430. In some embodiments, the interactionAtty. Dkt. No.: 115872-3415 handler 155 may identify the ROI 330’ within the section 430 or the classification 335 associated with the ROI 330’, or both, to include. The interaction handler 155 may send, transmit, or otherwise provide the supplemental output 435 to the administrative device 120 for display via the user interface 415. Upon receipt, the administrative device 120 may present the selected section 430 via the user interface 415. For instance, the selected section 430 may be presented within a window next to or on top of another window showing a three-dimensional view of the WBI 245’.[0122 J In this manner, the image processing system 105 may generate and provide an accessible three-dimensional image of a cell structures within the biological sample 210 in the volume of the sample block 215. This may address the drawbacks of cytology smears, which only provide a two-dimensional view of cell structures. The approaches using cytology smears can result in a loss of information for accurate diagnosis and treatment planning. The image processing system 105 can leverage advanced imaging techniques and machine learning (e.g., in the form of the ML model 160) to generate three-dimensional images of cell blocks. This can allow for a more comprehensive analysis of cell structures, enabling users (e.g., clinicians) to observe and analyze virtually created three-dimensional whole cells in the entire cell block. Furthermore, by having multiple functions provided by the ML model 160, the image processing system 105 can reduce the reliance on multiple sampling and staining, and different computer systems dedicated to particular imaging modalities. The image processing system 105 may thus provide efficiencies in the use of computing resources (e.g., processor and memory), relative to such systems.[0123J This imaging technique may also provide for non-destructive means of examining biological samples 210. This may allow for preservation of the biological sample 210 within the sample block 215, allowing for additional tests and analyses to be performed on the same sample. The integration of the ML model 160 further can enhance the capability of examining the biological samples. The digital conversion between imaging modalities (e.g., from CT to various types of stains) can enable improved visualization of cell structures that would otherwise be impossible if not difficult to analyze in the original imaging modality (or with just cytology slides). In addition, the automated generation ofAtty. Dkt. No.: 115872-3415 information derived from the outputs of the ML model 160 can provide a summary overview of the biological sample 210 and the images derived therefrom.[0124J Furthermore, the information presented via the administrative device 120 may be used by a clinician examining the subject 205 to make diagnostic or therapy decision about the subject 205. For example, the clinician may make an initial examination on a suspicious lesion that could potentially be cancerous in an organ of the subject 205. A biopsy of the organ may be performed to retrieve the biological sample 210, and the biological sample 210 may be embedded in the sample block 215 (e.g., a Formalin-Fixed Paraffin-Embedded (FFPE) cell block). The image processing system 105 may generate three-dimensional WBI 245 of the sample block 215 using image acquisitions by the CT scanner 110. The WBI 245 may processed using the ML model 160 to convert the WBI 245 into a digitally stained model, such as a hematoxylin and eosin (H&E), Papanicolaou stain, or an IHC stain. The clinician can view these digitally stained images on the user interface 415, allowing for detailed examination of the tissue sample without taking further invasive measures. With the generation as detailed herein, the WBI 245’ in the imaging modality 235” may include or identify at least one ROI corresponding to a feature in the biological sample 210 that would not be apparent or visible in another biomedical image derived from an individual section of the biological sample 210. In addition, the WBI 245’ in the imaging modality 235” may indicate or identify at least one condition that is not inferable from the biomedical image derived from an individual section of the biological sample 210.

[0125] By analyzing the images displayed through the user interface 415, the clinician can observe the cellular structures and identify ROIs 330’ that may indicate malignancy in tumors within the biological sample 210. The ML model 160 can further assist by classifying these ROIs 330’, providing information on whether the cells are benign or malignant, and identifying the type of cancerous tissue. The clinician can use this information to determine the stage and grade of the cancer. This may be used to decide the appropriate therapy. For example, if the cancer is in an early stage, the clinician might opt for surgical removal of the tumor. If the cancer is more advanced, the clinician might recommend a combination of surgery, chemotherapy, and radiation therapy. Additionally, the ability of the image processing system 105 to handle multiple imaging modalities mayAtty. Dkt. No.: 115872-3415 allow the clinician to view other stains such as IHC staining, to assess the expression of specific proteins that could influence treatment decisions. For example, if the IHC staining reveals that the tumor cells express a particular receptor, the clinician may prescribe targeted therapy that specifically attacks those cells.

[0126] Referring now to FIG. 15, depicted is a flow diagram of a method 500 of generating biomedical images of cell blocks for analysis across image modalities. The method 500 may be performed by or implemented using the system 100 described herein in conjunction with FIGs. 11-14 or the system 600 detailed herein in Section C. Under the method (500), a computing system can receive a set of biomedical images of sections (505). The computing system can generate a biomedical image of a volume (510). The computing system can apply a machine learning (ML) model to the biomedical image of the volume (515). Based on applying the ML model, the computing system can convert the biomedical image from an initial modality (or staining) to a target modality (520). The computing system can generate an output using the converted images (525). The computing system can provide a user interface for presentation of the output (530).B. Methods for Validating Three-Dimensional Cellular Arrangement Visualization Techniques for Use in Diagnostics

[0127] Described herein are validation methods for use of three-dimensional cellular imaging (e.g., WBI as detailed in Section A) to detect additional features and infer diagnoses that are not apparent in other techniques. Sub-section 1 describes a method to validate the use of a digitally created three-dimensional (3D) Papanicolaou stain image derived from hematoxylin and eosin (H&E) stained whole slide images to detect additional features and diagnoses. Sub-section 2 describes a method to validate 3D digitally stained image derived from computed tomography (CT) or stained slide scans to detect additional features and diagnoses.1. Validation Methods for Using Whole Block Images in Digital Papanicolaou Staining Derived from Images of Tissue Sections with Hematoxylin and Eosin (H&E) Staining to Detect Features and ConditionsSignificance of Cell Block Preparation31Aty. Dkt. No.: 115872-3415

[0128] Cell blocks are specimens prepared by aggregating and solidifying exfoliated or aspirated cells suspended in liquid-based cytology samples, followed by paraffin embedding. This technique enables the creation of tissue-like sections from cytological samples, thereby bridging the gap between cytology and histology. Cell blocks serve as valuable complementary tools in diagnostic cytology, as ancillary tests can be used such as immunohistochemistry (IHC), in situ hybridization (ISH), and molecular analyses.Furthermore, multiple sections can be obtained from a single cell block, allowing for repeated analyses and retrospective studies. Overall, cell blocks are a powerful tool in modern cytology, providing both structural and cytological information. Diagnostic utility can be further enhanced by optimal staining techniques and standardized preparation protocols. However, as evident from the preparation process, cells within cell blocks are distributed randomly in a liquid medium before being embedded. As a result, the spatial arrangement of cells in a cell block often resembles that of cytological samples rather than histological tissues. Therefore, cell blocks are considered to have cytology-like characteristics in terms of cellular arrangement, as opposed to the structured architecture seen in histological tissue sections.Characteristics and Limitations ofH&E and Papanicolaou Staining

[0129] Hematoxylin and eosin (H&E) staining is the standard method used for cell block evaluation, widely applied in histopathological analysis. H&E staining is effective for evaluating tissue morphology and pathology, particularly for observing architectural features such as glandular structures, connective tissue, and vascular distribution. It stains nuclei blue-purple and cytoplasm pink, making it suitable for assessing tissue structures and cell-to-cell interactions. In contrast, Papanicolaou staining is well-suited for the observation of individual cells in cytology. Papanicolaou staining allows for clear distinction between the nucleus and cytoplasm, providing high contrast and facilitating detailed assessment of cellular morphology, maturity, and atypia. The nucleus appears blue-purple, while the cytoplasm is stained pink or orange. This high contrast surpasses that ofH&E staining, making it easier to observe cellular structures and abnormalities, particularly when examining single-cell preparations such as smears or aspiration samples. Papanicolaou staining also clearly delineates cell borders, allowing individual cells to be distinguished even in densely packed areas. Considering the preparation process of cell blocks and theAtty. Dkt. No.: 115872-3415 cytology-like nature, Papanicolaou staining may offer a better representation of individual cell morphology than H&E staining. Nonetheless, depending on the diagnostic purpose and target, H&E may still be more appropriate in certain cases.Limitations in Applying Papanicolaou Staining to FFPE Cell Blocks

[0130] In practice, applying Papanicolaou staining to FFPE (formalin-fixed, paraffin-embedded) cell block sections often results in suboptimal staining quality.Papanicolaou staining was originally developed for alcohol-fixed smear specimens, and there are structural and chemical incompatibilities when used on FFPE tissue. Main issues include:10131] First issue is the reduced stainability due to fixation differences.Papanicolaou staining requires alcohol fixation for optimal dehydration and protein denaturation, crucial for vibrant cytoplasmic staining (using dyes such as Orange G and EA series). However, formalin fixation works primarily through protein cross-linking, leading to hardened and altered cellular structures. This affects dye uptake and may result in weak or uneven staining.(0132] Second issue is the impact of paraffin embedding and deparaffinization. The process of paraffin embedding, followed by deparaffinization and rehydration, further distorts the cellular architecture, reducing dye permeability and color development during Papanicolaou staining.

[0133] Third issue is poor nuclear staining. Hematoxylin-based nuclear staining, critical for cytological evaluation, may also be adversely affected by formalin-induced protein cross-links, resulting in uneven or faint nuclear staining. This compromises the visibility of chromatin patterns, which are essential for diagnostic cytology.

[0134] Fourth issue is difficulty in optimizing the staining protocol. Papanicolaou staining involves complex steps including precise timing, pH control, and dehydration sequences. These conditions are optimized for alcohol-fixed smears and are not easily transferable to FFPE sections, leading to issues with reproducibility and staining quality.Aty. Dkt. No.: 115872-3415

[0135] This phenomenon may be atributed to the staining mechanism of the Papanicolaou method, which relies on the molecular weight of dyes to sequentially bind to cellular components. Under normal conditions, these dyes bind uniformly to specific structures within intact cells. However, when cells are processed into paraffin-embedded cell blocks and subsequently sectioned, the original three-dimensional architecture is disrupted. As a result, the dyes may bind to unintended regions or fail to bind to the usual targets, leading to uneven or suboptimal staining (FIG. 16).[01 6 J Although cell blocks retain many cytological features, these limitations in staining reduce the effectiveness of conventional methods. In particular, H&E staining often fails to clearly delineate cell boundaries, and it remains technically difficult to reproduce the high contrast and detail of Papanicolaou staining in FFPE samples.Therefore, digital image processing was focused on as a novel approach to overcome these staining limitations.Objective[0137) The aim of this study was to improve the visualization of cellular morphology by applying digital Papanicolaou staining to H&E-stained images of cell block sections. While Papanicolaou staining provides excellent nuclear and cytoplasmic contrast, making it valuable for identifying cellular atypia and maturity in cytology, its application to FFPE samples suffers from poor reproducibility due to its optimization for alcohol-fixed smears. To address this limitation, a color transformation method was developed to digitally simulate Papanicolaou staining from existing H&E-stained images, enabling both enhanced visual contrast and improved structural clarity.

[0138] Additionally, because cell blocks originate from liquid-based cytology and maintain a non-architectural cell distribution, three-dimensional (3D) visualization is suitable at the cellular level. By utilizing serial sections and whole slide images (WSIs), the 3D spatial arrangement was aimed to keep cells within the block. This approach offers an analytical environment, allowing the assessment of individual cells’ spatial relationships and morphological features in three dimensions.Materials and MethodsAtty. Dkt. No.: 115872-3415 Cases, Staining Protocols, and Scanning Conditions

[0139] This study included a total of 10 cases of body cavity effusions, comprising five cases of adenocarcinoma, three cases of squamous cell carcinoma, and two cases of mesothelioma. Cell blocks were prepared from formalin-fixed effusion specimens (ascites and pleural effusion). From each cell block, 4-pm-thick sections were cut and stained with H&E for subsequent analysis. H&E stain sections were scanned with 11 layers (e.g., using 40 x mode: 0.23 pm / pixel, 11 layers 1 pm pitch. Hamamatsu Nano Zoomer S60 Digital slide scanner C13210-01).

[0140] Schematic representation of the principle underlying the digital Papanicolaou staining conversion process. In the equations, “HE” refers to H&E staining, and “PAP” refers to Papanicolaou staining.1. Color Transformation Algorithm: Mathematical Formulas and Optical Modeling1) Conversion of HE Image to Optical Density (OD) Space:

[0141] Before explaining each step of the proposed method, the optical model for color mixing in histological staining is briefly introduced. A color unmixing method based on the optical model is often used to separate a color image of multi-stained specimens into the components of stain intensities representing the individual stains, in this case eosin and hematoxylin. The detected intensities of light transmitted through a specimen are described by Lambert-Beer’s Law, known as the formula for light transport through a homogeneous absorbing medium.

[0142] First, a color image is converted to optical density space. The optical density of the sample is equal to the logarithm of the light transmitted through it and is proportional to the color intensity:

[0143] where OD = (ODRODGODBf is the RGB vector of the optical density at each pixel in the image, Io= (JORIOGIQB1is the RGB intensity vector of the incident light, I = IRIGIBf is the RGB intensity vector of the light passing through the specimen, M isAtty. Dkt. No.: 115872-3415 the stain matrix whose columns are the unit vectors of the RGB absorption coefficients of the stain colors, C is the vector whose each element represents the stain intensity, and 0 denotes the Hadamard division operator, respectively. In the vector-matrix notation here, spatial coordinates were omitted for simplicity. By normalizing with Ioselected from a glass area without a sample and taking a logarithm, the optical density for each wavelength is linear with the stain intensity.

[0144] After converting the image to optical density (OD) space as described in Eq. (1), each pixel’s color is represented as a linear combination of the stain-specific absorbance vectors. This representation is particularly advantageous because it transforms the nonlinear interaction of color channels in RGB space into a linear mixture model, which facilitates mathematical decomposition of the contributions from individual stains.

[0145] The assumption underlying this model is that each histological stain has a characteristic absorbance spectrum, which remains consistent across the tissue section. These spectra are encoded as the column vectors of the stain matrix MHE, and the goal is to recover the corresponding stain concentration vector C for each pixel. This formulation allows the separation of stain components using linear algebra techniques, which are robust and computationally efficient.

[0146] Importantly, this linearity in OD space forms the basis of stain unmixing, where the observed optical signal is mathematically decomposed into its underlying sources. The following section describes this unmixing process in detail, outlining how the stain intensities can be estimated and further manipulated for downstream applications such as virtual staining or stain normalization.2) Separation of H&E Components (Unmixing)C = MH+E■ OD (2)

[0147] where -+denotes the Moore-Penrose pseudoinverse operator. The number of stains was assumed to be three or fewer, and thus a unique solution can be obtained. If the _ -1number of stains is three, eq. (2) becomes C = MHEOD.Aty. Dkt. No.: 115872-3415

[0148] The resulting vector C contains the relative contributions of each stain to the observed color at that pixel, and can be interpreted as a quantitative representation of the histological components.

[0149] To enhance the visual representation or facilitate downstream processing, it is often desirable to adjust the contrast or intensity of each stain component. This is achieved by applying a component- wise scaling factor, denoted as cmp scl, yielding the adjusted component vector:C = C ■ cmp_scl (3)

[0150] This rescaling can be applied uniformly across all pixels or adapted locally, depending on the application, such as dynamic range normalization or emphasis of specific histological features.3) Reconstruction of Papanicolaou Components (Remixing)

[0151] Once the stain component vector C has been adjusted, it can be used to synthesize a virtual image in the color space of a different staining protocol, such as the Papanicolaou stain. This process, referred to as remixing, involves projecting the stain intensity vector onto a new stain matrix corresponding to the target staining method.

[0152] Let MPAPbe the Papanicolaou stain mixing matrix, in which each column represents the normalized RGB absorbance characteristics of each dye used in Papanicolaou staining (e.g., hematoxylin, OG-6, EA50 / EA65). The OD vector for Papanicolaou staining is computed by linearly combining the adjusted stain intensities:OD' = C ■ MPAP(4)

[0153] This operation maps the stain concentration values into a new OD space that reflects the optical behavior of Papanicolaou stains. The result MPAPis a 3-element vector at each pixel that characterizes how the pixel would absorb light under Papanicolaou staining conditions.Atty. Dkt. No.: 115872-3415

[0154] To obtain a final RGB image simulating Papanicolaou staining, an inverse transformation is performed to convert the OD values back to light transmittance, following the Lambert-Beer law. This is done using:= i0PAP■ 10- ' (5)

[0155] Here, Iodenotes the incident light intensity, typically set to [255,255,255] for white light. The exponential operation undoes the earlier logarithmic transformation, reconstructing the image in standard RGB space. The resulting image I' visually simulates how the tissue section would appear if it had been stained with the Papanicolaou protocol. The overall flow is shown in (FIG. 17).

[0156] Reference images used to design the digital Papanicolaou staining algorithm were obtained from two sources: conventional cytological smears stained using the Papanicolaou method, and FFPE cell block sections that had undergone physical Papanicolaou staining. From these images, representative regions of interest (ROIs) were manually selected to extract pixel-level color data corresponding to nuclei, cytoplasm, and erythrocytes. The extracted color values were used to define target hues and intensities for each cellular component. A Python-based transformation algorithm was then applied to the input H&E-stained images, which mapped source colors (e.g., hematoxylin-stained nuclei and eosin-stained cytoplasm) to the reference Pap-stained palette using hue replacement and intensity rescaling within the LAB space. This process preserved structural details while providing a visually faithful approximation of the Papanicolaou stain. The resulting images were reviewed for consistency and adjusted iteratively to optimize visual clarity and cytological relevance.

[0157] Based on the above principles, color transformation processing was performed using a Python program. The same region of interest from all layers was captured and saved using the scanner. Color transformation processing was then applied to all the images.2. 3D Image ConstructionAty. Dkt. No.: 115872-3415

[0158] After applying the color transformation to all images, a stack of images was created using the ImageJ software (version X) by utilizing its advanced stacking functionalities. The stack was constructed by aligning the images from all layers in the same ROI and saving in TIFF format, ensuring each layer was accurately preserved for further analysis. Following this, the stacked images were processed using Imaris (version Y, Oxford Instruments), a commercial 3D visualization software. Imaris was used to render the stack in three dimensions, employing its advanced rendering and segmentation tools to visualize the spatial relationships and structural features of the cells across all layers.Results

[0159] The digital Papanicolaou-stained images obtained after color transformation demonstrated significantly improved clarity of both nuclear and cytoplasmic structures compared to conventional H&E-stained images. Fine morphological details critical for cytological assessment, such as nuclear contours, chromatin texture, and cytoplasmic boundaries, were distinctly visualized. Artifacts commonly associated with formalin fixation, such as staining artifacts or chromatic turbidity, were not observed; instead, the images retained a natural and consistent color tone characteristic of traditional Papanicolaou staining. 3D reconstruction using the digital Papanicolaou images enabled detailed visualization of spatial arrangements of cellular clusters and the 3D architecture of nuclei. This was particularly effective in identifying the composition of small aggregates and visualizing mitotic figures with high resolution. Such 3D perspectives allowed for a more comprehensive understanding of cell-to-cell relationships and the distribution of nuclear atypia, which are features often obscured in conventional two-dimensional analysis.Importantly, the digital Papanicolaou images closely resembled routine cytology specimens stained by conventional Papanicolaou methods, both in color contrast and morphological clarity. This high visual similarity reduced cognitive burden for observers and allowed intuitive interpretation. These results suggest that digital Papanicolaou staining is a promising tool for enhancing cytological evaluation, particularly when applied to cell block preparations in digital pathology workflows (FIGs. 18A-C).

[0160] Distinguishing between non-keratinizing squamous cell carcinoma and adenocarcinoma can be challenging. When the images are rendered in 3D, the difference inAtty. Dkt. No.: 115872-3415 cell thickness between non-keratinizing squamous cell carcinoma and adenocarcinoma becomes visually apparent. Additionally, the texture of the cytoplasm in non-keratinizing squamous cell carcinoma appears opaque and gives a sense of thickness, whereas adenocarcinoma shows a more transparent appearance. Furthermore, in adenocarcinoma, the color of the nucleus is slightly paler, and the presence of nucleoli can be clearly observed (FIG. 19).10161 J Adenocarcinoma and malignant mesothelioma can also be difficult to differentiate. In this study, both clusters and individual cells were observed.Adenocarcinoma has a pale cytoplasm with a transparent appearance, while malignant mesothelioma exhibits very thick cytoplasm. There were also differences in the nuclei. In adenocarcinoma, the chromatin is very fine and transparent, making the nucleoli easily visible. In malignant mesothelioma, nucleoli are also easy to observe, but the chromatin is coarser and overall denser compared to adenocarcinoma (FIGs. 20 A and 20B).Discussion[0162) In H&E staining, distinguishing between the nucleus and the cytoplasm can be challenging due to the limited contrast and the similar staining intensity of both structures. However, in Papanicolaou staining, the distinction is much easier, as the cytoplasm is more vividly stained, often appearing as a light blue or greenish hue, while the nucleus is more intensely stained, providing a clearer separation between the two cellular components. In H&E staining, the nuclear and cytoplasmic structures are primarily evaluated based on morphology and size. This is because H&E provides a general overview of the tissue, highlighting the basic shapes and relative sizes of the cells and the components. The nuclear features, such as shape and size, and the cytoplasmic structures, including the general distribution, can be observed, but the detail is limited. In contrast, Papanicolaou staining allows for a more detailed examination of the nuclear and cytoplasmic structures at the ultrastructural level. The staining provides greater contrast, particularly highlighting finer details of the cellular components, such as the presence of cytoplasmic granules or the distinct texture of the nuclear chromatin. This makes Papanicolaou staining particularly useful for assessing the finer ultrastructural characteristics of cells (FIGs. 21A and 21B).Aty. Dkt. No.: 115872-3415

[0163] However, when performing conventional Papanicolaou staining on formalin-fixed paraffin-embedded sections, uneven staining or suboptimal coloration can occur due to the effects of fixation and the inherent limitations of the staining method. In this study, these limitations were overcome using digitally generated Papanicolaou images (digital Papanicolaou), which closely resembled the ideal cytologic appearance typically seen in routine cytology slides. Digital conversion allowed standardization of color rendering without being affected by physical fixation artifacts, enabling visualization with consistent and optimal coloration.

[0164] Moreover, by converting these digital Papanicolaou images into 3D renderings, the natural appearance of cells was approximated in three dimensions. This enhanced the ability to observe structures such as clusters and individual cell morphology that are difficult to appreciate in two-dimensional images. Unlike conventional microscopic observation, which is limited to viewing along a single Z-axis direction, 3D imaging enabled full 360-degree visualization of cells. This provided new insights into spatial arrangements and morphologic features of clusters and individual cells that were previously difficult to detect.

[0165] Taken together, the combination of digital Papanicolaou staining and 3D imaging represents a powerful approach for evaluating cell morphology in cell block preparations. This methodology holds significant promise for enhancing cytology -based diagnostic capabilities in surgical pathology.Conclusion

[0016] This study demonstrates that digital color transformation applied to H&E-stained FFPE cell block sections can effectively replicate the cytomorphological clarity of conventional Papanicolaou staining. Furthermore, three-dimensional reconstruction using serial whole slide images enabled spatial visualization of cellular architecture, which is challenging to achieve with conventional two-dimensional observation. These technological advancements enhance the diagnostic value of routinely prepared histopathological specimens and highlight the potential utility of this approach as a novel diagnostic aid in digital cytopathology. Ongoing investigations are being conducted using aAtty. Dkt. No.: 115872-3415 variety of cases and specimen types to evaluate the generalizability and robustness of the proposed methodology.Supplemental MaterialsObjective|0167j The objective of this study was to observe cytology specimens in three dimensions and analyze the details of nuclear and cytoplasmic features of cells that cannot be captured by conventional 2D images. By exploring the advantages of capturing cells three-dimensionally, the study aims to investigate whether it is possible to improve the accuracy of cytological evaluation and diagnosis as a new auxiliary tool.Cases and original reportAtty. Dkt. No.: 115872-3415Results

[0168] By using water immersion to create 3D images from multiple layers, detailed observation of the chromatin inside highly overlapping malignant tumor cell clusters was possible. New findings were obtained in 5 cases, and more accurate assessments were made in 4 cases.Atty. Dkt. No.: 115872-3415>|0169 J Referring now to FIGs. 22A and 22B, shown is a comparison of H&E-stained cell block images and corresponding digitally generated Papanicolaou (Pap) images from the same specimen.

[0170] H&E Characteristics: HE staining is widely used for observing the overall structure of tissues and is particularly helpful in understanding the anatomical arrangement of cells and tissues. It allows for clear distinction between nuclei and cytoplasm, making it easy to observe the shape and size of nuclei, which is valuable for pathological diagnosis.[0171 J Pap Characteristics: Papanicolaou staining emphasizes the morphology and content of the cells, making it particularly useful for assessing cell maturity andAtty. Dkt. No.: 115872-3415 abnormalities. It is commonly used for cytological screening, such as Pap smears for cervical cancer diagnosis, as it highlights individual cell structures.Differences in the Appearance of Nuclei and Cells[0172} In HE staining, nuclei appear dark blue or purple, while the cytoplasm appears pink, allowing clear observation of tissue structure and abnormalities. In Papanicolaou staining, nuclei appear blue or green, and the cytoplasm can range from pink to bluish-green. This staining method highlights finer cellular details and is useful for observing individual cell conditions and differentiation.

[0173] Referring now to FIG. 23, depicted is a comparison of conventional H&E (left) and digitally generated Papanicolaou (right) images from the same cell block region. As illustrated, the digital Pap conversion may improve distinction between nucleus and cytoplasm and reveals fine ultrastructural details.

[0174] Referring now to FIG. 24A, depicted is a comparison of non-keratinizing squamous cell carcinoma (left) and adenocarcinoma (right) in H&E-stained cell slide images and corresponding cell diagrams. These two histological types can be difficult to differentiate in some cases. It is believed that 3D images can also be useful for diagnosis. For example, it can be difficult to distinguish between non-keratinizing squamous cell carcinoma and adenocarcinoma. By rendering the images in 3D, the shape of the nuclei and cells can be observed, which aids in diagnosis.

[0175] Referring now to FIG. 24B, depicted is a comparison of non-keratinizing squamous cell carcinoma (left) and adenocarcinoma (right) in Papanicolaou (Pap) stained slide images and corresponding cell diagrams. These two histological types can be difficult to differentiate in some cases.

[0176] When observing 3D structures, cytology specimens are typically used.Therefore, PAP staining was accustomed to observing, which makes it easier to examine the 3D structure of cell blocks as well.

[0177] Referring now to FIG. 24C, depicted is a comparison of adenocarcinoma (left) and malignant mesothelioma (right) in Papanicolaou (Pap) stained slide images andAty. Dkt. No.: 115872-3415 corresponding cell diagram. These two histological types can be difficult to differentiate in some cases. It can also be difficult to distinguish between adenocarcinoma and malignant mesothelioma. By viewing the 3D images, differences in cytoplasmic thickness can be observed, which is useful for differentiation.

[0178] Referring now to FIG. 24D, depicted is a comparison of adenocarcinoma (left) and malignant mesothelioma (right) in Papanicolaou (Pap) stained slide images and corresponding cell diagrams. These two histological types can be difficult to differentiate in some cases. It is often difficult to differentiate foamy cells, but when viewed in 3D, the differences become more apparent.

[0179] Cell blocks are stained with HE staining, but if they can be converted to Pap staining, it becomes easier to observe the 3D structure. Furthermore, by converting a single slide, it is possible to observe multiple staining images of the exact same cells. This is difficult to achieve with serial sections. It is useful for pathological diagnosis.

[0180] Referring now to FIG. 25, depicted is a schematic overview of the cell block concept and its optimization by the proposed digital staining method. Cells suspended in effusion fluids may be aggregated, fixed, and embedded to form a cell block, enabling direct observation of the cells themselves rather than background tissue Cell blocks collect and solidify cells floating in the liquid. In other words, it allows for the observation of the cells themselves, rather than the background structure. Pap is better for observing cellular microstructure than HE for observing tissue architecture.

[0181] Referring now to FIG. 26, depicted is a block diagram of a cell block being sectioned and then imaged to acquire H&E-stained cell slide images. The number of cells contained in the cell block is small. Only a limited number of sections are available for tests such as IHC. Since the cell block collects cells floating in the liquid, the background structure cannot be observed. In other words, it is better to perform Papanicolaou (Pap) staining, which is excellent for observing cells, rather than Hematoxylin and Eosin (HE) staining. There are a number of considerations with respect to straining with Pap staining.

[0182] 1. Nuclear staining: Formalin fixation can affect nuclear staining, making it less intense compared to alcohol-fixed specimens. The formalin fixative can cause theAtty. Dkt. No.: 115872-3415 chromatin to become more condensed and can interfere with the ability of hematoxylin to stain the nuclei deeply, resulting in a lighter or more washed-out blue or green color in the nuclei. This occurs because formalin forms cross-links with proteins and nucleic acids, which may prevent the proper binding of the stain.

[0183] 2. Cytoplasmic staining: the cytoplasmic staining can be less vivid in formalin-fixed specimens. Formalin-induced changes in the cytoplasm, such as protein denaturation and hardening, can reduce the uptake of eosin, which usually stains the cytoplasm a bright pink. As a result, the cytoplasm may appear paler or more unevenly stained.

[0184] 3. Overall staining quality: the overall quality of Pap staining in formalin-fixed specimens can be reduced due to the cross-linking effects of formalin, which affects the morphology and staining properties of cells. This can lead to less clarity in the visualization of fine cellular details compared to alcohol-fixed specimens.

[0185] In summary, formalin fixation can cause differences in both the intensity and color of Papanicolaou staining. Specifically, the nuclear staining may be lighter, the cytoplasm may appear paler, and the background structure can become more diffuse. These effects are due to the chemical interactions between formalin and cellular components, which can alter the way stains bind to the tissue.

[0186] Even if cell block sections are stained with Papanicolaou staining, the original coloration may not be retained. By converting the HE-stained image into a standard Papanicolaou-stained image, it becomes easier to observe the cellular structure in detail. Furthermore, by reconstructing the flat image of the cell block into a 3D structure, it approaches the original cell appearance, allowing for more accurate evaluation and diagnosis.

[0187] Referring now to FIG. 27, depicted a digital conversion of a H&E-stained slide image to a digital Papanicolaou (Pap)-stained slide image via a reference image. Referring now to FIGs. 28A and 28B, depicted is a comparison between H&E-stained slide images to their corresponding digital Papanicolaou (Pap)-stained slide images. Referring now to FIG. 29, depicted is a comparison between a H&E-stained slide image to a digitalAty. Dkt. No.: 115872-3415 Papanicolaou (Pap)-stained slide image. With serial sections, the process of stacking serial section images to create a 3D structure can build a three-dimensional space, similar to cytology. In observing cellular images in three-dimensional space, it is more common to use Pap stain rather than HE stain. Therefore, converting to Pap stain adds more value.2. Validation Methods for Using Whole Block Images with Digital Staining Derived from Computed Tomography Scans of Cell Blocks to Detect Features and Conditions[01881 Cytology is one of specialties in pathology. Samples are collected non-invasive or minimally invasive method and initial screening for abnormal cells with observation of individual cells and cell clusters. While relatively simple and quick compared to invasive methods, the accuracy may be lower than histology and sample can be used only once, no additional stain can be performed. Sample on a slide is between 1-30 micrometers thick. A whole slide (e.g., stained with hematoxylin and eosin (H&E) staining) is typically 4 micrometer across the tissue. Other non-destructive imaging modality are being used for cell biology but not for cytology.[0189) In addition, scanning thick cytology slides has been a challenge in Digital Pathology. A technique called Z-stack may be used to capture multiple images of the specimen at different focal planes to capture a three-dimensional representation. It is very useful but still missing some information on cytology slide. The focus level can be changed 1 to 50 micrometers to observe all cells using a microscope. But this is not practicable or possible with whole slide images even with the Z-stack technique.[0190 J Cell block (CB) cytology is a technique that processes cellular samples (e.g., from fine needle aspirations or fluids) into a paraffin block, similar to surgical tissue. This may allow for detailed microscopic examination and additional tests, such as immunohistochemistry (IHC) or molecular studies, offering better architectural detail and diagnostic certainty than smears alone, especially for scant material. To prepare, cells may be coagulated with plasma, thrombin, or gels (e.g., Histogel), then embedded and sectioned.

[0191] Formalin-fixed, paraffin-embedded (FFPE) sections that resemble histology may be used, allowing for better architectural preservation and multiple immunohistochemical (IHC) tests. CBs may offer higher diagnostic yield, especially inAtty. Dkt. No.: 115872-3415 fluid samples. The size of sample in cell block may be smaller than 1 centimeter x 1 centimeter x 5 millimeter. The processed tissue sections may be typically cut at a thickness of about 4 micrometers and H&E stain slide may be prepared. It can be used to examine both cell cluster and morphology.

[0192] However, individual size of cell in cell block is up to 30 to 50 micrometers that can be examined using a smear sample. Once the sample is sectioned, the detailed information of each cell and cell cluster may be missed. When a cytopathologist observes a smear slide, the stain may be Papanicolaou stain. When a cell block 3D is generated, digital Papanicolaou stain may be used in conjunction to H&E. The cell block may be prepared using actual Papanicolaou stain. However, since sample processing is very different between smear and cell block, the physical PAP stain on cell block slide may show less information than regular Papanicolaou stain on smear or liquid-based cytology (LBC).

[0193] To address these issues, presented herein are techniques to generate three-dimensional (3D) whole block images (WBIs) from whole cell blocks, using 3D images from serial sections (e.g., hundreds) of whole slide images (WSIs), and two-dimensional or three-dimensional (2D / 3D) from one slide of single, Z-stack image. It can be applied on the images from other non-invasive system such as a light sheet microscope as well. The use of Whole Cell Block 3D image and histology slides (e.g., as detailed herein) is expected to keep both cytology information and histological information.

[0194] To validate the technology as detailed herein (e.g., Section A), the technology will be evaluated using grand truth which is pathology diagnosis and then accuracy compared to the diagnosis using the technology will be compared. It is expected that creating three-dimensional (3D) images of cell blocks with a digital histology stain (e.g., Papanicolaou stain) from computed tomography (CT) (e.g., micro-CT or nano-CT) scans of a tissue or cell blocks will lead to detection and inference of additional features and diagnoses that are not detectable with other techniques. It is expected that creating three-dimensional (3D) images of cell blocks with a digital histology stain (e.g., Papanicolaou stain) from scans of a tissue or cell blocks with another histology stain (e.g., H&E stain) will lead to detection and inference of additional features and diagnoses that are not detectable with other techniques.Atty. Dkt. No.: 115872-3415 Basis

[0195] Three-dimensional images will provide more accurate information and additional information than the grand truth diagnosis. Using the 3D information, evaluation can be done as a case or as a patient level using all resected tissue sample from a subject, it is expected that the inclusion of the information not on a glass slide sample will result in new findings, such as regions of interest (ROI) and conditions. In addition, new features of disease will be discovered. In some instances, new findings will be confirmed during clinical practice. While the ground truth may not always be present to compare against, it is expected that the diagnosis using the present technique will be correlated.

[0196] When validated, clinical diagnosis will be compared versus diagnosis in accordance with the present technique. The additional findings will be verified at consensus meeting. The three-dimensional WBI may be compared to WSIs for the corresponding sample. The three-dimensional WBI will correspond to a FFPE block with a thickness of between 1 millimeter and 10 millimeters (e.g., between 3 and 7 millimeters). The tissues or cells in the block may be between 1 millimeter to 5 millimeters. The thickness of a tissue section for the WSI will have a thickness of between 1 micrometers and 30 micrometers.10197] The ROIs to be evaluated will include at least one of a blood vessel, a tissue, tumor, or a cell nucleus, among others. The condition to be evaluated will include at least one of cancer, an infection, an inflammation, or a cellular degeneration. In some embodiments, the cancer comprises at least one of carcinoma, sarcoma, hematopoietic cancer, adrenal cancer, bladder cancer, bone cancer, brain cancer, breast cancer, cervical cancer, colon cancer, colorectal cancer, corpus uterine cancer, ear, nose and throat (ENT) cancer, endometrial cancer, esophageal cancer, gastrointestinal cancer, head and neck cancer, Hodgkin’s disease, intestinal cancer, kidney cancer, larynx cancer, leukemia, liver cancer, lymph node cancer, lymphoma, lung cancer, melanoma, mesothelioma, myeloma, nasopharynx cancer, neuroblastoma, non-Hodgkin’s lymphoma, oral cancer, ovarian cancer, pancreatic cancer, penile cancer, pharynx cancer, prostate cancer, rectal cancer, seminoma, skin cancer, stomach cancer, teratoma, testicular cancer, thyroid cancer, uterine cancer, vaginal cancer, vascular tumor, or metastases thereof. In some embodiments, the infection may include at least one of a bacterial infection, a viral infection, a fungal infection, or aAty. Dkt. No.: 115872-3415 parasitic infection. In some embodiments, the inflammation may include at least one of an acute inflammation or a chronic inflammation. In some embodiments, the cellular degeneration may include at least one of nuclear degeneration, cytoplasmic vacuolation, cytolysis, or cell atrophy.

[0198] The condition may be associated with any number of organs in a subject, such as at least one of lung, breast, colon, stomach, liver, pancreas, prostate, cervix, ovary, skin, esophagus, kidney, urinary bladder, head and neck mucosa, femur, tibia, pelvis, spine, brain, uterine cervix, rectum, uterine corpus, endometrium, myometrium, ear, nasal cavity, paranasal sinuses, pharynx, larynx, gastrointestinal system, anus, oral cavity, salivary glands, spleen, lymphatic system, lymph nodes, pleura, peritoneum, pericardium, tunica vaginalis, bone marrow, eye, uvea, adrenal medulla, sympathetic nervous system, paraspinal ganglia, lips, tongue, floor of mouth, buccal mucosa, hard palate, gingiva, penis, nasopharynx, oropharynx, hypopharynx, testes, mediastinum, sacrococcygeal region, thyroid gland, vagina, blood vessels, lymphatic vessels, or adrenal gland, among others.Example 1: Validation of Whole Tissue / Cell Block 3D Imaging Derived from Computed Tomography (CT) Scans to Detect Additional Features and Conditions, Relative to H&E Whole Slide Imaging

[0199] To validate the use of 3D images of whole tissue or cell blocks to detect additional features (e.g., ROIs) and conditions (e.g., cancer, infection, inflammation, or cellular degradation), the following method will be used:

[0200] (1) Case selection: a number of tissue or cell blocks (e.g., 10-30 blocks) will be obtained from an organ associated with the condition in one or more subjects. Blocks with high cellular density (e.g., more than 30-50% of evaluated area or volume or more than 200-300 nucleated cells over a set area or volume) will be selected. The cellular density will be checked via computed tomography (e.g., micro-CT or nano-CT). The block will have a thickness between 3 and 7 millimeters.

[0201] (2) CT Scanning: The selected tissue or cell blocks will be scanned via micro-CT at a resolution (e.g., 1-10 microns per voxel for high resolution or 10-100 microns per voxel for standard resolution) or via nano-CT (e.g., 50 nm to 400 nm per voxelAty. Dkt. No.: 115872-3415 for high resolution or 29 nm to 50 nm per voxel for standard resolution). Multiple sections of the block will be scanned for a whole block image (WBI).[0202J (3) Preparation of Slides: Each of the tissue or cell blocks will be sectioned and then placed onto respective slides for slide imaging. Each section will be stained using H&E staining. The section of the sample on the slide will have a thickness between 1 to 30 micrometers.

[0203] (4) WSI Scanning: Each H&E-stained slide will be imaged via microscopy to generate a two-dimensional WSI image.

[0204] (5) Conversion: Each whole block image (WBI) will be digitally converted from CT imaging to Papanicolaou staining.[0205J (6) Assessment of WBI: A pathologist (e.g., cytopathologist) will assess each WBI and will create a report regarding various features (e.g., ROIs) or conditions (e.g., cancer, infection, inflammation, or cellular degradation) from the WBI.

[0206] (7) Assessment of WSI: After the wash-out period (e.g., one to three weeks), a pathologist will assess the WSI of the H&E-stained slide and will create a report regarding various features (e.g., ROIs) or conditions (e.g., cancer, infection, inflammation, or cellular degradation) from the WSI.(0207 J (8) Comparison: The findings of the report from WBI will be compared with the findings of the report from WSI.

[0208] From the comparison, it is expected that the WBI (e.g., derived from CT scans and digital conversion to Papanicolaou stain) will result in detection or inference of additional ROIs or conditions that were not inferable from the WSI (e.g., in H&E stain).Example 2: Validation of Whole Tissue / Cell Block 3D Imaging Derived from H&E-Stained Images to Detect Additional Features and Conditions, Relative to H&E Whole Slide ImagingAtty. Dkt. No.: 115872-3415

[0209] To validate the use of 3D images of whole tissue or cell blocks to detect additional features (e.g., ROIs) and conditions (e.g., cancer, infection, inflammation, or cellular degradation), the following method will be used:

[0210] (1) Case Selection: a number of tissue or cell blocks (e.g., 10-30 blocks) will be obtained from an organ associated with the condition in one or more subjects. Blocks with high cellular density (e.g., more than 30-50% of evaluated area or volume or more than 200-300 nucleated cells over a set area or volume) will be selected. The block will have a thickness between 3 and 7 millimeters.

[0211] (2) Sectioning: At least a subset of the cases will be sectioned for whole slide imaging (WSI). To preserve material for IHC and other clinical tests, a limited number of additional sections will be taken from each block. Each tissue or sample section will be placed on slides and then stained using H&E staining. The section of the sample on the slide will have a thickness between 1 to 30 micrometers.

[0212] (3) Scanning: Each section with the tissue or cell sample will be imaged. The imaging will be at a set magnification factor (e.g., 40x or higher, or 0.25 micrometers per pixel or higher). Multiple sections from the same tissue or cell sample will be scanned to form a whole block image (WBI).

[0213] (4) Conversion: Each whole block image (WBI) will be digitally converted from H&E stain to Papanicolaou stain.

[0214] (5) Assessment of WBI: A pathologist (e.g., cytopathologist) will assess each WBI and will create a report regarding various features (e.g., ROIs) or conditions (e.g., cancer, infection, inflammation, or cellular degradation) from the WBI.

[0215] (6) Assessment of WSI: After the wash-out period (e.g., one to three weeks), a pathologist will assess the WSI of the H&E-stained slide and will create a report regarding various features (e.g., ROIs) or conditions (e.g., cancer, infection, inflammation, or cellular degradation) from the WSI.

[0216] (7) Comparison: The findings of the report from WBI will be compared with the findings of the report from WSI.Aty. Dkt. No.: 115872-3415

[0217] From the comparison, it is expected that the WBI (e.g., derived from H&E-stained WSI scans and digital conversion to Papanicolaou stain) will result in detection or inference of additional ROIs or conditions that were not inferable from the WSI (e.g., in H&E stain).Example 3: Validation of Whole Tissue / Cell Block 3D Imaging Derived from Computed Tomography (CT) Scans to Detect Additional Features and Conditions, Compared to Z-Stack 3D Imaging10218] To validate the use of 3D images of whole tissue or cell blocks to detect additional features (e.g., ROIs) and conditions (e.g., cancer, infection, inflammation, or cellular degradation), the following method will be used:

[0219] (1) Case selection: a number of tissue or cell blocks (e.g., 10-30 blocks) will be obtained from an organ associated with the condition in one or more subjects. Blocks with high cellular density (e.g., more than 30-50% of evaluated area or volume or more than 200-300 nucleated cells over a set area or volume) will be selected. The cellular density will be checked via computed tomography (e.g., micro-CT or nano-CT). The block will have a thickness between 3 and 7 millimeters.

[0220] (2) CT Scanning: The selected tissue or cell blocks will be scanned via micro-CT at a resolution (e.g., 1-10 microns per voxel for high resolution or 10-100 microns per voxel for standard resolution) or via nano-CT (e.g., 50 nm to 400 nm per voxel for high resolution or 29 nm to 50 nm per voxel for standard resolution). Multiple sections of the block will be scanned for a whole block image (WBI).

[0221] (3) Preparation of Slides: Each of the tissue or cell blocks will be sectioned and then placed onto respective slides for slide imaging. Each section will be stained using H&E staining. The section of the sample on the slide will have a thickness between 1 to 30 micrometers.

[0222] (4) WSI Scanning: Each H&E-stained slide will be imaged via microscopy. Z-stacking will be used to generate a three-dimensional WSI image.Atty. Dkt. No.: 115872-3415

[0223] (5) Conversion: Each whole block image (WBI) will be digitally converted from CT imaging to Papanicolaou staining. The digital WBI converted Papanicolaou stain will be used as supplemental information for H&E slides or slides of other staining (e.g., IHC staining).

[0224] (6) Assessment of WBI: A pathologist (e.g., cytopathologist) will assess each WBI and will create a report regarding various features (e.g., ROIs) or conditions (e.g., cancer, infection, inflammation, or cellular degradation) from the WBI.

[0225] (7) Assessment of Z-Stacked WSI: A pathologist will assess the Z-stack WSI of the H&E-stained slide and will create a report regarding various features (e.g., ROIs) or conditions (e.g., cancer, infection, inflammation, or cellular degradation) from the WSI.

[0226] (8) Comparison: The findings of the report from WBI will be compared with the findings of the report from WSI.

[0227] From the comparison, it is expected that the WBI (e.g., derived from CT scans and digital conversion to Papanicolaou stain) will result in detection or inference of additional ROIs or conditions that were not inferable from the Z-stacked WSI (e.g., in H&E stain).Example 4: Validation of Whole Tissue / Cell Block 3D Imaging Derived from H&E-Stained Images Scans to Detect Additional Features and Conditions, Compared to Z-Stack 3D Imaging

[0228] To validate the use of 3D images of whole tissue or cell blocks to detect additional features (e.g., ROIs) and conditions (e.g., cancer, infection, inflammation, or cellular degradation), the following method will be used:

[0229] (1) Case Selection: a number of tissue or cell blocks (e.g., 10-30 blocks) will be obtained from an organ associated with the condition in one or more subjects. Blocks with high cellular density (e.g., more than 30-50% of evaluated area or volume or more than 200-300 nucleated cells over a set area or volume) will be selected. The block will have a thickness between 3 and 7 millimeters.Atty. Dkt. No.: 115872-3415

[0230] (2) Sectioning: At least a subset of the cases will be sectioned for whole slide imaging (WSI). To preserve material for IHC and other clinical tests, a limited number of additional sections will be taken from each block. Each tissue or sample section will be placed on slides and then stained using H&E staining. The section of the sample on the slide will have a thickness between 1 to 30 micrometers.[0231| (3) Scanning for WBI: Each section with the tissue or cell sample will be imaged. The imaging will be at a set magnification factor (e.g., 40x or higher, or 0.25 micrometers per pixel or higher). Multiple sections from the same tissue or cell sample will be scanned to form a whole block image (WBI).

[0232] (4) Z-Stack WSI Scanning: Each H&E-stained slide will be imaged via microscopy. Z-stacking will be used to generate a three-dimensional WSI image.

[0233] (5) Conversion: Each whole block image (WBI) will be digitally converted from H&E stain to Papanicolaou stain. The digital WBI converted Papanicolaou stain will be used as supplemental information for H&E slides or slides of other staining (e.g., IHC staining).

[0234] (6) Assessment of WBI: A pathologist (e.g., cytopathologist) will assess each WBI and will create a report regarding various features (e.g., ROIs) or conditions (e.g., cancer, infection, inflammation, or cellular degradation) from the WBI.

[0235] (7) Assessment of WSI: A pathologist will assess the Z-stacked WSI of the H&E-stained slide and will create a report regarding various features (e.g., ROIs) or conditions (e.g., cancer, infection, inflammation, or cellular degradation) from the WSI.

[0236] (8) Comparison: The findings of the report from WBI will be compared with the findings of the report from WSI.

[0237] From the comparison, it is expected that the WBI (e.g., derived from H&E-stained WSI scans and digital conversion to Papanicolaou stain) will result in detection or inference of additional ROIs or conditions that were not inferable from the Z-stacked WSI (e.g., in H&E stain).Atty. Dkt. No.: 115872-3415 C. Computing and Network Environment

[0238] Various operations described herein can be implemented on computer systems. FIG. 30 shows a simplified block diagram of a representative server system 600, client computing system 614, and network 626 usable to implement certain embodiments of the present disclosure. In various embodiments, server system 600 or similar systems can implement services or servers described herein or portions thereof. Client computing system 614 or similar systems can implement clients described herein. The system 600 described herein can be similar to the server system 600. Server system 600 can have a modular design that incorporates a number of modules 602 (e.g., blades in a blade server embodiment); while two modules 602 are shown, any number can be provided. Each module 602 can include processing unit(s) 604 and local storage 606.

[0239] Processing unit(s) 604 can include a single processor, which can have one or more cores, or multiple processors. In some embodiments, processing unit(s) 604 can include a general-purpose primary processor as well as one or more special-purpose coprocessors such as graphics processors, digital signal processors, or the like. In some embodiments, some or all processing units 604 can be implemented using customized circuits, such as application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions that are stored on the circuit itself. In other embodiments, processing unit(s) 604 can execute instructions stored in local storage 606. Any type of processors in any combination can be included in processing unit(s) 604.

[0240] Local storage 606 can include volatile storage media (e.g., DRAM, SRAM, SDRAM, or the like) and / or non-volatile storage media (e.g., magnetic or optical disk, flash memory, or the like). Storage media incorporated in local storage 606 can be fixed, removable, or upgradable as desired. Local storage 606 can be physically or logically divided into various subunits such as a system memory, a read-only memory (ROM), and a permanent storage device. The system memory can be a read-and-write memory device or a volatile read-and-write memory, such as dynamic random-access memory. The system memory can store some or all of the instructions and data that processing unit(s) 604 need at runtime. The ROM can store static data and instructions that are needed by processingAtty. Dkt. No.: 115872-3415 unit(s) 604. The permanent storage device can be a non-volatile read-and-write memory device that can store instructions and data even when module 602 is powered down. The term “storage medium” as used herein includes any medium in which data can be stored indefinitely (subject to overwriting, electrical disturbance, power loss, or the like) and does not include carrier waves and transitory electronic signals propagating wirelessly or over wired connections.[0241 J In some embodiments, local storage 606 can store one or more software programs to be executed by processing unit(s) 604, such as an operating system and / or programs implementing various server functions, such as functions of the system 100 of FIG. 11 or any other system described herein, or any other server(s) associated with system 100 or any other system described herein.10242] Software” refers generally to sequences of instructions that, when executed by processing unit(s) 604, cause server system 600 (or portions thereof) to perform various operations, thus defining one or more specific machine embodiments that execute and perform the operations of the software programs. The instructions can be stored as firmware residing in read-only memory and / or program code stored in non-volatile storage media that can be read into volatile working memory for execution by processing unit(s) 604. Software can be implemented as a single program or a collection of separate programs or program modules that interact as desired. From local storage 606 (or non-local storage described below), processing unit(s) 604 can retrieve program instructions to execute and data to process in order to execute various operations described above.

[0243] In some server systems 600, multiple modules 602 can be interconnected via a bus or other interconnect 608, forming a local area network that supports communication between modules 602 and other components of server system 600. Interconnect 608 can be implemented using various technologies, including server racks, hubs, routers, etc.

[0244] A wide area network (WAN) interface 610 can provide data communication capability between the local area network (interconnect 608) and the network 626, such as the Internet. Technologies can be used, including wired (e.g., Ethernet, IEEE 602.3 standards) and / or wireless technologies (e.g., Wi-Fi, IEEE 602.11 standards).Atty. Dkt. No.: 115872-3415

[0245] In some embodiments, local storage 606 is intended to provide working memory for processing unit(s) 604, providing fast access to programs and / or data to be processed while reducing traffic on interconnect 608. Storage for larger quantities of data can be provided on the local area network by one or more mass storage subsystems 612 that can be connected to interconnect 608. Mass storage subsystem 612 can be based on magnetic, optical, semiconductor, or other data storage media. Direct-attached storage, storage area networks, network-attached storage, and the like can be used. Any data stores or other collections of data described herein as being produced, consumed, or maintained by a service or server can be stored in mass storage subsystem 612. In some embodiments, additional data storage resources may be accessible via WAN interface 610 (potentially with increased latency).

[0246] Server system 600 can operate in response to requests received via WAN interface 610. For example, one of the modules 602 can implement a supervisory function and assign discrete tasks to other modules 602 in response to received requests. Work allocation techniques can be used. As requests are processed, results can be returned to the requester via WAN interface 610. Such operation can generally be automated. Further, in some embodiments, WAN interface 610 can connect multiple server systems 600 to each other, providing scalable systems capable of managing high volumes of activity. Other techniques for managing server systems and server farms (collections of server systems that cooperate) can be used, including dynamic resource allocation and reallocation.

[0247] Server system 600 can interact with various user-owned or user-operated devices via a wide-area network such as the Internet. An example of a user-operated device is shown in FIG. 6 as client computing system 614. Client computing system 614 can be implemented, for example, as a consumer device such as a smartphone, other mobile phone, tablet computer, wearable computing device (e.g., smartwatch, eyeglasses), desktop computer, laptop computer, and so on.

[0248] For example, client computing system 614 can communicate via WAN interface 610. Client computing system 614 can include computer components such as processing unit(s) 616, storage device 618, network interface 620, user input device 622, and user output device 637. Client computing system 614 can be a computing deviceAtty. Dkt. No.: 115872-3415 implemented in a variety of form factors, such as a desktop computer, laptop computer, tablet computer, smartphone, other mobile computing device, wearable computing device, or the like.[0249 J Processing unit(s) 616 and storage device 618 can be similar to processing unit(s) 604 and local storage 606 described above. Suitable devices can be selected based on the demands to be placed on client computing system 614; for example, client computing system 614 can be implemented as a “thin” client with limited processing capability or as a high-powered computing device. Client computing system 614 can be provisioned with program code executable by processing unit(s) 616 to enable various interactions with server system 600.

[0250] Network interface 620 can provide a connection to the network 626, such as a wide area network (e.g., the Internet), to which WAN interface 610 of server system 600 is also connected. In various embodiments, network interface 620 can include a wired interface (e.g., Ethernet) and / or a wireless interface implementing various RF data communication standards such as Wi-Fi, Bluetooth, or cellular data network standards (e.g., 3G, 4G, LTE, etc ).10251] User input device 622 can include any device (or devices) via which a user can provide signals to client computing system 614; client computing system 614 can interpret the signals as indicative of particular user requests or information. In various embodiments, user input device 622 can include any or all of a keyboard, touch pad, touch screen, mouse or other pointing device, scroll wheel, click wheel, dial, button, switch, keypad, microphone, and so on.

[0252] User output device 637 can include any device via which client computing system 614 can provide information to a user. For example, user output device 637 can include a display-to-display image generated by or delivered to client computing system 614. The display can incorporate various image generation technologies, e.g., a liquid crystal display (LCD), light-emitting diode (LED), including organic light-emitting diodes (OLED), projection system, cathode ray tube (CRT), or the like, together with supporting electronics (e.g., digital-to-analog or analog-to-digital converters, signal processors, or the like). Some embodiments can include a device such as a touchscreen that functions as bothAtty. Dkt. No.: 115872-3415 an input and output device. In some embodiments, other user output devices 637 can be provided in addition to or instead of a display. Examples include indicator lights, speakers, tactile “display” devices, printers, and so on.

[0253] Some embodiments include electronic components, such as microprocessors, storage, and memory that store computer program instructions in a computer-readable storage medium. Many of the features described in this specification can be implemented as processes that are specified as a set of program instructions encoded on a computer-readable storage medium. When these program instructions are executed by one or more processing units, the processing unit(s) are caused to perform various operations indicated in the program instructions. Examples of program instructions or computer code include machine code, such as that produced by a compiler, and files including higher-level code that are executed by a computer, an electronic component, or a microprocessor using an interpreter. Through suitable programming, processing unit(s) 604 and 616 can provide various functionality for server system 600 and client computing system 614, including any of the functionality described herein as being performed by a server or client, or other functionality.

[0254] It will be appreciated that server system 600 and client computing system 614 are illustrative and that variations and modifications are possible. Computer systems used in connection with embodiments of the present disclosure can have other capabilities not specifically described here. Further, while server system 600 and client computing system 614 are described with reference to particular blocks, it is to be understood that these blocks are defined for convenience of description and are not intended to imply a particular physical arrangement of component parts. For instance, different blocks can be, but need not be, located in the same facility, in the same server rack, or on the same motherboard. Further, the blocks need not correspond to physically distinct components. Blocks can be configured to perform various operations, e.g., by programming a processor or providing appropriate control circuitry, and various blocks might or might not be reconfigurable depending on how the initial configuration is obtained. Embodiments of the present disclosure can be realized in a variety of apparatus, including electronic devices implemented using any combination of circuitry and software.Atty. Dkt. No.: 115872-3415

[0255] While the disclosure has been described with respect to specific embodiments, one skilled in the art will recognize that numerous modifications are possible. Embodiments of the disclosure can be realized using a variety of computer systems and communication technologies, including but not limited to the specific examples described herein. Embodiments of the present disclosure can be realized using any combination of dedicated components and / or programmable processors and / or other programmable devices. The various processes described herein can be implemented on the same processor or different processors in any combination. Where components are described as being configured to perform certain operations, such configuration can be accomplished, e.g., by designing electronic circuits to perform the operation, by programming programmable electronic circuits (such as microprocessors) to perform the operation, or any combination thereof. Further, while the embodiments described above may make reference to specific hardware and software components, those skilled in the art will appreciate that different combinations of hardware and / or software components may also be used and that particular operations described as being implemented in hardware might also be implemented in software or vice versa.

[0256] Computer programs incorporating various features of the present disclosure may be encoded and stored on various computer-readable storage media; suitable media include magnetic disk or tape, optical storage media such as compact disk (CD) or DVD (digital versatile disk), flash memory, and other non-transitory media. Computer-readable media encoded with the program code may be packaged with a compatible electronic device, or the program code may be provided separately from electronic devices (e.g., via Internet download or as a separately packaged computer-readable storage medium).

[0257] Thus, although the disclosure has been described with respect to specific embodiments, it will be appreciated that the disclosure is intended to cover all modifications and equivalents within the scope of the following claims.

Claims

Atty. Dkt. No.: 115872-3415 WHAT IS CLAIMED IS:

1. A method of generating biomedical images of cell or tissue blocks, comprising:receiving, by one or more processors, for a subject at risk of or diagnosed with a condition, a plurality of first biomedical images of a volume with a biological sample having a plurality of cells from the subject in a first imaging modality, each first biomedical image of the plurality of first biomedical images corresponding to a respective section of a plurality of sections of the volume;generating, by the one or more processors, using the plurality of first biomedical images, a second biomedical image corresponding to the volume with the biological sample in the first imaging modality;applying, by the one or more processors, a machine learning (ML) model to the second biomedical image to convert from the first imaging modality to a second imaging modality, wherein the ML model is established using a plurality of examples, each of the plurality of examples comprising (i) an example first image of an example biological sample having an example plurality of cells in the first imaging modality and (ii) an example second image of the example biological sample applied with a stain associated with the second imaging modality to modify example color corresponding to the example plurality of cells;converting, by the one or more processors, based on applying the ML model, the second biomedical image from the first imaging modality to the second imaging modality to modify color corresponding to the plurality of cells in the second biomedical image; and storing, by the one or more processors, using one or more data structures, an association between the subject and the second biomedical image in the second imaging modality.

2. The method of claim 1, further comprising providing, by the one or more processors, a user interface to display the second biomedical image corresponding to the volume with the biological sample having the plurality of cells in at least one of the first imaging modality or the second imaging modality.

3. The method of claim 2, further comprising:Atty. Dkt. No.: 115872-3415 receiving, by the one or more processors, via the user interface, a selection of a section of the plurality of sections of the second biomedical image;identifying, by the one or more processors, from the second biomedical image, a two-dimensional slice corresponding to the section selected from the plurality of sections; andproviding, by the one or more processors, for display via the user interface, the two-dimensional slice.

4. The method of any one of claims 1-3, further comprising:receiving, by the one or more processors, a plurality of third biomedical images in a third imaging modality, each third biomedical image of the plurality of third biomedical images corresponding to a slice of a plurality of slices of the biological sample;generating, by the one or more processors, using the plurality of third biomedical images, a fourth biomedical image corresponding to the volume with the biological sample in the third imaging modality; andproviding, by the one or more processors, a user interface to display at least one of the second biomedical image in the second imaging modality or the fourth biomedical image in the third imaging modality.

5. The method of any one of claims 1-4, wherein the ML model is established using a second plurality of examples, each of the second plurality of examples comprising (i) an example third image of a respective example biological sample and (ii) an annotation identifying at least an example portion of the example third image as an example region of interest (ROI), and further comprising:generating, by the one or more processors, based on applying the ML model to the second biomedical image, a third biomedical image identifying at least a portion of the second biomedical image as corresponding to an ROI.

6. The method of claim 5, wherein at least one of the second plurality of examples comprises the annotation identifies at least the example portion of the example third image as at least one of a plurality of classifications for the example ROI, wherein the plurality of classification comprises at least one of a blood vessel, a tissue, tumor, or a cell nucleus, andAtty. Dkt. No.: 115872-3415 wherein generating the third biomedical image further comprises generating the third biomedical image identifying at least the portion of the second biomedical image as corresponding to a classification of the plurality of classification for the ROI.

7. The method of any one of claims 5 or 6, further comprising:generating, by the one or more processors, a report using the ROI corresponding to at least the portion of the second biomedical image identified by the third biomedical image; andproviding, by the one or more processors, a user interface to display at least one of the third biomedical image or the report.

8. The method of claim 7, wherein the report comprises at least one of: (i) a first identifier for the first plurality of biomedical images, (ii) a second identifier for the second biomedical image, (iii) a third identifier for the ROI, or (iv) the classification for the ROI.

9. The method of any one of claims 6-8, further comprising:identifying, by the one or more processors, a section of interest (SOI) from a plurality of sections of the second biomedical image based on the ROI on the section; and providing, by the one or more processors, a user interface to display an identification of the SOI in the second biomedical image.

10. The method of any one of claims 1-9, further comprising:receiving, by the one or more processors, a plurality of third biomedical images in at least one of the second imaging modality or a third imaging modality, each third biomedical image of the plurality of third biomedical images corresponding to a slice of a plurality of slices of the biological sample; andadding, by the one or more processors, at least a portion of the plurality of third biomedical images to generate the second biomedical image in the second imaging modality.

11. The method of any one of claims 1-10, wherein the second biomedical image in the second imaging modality identifies at least one ROI corresponding to a feature in theAtty. Dkt. No.: 115872-3415 biological sample that is not apparent in a fifth biomedical image in the third imaging modality, the fifth biomedical image corresponding to a section of the biological sample.

12. The method of claim 11, wherein the second biomedical image in the second imaging modality identifies at least one condition not inferable from the fifth biomedical image in the third imaging modality.

13. The method of any one of claims 1-12, wherein the first imaging modality comprises at least one of a micro computed tomography (micro-CT) imaging modality or a nano computed tomography (nano-CT) imaging modality.

14. The method of any one of claims 1-13, wherein the second imaging modality comprises a microscopy imaging modality including at least one of a hematoxylin and eosin (H&E) stain, a Papanicolaou stain, or immunohistochemical (IHC) stain, to differentiate the color of the plurality of cells from a remainder of the second biomedical image,optionally wherein the third imaging modality comprises the microscopy imaging modality.

15. The method of any one of claims 1-1 , wherein receiving the first plurality of biomedical images further comprises receiving the first plurality of biomedical images of the biological sample housed in a Formalin-Fixed Paraffin-Embedded (FFPE) cell block and scanned from within the FFPE cell block via an imaging device.

16. The method of any one of claims 1-14, wherein the condition comprises at least one of cancer, an infection, an inflammation, or a cellular degeneration.

17. The method of claim 16, wherein the cancer comprises at least one of carcinoma, sarcoma, hematopoietic cancer, adrenal cancer, bladder cancer, bone cancer, brain cancer, breast cancer, cervical cancer, colon cancer, colorectal cancer, corpus uterine cancer, ear, nose and throat (ENT) cancer, endometrial cancer, esophageal cancer, gastrointestinal cancer, head and neck cancer, Hodgkin's disease, intestinal cancer, kidney cancer, larynx cancer, leukemia, liver cancer, lymph node cancer, lymphoma, lung cancer, melanoma,Atty. Dkt. No.: 115872-3415 mesothelioma, myeloma, nasopharynx cancer, neuroblastoma, non-Hodgkin’s lymphoma, oral cancer, ovarian cancer, pancreatic cancer, penile cancer, pharynx cancer, prostate cancer, rectal cancer, seminoma, skin cancer, stomach cancer, teratoma, testicular cancer, thyroid cancer, uterine cancer, vaginal cancer, vascular tumor, or metastases thereof..

18. The method of any one of claims 16 or 17, wherein the infection comprises at least one of a bacterial infection, a viral infection, a fungal infection, or a parasitic infection.

19. The method of any one of claims 16-18, wherein the inflammation comprises at least one of an acute inflammation or a chronic inflammation.

20. The method of any one of claims 16-19, wherein the cellular degeneration comprises at least one of nuclear degeneration, cytoplasmic vacuolation, cytolysis, or cell atrophy.

21. The method of any one of claims 1-20, wherein the biological sample is obtained from an organ associated with the condition within the subject.

22. The method of any one of claims 1-21, wherein a section of the biological sample has a thickness of between 1 micrometers and 30 micrometers.

23. The method of any one of claims 1-22, wherein the volume corresponding to the second biomedical image in the second imaging modality has a thickness between 1 millimeter and 10 millimeters.

24. A system for generating biomedical images of cell or tissue blocks, comprising:one or more processors coupled with memory, configured to:receive, for a subject at risk of or diagnosed with a condition, a plurality of first biomedical images of a volume with a biological sample having a plurality of cells from the subject in a first imaging modality, each first biomedical image of the plurality of first biomedical images corresponding to a respective section of a plurality of sections of the volume;Aty. Dkt. No.: 115872-3415 generate, using the plurality of first biomedical images, a second biomedical image corresponding to the volume with the biological sample in the first imaging modality;apply a machine learning (ML) to the second biomedical image to convert from the first imaging modality to a second imaging modality, wherein the ML model is established using a plurality of examples, each of the plurality of examples comprising (i) an example first image of an example biological sample having an example plurality of cells in the first imaging modality and (ii) an example second image of the example biological sample applied with a stain associated with the second imaging modality to modify example color corresponding to the example plurality of cells;convert, based on applying the ML model, the second biomedical image from the first imaging modality to the second imaging modality to modify color corresponding to the plurality of cells; andstore, using one or more data structures, an association between the subject and the second biomedical image in the second imaging modality.

25. The system of claim 24, wherein the one or more processors are further configured to provide a user interface to display the second biomedical image corresponding to the volume with the biological sample having the plurality of cells in at least one of the first imaging modality or the second imaging modality.

26. The system of claim 25, wherein the one or more processors are further configured to:receive, via the user interface, a selection of a section of the plurality of sections of the second biomedical image;identify, from the second biomedical image, a two-dimensional slice corresponding to the section selected from the plurality of sections; andprovide, for display via the user interface, the two-dimensional slice.

27. The system of any one of claims 24-26, wherein the one or more processors are further configured to:receive a plurality of third biomedical images in a third imaging modality, each third biomedical image of the plurality of third biomedical images corresponding to a slice of a plurality of slices of the biological sample;Aty. Dkt. No.: 115872-3415 generate, using the plurality of third biomedical images, a fourth biomedical image corresponding to the volume with the biological sample in the third imaging modality; and provide a user interface to display at least one of the second biomedical image in the second imaging modality or the fourth biomedical image in the third imaging modality.

28. The system of any one of claims 24-27, wherein the ML model is established using a second plurality of examples, each of the second plurality of examples comprising (i) an example third image of a respective example biological sample and (ii) an annotation identifying at least an example portion of the example third image as an example region of interest (ROI),wherein the one or more processors are further configured to generate, based on applying the ML model to the second biomedical image, a third biomedical image identifying at least a portion of the second biomedical image as corresponding to an ROI.

29. The system of any one of claims 24-28, wherein at least one of the second plurality of examples comprises the annotation identifies at least the example portion of the example third image as at least one of a plurality of classifications for the example ROI, wherein the plurality of classification comprises at least one of a blood vessel, a tissue, or tumor, wherein the one or more processors are further configured to generate the third biomedical image identifying at least the portion of the second biomedical image as corresponding to a classification of the plurality of classification for the ROI.

30. The system of claim 29, wherein the one or more processors are further configured to:generate a report using the ROI corresponding to at least the portion of the second biomedical image identified by the third biomedical image; andprovide a user interface to display at least one of the third biomedical image or the report.

31. The system of claim 30, wherein the report comprises at least one of: (i) a first identifier for the first plurality of biomedical images, (ii) a second identifier for the second biomedical image, (iii) a third identifier for the ROI, or (iv) the classification for the ROI.Aty. Dkt. No.: 115872-3415 32. The system of any one of claims 29-31, wherein the one or more processors are further configured to:identify a section of interest (SOI) from a plurality of sections of the second biomedical image based on the ROI on the section; andprovide a user interface to display an identification of the SOI in the second biomedical image.

33. The system of any one of claims 24-32, wherein the one or more processors are further configured to:receive a plurality of third biomedical images in at least one of the second imaging modality or a third imaging modality, each third biomedical image of the plurality of third biomedical images corresponding to a slice of a plurality of slices of the biological sample; andadd at least a portion of the plurality of third biomedical images to generate the second biomedical image in the second imaging modality.

34. The system of any one of claims 24-33, wherein the second biomedical image in the second imaging modality identifies at least one ROI corresponding to a feature in the biological sample that is not apparent in a fifth biomedical image in the third imaging modality, the fifth biomedical image corresponding to a section of the biological sample.

35. The system of claim 34, wherein the second biomedical image in the second imaging modality identifies at least one condition not inferable from the fifth biomedical image in the third imaging modality.

36. The system of claim 24-35, wherein the first imaging modality comprises at least one of a micro computed tomography (micro-CT) imaging modality or a nano computed tomography (nano-CT) imaging modality.

37. The system of any one of claims 24-36, wherein the second imaging modality comprises a microscopy imaging modality for at least one of a hematoxylin and eosinAty. Dkt. No.: 115872-3415 (H&E) stain, a Papanicolaou stain, or immunohistochemical (IHC) stain to differentiate the color of the plurality of cells from a remainder of the second biomedical image, optionally wherein the third imaging modality comprises the microscopy imaging modality.

38. The system of any one of claims 24-37, wherein the one or more processors are further configured to receive the first plurality of biomedical images of the biological sample housed in a Formalin-Fixed Paraffin-Embedded (FFPE) cell block and scanned from within the FFPE cell block via an imaging device.

39. The system of any one of claims 24-38, wherein the condition comprises at least one of cancer, an infection, an inflammation, or a cellular degeneration.

40. The system of claim 39, wherein the cancer comprises at least one of carcinoma, sarcoma, hematopoietic cancer, adrenal cancer, bladder cancer, bone cancer, brain cancer, breast cancer, cervical cancer, colon cancer, colorectal cancer, corpus uterine cancer, ear, nose and throat (ENT) cancer, endometrial cancer, esophageal cancer, gastrointestinal cancer, head and neck cancer, Hodgkin's disease, intestinal cancer, kidney cancer, larynx cancer, leukemia, liver cancer, lymph node cancer, lymphoma, lung cancer, melanoma, mesothelioma, myeloma, nasopharynx cancer, neuroblastoma, non-Hodgkin’s lymphoma, oral cancer, ovarian cancer, pancreatic cancer, penile cancer, pharynx cancer, prostate cancer, rectal cancer, seminoma, skin cancer, stomach cancer, teratoma, testicular cancer, thyroid cancer, uterine cancer, vaginal cancer, vascular tumor, or metastases thereof.

41. The system of claim 39 or 40, wherein the infection comprises at least one of a bacterial infection, a viral infection, a fungal infection, or a parasitic infection.

42. The system of claims 39-41, wherein the inflammation comprises at least one of an acute inflammation or a chronic inflammation.

43. The system of claims 39-42, wherein the cellular degeneration comprises at least one of nuclear degeneration, cytoplasmic vacuolation, cytolysis, or cell atrophy.Atty. Dkt. No.: 115872-341544. The system of any one of claims 24-43, wherein the biological sample is obtained from an organ associated with the condition within the subject.

45. The system of any one of claims 24-44, wherein a section of the biological sample has a thickness of between 1 micrometers and 30 micrometers.

46. The system of any one of claims 24-45, wherein the volume corresponding to the second biomedical image in the second imaging modality has a thickness between 1 millimeter and 10 millimeters.

Images