System and method for training machine learning on noisy and inaccurate immunostain

Abstract

To provide a method of clearing false positive immunostaining signals and noise, to perfect incomplete staining.SOLUTION: A method for refining virtually-stained image annotations includes separating an image of a stained tissue section into an image of a counterstain and an image of immunostaining visual signals via linear transformation, to identify and segment nuclei in the image of the counterstain to generate an image of a refined counterstain with single-cell segments. The method further includes comparing immunostaining visual signals in the single-cell segments of the image of the refined counterstain to a user-defined threshold to determine true positive cells. The method further includes overlaying the image of the counterstain and the image of refined immunostaining visual signals to generate a refined virtually-stained image annotation.SELECTED DRAWING: None

Description

[Technical field]

[0001] The present disclosure relates to virtual staining models, and more particularly to refining digitally scanned immunostained slides to train machine learning models to perform better in terms of accuracy and reproducibility in biomarker discovery, tissue-based research, and diagnostic testing. [Background technology]

[0002] Across many fields of medicine and biology, immunostaining analyses such as immunofluorescence (IF) and immunohistochemistry (IHC) are frequently used by clinicians and researchers to evaluate and diagnose various diseases, including cancer. Immunostaining methods use antibodies to identify specific types of molecules in a sample, and the visual signal can be amplified using dyes, including but not limited to fluorescent tags, fluorescently expressed proteins, and 3,3'-diaminobenzidine (DAB). The resulting staining pattern can reflect the localization of the target molecule. Due to their strong molecular specificity, there is an increasing demand for the use of biomarker information to evaluate in situ protein expression in tumor tissues.

[0003] However, this method requires physical staining of individual tissue samples, which may result in more or less non-specific staining, granular staining due to clumping or pre-analysis, incomplete staining, and uneven color. Variations in staining quality may lead to misdiagnosis and research analysis, leading to misleading medical decisions in future treatment steps. More importantly, inconsistent staining quality and staining noise pose major challenges when using stains as ground truth labels to train virtual staining machine learning models. Summary of the Invention

[0004] In view of the above, there is a need in the art for a method to remove false positive immunostaining signals and noise, and to perfect incomplete staining. In this regard, the present disclosure relates to a method for refining a virtual stained image annotation, the method comprising: separating an image of a stained tissue section into an image of a counterstain and an image of an immunostained visual signal via a linear transformation; identifying and segmenting nuclei in the image of the counterstain to generate a refined counterstained image with single cell segments; in addition, the method comprises comparing the immunostained visual signal in the single cell segment of the refined counterstained image with a user-defined threshold to determine true positive cells; thereafter, the method comprises refining the image of the immunostained visual signal to generate a refined image of the immunostained visual signal; the refinement is based on the protein localization or morphology of the stained tissue section and the determined true positive cells; finally, the method comprises overlaying the image of the counterstain and the image of the refined immunostained visual signal to generate a refined virtual stained image annotation.

[0005] In another embodiment, the counterstain may be selected from the group consisting of hematoxylin, Hoechst, and 4',6-diamidino-2-phenylindole (DAPI).Similarly, the immunostaining visual signal may be selected from the group consisting of a fluorescent tag, a fluorescently expressed protein, and 3,3'-diaminobenzidine (DAB) stain.

[0006] In another embodiment, the refinement of the immunostained visual signal image may include burring using a Gaussian filter to simulate membrane staining. In yet another embodiment, overlaying the refined counterstained image onto the refined immunostained visual signal image may include concatenating the refined counterstained image and the immunostained visual signal image in a third dimension to generate a concatenation. Additionally, overlaying the refined counterstained image onto the refined immunostained visual signal image may include performing a linear transformation to convert the concatenation into the image native color space for display purposes.

[0007] In yet another embodiment, comparing the immunostained visual signal to a user-defined threshold in the single cell segments of the refined counterstained image may include summing the number of pixels in the image of the immunostained visual signal that have an immunostained visual signal greater than 0 in each single cell segment. Comparing the immunostained visual signals may additionally or alternatively include summing the immunostained visual signal intensity in each single cell segment or analyzing the maximum intensity of the immunostained visual signal in each single cell segment. Still further, comparing the immunostained visual signals may additionally or alternatively include defining true positive cells by dividing the total amount of immunostained visual signal in the cell area by the area of ​​the cell segment area.

[0008] The method may further include training a neural network to generate refined virtual stained image annotations, where the image of the unstained tissue section is input to the neural network.

[0009] In a further embodiment, segmenting the nucleus in the counterstained image may include expanding the nucleus to include the cytoplasmic region. The present disclosure relates to a method for refining a virtual stained image annotation. The method includes separating an image of a stained tissue section into an image of a counterstain and an image of an immunostained visual signal through a linear transformation. The method includes identifying and segmenting nuclei in the counterstained image to generate a refined counterstained image with single cell segments. The method includes comparing the immunostained visual signal in the single cell segment of the refined counterstained image with a user-defined threshold to determine true positive cells. The method includes refining the image of the immunostained visual signal to generate a refined image of the immunostained visual signal, the image refinement being based on the protein localization or morphology of the stained tissue section and the determined true positive cells. Finally, the method includes training a neural network to generate a refined virtual stained image annotation, and an image of an unstained tissue section is input to the neural network.

[0010] The present disclosure also relates to a method for refining a virtual stained image annotation, the method comprising: separating an image of a stained tissue section into two separate staining channels via a linear transformation. The two separate staining channels include an image of a counterstain and an image of 3,3'-diaminobenzidine (DAB) stain. The method comprises segmenting nuclei in the image of the counterstain to generate an image of a single cell segment. The method comprises comparing the DAB signal of the image of the single cell segment with a threshold to determine true positive cells. The method comprises refining the image of the DAB stain according to the determined true positive cells, and generating an image of a refined immunostained visual signal. Finally, the method comprises training a neural network to generate a refined virtual stained image annotation, and the image of the refined immunostained visual signal is an input to the neural network.

[0011] Various aspects of the present disclosure are described herein with reference to the accompanying drawings. For simplicity and clarity of illustration, elements shown in the drawings have not necessarily been drawn accurately or to scale. For example, dimensions of some elements may be exaggerated relative to other elements for clarity and some physical components may be subsumed within a single functional block or element. Further, where considered appropriate, the same reference numbers may be used between the drawings to indicate corresponding or similar elements. However, for clarity, not every component may be labeled in every drawing. The drawings are provided for illustration and description and are not intended to define the limits of the present disclosure. [Brief description of the drawings]

[0012] [Figure 1] 1 shows an IHC stained tissue specimen according to the present disclosure. [Diagram 2] Figure 1 shows an IHC stained tissue specimen separated into two channels: an image of the counterstain and an image of the immunostained visual signal. [Diagram 3]Shown is the counterstained image of FIG. 2 refined so that each nucleus is identified and segmented. [Figure 4] Single cell segmentations in Figure 3 are overlaid on the immunostained visual signal of true positive cells. [Diagram 5] We show the immunostaining visual signals of true positive cells for the single cell segmentations in Figures 3 and 4, which were refined based on prior knowledge of the target molecule. [Figure 6] Shown is the refined true positive cell staining in Figure 5 overlaid with the counterstain in Figure 2 to generate the final IHC / IF staining output. [Figure 7] 1 illustrates training of a neural network according to the present disclosure. [Figure 8] 1 illustrates a method for refining virtual stain image annotation according to the present disclosure. [Figure 9] FIG. 1 is a block diagram of a computer system configured to perform neural network-related processing for refining virtual stain image annotations, as described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The present technology overcomes many of the problems of the prior art associated with immunostaining analysis. Advantages and other features of the technology disclosed herein will be readily apparent to those skilled in the art from the following detailed description of certain exemplary embodiments in combination with the drawings, in which like reference numerals indicate like structural elements. It should be noted that directional designations such as vertical, horizontal, upward, downward, right, left, etc. are used with respect to the figures and are not meant to be limiting.

[0014] Presented herein is a framework for refining digitally scanned immunostained slides to train machine learning models to perform better in terms of accuracy and reproducibility, removing false positive immunostaining signals and noise, and perfecting incomplete staining, based on prior knowledge from relevant literature and biotechnology and medical experts such as tissue technicians and histopathologists.

[0015] The workflow described herein facilitates accurate virtual immunostaining. In the following description, immunohistochemistry (IHC) staining is used as an example to explain the workflow, but the workflow is not limited to IHC and can be applied to any immunostaining method, such as immunofluorescence staining.

[0016] 1, there is shown an IHC stained tissue specimen 100. The principles described with respect to the IHC stained tissue specimen 100 apply equally to immunofluorescence stained tissue specimens. IHC stained tissue specimen 100 slides are prepared according to different protocols depending on the sample and purpose. For immunohistopathology, the first step in preparing an IHC stained tissue specimen 100 may be to fix the tissue to avoid tissue degradation, which is usually completed by immersing the tissue in formalin. The tissue may then be processed by dehydrating, clearing, and infiltrating with paraffin wax. The tissue is then transferred to a cold room where it hardens to form a paraffin block. The tissue block may then be sliced ​​into 2-5 micron thick sections using a microtome, which may then be placed on a rectangular glass surface. The paraffin may then be removed using a solvent, for example xylene.

[0017] Depending on the type of biomarker desired and the type of antibody cell desired, various techniques and processes are then performed. After labeling with the desired antibodies, the IHC stained tissue specimen 100 is imaged using a scanning microscope (usually using different frequencies of visible and invisible light, allowing different layers of tissue to be detected by phenomena such as simple reflection and / or autofluorescence).

[0018] Specifically, in our setup, the IHC stained tissue specimen 100 in FIG. 1 is imaged using a Zeiss Axio Scan Z1 or Zeiss AxioScan 7 automated microscope equipped with a Hamamatsu Photonics OrcaFlash camera. The automated microscope has a magnification of 20x and a numerical aperture of 0.8. The IHC stained tissue specimen 100 has a pixel resolution of 16-bit and a size of 0.324 μm × 0.324 μm. Modern microscopes are equipped with digital cameras with different types of sensors (usually called CCD sensors or CMOS sensors), so the sensitivity and resolution can affect the quality and size of the IHC stained tissue specimen 100.

[0019] 2, the IHC stained tissue specimen 100 of FIG. 1 is separated into an image 200 of a counterstain, such as a hematoxylin stain used to stain cell nuclei, and an image 250 of an immunostained visual signal, such as a 3,3'-diaminobenzidine (DAB) stain that marks certain proteins or molecules. When used for immunofluorescence, the stained tissue specimen is separated into the counterstain and the immunostained visual signal 250, which includes a fluorescent tag or fluorescently expressed protein.

[0020] Stain separation is achieved by a linear transformation. Linear transformation requires a mapping between two vector spaces that preserves the operations of vector addition and scalar multiplication. Stain separation is achieved by a linear transformation. Thus, multiplying an image of an IHC stained tissue specimen 100 by a 3x3 matrix in logarithmic scale results in an image of the counterstain 200 and an image of the immunostained visual signal 250 separated into different channels.

[0021] In FIG. 2, the following 3×3 matrix was applied in logarithmic scale to an image of an IHC stained tissue specimen 100 to obtain an image of counterstain 200 and an image of immunostained visual signal 250: [1.88,-0.07,-0.60] [-1.02,1.13,-0.48] [-0.55,-0.13,1.57] As a result, the image manipulation of the IHC stained tissue specimen 100 is log(<original image>) multiplied by (<the 3x3 matrix provided above>).

[0022] Referring now to FIG. 3, each nucleus in the counterstain 200 image is identified and segmented to generate a refined counterstain 300 image with single cell segments. Each nucleus in the refined counterstain 300 is colored a different color and expanded to include the cytoplasmic region of each cell via a single or series of morphological operators. The identification and segmentation can be completed by a deep learning based segmentation algorithm such as Cellpose. Cellpose can accurately segment cells from a wide variety of images and does not require model retraining or parameter tuning. See Stringer, C., Wang, T., Michaelos, M. et al. Cellpose: a generalist algorithm for cellular segmentation. Nat Methods 18, 100-106 (2021), the entirety of which is incorporated herein by reference for any purpose.

[0023] Referring now to Figure 4, a silhouette of a single cell segmentation 400 is shown overlaid with the immunostaining visual signal of a true positive cell. Each single cell segmentation of the refined counterstain 300 of Figure 3 is extrapolated and the immunostaining visual signal, such as the DAB signal, of each single cell segmentation of the refined counterstain 300 is compared to a user-defined threshold T to determine true positive cells. Among the single cell segments, those that contain a DAB signal higher than the user-defined threshold T are considered to be true positive cells.

[0024] There are multiple ways to set the threshold T, but each case is unique. For example, the amount of immunostaining visual signal can be evaluated by i) counting the number of pixels of the elaborated counterstain 300 where the immunostaining visual signal is higher than 0 in each segmentation area, ii) summing the intensity of the immunostaining visual signal in each segmentation area of ​​the elaborated counterstain 300, or iii) reading the maximum intensity of the immunostaining visual signal in each segmentation area of ​​the elaborated counterstain 300. Then, to determine positive cells, a) the total amount of immunostaining visual signal in the area of ​​the cells, or the value obtained by dividing <the total amount of immunostaining visual signal in the area of ​​the cells> by <the area of ​​the segmentation area of ​​the single cell> can be evaluated.

[0025] In Figure 4, the ratio of the DAB area to the total cell area is considered to determine true positive cells. In this example, cells with a ratio of DAB to cell area of ​​more than 20% are considered true positive cells.

[0026] Referring now to FIG. 5, the immunostaining visual signal 500, such as the DAB signal of the true positive cell of the single cell segmentation of FIG. 3 and FIG. 4, is refined based on prior knowledge about the target molecule. For example, if prior knowledge about the protein localization and morphology of the target molecule is known, a hypothetical "perfect stain" can be generated. In FIG. 5, the target molecule is known to be localized on the membrane. Therefore, the perimeter line of the true positive cell of the single cell segmentation of FIG. 3 and FIG. 4 is blurred using a conventional Gaussian filter to simulate membrane staining to obtain the refined true positive cell 550. Gaussian filtering is a common technique in image processing that is utilized for a number of purposes, such as noise removal and smoothing.

[0027] In the previous example, the following Gaussian filter, a 3x3 matrix, was applied to the log-scaled immunostaining visual signal 500 to simulate membrane staining: [0.65,0.70,0.29] [0.07,0.99,0.11] [0.27,0.57,0.78] It should be understood that the Gaussian filtering operation described with reference to FIG. 5 is not limited to membrane proteins. For example, if the target protein is known to be localized in the nucleus, the immunostaining visual signal can be limited to the nucleus only. If the target protein is known to be localized in the cytoplasmic region, the immunostaining visual signal can be limited to the cytoplasmic region only. Other staining patterns can also be applied, such as, but not limited to, other organelles such as nucleoli, ribosomes, vesicles, endoplasmic reticulum, Golgi apparatus, cytoskeleton, mitochondria, vacuoles, lysosomes, centrosomes, etc.

[0028] Referring now to FIG. 6, the staining of the refined true positive cells 550 of FIG. 5 is shown overlaid with a counterstain 200 (e.g., DAPI, hematoxylin) to generate the final IHC / IF staining output 600. Specifically, the original counterstain 200 image shown in FIG. 2 and the refined true positive cells 550 image of FIG. 5 are concatenated in the third dimension. For display purposes, a linear transformation is applied to convert the overlaid image to the image native color space, such as the RGB (Red, Green, Blue) color space, to generate the final IHC / IF staining output 600. For example, since the (X,Y) location of each pixel detected in the stained image of the refined true positive cells 550 is known, these pixels can be colored with different intensities and thus "overlaid" on the counterstain 200.

[0029] 7, the training of a neural network 725 is illustrated. The neural network 725 is designed and trained using as input one or a combination of the following: 1) an unstained tissue section 700, 2) a stained tissue section 100, 3) an image of a counterstain 200 of the stained tissue section, 4) an image of an immunostain visual signal of the stained tissue section 250, or 5) a final IHC / IF staining output 600. From the aforementioned inputs, the neural network 725 is trained to generate a final IHC / IF staining output 750 similar to the final IHC / IF staining output 600 of FIG. 6. Thus, the neural network is configured to avoid false positive immunostaining signals and noise and to complete incomplete staining.

[0030] Referring now to FIG. 8, a method 800 for refining a virtual stained image annotation is shown. According to the method, first, in step S802, an image of a stained tissue section is separated into an image of a counterstain and an image of an immunostained visual signal through a linear transformation. Then, in step S804, nuclei are identified and segmented in the counterstained image to generate a refined counterstained image having single cell segments. Then, in step S806, the immunostained visual signal in the single cell segment of the refined counterstained image is compared with a user-defined threshold to determine true positive cells. In step S808, the image of the immunostained visual signal is refined to generate a refined image of the immunostained visual signal. The refinement is based on the protein localization or morphology of the stained tissue section and the determined true positive cells. Finally, in step S810, the image of the counterstain and the refined image of the immunostained visual signal are superimposed to generate a refined virtual stained image annotation.

[0031] Referring now to FIG. 9, FIG. 9 is a block diagram of a computer system 900 configured to perform processing related to a neural network for refining virtual stain image annotations as described herein. The exemplary computer system 900 includes a central processing unit (CPU) 902, a memory 904, and an interconnecting bus 906. The CPU 902 may include a single microprocessor or multiple microprocessors or special purpose processors to configure the computer system 900 as a multiprocessor system. The memory 904 includes, for example, a main memory and a read-only memory. The computer 900 also includes a mass storage device 908 having, for example, various disk drives, tape drives, and the like. The memory 904 also includes a dynamic random access memory (DRAM) and / or a high-speed cache memory. During operation, the memory 904 stores at least a portion of instructions and data for execution by the CPU 902. The memory 904 may also include computational elements such as Deep In-Memory Architectures (DIMA), where data is sent to the memory and a function of the data (eg, matrix vector multiplication) is read by the CPU 902.

[0032] Mass storage 908 may include one or more magnetic disks, optical disk drives, and / or solid state memory for storing data and instructions used by CPU 902. At least one component of mass storage 908 is preferably in the form of a non-volatile disk drive, solid state, or tape drive, and stores a database used to process data and control the function of the neural network to refine the virtual stain image annotation. Mass storage 908 may also include one or more drives for various portable media, such as floppy disks, flash drives, compact disk read-only memories (CD-ROM, DVD, CD-RW, etc.), memory sticks, or integrated circuit non-volatile memory adapters (i.e., PC-MCIA adapters), for inputting and outputting data and code to and from computer system 200.

[0033] The computer system 900 may also include one or more input / output interfaces for communication, shown as, by way of example, an interface 910 and / or a transceiver for data communication over a network 912. The data interface 910 may be a modem, an Ethernet card, or any other suitable data communication device. To provide the functionality of the processor executing the neural network for refining the virtual stain image annotation, the data interface 910 may provide a relatively high-speed link to a network 912, such as an intranet, the Internet, an Aegis network, or the Internet, directly or through another external interface. The communication link to the network 912 may be, for example, optical, wired, or wireless (e.g., via a satellite or cellular network). The computer system 900 may also connect to at least one other computer system via the data interface 910 and the network 912 to perform remote or distributed multi-sensor processing, for example, in connection with a common operational picture (COP). Alternatively, the computer system 900 may include a mainframe or other type of host computer system capable of web-based communication over the network 912. Computer system 900 may include software for operating network applications, such as a web server and / or a web client.

[0034] Computer system 900 may also include suitable input / output ports, which may interface with portable data storage devices, or may use an interconnect bus 906 for interconnection with a local display 916 and keyboard 914, etc., as a local user interface for programming and / or data acquisition purposes. The display 916 may include touch screen capabilities that allow a user to interface with system 900 by touching a portion of the surface of the display 916. Server operators may interact with system 900 to control and / or program the system from a remote terminal device via a user interface 912.

[0035] The computer system 900 may execute various application programs and store associated data in a database in the mass storage device 908. One or more of such applications may include neural networks for refining virtual stain image annotations, such as those described with respect to Figures 1-8.

[0036] Depending on the components included in computer system 900, the computer system may be used as a server, a workstation, a personal computer, a network terminal, a mobile computing device, a mobile phone, a system on a chip (SoC), etc. System 900 may include software and / or hardware implementing a web server application. The web server application may include software such as HTML, XML, WML, SGML, PHP (Hypertext Preprocessor), CGI languages, etc.

[0037] The aforementioned features of the present disclosure may be implemented as software components operating on the system 900, where the system 900 includes a Unix workstation, a Windows workstation, a LINUX workstation, or other types of workstations. Other operating systems may be employed, such as, but not limited to, Windows, MAC OS, LINUX. In some embodiments, the software may be optionally implemented as a computer program written in C language computer programs or any high-level language, including, but not limited to, JavaScript, Java, CSS, Python, Keras, TensorFlow, PHP, Ruby, C++, C, Shell, C#, Objective-C, Go, R, TeX, VimL, ​​Perl, Scala, CoffeeScript, Emacs Lisp, Swift, Fortran, or Visual BASIC. Certain script-based programs, such as XML, WML, PHP, etc. may be employed. The system 200 may use a digital signal processor (DSP).

[0038] As previously mentioned, the mass storage device 908 may include a database. The database may be any suitable database system, including the commercially available Microsoft Access database, and may be a local or distributed database system. The database system may implement Sybase or SQL Server. The database may be supported by any suitable persistent data memory, such as a hard disk drive, a RAID system, a tape drive system, a floppy diskette, or any other suitable system. It will be appreciated that the system 900 may include a database integrated with a neural network for refining the virtual stain image annotation, although in other embodiments the database and mass storage device 908 may be external elements.

[0039] In certain embodiments, the system 900 may include an Internet browser program and / or may be configured to operate as a web server. In some configurations, the client and / or web server may be configured to recognize and interpret various network protocols that may be used by the client or server programs. Examples of commonly used protocols include HTTP (Hypertext Transfer Protocol), FTP (File Transfer Protocol), Telnet, SSL (Secure Sockets Layer), and TLS (Transport Layer Security). However, new protocols and revisions to existing protocols may be frequently introduced. Thus, new revisions of server and / or client applications may be continually developed and released to support the new and revised protocols.

[0040] In one embodiment, the neural network includes a network-based, e.g., Internet-based, application that may be configured and executed on any combination of the system 900 and / or other components of the neural network for refining virtual stain image annotations. The computer system 900 may include a web server that executes Web 2.0 applications, etc. The web application that executes on the neural network may use server-side dynamic content generation mechanisms, such as, but not limited to, Java Servlets, CGI, PHP, ASP, etc. In certain embodiments, the mashed-up content may be generated by, for example, a web browser executing client-side scripting, including, but not limited to, JavaScript and / or applets, on a wireless device.

[0041] In certain embodiments, the neural network or computer system 900 for refining virtual stain image annotations may include applications employing asynchronous JavaScript+XML (Ajax) and similar techniques using asynchronous loading and content presentation techniques, including, but not limited to, XHTML and CSS for style presentation, Document Object Model (DOM) APIs exposed by web browsers, asynchronous data exchange of XML data, and web browser side scripting (e.g., JavaScript). Certain web-based applications and services may utilize web protocols, including, but not limited to, services-oriented access protocol (SOAP) and representational state transfer (REST). REST may utilize HTTP with XML.

[0042] The neural network for refining virtual stain image annotations, the computer system 900, or another component of the neural network may also provide enhanced security and data encryption. Enhanced security may include access control, biometric authentication, cryptographic authentication, message integrity checks, encryption, digital rights management services, and / or other similar security services. Security may include protocols such as IPSEC and IKE. Encryption may include, but is not limited to, DES, 3DES, AES, RSA, ECC, and any similar public or private key based schemes.

[0043] Those skilled in the art will appreciate that in alternative embodiments the functionality of some elements may be accomplished by fewer elements or by a single element. Similarly, in some embodiments any functional element may perform fewer or different operations than those described with respect to the illustrated embodiment. Also, functional elements shown as separate for purposes of illustration may be combined within other functional elements in particular embodiments.

[0044] While the present technology has been described with respect to various embodiments, those skilled in the art will readily appreciate that various changes and / or modifications can be made to the present technology without departing from the scope of the disclosure.

Claims

1. 1. A method for refining a virtual stain image annotation, comprising: Separating the image of the stained tissue section into an image of counterstain and an image of immunostained visual signals via a linear transformation; identifying and segmenting nuclei in said counterstained image to generate a refined counterstained image having single cell segments; comparing the immunostaining visual signal in the single cell segment of the refined counterstained image with a user-defined threshold to determine true positive cells; and refining the immunostained visual signal image based on protein localization or morphology of the stained tissue section and the determined true positive cells to generate a refined immunostained visual signal image; The method includes:

2. 2. The method of claim 1, wherein the counterstain is selected from the group consisting of hematoxylin, Hoechst, and 4',6-diamidino-2-phenylindole (DAPI).

3. 3. The method of claim 1 or 2, wherein refining the image of the immunostained visual signal comprises blurring with a Gaussian filter to simulate membrane, cytoplasmic, or nuclear staining.

4. 3. The method of claim 1 or 2, further comprising the step of training a neural network to generate refined virtual stained image annotations, wherein images of unstained tissue sections are input to the neural network.

5. overlaying the refined counterstained image with the refined immunostained visual signal image; Concatenating the refined counterstained image and the immunostained visual signal image in a third dimension to generate a concatenation; and performing a linear transformation to convert said concatenation into an image native color space for display purposes; The method of claim 1 or 2, comprising:

6. comparing the immunostained visual signal in the single cell segment of the refined counterstained image to a user defined threshold; 3. The method of claim 1 or 2, comprising summing the number of pixels in the image of the immunostaining visual signal that have an immunostaining visual signal greater than 0 within each single cell segment.

7. comparing the immunostained visual signal in the single cell segment of the refined counterstained image to a user defined threshold; The method of claim 1 or 2, comprising summing the intensities of the immunostaining visual signals within each single cell segment.

8. comparing the immunostained visual signal in the single cell segment of the refined counterstained image to a user defined threshold; The method of claim 1 or 2, comprising analyzing the maximum intensity of the immunostaining visual signal within each single cell segment.

9. comparing the immunostained visual signal in the single cell segment of the refined counterstained image to a user defined threshold; The method of claim 1 or 2, comprising defining a true positive cell by dividing the total amount of immunostaining visual signal in a cell region by the area of ​​the segmentation region of the cell.

10. The method of claim 1 or 2, wherein segmenting the nucleus in the counterstained image comprises expanding the nucleus to include a cytoplasmic region.

11. The method of claim 1 or 2, wherein the immunostaining visual signal is selected from the group consisting of a fluorescent tag, a fluorescently expressed protein, and 3,3'-diaminobenzidine (DAB) staining.

12. 3. The method of claim 1 or 2, further comprising overlaying the counterstained image and the refined immunostained visual signal image to generate a refined virtual stained image annotation.

13. 1. A method for refining a virtual stain image annotation, comprising: Separating an image of a stained tissue section into two separate staining channels via a linear transformation, the two separate staining channels including an image of a counterstain and an image of 3,3′-diaminobenzidine (DAB) stain; segmenting nuclei in said counterstained image to generate an image of single cell segments; comparing the DAB signal of the image of the single cell segment with a threshold to determine true positive cells; refining the DAB staining image according to the determined true positive cells, and generating a refined immunostaining visual signal image; and training a neural network to generate refined virtual stain image annotations, the refined images of the immunostained visual signals being input to the neural network; The method includes:

Images