Escalating high dimensional imaging using channel multiplexing and deep learning

The combination of combinatorial staining and deep learning enables the simultaneous visualization of multiple proteins at single-cell resolution, overcoming the limitations of existing technologies by decompressing multiplexed images to reconstruct individual protein images, thus enhancing imaging capabilities without requiring specialized equipment.

WO2024236574A1PCT designated stage expired Publication Date: 2024-11-21YEDA RES & DEV CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
PCT/IL2024/050477
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-18
Filing Date
2024-05-16
Publication Date
2024-11-21

AI Technical Summary

Technical Problem

Current imaging technologies are limited in their ability to visualize multiple proteins simultaneously in tissue sections due to the need for separate channels for each protein, restricting throughput and scalability, and existing combinatorial staining methods are not applicable to proteins due to the assumption of 'spot counting' which is invalidated for proteins under standard magnifications.

Method used

A combinatorial staining platform coupled with a deep learning algorithm that allows each protein to be imaged in multiple channels, with combinatorically-compressed images decompressed using a neural network to reconstruct individual protein images, enabling the visualization of hundreds of proteins at single-cell resolution without the need for specialized instrumentation.

Benefits of technology

This approach significantly increases the number of proteins that can be measured simultaneously, achieving accurate reconstruction with high fidelity, as demonstrated by successful experiments in both fluorescence microscopy and mass-based imaging, and is applicable to various imaging modalities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure IL2024050477_21112024_PF_FP_ABST
    Figure IL2024050477_21112024_PF_FP_ABST
Patent Text Reader

Abstract

Methods for decompressing a multiplexed-image, of C different channels, acquired to identify P targets in a tissue sample, where the targets are combinatorically labeled by at least C different labeling-elements, which are configured to be detected by the C different channels, and where P > C. The method comprising processor implemented step of: applying the multiplexed-image as an input to a trained NN. The NN is configured to output a tensor B, having P binary -images each having binary-pixels, each binary -image p per a different molecule p of the P molecules (where p = l,2,...,Py, wherein a pixel (ij; i = 1,2, = 7,2, in a binary-image p having a value of one (7) indicates that pixel (zj) is expected to contain a signal of its associated molecule p.
Need to check novelty before this filing date? Find Prior Art

Description

ESCALATING HIGH DIMENSIONAL IMAGING USING CHANNEL MULTIPLEXING AND DEEP LEARNINGBACKGROUND OF THE INVENTION

[0001] Tissue pathology is essential for clinical diagnosis of tumors and is routinely used in discovery studies. Immunohistochemistry, whereby antibodies are used to quantify the single-cell spatial distribution of proteins in tissue sections, is a pillar of pathology, used for patient diagnosis, clinical decision making, as well as discovery studies. Traditional chromogenic histochemistry is limited to visualizing one or two proteins at a time, whereas standard immunofluorescence methods are limited to three to five proteins, due to overlapping emission spectra. Accordingly, there is a need, to uncover complex cellular structures, interactions, and functions, and to simultaneously visualize many proteins in situ.SUMMARY OF THE INVENTION

[0002] Understanding tissue structure and function requires tools that quantify the expression of multiple proteins at single-cell resolution while preserving spatial information. Current imaging technologies use a separate channel for each individual protein, inherently limiting their throughput and scalability. In the following method embodiments, a combinatorial staining platform is provided, coupled with an algorithmic framework to exponentially increase the number of proteins that can be measured from C up to 2C- 1. In this new method, every protein can be imaged in several channels, and every channel contains agglomerated images of several proteins. These combinatorically-compressed images are then decompressed to individual protein-images using deep learning. In the following examples, accurate reconstruction is achieved when compressing the stains of twenty-two proteins to five imaging channels and it is demonstrated that the approach works in both fluorescence microscopy and in mass-based imaging. Combinatorial staining coupled with deep-learning decompression can escalate the number of proteins measured using any imaging modality, without the need for specialized instrumentation. Coupling the new method with instruments for high-dimensional imaging paves the way to image hundreds of proteins at single-cell resolution in intact tissue sections.

[0003] According to some embodiments of the invention, a new method is provided for decompressing a multiplexed-image (y), of C different channels, acquired to identify P molecules in a tissue sample; the method comprising steps of combinatorically labeling (A) the molecules by at least C different labeling-elements, which are configured to be detected by at least one of said C different channels; accordingly, each one of the molecules is differently labeled by at least one of the labeling-elements; wherein P > C; and processor implemented step of: applying the multiplexed-image (y) as an input to at least one trained neural network (NN); the NN is configured to output a tensor B, having P binary-images each having (IxJ) binary-pixels; each binary-image p per a different molecule p of the P molecules (where p = 1,2,...,P); wherein a pixel (i,j; i = 1,2,. . .,1; j = 1,2,. . ,,J) in a binary-image p having a value of one (1) indicates that pixel (i,j) is expected to contain a signal of its associated molecule p.

[0004] According to some embodiments of the invention, a new method is provided for decompressing a multiplexed-image (y), of C different channels, acquired to identify P molecules in a tissue sample, wherein the molecules are combinatorically labeled (A) by at least C different labeling-elements which are configured to be detected by at least one of the C different channels; accordingly, each one of the molecules is differently labeled by at least one of the labeling-elements, and wherein one of the molecules; wherein P > C; the method comprising processor implemented step of applying the multiplexed-image (y) as an input to at least one trained neural network (NN); the NN is configured to output a tensor B, having P binary -images each having (I* J) binary-pixels; each binary-image p per a different molecule p of the P molecules (where p = 1,2, ...,Py, wherein a pixel (i,j; i = 1,2,= 1,2, ..., J) in a binary -image p having a value of one (7) indicates that pixel (zj) is expected to contain a signal of its associated molecule p.

[0005] According to some embodiments, the NN is a supervised NN. According to some embodiments, the supervised NN is CNN.

[0006] According to some embodiments, the method further comprising applying an optimization algorithm to the NN’s output tensor 7>; the optimization algorithm is configured to provide each of the pixels (z ) of a binary-image p with an intensity value; wherein the intensity value indicates the associated molecule p in pixel (ij).

[0007] According to some embodiments, the method further comprising combinatorically labeling the P molecules, via the C labeling-elements, wherein each molecule is uniquely labeled via one or several labeling-elements, and wherein at least one molecule is labeled via at least two labeling-elements.

[0008] According to some embodiments, the labeling-elements comprise at least one of: fluorescent dyes, chromophores, heavy-metal isotopes, chemiluminescent substances, radioactive substances, fluorescent proteins, nanoparticles, any detectable moiety, and any combination thereof

[0009] According to some embodiments, the method further comprising acquiring the multiplexed-image (y), of C channels, of the P labeled molecules, configured to be applied as the input to the trained NN at the decompression step.

[0010] According to some embodiments, the method further comprising unsupervised learning of the NN.

[0011] According to some embodiments, the method further comprising supervised training of the NN, via:• at least one training multiplexed-image, to be applied as a training input-image to the NN; and• a known desired-tensor b, of the training multiplexed-image, having P binaryimages with binary -pixels, to be applied as a training’s desired-output for the NN.

[0012] According to some embodiments, the method further comprising acquiring the at least one training multiplexed-image.

[0013] According to some embodiments, the method further comprising:• acquiring P ground truth individual-images (x), of the P separately-labeled molecules, wherein each individual-image p is configured to depict a single molecule p of the P molecules; and• converting the P individual -images to P binary -images having (I *J) binary -pixels, configured to be applied as the known desired-tensor b.

[0014] According to some embodiments, the method further comprising simulating the at least one training multiplexed-image, from the P individual-images (x), based on the combinatorial labeling design.

[0015] According to some embodiments, the method further comprising convolving matrix A with the P individual-images (x).

[0016] According to some embodiments of the invention, a new method is provided for detecting targets in a sample comprising:• combinatorically labeling P targets in a sample with at least C different labelingelements which are configured to be detected by at least one of C different channels (only C); accordingly, each one of the targets is differently labeled by at least one of the labeling-elements; wherein C < P• imaging the sample to acquire at least one multiplexed-image of the sample; and• decompressing the multiplexed-image, according to the method of any one of the above mentioned embodiments, to extract P different images each of a different target of the P targets.

[0017] According to some embodiments, the targets comprise biological molecules.

[0018] According to some embodiments, the biological molecules are selected from the group consisting of proteins, peptides, polypeptides, antibodies, antigens, enzymes, enzyme substrates, ligands, receptors, polynucleotides, RNA, DNA, metabolites, polysaccharides, lectins, sugars, glycoproteins, lipids, fats, vitamins, hormones, and cellular compartments.

[0019] According to some embodiments, the one or more biological molecules are biomarkers.

[0020] According to some embodiments, the labeling-elements are attached to the P targets by P reporters; and wherein the reporters are selected from the group consisting of antibodies, peptides, proteins, polynucleotides, oligonucleotides, polysaccharides, lipids, enzymes, enzyme substrates, ligands, receptors, antigens, haptens, and aptamers.

[0021] According to some embodiments, the labeling-element comprises at least one detectable moiety.

[0022] According to some embodiments, the detectable moiety is selected from the group consisting of a dye, fluorescent dye, heavy -metal isotope, chromophore, chemiluminescent substance, radioactive substance, nanoparticle, surface plasmon resonant particle (PRP), and resonance light scattering particle (RLS).

[0023] According to some embodiments, the sample comprises a tissue sample. According to some embodiments, the tissue sample comprises a mammalian tissue. According to some embodiments, the tissue sample comprises a human tissue. According to some embodiments, the tissue sample comprises a biopsy from a subject.

[0024] According to some embodiments, wherein the sample comprises cells.

[0025] According to some embodiments of the invention, a new device is provided, configured to execute the method steps according to any one of the above mentioned method steps, the device comprising:• at least one processor configured to execute the method steps according to any one of the above-mentioned method steps; and• at least one display -device, configured to display to a user data related to one any of the above-mentioned embodiments.

[0026] According to some embodiments, the device further comprising at least one inputdevice, configured to collect and interpret data, according to one any of the above- mentioned embodiments.

[0027] According to some embodiments, the device further comprising an apparatus selected from: a computer, a smart phone, a tablet and any combination thereof.

[0028] According to some embodiments, the device further comprising at least one of data storage, an input device, a computer mouse; and any combination thereof.BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The subject matter regarded as the invention is particularly pointed out and distinctly claimed in the concluding portion of the specification. The invention, however, both as to organization and method of operation, together with objects, features, and advantages thereof, may best be understood by reference to the following detailed description when read with the accompanying drawings in which:

[0030] Figs. 1A and IB schematically illustrate an example for a protein staining process (Fig. 1 A), and its resulted multiplexed-image comprising several single-color images (Fig. IB), according to some embodiments of the invention;

[0031] Fig. 2A, schematically illustrates simple mapping, according to some embodiments of the invention;

[0032] Fig. 2B, schematically illustrates combinatorial mapping, according to some embodiments of the invention;

[0033] Fig. 3 schematically illustrates reconstruction method steps, according to some embodiments of the invention;

[0034] Fig. 4 schematically illustrates an overall procedure scheme, according to some embodiments of the invention;

[0035] Fig. 5A schematically illustrates a training scheme for the decompression step, according to some embodiments of the invention;

[0036] Fig. 5B schematically illustrates an optimization phase, after the decompression phase, according to some embodiments of the invention;

[0037] Fig. 5C schematically demonstrates a simulation process for providing a multiplexed -image, as input for training the decompression neural network (NN), according to some embodiments of the invention;

[0038] Figs. 6A and 6B schematically illustrate an experiment (TEST 1) to validate the approach, by measuring both a multiplexed staining table configured for three channels (Fig.6A) and the six individual proteins (Fig. 6B), according to some embodiments of the invention;

[0039] Fig. 7A schematically illustrates six antibodies labeled with three different metal isotopes (shown in red, green and blue), according to some embodiments of the invention;

[0040] Fig. 7B schematically demonstrates the multi-metal polymer conjugation: where unlabeled polymers are loaded with one or multiple metal isotopes and conjugated to a reduced antibody, according to some embodiments of the invention;

[0041] Figs 8A, 8B, 8C, 8D, 8E, 8F and 8G demonstrate an evaluated compression via overlaying the ground truth individual images with the combinatorically multiplexed- images, according to some embodiments of the invention;

[0042] Fig. 9 demonstrates recovery results (TEST 1), of 17 multiplexed-images (test images) of six proteins, compressed to three channels and measured by mass-based imaging (according to Map in Fig. 6A), according to some embodiments of the invention;

[0043] Figs 10A, 10B, 10C, 10D, 10E, and 10F demonstrate recovery results vs. ground truth images (TEST 1), according to some embodiments of the invention.

[0044] Figs. 11 A and 11B schematically illustrate an experiment (TEST 2) to further validate the approach, by measuring both a multiplexed staining table configured for three channels (Fig. 11 A) and the seven individual proteins using fluorescent microscopy (Fig.1 IB), according to some embodiments of the invention;

[0045] Fig. 12 demonstrates recovery results (TEST 2), of 32 multiplexed-images (test images) of seven proteins, compressed to three channels (according to map in Fig. 11 A), according to some embodiments of the invention;

[0046] Figs. 13A, 13B, 13C, 13D, 13E and 13F demonstrate recovery results vs. ground truth images (TEST 2), according to some embodiments of the invention;

[0047] Figs. 14A, 14B, 14C, 14D, and 14E demonstrate examples of properties of protein images that can serve as constraints during signal reconstruction;

[0048] Figs. 15A, 15B, 15C, 15D, 15E, 15F, 15G, 15H, 151, 15J, 15K, and 15L demonstrate decompression results of a simulation for combinatorically-stained images, via various methods, as in the following TEST 3;

[0049] Figs. 16A, 16B, 16C, 16D, 16E, 16F, and 16G demonstrate that CombPlex accurately reconstructs compressed cancer images measured by fluorescent microscopy, as in the following TEST 4;

[0050] Figs. 17A, 17B, 17C, 17D, 17E, 17F, 17G, 17H, 171, and 17J, demonstrate that CombPlex can support large-scale compressions of 10 antibodies per channel, as in the following TEST 5;

[0051] Figs. 18A 18B, 18C, 18D, 18E, 18F, 18G, and 18H demonstrate that CombPlex accurately reconstructs compressed images measured by mass-spec imaging, as in the following TEST 6;

[0052] Figs. 19A 19B, 19C, 19D, 19E, 19F, 19G, 19H, 191, 19J, and 19K demonstrate features contributing to CombPlex performance, strengths and limitations;

[0053] Fig. 20A demonstrates an illustration of the simulation process of compressed images, according to some embodiments of the invention;

[0054] Fig. 20B demonstrates an illustration of the training process involving simulated compressed images; BCE loss is calculated by comparing the binarized masks of the singleprotein images and the binary mask predictions generated by the neural network / s, using the compressed images as input; MSE loss is calculated by comparing the single-protein images and the predictions generated by the neural network / s, using the compressed images as input, according to some embodiments of the invention; and

[0055] Fig. 20C demonstrates an illustration of the inference / application procedure; compressed images are provided as input to the masking and to the value reconstruction networks; the prediction from both is intersected to produce the final prediction, according to some embodiments of the invention.

[0056] It will be appreciated that for simplicity and clarity of illustration, elements shown in the figures have not necessarily been drawn to scale. For example, the dimensions of some of the elements may be exaggerated relative to other elements for clarity. Further, where considered appropriate, reference numerals may be repeated among the figures to indicate corresponding or analogous elements.DETAILED DESCRIPTION OF THE PRESENT INVENTION

[0057] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, it will be understood by those skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, and components have not been described in detail so as not to obscure the present invention.

[0058] As used herein, the term “about” refers to ± 10% of the related value, ± 5%, ± 1%, and any combination thereof.

[0059] Immunohistochemistry, whereby antibodies are used to quantify the single-cell spatial distribution of proteins in tissue sections, is a pillar of pathology, informing patient diagnosis, clinical decision making as well as discovery studies. Traditional chromogenic immunohistochemistry is limited to visualizing one or two proteins at a time, whereas standard immunofluorescence methods are limited to three to five proteins due to overlapping emission spectra. However, to uncover complex cellular structures, interactions, and functions, there is a need to simultaneously visualize many proteins in situ.

[0060] While there are many multiplexed imaging methods, differing in the antibody tagging and detection strategies used, they can be broadly grouped into two classes. Massbased approaches use antibodies conjugated to heavy metals, which can be resolved using a mass spectrometer. These methods require expensive instrumentation, and their multiplexing capability is limited to a few dozen proteins by the number of available metals for conjugation and detection. Cyclic fluorescence-based approaches increase the number of proteins measured by performing consecutive imaging cycles, each illuminating two or three proteins, followed by removal of the signal by chemical or enzymatic reactions. These methods are also effectively limited since the number of imaging cycles increases linearly with the number of proteins visualized, which in turn can result in tissue deterioration and prohibitively long imaging times. To date, there remains a need to increase the multiplexing capacity in imaging for any instrument. One common characteristic of existing multiplexed imaging approaches is the use of a separate channel (e.g., fluorescent color, mass tag etc.) for each individual protein, inherently limiting the scalability of these methods (Fig. 2A). Combinatorial approaches, wherebyeach target is encoded by one or more channels (e.g., a sequence of colors over multiple rounds), can dramatically increase throughput (Fig. 2B). Such methods effectively generate a target-specific barcode and have been successfully used to measure thousands of mRNAs. One assumption that lies at the basis of these combinatorial approaches is the ability to optically resolve individual molecules (‘spot counting’) to identify the underlying barcodes. This assumption can be upheld for the mRNAs of many genes if using a high enough magnification (400-1000X). If this assumption is invalidated and barcodes spatially overlap, it becomes challenging to differentiate between different genes. Since proteins are on average 10,000-fold more abundant than mRNAs, the spot counting assumption is invalidated for most proteins under standard magnifications, and therefore to date combinatorial staining methods have not been applied to imaging proteins.

[0061] Mathematically, resolving the spatial distribution of P proteins from combinatorial staining using C channels (P » C) while allowing spatial overlap is underdetermined, meaning that it has an infinite number of possible solutions (see details in the results). Work in signal processing has shown that solutions for underdetermined linear systems can be recovered if the underlying data is structured. For example, in compressed sensing, the assumption that the underlying data is sparse is often used to constrain the infinite plausible solution space. Previous work has leveraged the structure of gene expression modules to apply compressed sensing to increase the throughput of both single-cell RNA sequencing and single molecule mRNA imaging. However, one limitation of compressed sensing is that it explicitly formulates the constraints on the solution space, and as such is highly sensitive to the predefined structure of the data. Thus, these methods require elaborate setups such as performing single cell RNA sequencing (scRNASeq) prior to imaging, which is expensive and often incompatible with the availability of clinical material. While similar approaches have been suggested for proteins in effect they have never materialized beyond simulations of cells in suspension, presumably due to the inherent limitations of the method.

[0062] According to some embodiments, and as elaborated in the following, a new combinatorial multiplexing (CombPlex) method is presented, providing an experimental foundation for combinatorial staining of proteins in tissues coupled with an algorithmic framework for accurately decompressing the combined signal back to individual proteins.In CombPlex, each protein can be measured in several channels, and each acquired image captures a combination of signals from several proteins. CombPlex utilizes a deep neural network / s to learn meaningful priors on the underlying protein images and reconstruct individual protein images from combinatorically-compressed measurements.

[0063] In the following RESULTS section, experimental protocols are demonstrated to perform combinatorial staining and show that multiple proteins can be accurately multiplexed on a single channel, without loss of signal. Accurate (Fl>0.97, R>0.93) reconstruction is demonstrated, when staining for seven (7) proteins using three (3) fluorescent channels, indicating that the approach is compatible with conventional fluorescent microscopes that are widely used in research and in the clinic. The method is further tested in cyclic fluorescence (CODEX) and mass-based imaging (MIBI), using up to 5 channels to accurately measure 22 proteins, showing that the method is applicable to a wide variety of platforms and tissue specimens. Using simulations, it is further demonstrated that the neural network (NN) learns several degrees of structure in the data including the compression scheme, protein co-expression and spatial expression patterns. Altogether, it is demonstrated the combinatorial staining coupled with deep-learning decompression can serve to escalate the number of proteins measured using any imaging modality, without the need for specialized instrumentation. Coupling CombPlex with instruments for high-dimensional imaging paves the way to imaging hundreds of proteins at single-cell resolution in intact tissue sections.

[0064] In the digital realm, there can be any number of conventional primary colors making up a multiplexed-image; a channel, in this case, is extended to be the grayscale image, based on any such conventional primary color. A channel is a conventional term used to refer to a certain component (e.g., color) of a multiplexed-image. The concept of channels can be extended beyond the visible spectrum in multispectral and hyperspectral imaging. In that context, each channel corresponds to a range of wavelengths and contains spectroscopic information. The channels can have multiple widths and ranges. Accordingly, channels may be separated according to different spectrums in distinct imaging approaches, for example, distinct mass to charge ratio (m / z) in Mass Spectrometry Imaging (MSI).

[0065] A skilled artisan would appreciate that the terms “image” and “channel” and “channel-image” and “compressed-image”, used herein, may in some embodiments and / or examples be used interchangeably all counted herein via variable “C”. According to some embodiments the term “compressed-image” is an image and / or channel generated by several targets.

[0066] A skilled artisan would appreciate that the term “multiplexed-image”, used herein, may in some embodiments, refer to an entity combined from a number (C) of images of the same physical specimen. Each channel in the multiplexed-image is a grayscale image depicting measurement of a different molecular property. For a non-limiting example, a multiplexed-image of a tissue specimen can comprise a set of C grayscale images, each measuring the location and abundance of at least one protein in the tissue.

[0067] A skilled artisan would appreciate that the term “measurement-channel”, used herein, refers to the measurement of a predetermined channel; for a non-limiting example the measurement of predetermined color.

[0068] A skilled artisan would appreciate that the term “target”, as used herein, may in some embodiments, refer to an element- or substance- or component- of interest. Nonlimiting examples include a specific “molecule”, or specific a “biological molecule”, such as a specific “protein” or “mRNA”, in a tissue sample. The terms “target”, “molecule”, “biological molecule”, and “protein” may in some embodiments and / or examples be used interchangeably.

[0069] A skilled artisan would appreciate that the term “desired output”, as used herein, may in some embodiments refer to a desired output (and / or output tensor), given its associated input (and / or input tensor); for a non-limiting example, at a process of supervised-training of a neural network (NN), demonstrated in Fig. 3, step (350), tensor b is provided as a desired output, for the training process (350), where y is provided as the associated training input.

[0070] A skilled artisan would appreciate that “Fl score” refers to a measure of the harmonic mean of precision and recall. Commonly used as an evaluation metric in binary and multi-class classification and LLM evaluation, the Fl score integrates precision and recall into a single metric to gain a better understanding of model performance.

[0071] A skilled artisan would appreciate that “R score” refers to the most common correlation coefficient, which indicates the strength of the linear relationship between two variables. The value of R has a range of -1 to 1 (0 indicates no relationship). Values of R closer to -1 or 1 indicate a stronger relationship and values closer to 0 indicate a weaker relationship.

[0072] According to some embodiments and as schematically demonstrated in Figs. 1A and IB, the present invention is in the field of biological imaging (100) of samples, such as tissues (101), where P reporters (121,122) (e.g., two antibodies) are used to infer the location of different P targets or molecules (111,112) (e.g., two proteins) in the tissue sample. According to some embodiments, and as further demonstrated in Fig. 1A and Table 1, these reporters can be:• antibodies, which are each directed to a specific protein; or• oligonucleotides, which are each directed to a specific mRNA.

[0073] According to some embodiments, these reporters are each chemically attached or conjugated to at least one associated detectable moiety (a measurable marker) (131,132) that can be detected using various measurement systems. According to some embodiments, this moiety is at least one selected from:• chromophores, which can be detected using a light microscope;• fluorescent dyes, which can be detected using a fluorescent microscope; and• heavy-metal isotopes, which can be detected using a mass-spectrometer.

[0074] Table 1: non limiting examples for molecules, reporters and their measurable markers.

[0075] According to some embodiments, in the imaging process (100), different channels are acquired as separate, co-registered grayscale images (211,212). In this example the size of each of the images (211,212) is of 400 / 7 / 7? x 400« / ??. These channel-images are combined to generate a multiplexed-image (200). In other words, multiplexed-images (200) are comprised from a variable number (C) of grayscale images (211,212), each measured using a different measurement-channel.

[0076] According to some embodiments of the invention, multiplexed imaging enables the visualization of any tissue, including but not limited to: tumor microenvironment, pathological markers, spatial interactions between proteins, cancer, neurodegenerative disorders, autoimmune disorders, inflammation etc. While the different state-of-the-art technologies, such as MIBI- TOF, IMC and CODEX allow for imaging dozens of targets in situ, most labs use three-channel fluorescent microscopy.

[0077] According to some embodiments, multiplexed imaging methods are configured for antibody tagging and detection strategies. Multiplexed imaging methods can be broadly grouped into two classes: fluorescence-based approaches (CODEX) and mass-based approaches, such as Imaging Mass Cytometry (IMC), and Multiplexed Ion Beam Imaging by Time-Of-Flight (MIBI-TOF) using antibodies conjugated to heavy metals, which can be detected with a mass spectrometer. These methods require expensive instrumentation, and their multiplexing capabilities are limited to a few dozen proteins, by the number of available metals for conjugation and detection.

[0078] According to some embodiments, cyclic fluorescence-based approaches increase the number of proteins measured by performing consecutive imaging cycles, each illuminating two or three proteins, followed by removal of the signal. Many such methodologies exist, differing in how the signal is removed, include antibody stripping, enzymatic reactions, chemical fluorophore bleaching and washing of secondary oligos that bind to probes chemically attached to the primary antibodies. These methods require repetitive and lengthy steps and thus are also effectively limited to a few dozen proteins since the number of imaging cycles increases linearly with the number of proteins visualized.

[0079] According to some embodiments, one common theme of multiplexed imaging approaches is the use of a separate channel (e.g., separate fluorescent color, separate masstag etc.) for each individual target, as demonstrated for example in Fig. 2A. This inherently limits the scalability of these methods, because each additional target requires adding a channel. For the mass-spectrometry-based methods, this entails identifying a new isotope that can be efficiently conjugated to antibodies and sensitively detected by the systems. For the cyclic-fluorescence-based methods, this entails adding a staining and removal cycle, and each such cycle potentially deteriorates the tissue and compromises further staining.

[0080] According to some embodiments of the invention, combinatorial approaches, whereby each target e.g., molecule, RNA or protein) is encoded by one or several channels e.g., a sequence of colors over multiple rounds), can dramatically increase throughput, as demonstrated for example in Fig. 2B. As demonstrated for example, three channels can be used to measure three targets (Fig. 2A), in a linear scheme, and seven targets, in a combinatorial scheme (Fig. 2B). According to some embodiments, in the example of mass-based methods, the combinatorial staining could be achieved by conjugating several antibodies to each isotope; in cyclic fluorescence-based methods, this could be done by staining several proteins using one color in each round, and the same protein in several rounds.

[0081] According to some embodiments, one assumption that currently lies at the basis of these combinatorial approaches, is the ability to optically resolve individual molecules (‘spot counting’). This assumption can be upheld for the mRNAs of a good number of genes if using a high enough magnification (* 100), so that individual RNA molecules appear as bright, well-separated spots. If this assumption is invalidated, barcodes overlap, and it becomes infeasible to differentiate between different genes. This problem limits throughput in two important ways: A) Imaging at high magnification is slow and limits acquisition of large cohorts; B) It restricts the number of genes measured, even at high magnification (x 100), as optical crowding scales with the number of genes, and prohibits measurements of highly abundant mRNAs. Since proteins are on average 10,000-fold more abundant than mRNAs, the spot counting assumption is invalidated for most proteins under standard magnifications.

[0082] According to some embodiments of the invention, a combinatorial staining method is provided, coupled with a new algorithmic framework, configured to increasechannel combinatorial multiplexing (Combplex), according to some embodiments, even in cases where the spot counting assumption is invalidated.

[0083] According to some embodiments, Combplex is configured to image up to D = 2C- 1 proteins, in one round of imaging with C channels. According to some embodiments, biological and practical constraints may decrease this number.

[0084] According to some embodiments, Combplex is configured for the escalation of multiplexed imaging, for any imaging platform.

[0085] According to some embodiments, a combinatoric labeling method of P targets is provided, using at least C different labeling-elements which are configured to be detected by C different channels, where (P > C), which allows for spatial overlap between the P targets; this method is known to be underdefined in the field of mathematics and signal processing. According to some embodiments, the labeling-elements comprise selected measurable markers, as listed in Table 1.

[0086] According to some embodiments, the multiplexed-image compression can be written as: y = A * x where:- y is a multiplexed-image with C channels (dimensions: width x height x C),- A defines the compression matrix of size Cand- x is the set of P single-protein images;- * is the convolution operator.

[0087] According to some embodiments, given a multiplexed-image y, of C channelimages and a pre-defined compression mapping-matrix^, one would like to find a solution for x. Altogether this defines a set of C linear equations with P variables, where P > C, and accordingly has an infinite number of solutions.

[0088] According to some embodiments, in the fields of signal processing and electrical engineering, solutions for underdetermined linear systems can be approximated, if the underlying data is structured, using approaches, like for example, compressed sensing. A limitation of compressed sensing is that it is highly sensitive to the predefined structure of the data. Thus, these methods require elaborate setups, such as for the example ofRNA, performing Single-Cell RNA sequencing, prior to imaging, which is expensive and often incompatible with availability of clinical material.

[0089] According to some embodiments, given a set of compressed images y and a predefined compression matrix A, it is a goal to recover the original protein images, i.e., solve for x (Fig. 4 442-448). Altogether this defines a set of C linear equations with P variables for each pixel. Since P > C, this problem is underdetermined and has a degenerate solution space with an infinite number of solutions. According to some embodiments, although the mathematical problem of compressing protein images is underdetermined, additional properties of the images are leveraged to constrain the solution space and recover the true x. According to some embodiments, images of proteins are often sparse, with only a minority of pixels containing a signal for any particular protein, as demonstrated in Fig. 14A for CD8 that exhibits sparse staining in tissues. According to some embodiments, protein expression is continuous in space. If a pixel stains positive for a particular protein, it is likely that nearby pixels will also stain positive for that same protein , as demonstrated in Fig. 14B for Pan-Keratin that exhibit continuous signal. According to some embodiments, protein expression is structured, with particular proteins tending to cooccur together or be mutually exclusive. For example, pixels that stain positive for CD8, expressed on T cells, will also stain positively for CD45, expressed on all immune cells, but not vice-versa, as demonstrated in Fig. 14C where CD8 (blue) is co-expressed with CD45 (red). According to some embodiments, different proteins vary in their staining patterns (e.g., membranous vs. nuclear) and their overall abundance (strong vs. weak staining), as demonstrated in Fig. 14D where Pan- Keratin (red) exhibits cytoplasmic staining while Ki-67 exhibits nuclear staining (green). The unique staining pattern of proteins enables to recognize a variety of widely used antibodies, even without labels. For example, visually separating the signals of Ki-67, a nuclear protein, from keratins, which are expressed in the cytoplasm, even when both are stained together in a single image / channel, as demonstrated Fig. 14E, that has similar layout as Fig. 14D; although both proteins are shown in grayscale, they can be differentiated by their unique staining patterns.

[0090] Accordingly, according to some embodiments, an algorithm is trained to recognize a variety of widely used antibodies. According to some embodiments it ishypothesized that the various layers of structure in biological images could serve as priors to accurately reconstruct the images of single proteins from combinatorically-compressed images, even though equation y = A * x is underdetermined.

[0091] According to some embodiments, combinatorial multiplexing (Combplex) is provided as an experimental foundation for combinatorial staining or labeling of targets in tissues. According to some embodiments, it is further coupled with an algorithmic framework, for accurately decompressing the multiplexed-images, having combined signal, back to individual-images. According to such embodiments, each molecule is uniquely labeled by one or several labeling-elements, out of C labeling-elements, wherein at least one molecule is labeled by at least two labeling-elements. Consequently, after the tissue is imaged using C channels, each molecule is measured via one or several channels, thus at least one molecule can be measured via at least two channels, and at least one channel may include a measurement of at least two molecules.

[0092] According to some embodiments, an artificial intelligence pipeline is provided, configured to decompress the C images, back to their individual signals x. This process is configured to exponentially increase the number P of targets (e.g., molecules) that can be imaged using a fixed number of C channels.

[0093] Fig. 2B demonstrates, a non-limiting example, for a case where C = 3 channels (R,G,B), are selected to label P= rl targets (e.g., molecules) to be measured. An example case is provided in the following, as proof of concept experiments. According to some embodiments, the number of C = 3 measuring channels can be higher, to apply higher dimensions.

[0094] According to some embodiments, and as demonstrated in Figs. 3, 4 and 5A-5C an image reconstruction method (300) is provided, comprising: a decompression phase (370,440), configured to decompress a multiplexed-image (y) of C compressed-images (431- 433) to P individual-images (442-448 or 452-458), where P > C. In Figs. 3, 4 and 5A-5C the size of each of the images (431-433,442-448,452-458,531-533,542-548, 562-568,581- 583) is about 400 / 7 / 7? x 400« / / ?.

[0095] According to some embodiments, and as demonstrated in Figs. 3 and 5B, the method further comprises an optional optimization phase (380), configured to recover and / or improve the solution provided by the decompression phase (370).

[0096] According to some embodiments, and as demonstrated in Figs. 3 and 5C, the method further comprises a simulation phase (331), configured to simulate and provide at least one multiplexed-image (580) of C compressed-images (581,582,583), generated from P ground-truth individual -images (542-548) of the P labeled molecules. The simulation phase is provided for the purpose of training the decompression NN (441), at the training phase (350).

[0097] A skilled artisan would appreciate that the term “ground truth individual-images” as used herein, may in some embodiments, refer to P individual-images of the P labeled molecules / proteins / targets that are known to be real and / or true, provided by direct observation and measurement (i.e., empirical evidence) as opposed to images provided by inference.

[0098] According to some embodiments, and as shown in Fig. 4, at the decompression phase, a trained Masking NN Me (441) is provided, configured to reconstruct the C compressed-images (431-433) and output a tensor B having a set of P binary -images (442- 448), as a binary support for x (the P individual molecule images (452-458), shown in Fig 5B). The input to this NN is tensor y (430), having C (for example C = 3) compressed- images (431-433), and the output is tensor B, having a set of P (for example P = 7) binaryimages (442-448), with binary values; each binary-image is for a different molecule. According to some embodiments, for the binary-images: a value of “7” in a pixel of a binary -image p (where p = 1,2, ...,P) indicates that this pixel is expected to contain a signal of target molecule p.

[0099] Training the Masking Network for the decompression phase:

[0100] According to some embodiments, given a multiplexed-image y G / ?Hx KxC, which is a compressed measurement of an underlying signal x G RH W P(where C< P), the goal is to reconstruct x.• H - number of height pixels;• W - number of width pixels;• C- number of measured channels; and• P - number of searched proteins / antibodies / mRNAs / molecules or any other searched target.

[0101] According to some embodiments, the goal of the NN is to classify each pixel, in the C (for example C = 3) compressed-images, as positive or negative for each of the (for example P = 7) classes, where a class refers to an individual target molecule.

[0102] According to some embodiments, for this semantic training task, a supervised learning model is utilised; according to some embodiments, using at least one convolutional NN (a deep learning neural network (DLNN)). According to some embodiments, an unsupervised learning is utilised; according to some embodiments, a transformer deep learning model is utilised, which adopts the mechanism of selfattention, by differentially weighting the significance of each part of the input data.

[0103] According to some embodiments, the decompression phase comprises training of the Masking NN, Me, to reconstruct and output the binary support B of x. Formally,

[0104] According to some embodiments and as shown in Fig. 5A, for the training phase (350), a supervised learning approach is used, where the NN is configured to decompress the multiplexed-image (530,580) y into binary -masks B of individual molecules; i.e., B is the NN output tensor having a set of P individual binary-images (562-568)). Consequently, training of the network requires a collection of C compressed-images (531-533 and / or 581-583) (per each training example), where the underlying staining of the P individual targets (e.g., of the searched molecules) is known (542-548) (ground truth individualimages), and accordingly it’s binary values b (540).

[0105] According to some embodiments and as demonstrated in Figs. 3 and 5A, the NN training (350) is provided via at least one multiplexed-image (530,580) provided as a training-input to the NN. According to some embodiments, the training-input is provided by physically acquiring (332) (e.g., imaging) the multiplexed-image (530). According to some embodiments, training-input is provided by simulating (331) the multiplexed-image (580), based on P ground-truth images (542-548) of the P target molecules.

[0106] Objective Function

[0107] According to some embodiments, the task of a supervised NN Me is to reconstruct a support B of x, therefore a binary classification loss is employed by:L = -5(x) log(Me(y)) - (1 - 5(x)) log(l -Me )

[0108] According to some embodiments, the implemented framework is PyTorch. According to some embodiments, the full objective function is optimized with the MADGRAD optimizer, with a constant learning rate of 2.5 - 10-3, weight decay of 0.01 and momentum 0.9.

[0109] According to some embodiments, the convolutional NN is selected from the known: U-Net architecture, binary cross-entropy loss, and ADAM optimizer and any combination thereof.

[0110] Masking Network Architecture

[0111] According to some embodiments, the network Me is based on the U-Net architecture, with a 7-layer encoder and a symmetrical decoder. According to some embodiments, all layers comprise 3x 3 Convolutional layers, followed by BatchNorm, and LeakyReLU activation. According to some embodiments, intermediate channels dimension is 128. According to some embodiments, in each level of the encoder, an additional U1 Convolutional layer is added and concatenated to the output features to the corresponding level of the decoder. According to some embodiments, a 1 x 1 Convolutional layer is added, followed by Sigmoid activation to get the final probabilities.

[0112] According to some embodiments, although the NN is trained in a supervised manner, training data does not require to physically acquire the multiplexed-images. In the following data simulation section, a new simulation method is configured to circumvent the need for acquiring multiplexed-images, for the training phase.

[0113] According to some embodiments and as demonstrated in Fig. 5C, a data simulation method (331) is provided, for supplying at least one multiplexed-image y, as training-input / s, configured to train the NN.

[0114] According to some embodiments, for the case of unsupervised training, a straightforward way to obtain the training input would be to acquire it , / .< ., to image the tissue section and acquire a multiplexed-image y.

[0115] According to some embodiments, for the case of supervised training, a straightforward way to obtain the training input (y) and its desired-output data (Z>) would be to acquire both (332,320), / .< ., to image (320) the same tissue section per each of the individual molecules, via -channels (for the b training desired-output) and to image the same tissue section (332) via C-channels of the multiplexed-image y, as a corresponding training input.

[0116] However, generating such data is tedious and time-consuming, especially if one is interested in training the NN for various staining / labeling combinations (meaning plurality of mapping A matrices).

[0117] According to some embodiments, to overcome this challenge, for the purpose of training the NN, the multiplexed training image / s (y) (580) are simulated from ground truth individual images of the P molecules (542-548).

[0118] According to some embodiments, the simulation process is optimized and configured to generate data that resembles actual C channel-images.

[0119] According to some embodiments, the simulation process (generating the training multiplexed-input y (580)) can be initiated by at least one of the following sections 1 to 3:1. For all individual molecule images, apply same augmentation / s selected from: random crop, random flip, random rotation and any combination thereof;2. For all individual molecule images, apply same average pooling, to smooth their signals; and3. Apply Gaussian noise on the known compression matrix A .

[0120] Then, after the optional initiation according to any one of sections 1, 2, and / or 3, the stimulation process comprises: the step of section 4 and optionally section 5:4. convolve matrix A (optionally noised, as in section 3) with the signals of x (optionally transformed, as in sections 1 and / or 2); and optionally5. apply Poisson sampling pixel -wise on the smooth compressed-images of section4, and get the final multiplexed-image y, as input for the training phase.

[0121] According to some embodiments, this simulation process is configured to enrich the training data with augmentations that are relevant to the domain measurement systems. According to some embodiments, this simulation-based training scheme increases the modularity of the approach. According to some embodiments, given the training data of P individual images that were measured individually, this simulation can accommodate a variety of combinatorial compression schemes, on all channels and / or a subset of them. According to some embodiments, the main advantage of the simulated training inputs y, is the ability to simulate many mapping combinations A, and to learn which combination would be best for the specific molecules and / or their combination, in any tissue.

[0122] An example is provided for such embodiments, where the network is trained using simulated data of the compression process. Given a training set of images [%, a training input is createdthe number of training examples.

[0123] According to some embodiments, assuming the compression can be described by the following process: y = Ps[A * h(x)]Where:• Ps(-) denotes a Poisson process, and is applied independently pixel -wise,• h(-) is an average filter applied on the spatial dimensions, and• A G RCxPis a known linear compression matrix.

[0124] According to some embodiments, the dataset is further enhanced by applying at least one augmentation to x before calculating y = Ps[A * h(X)] , selected from:• random spatial square crops of size 256 pixels;• random horizontal-flipping is applied with probability of 0.5;• random axis-aligned rotation is applied with probability of 0.5; and• any combination thereof.

[0125] Post-Decompression, Optimization Phase

[0126] According to some embodiments, after the decompression Phase, an optimization method is provided, configured to recover the solution for output tensor x, given the NNoutput binary support tensor B. In this phase, the precise intensity value of each target in each pixel is recovered, using B as a constraint.

[0127] According to some embodiments, the optimization phase comprises solving a constrained least-square problem, to recover x\B(x). This allows to significantly simplify the data-driven learning task, while still achieving sufficient constraints to accurately recover x.

[0128] According to some embodiments, this optimization is provided by minimizing the following reconstruction term:where:- Me is the masking network;- A is the labeling compression matrix;- y is the multiplexed-image data;- co is the optimal solution for X; and- O is a symmetric tensor product.

[0129] According to some embodiments, a random initialization is provided, and a gradient descent is iteratively used to find Xpred= M0(y) O the reconstructed images X, derived from the compressed-images ofy.

[0130] According to some embodiments, Fig. 3 schematically illustrates an overall procedure scheme (300) , according to some embodiments of the invention.

[0131] According to some embodiments the method (300) is provided for decompressing a multiplexed-image (y), of C different channels (431-433), acquired to identify P molecules in a tissue sample (101), where the molecules are combinatorically labeled (A) by at least C different labeling-elements which are configured to be detected by the C different channels, and where P > C. The method comprising processor (390) implemented step of applying (370) the multiplexed-image (y) as an input to a trained NN (441); the NN is configured to output a tensor B, having P binary-images (442-448) each having (I J) binary -pixels, each binary -image p per a different molecule p of the P molecules (where p = 1,2, ...,Py, wherein a pixel (i,j; i = 1,2, j = 1,2, in a binary-image p having a value of one (7) indicates that pixel (zj) is expected to contain a signal of its associated molecule p.

[0132] According to some embodiments, the NN is supervised NN. According to some embodiments, the supervised NN is a convolutional neural network (CNN).

[0133] According to some embodiments, the method (300) further comprising a step of applying an optional optimization algorithm (380) to the NN’s output tensor B (442-448). The optimization algorithm is configured to provide each of the pixels (zj) of a binary-image p with an intensity value; wherein the intensity value indicates the associated molecule p in pixel (i,j).

[0134] According to some embodiments, the method (300) further comprising a step of combinatorically labeling (310) the P molecules, via the C labeling-elements, wherein each molecule is uniquely labeled via one or several labeling-elements, and wherein at least one molecule is labeled via at least two labeling-elements.

[0135] According to some embodiments, labeling-elements comprise: fluorescent dyes, chromophores, heavy-metal isotopes, chemiluminescent substances, radioactive substances, fluorescent proteins, nanoparticles, any detectable moiety, and any combination thereof.

[0136] According to some embodiments, the method (300) further comprising acquiring (360) the multiplexed-image (y), of C channels (431-433), of the P labeled molecules, configured to be applied as the input to the trained NN (441) at decompression step (370).

[0137] According to some embodiments, the method (300) further comprising unsupervised learning of the NN (not shown).

[0138] According to some embodiments, the method (300) further comprising supervised training (350) of the NN, via:• at least one training multiplexed-image (530,580), to be applied as a training inputimage to the NN; According to some embodiments, the at least one training multiplexed-image (530) can be acquired, via C measured channels (531-533) and / or the at least one training multiplexed-image (580) can be simulated via C simulated channels (581-583); and a known desired-tensor b (540), of the training multiplexed-image, having P binaryimages with binary-pixels, to be applied as a training’s desired-output for the NN.

[0139] According to some embodiments, the method (300) further comprising acquiring (332), by means of measuring, the at least one training multiplexed-image (530) via C measured channels (531-533).

[0140] According to some embodiments, the method (300) further comprising:• acquiring (320) P ground truth individual-images (x) (542-548), of the P separately- labeled molecules, wherein each individual-image p is configured to depict a single molecule p of the P molecules; and• converting (340) the P individual -images to P binary -images having (I J) binarypixels, configured to be applied as the known desired-tensor b.

[0141] According to some embodiments, the method (300) further comprising simulating (331) the at least one training multiplexed-image (580), from the P individual-images (x) (542-548), based on the combinatorial labeling design (matrix A). According to some embodiments, the simulating further comprising convolving matrix A with the P individualimages (x).

[0142] According to some embodiments, a device (390) is provided configured to execute the method steps according to any one of the above-mentioned method steps, comprising:• at least one processor (391) configured to execute the method steps according to according to any one of the above-mentioned method steps;• at least one display-device (392), configured to display data to a user, according to one any of the above-mentioned embodiments.

[0143] According to some embodiments, the device further comprises at least one inputdevice (393), for a non-limiting example an imaging device (395), configured to collect and interpret data, according to any one of the above-mentioned method steps.

[0144] According to some embodiments, the device comprises an apparatus selected from: a computer, a smart phone, a tablet and any combination thereof.

[0145] According to some embodiments, the device further comprises at least one of: data storage (394), configured to store, according to any one of the above-mentioned method steps; an input device; a speaker device; a microphone device; a computer mouse; and any combination thereof.

[0146] According to some embodiments, a method of detecting targets in a sample is provided.

[0147] According to some embodiments, the method of detecting targets in a sample comprising: a) combinatorically labeling P targets in a sample with at least C different labelingelements which are configured to be detected by at least one of C different channels, wherein C < P b) imaging the sample to acquire at least one multiplexed-image of the sample; and c) decompressing the multiplexed-image, according to the method and / or device of any one of the above mentioned embodiments, to extract P different images each of a different target of the P targets.

[0148] According to some embodiments, the targets comprise molecules. According to some embodiments, the targets comprise biological molecules. According to some embodiments, the targets comprise biomarkers.

[0149] In some embodiments, at least two targets are imaged or detected. In some embodiments, least two, at least three, at least four, at least five, at least six, at least seven, at least eight, at least nine, at least 10, at least 20, at least 30, at least 40, at least 50, at least 60, at least 70, at least 80, at least 90, or at least 100 targets are imaged or detected.

[0150] According to some embodiments, the method of imaging biological molecules in a sample may be used in analytical, diagnostic or prognostic applications. According to some embodiments, the step of labeling one or more biological molecules in a sample with one or more labeling-elements comprises analyte detection, histochemistry, immunohistochemistry, immunofluorescence, color development in situ hybridization or fluorescence in situ hybridization (FISH). According to some embodiments, the step of labeling one or more biological molecules in a sample with one or more labeling-elements comprises histochemistry. According to some embodiments, the step of labeling one or more biological molecules in a sample with one or more labeling-elements comprises immunostaining. According to some embodiments, the step of labeling one or more biological molecules in a sample with one or more labeling-elements comprises immunohistochemistry. According to some embodiments, the step of labeling one or morebiological molecules in a sample with one or more labeling-elements comprises immunoassay. According to some embodiments, the step of labeling one or more biological molecules in a sample with one or more labeling-elements comprises immunofluorescence. According to some embodiments, the step of labeling one or more biological molecules in a sample with one or more labeling-elements comprises immunoblotting. According to some embodiments, the step of labeling one or more biological molecules in a sample with one or more labeling-elements comprises immunoassays. According to some embodiments, the step of labeling one or more biological molecules in a sample with one or more labelingelements comprises enzyme linked immunosorbent assay (ELISA). According to some embodiments, the step of labeling one or more biological molecules in a sample with one or more labeling-elements comprises radioimmunoassay (RIA).

[0151] According to some embodiments, the one or more biological molecules are selected from the group consisting of proteins, peptides, polypeptides, antibodies, antigens, enzymes, enzyme substrates, ligands, receptors, polynucleotides, RNA, DNA, metabolites, polysaccharides, lectins, sugars, glycoproteins, lipids, fats, vitamins, hormones, and cellular compartments.

[0152] As used herein, the term “biological molecule” may encompass a component of a biological sample or cell. According to some embodiments, the biological molecule comprises a peptide. According to some embodiments, the biological molecule comprises a peptide. According to some embodiments, the biological molecule comprises a protein. According to some embodiments, the biological molecule comprises an antibody. According to some embodiments, the biological molecule comprises a nucleic acid. According to some embodiments, the biological molecule comprises a polynucleotide. According to some embodiments, the biological molecule comprises a DNA molecule. According to some embodiments, the biological molecule comprises an RNA molecule. According to some embodiments, the RNA molecule comprises an mRNA molecule. According to some embodiments, the biological molecule comprises a polysaccharide. According to some embodiments, the biological molecule comprises a lectin. According to some embodiments, the biological molecule comprises a sugar. According to some embodiments, the biological molecule comprises a lipid. According to some embodiments, the biological molecule comprises an enzyme. According to some embodiments, thebiological molecule comprises an enzyme substrate. According to some embodiments, the biological molecule comprises a ligand. According to some embodiments, the biological molecule comprises a receptor. According to some embodiments, the biological molecule comprises an antigen. According to some embodiments, the biological molecule comprises a metabolite. According to some embodiments, the biological molecule comprises a polysaccharide. According to some embodiments, the biological molecule comprises a glycoprotein. According to some embodiments, the biological molecule comprises a fat. According to some embodiments, the biological molecule comprises a hormone. According to some embodiments, the biological molecule comprises a vitamin. According to some embodiments, the biological molecule comprises a cellular compartment.

[0153] The term “protein” as used herein, may encompass full length proteins, protein fragments, proteins in their native state or denatured proteins.

[0154] According to some embodiments, the biological molecule comprises a biomarker. A skilled artisan would appreciate that the term “biomarker” comprises any measurable substance in an organism, whose presence is indicative of a biological state or a condition of interest. In some embodiments, the presence of a biomarker is indicative of the presence of a compound or a group of compounds of interest. In some embodiments, the concentration of a biomarker is indicative of the concentration of a compound or a group of compounds of interest.

[0155] An artisan would appreciate that the term “cellular compartment” may encompass cellular organelles, including the nucleus, nucleolus, ribosome, mitochondria, and chloroplast, and other cellular structures or sub-cellular structures such as Golgi apparatus, endoplasmic reticulum, centrioles, membranes, the cytoplasm, cytoskeleton, microtubules, cellular filaments, and cell scaffolding.

[0156] According to some embodiments, the term "lipid" may encompass fats or fat-like substances, including, esters of fatty acids and glycerol, lipoids (i.e., phospholipids, cerebrosides, waxes) and sterols (i.e., cholesterol, ergostrol). According to some embodiments, the lipid comprises lipoprotein molecules (HDL, LDL, IDL or VLDL), cholesterol and triglycerides. According to some embodiments, the term “fat” mayencompass any of the glyceryl esters of fatty acids, e.g., the monoacylglycerol, di acylglycerol and tri acylglycerol fatty acids.

[0157] According to some embodiments, the reporter comprises an antibody, oligonucleotide, or aptamer. According to some embodiments, the reporters are selected from the group consisting of antibodies, peptides, proteins, oligonucleotides, polynucleotides, polysaccharides, lipids, enzymes, enzyme substrates, ligands, receptors, antigens, haptens, and aptamers.

[0158] As used herein, the term “reporter” may encompass a molecule that can bind to one or more biological molecules (targets) in a biological sample. The reporter can specifically bind to the biological molecule. According to some embodiments, the reporter comprises one or more natural or modified peptides, proteins (e.g., antibodies, affibodies or aptamers), nucleic acids (e.g., oligonucleotides, polynucleotides, DNA, RNA or aptamers); polysaccharides (e.g., lectins, sugars), lipids, enzymes, enzyme substrates or inhibitors, ligands, receptors, antigens or haptens. According to some embodiments, the reporter comprises an antibody. According to some embodiments, the reporter comprises an oligonucleotide. According to some embodiments, the reporter comprises an aptamer.

[0159] An artisan would appreciate that the term “antibody” may encompass an immunoglobulin that is defined as complementary to a particular spatial and polar organization of another molecule by being specifically bound to that molecule. According to some embodiments, the antibody is monoclonal or polyclonal. According to some embodiments, the antibody may comprise intact immunoglobulin or a fragment thereof, for example, Fab, Fv and F (ab ') 2 or Fab ' which retain binding with affinity similar to a full- length antibody.

[0160] According to some embodiments, the labeling-elements are attached to the P targets by P reporters; and wherein the reporters are selected from the group consisting of antibodies, peptides, proteins, polynucleotides, oligonucleotides, polysaccharides, lipids, enzymes, enzyme substrates, ligands, receptors, antigens, haptens, and aptamers

[0161] According to some embodiments, the reporter comprises at least one detectable moiety. In some embodiments, the reporter comprises at least two detectable moieties. Insome embodiments, the reporter comprises at least two, at least three, at least four, at least five, at least six, at least seven, at least eight, at least nine, or at least 10 detectable moi eties.

[0162] According to some embodiments, the detectable moiety is selected from the group consisting of a fluorescent dye, heavy-metal isotope, chromophore, chemiluminescent substance, radioactive substance, nanoparticle, surface plasmon resonant particle (PRP), and resonance light scattering particle (RLS). According to some embodiments, the detectable moiety comprises a fluorescent dye. According to some embodiments, the detectable moiety comprises a heavy-metal isotope. According to some embodiments, the detectable moiety comprises a chromophore. According to some embodiments, the detectable moiety comprises a chemiluminescent substance. According to some embodiments, the detectable moiety comprises a radioactive substance. According to some embodiments, the detectable moiety comprises a nanoparticle. According to some embodiments, the detectable moiety comprises a surface plasmon resonant particle (PRP). According to some embodiments, the detectable moiety comprises a resonance light scattering particle (RLS).

[0163] According to some embodiments, the labeling-element comprises at least one detectable moiety. In some embodiments, the detectable moiety comprises a measurable marker. As used herein, the terms “detectable moiety” and “measurable marker” are used interchangeably to refer to a moiety that is associated with (e.g., conjugated to) the labelingelement and allows direct or indirect detection, and include without limitation, radioisotopes, fluorophores, chemiluminescent compounds, bioluminescent compounds, and dyes. According to some embodiments, the detectable moiety can be qualitatively detected (e.g., optically or spectrally), or it can be quantified. According to some embodiments, a plurality of detectable moieties can be attached to a labeling-element.

[0164] In some embodiments, the detectable moiety comprises dyes, optically active dyes, such as fluorescent dyes, heavy-metal isotopes, chromophores, chemiluminescent substances, radioactive substances, nanoparticles such as fluorospheres and quantum dots, rods or nano-bars, and surface plasmon resonant particles (PRPs) or resonance light scattering particles (RLSs) — particles of silver or gold that scatter light.

[0165] The term “dye” as used herein refers to a compound that emits light to produce an observable detectable signal.

[0166] The term “fluorophore” or “fluorescent dye”, as used herein, refers to a compound, chemical group, or composition that is inherently fluorescent. Fluorophores may contain substituents that alter the solubility, spectral properties or physical properties of the fluorophore. Numerous fluorophores are known to those skilled in the art and include, without limitation, coumarin, cyanine, benzofuran, a quinoline, a quinazolinone, an indole, a furan, a benzazole, a borapoly azaindacene and xanthenes including fluorescein, rhodamine and rhodol.

[0167] As used herein, the term “chromophore” means the part of a molecule in which the energy difference between the two types of molecular activations is within the visible spectrum. A chromophore can provide a molecular color that results from the absorption of certain wavelengths of visible light and the transmission or reflection of other wavelengths.

[0168] In some embodiments, the detectable moiety comprises radioisotopes (e.g., 32P, 14C, 1251, 3H, and 1311), fluorescein, rhodamine, dansyl chloride, umbelliferone, luciferase, peroxidase, alkaline phosphatase, [3-galactosidase, [3-glucosidase, horseradish peroxidase, glucoamylase, lysozyme, saccharide oxidase, microperoxidase, biotin, and ruthenium.

[0169] In some embodiments, the detectable moiety comprises Atto dyes, 4-acetamido- 4'-isothiocyanatostilbene-2,2'disulfonic acid; acridine and derivatives: acridine, acridine isothiocyanate; 5-(2'-aminoethyl)aminonaphthalene-l -sulfonic acid (EDANS); 4-amino-N- (3-vinyl sulfonyl)phenyl)naphthalimide-3,5 disulfonate; N-(4-anilino-l -naphthyl) mal eimide; anthranilamide; BODIPY; Brilliant Yellow; coumarin and derivatives; coumarin, 7-amino-4-methylcoumarin (AMC, Coumarin 120), 7-amino-4- trifluoromethylcouluarin (Coumaran 151); cyanine dyes; cyanosine; 4',6-diaminidino-2- phenylindole (DAPI); 5'5''-dibromopyrogallol-sulfonaphthalein (Bromopyrogallol Red); 7- diethylamino-3-(4'-isothiocyanatophenyl)-4-methylcoumarin; di ethylenetriamine pentaacetate; 4,4'-diisothiocyanatodihydro-stilbene-2,2'-disulfonicacid; 4,4'- diisothiocyanatostilbene-2,2'-disulfonic acid; 5-(dimethylamino)naphthalene-l-sulfonyl chloride (DNS, dansylchloride); 4-dimethylaminophenylazophenyl-4'-isothiocyanate(DABITC); eosin and derivatives; eosin, eosin isothiocyanate, erythrosin and derivatives; erythrosin B, erythrosin, isothiocyanate; ethidium; fluorescein and derivatives; 5- carboxyfluorescein (FAM), 5-(4,6-dichlorotriazin-2-yl) aminofluorescein (DTAF), 2', 7'- dimethoxy-4'5'-dichloro-6-carboxyfluorescein, fluorescein, fluorescein isothiocyanate, QFITC, (XRITC); fluorescamine; IR144; IR1446; Malachite Green isothiocyanate; 4- methylumbelliferoneortho cresolphthalein; nitrotyrosine; pararosaniline; Phenol Red; B- phycoerythrin; o-phthaldialdehyde; pyrene and derivatives: pyrene, pyrene butyrate, succinimidyl 1 -pyrene; butyrate quantum dots; Reactive Red 4 (Cibacron™ Brilliant Red 3B-A) rhodamine and derivatives: 6-carboxy-X-rhodamine (ROX), 6-carboxyrhodamine (R6G), lissamine rhodamine B sulfonyl chloride rhodamine (Rhod), rhodamine B, rhodamine 123, rhodamine X isothiocyanate, sulforhodamine B, sulforhodamine 101, sulfonyl chloride derivative of sulforhodamine 101 (Texas Red); N,N,N',N' tetramethyl-6- carboxyrhodamine (TAMRA); tetramethyl rhodamine; tetramethyl rhodamine isothiocyanate (TRITC); riboflavin; rosolic acid; terbium chelate derivatives; Cy3; Cy5; Cy5.5; Cy7; IRD 700; IRD 800; La Jolta Blue; phthalo cyanine; and naphthalo cyanine.

[0170] According to some embodiments, the detectable moiety comprises a blue fluorescent protein, cyan fluorescent protein, green fluorescent protein, red fluorescent protein, yellow fluorescent protein or any photoconvertible protein.

[0171] According to some embodiments, the detectable moiety comprises a heavy metal. According to some embodiments, the detectable moiety comprises uranyl acetate. According to some embodiments, the detectable moiety comprises lead citrate. According to some embodiments, the detectable moiety comprises Lanthanum. According to some embodiments, the detectable moiety comprises Cerium. According to some embodiments, the detectable moiety comprises Praseodymium. According to some embodiments, the detectable moiety comprises Neodymium. According to some embodiments, the detectable moiety comprises Promethium. According to some embodiments, the detectable moiety comprises Samarium. According to some embodiments, the detectable moiety comprises Europium. According to some embodiments, the detectable moiety comprises Gadolinium. According to some embodiments, the detectable moiety comprises Terbium. According to some embodiments, the detectable moiety comprises Dysprosium. According to some embodiments, the detectable moiety comprises Holmium. According to some embodiments,the detectable moiety comprises Erbium. According to some embodiments, the detectable moiety comprises Thulium. According to some embodiments, the detectable moiety comprises Ytterbium. According to some embodiments, the detectable moiety comprises Lutetium.

[0172] According to some embodiments, the terms “sample”, “tissue sample” or “biological sample” may encompass a sample obtained from a subject, such as a biological tissue or a fluid sample. According to some embodiments, the sample comprises body fluids e.g., blood, saliva, plasma, serum, feces, or urine), organs, tissues, fragments, and cells isolated from mammals, such as humans. According to some embodiments, the sample can also include a section of a biological sample containing tissue (e.g., a cross-section of an organ or tissue).

[0173] According to some embodiments, the sample comprises a tissue sample. According to some embodiments, the tissue comprises a mammalian tissue. According to some embodiments, the tissue comprises a human tissue. According to some embodiments, the tissue comprises a biopsy from a subject. According to some embodiments, the sample comprises cells. According to some embodiments, the sample comprises immune cells, tumor cells, cancer cells, diseased cells, stem cells, fetal cells, embryonic cells, living cells, cultured cells, or blood cells.

[0174] According to some embodiments, the sample comprises an organism. According to some embodiments, the sample comprises can be obtained from non-mammalian organisms (e.g., plants, insects, arachnids, nematodes, fungi, amphibians, or fish. According to some embodiments, the sample comprises a prokaryote such as a bacterium. According to some embodiments, the sample comprises archaea. According to some embodiments, the sample comprises a virus. According to some embodiments, the sample comprises a biofilm. According to some embodiments, the sample comprises a plant. According to some embodiments, the sample comprises a fossil.

[0175] According to some embodiments, the term “tissue” may encompass an aggregate of cells. According to some embodiments, the tissue is selected from the group consisting of connective tissue (e.g., areolar connective tissue, dense connective tissue, elastic tissue, reticular connective tissue, and adipose tissue), muscle tissue (e.g., skeletal muscle, smoothmuscle and cardiac muscle), genitourinary tissue, gastrointestinal tissue, pulmonary tissue, bone tissue, nervous tissue, and epithelial tissue (e.g., simple epithelium and stratified epithelium), endoderm-derived tissue, mesoderm-derived tissue, and ectoderm-derived tissue. According to some embodiments, the tissue is liver tissue. According to some embodiments, the tissue is spleen tissue. According to some embodiments, the tissue is intestine tissue. According to some embodiments, the tissue is colon tissue. According to some embodiments, the tissue is bone marrow tissue. According to some embodiments, the tissue is an immune tissue. According to some embodiments, the tissue is gastrointestinal track tissue. According to some embodiments, the tissue is an immune system tissue, which is, without limitation, lymphoid organ tissue, bone marrow, thymus, and lymph node tissue. According to some embodiments, the tissue is lung tissue. According to some embodiments, the tissue is breast tissue. According to some embodiments, the tissue is skin.

[0176] Accordingto some embodiments, the cells may be of atissue or organ. An artisan would appreciate that the term, “organ” may encompass a collection of tissues and cells which together form a structural unit having a function. According to some embodiments, the organ is selected from the group consisting of skin, sweat glands, sebaceous glands, mammary glands, bone, brain, hypothalamus, pituitary gland, pineal body, heart, blood vessels, larynx, trachea, bronchus, lung, lymphatic vessel, salivary glands, mucous glands, esophagus, stomach, gallbladder, liver, pancreas, small intestine, large intestine, colon, urethra, kidney, adrenal gland, conduit, ureter, bladder, fallopian tube, uterus, ovaries, testes, prostate, thyroid, parathyroid, meibomian gland, parotid gland, tonsil, adenoid, thymus, and spleen.

[0177] According to some embodiments, the methods described herein can be used for the detection of multiple targets in a biological sample. According to some embodiments, the methods disclosed herein may allow detection of multiple targets in the same biological sample. According to some embodiments, the same channel can be used for the detection of multiple targets in a sample. According to some embodiments, the methods described herein are used to diagnose, prognose, monitor or observe cancers or other diseases.

[0178] EXAMPLES and TEST RESULTS

[0179] According to some embodiments, the above-mentioned methods provide promising preliminary results, both in data simulations and decompression experiments. The size of each of the images in Figs. 8 A, 8B, 8C, 8D, 8E, 8F, 8G, 10A, 1 OB, 10C, 10D, 10E and 10F is about 400 / 7 / 7? 400 / / ??.

[0180] To test the method experimentally, Multiplexed Ion Beam Imaging were used to compress the staining of six target proteins (P = 6), into three images of three measurementchannels (C = 3). Figs. 6A demonstrates the combinatorial staining combinations and Fig. 6B the simple mapping. Images were acquired on a single tissue section, both for the ground truth images (x) of the individual P = 6 target proteins, according to the simple mapping as in Fig. 6B, and the multiplexed-images (y), stained according to combinatorial mapping as in Fig. 6A, where the same six target proteins were measured in a compressed combinatorial manner.

[0181] In order to generate the three compressed-images of the multiplexed-image, antibodies were conjugated to three different labeling-metals (labeling elements), according to the staining matrix A of Fig. 6A, while splitting some of the antibody signals between channels ( / .<?., Keratin, HLA-DR, and SMA).

[0182] According to some embodiments, the selected reporters (forthis example antibodies) do not need to be mutually exclusive, and their corresponding biological targets can be coexpressed.

[0183] According to some embodiments, targets may be chosen according to at least one of: expression in the tissue, clinical significance, dynamic range (intensity), and any combination thereof.

[0184] According to some embodiments, the reporter- (in this example antibody-) labeling, or staining method is configured to split the reporter- (in this example antibody-) signal across multiple channels, according to the combinatorial staining matrix A. According to some embodiments, for mass-based imaging systems, this labeling method is achieved by labeling polymers with different metal isotopes, before conjugating them to the antibody, as demonstrated in Fig. 7A.

[0185] Fig. 7B demonstrates the multi-metal polymer conjugation, according to some embodiments of the invention, in which unlabeled polymers are loaded with one or multiple metal isotopes and conjugated to a reduced antibody.

[0186] The three compressed-images and six ground truth images (x) were produced simultaneously, from the same tissue, according to the mapping demonstrated in Fig. 6A. An evaluation of successful compression is demonstrated in Figs. 8A-8G, by overlaying the ground truth signal (x) with the corresponding C compressed-images of the multiplexed-image (y). Fig. 8A demonstrates three MIBI-TOF images (C 3) of a sample, stained by six different antibodies (P = 6), shown individually in Fig. 8B, combined according to the compressed staining matrix A (shown in Fig. 7A).

[0187] Figs. 8C-8G demonstrate an assay validation for channel No. 3 of Fig. 8A (left image), which is also shown in Fig. 8C. The image color-overlays demonstrate the ground truth for Keratin (red color, as in Fig. 8D) and for CD45 (red color, as in Fig. 8F) together with the compressed- image Fig. 8C (green color). The overlay is indicated by a yellow color shown in Fig. 8E for the Keratin, and in Fig. 8G for CD45.

[0188] TEST I:

[0189] To further test the method experimentally, Multiplexed Ion Beam Imaging by Time-Of-Flight (MIBI-TOF) was used to compress the staining of six target proteins (P = 6), into three images of three measurement-channels (C = 3). Figs. 6A demonstrates the combinatorial staining combinations and Fig. 6B the simple mapping. Images were acquired on a single tissue section, both for the ground truth images (x) of the individual P = 6 target proteins, according to the simple mapping as in Fig. 6B, and the multiplexed- images (y), stained according to combinatorial mapping as in Fig. 6A, where the same six target proteins were measured in a compressed combinatorial manner

[0190] To validate the strength of the above mentioned method / s, the masking network Me was trained on N = 131 simulated training-input multiplexed-images y. Each such multiplexed-image has three channels (C = 3), resulting from the compression of the same six (P=6) target-proteins. Images for the individual proteins (ground truth images) were taken both from publicly available data and from internal, unpublished data. The trainingdata includes images acquired at: different times, using different antibody batches and a variety of tissues, including breast cancer, lung cancer and melanoma.

[0191] After training the NN, the performance of the above mentioned methods was tested (at testing phase) via 17 new multiplexed-images (test input-images), each of C = 3 channels. Both the multiplexed-image y (C = 3) and the ground truth individual images x (P = 6), were measured for the purpose of evaluating the 17 test experiments.

[0192] Results achieved an average an Fl of 0.83±0.07 across channels, with the median per channel ranging from 0.82 to 0.93, as shown in Fig. 9, which demonstrates the recovery of the six target proteins. Altogether, Combplex is shown to increase the number of proteins measured, for any given number of channels, using channel multiplexing and deep-1 earningbased decompression.

[0193] Example recovery results are shown in Figs. 10A-10F, for three target ground truth images, CD8 (Fig. 10A), CD45 (Fig. 10B) and HLA-DR (Fig. 10C), measured individually (red color), where their recovered images X out of the compressed images are respectively show in Figs. 10D-10F (green color), together with a percentage that indicates the Fl score of their prediction.

[0194] TEST 2:

[0195] To further test the method experimentally, Co-detection By Indexing (CODEX) was used to compress the staining of seven target proteins (P = 7), into three images of three measurement-channels (C = 3). Figs. 11A demonstrates the combinatorial staining combinations and Fig. 11B the simple mapping. Images were acquired on a single tissue section, both for the ground truth images (x) of the individual P = 7 target proteins, according to the simple mapping as in Fig. 11B, and the multiplexed-images (y), stained according to combinatorial mapping as in Fig. 11 A, where the same seven target proteins were measured in a compressed combinatorial manner.

[0196] To validate the strength of the above-mentioned method / s, the masking network Me was trained on N = 100 fluorescent simulated training-input multiplexed-images y. Each such multiplexed-image has three channels (C = 3), resulting from the compression of seven (P = 7) target-proteins. Images for the individual proteins (ground truth images) were taken from internal unpublished date.

[0197] After training the NN, the performance of the above-mentioned methods was tested (at testing phase) via 32 new multiplexed-images (test input-images), each of C = 3 channels. Both the multiplexed-image y (C = 3) and the ground truth individual images x (P = 7), were measured for the purpose of evaluating the 32 test experiments.

[0198] Results achieved an average Fl of 0.90±0.08 across channels, with the median per channel ranging from 0.81 to 0.99, as shown in Fig. 12, which demonstrates the recovery of the seven target proteins. Altogether, Combplex is shown to increase the number of proteins measured, for any given number of channels, across different modalities, using channel multiplexing and deep-leaming-based decompression.

[0199] Example recovery results are shown in Figs. 13A-13F, for three target ground truth images, Ki67 (Fig. 13A), CD45 (Fig 13B), and Pan-Keratin (Fig. 13C), measured individually (red color), where their recovered images X out of the compressed images are respectively shown in Figs. 13D-13F (green color), together with a percentage that indicates the Fl score of their prediction.

[0200] TEST 3: CombPlex, a deep learning approach to decompress combinatorically- stained images, accurately reconstructs compressed images in simulations.

[0201] To devise a computational framework for decompressing combinatorically- acquired images, we performed in silico simulations. We downloaded a published dataset, in which 57 proteins were measured in colorectal cancer (CRC) specimens using co-detection by indexing (CODEX) cyclic fluorescence (Schiirch, C. M. et al. Coordinated Cellular Neighborhoods Orchestrate Antitumoral Immunity at the Colorectal Cancer Invasive Front. Cell 182, 1341-1359.el9 (2020). From this data, we chose 41 fields of view (FOVs) and 22 proteins that had specific staining, with good signal to noise ratio. We then simulated combinatorial measurements in silico by a linear combination of the protein-images according to the compression matrix illustrated in Fig. 15A (the compression matrix employed to condense the data). According to some embodiments of the invention, the matrix was designed to minimize the partition of proteins between channels and balance the number of proteins compressed in each channel . Overall, this resulted in 41 multiplexed images, each containing 5 channels, depicting the expression of 22 proteins, as demonstrated in Fig. 15B.

[0202] We tested different approaches to decompress the simulated images and recover the original input images. We first tried to use compressed sensing, following literature that demonstrated success for similar problems in the analysis of gene expression data (Cleary, B., Cong, L., Cheung, A., Lander, E. S. & Regev, A. Efficient Generation of Transcriptomic Profiles by Random Composite Measurements. Cell 171, 1424-1436.el8 (2017); and Cleary, B. et al. Compressed sensing for highly efficient imaging transcriptomics. Nature Biotechnology 2021 39:839, 936-942 (2021)). However, using constraints for sparsity and signal continuity, compressed sensing failed to accurately recover the images of individual proteins; Median Fl scores of 0.28±0.26. Fig. 15C demonstrates combinatorically- compressed image (red) is decompressed into individual protein images using compressed sensing. Accuracy is evaluated by calculating the Fl scores of the recovered images (magenta) compared to experimentally-measured single protein images (ground truth, green); white pixels were accurately recovered, whereas green and magenta indicate false negatives and false positives, respectively). 15D demonstrates Fl scores (y-axis) for each one of the 22 proteins (x-axis) recovered by compressed sensing on eight test FOVs.

[0203] We reasoned that more informative priors on the patterns and properties of protein expression could address the problem, but such priors are difficult to formulate mathematically. We therefore turned to deep learning, in which a neural network can leverage intricate structures of the underlying data to reconstruct the single-protein images. We posed the problem as a supervised learning task and used deep learning to solve it. We trained a convolutional neural network (CNN), which, given a compressed image with C channels (y), recovers the underlying single-protein images, x (Methods). We trained the network on the simulated compressed images described above. We used 33 random FOVs for training and evaluated the results on 8 images. This approach dramatically improved the overall results, Fl = 0.91±0.16, , but performance varied significantly between proteins and FOVs. Fig. 15E demonstration has similar layout as Fig. 15C, using a convolutional neural network (CNN) for reconstruction; the network uses MSE loss to recover the signal in a single step. Fig. 15F demonstrates Fl scores (y-axis) for each one of the 22 proteins (x-axis) recovered by a single-step CNN on the same eight FOVs shown in Fig. 15E. Investigation of the errors over training suggested that the network had a high incentive to maximize the reconstruction of highly abundant proteins. From a biologist's point of view, false predictions of signals are more crucial than the precise reconstruction of abundant signals.

[0204] We therefore implemented a hybrid approach, according to some embodiments of the invention, using the intersection of two U-Nets, one which predicts where different proteins are expressed in the image, and the other which aims to recover the actual intensity values as demonstrated in Figs. 15G and 15H. The complete pipeline is described in the methods section and illustrated in Figs. 20A, 20B and 20C, according to some embodiments. Fig. 15H demonstration has similar layout as Fig. 15C, using the CombPlex algorithmic framework for reconstruction; CombPlex is designed as a two-step process, using the combination of a CNN that outputs binary predictions, and a CNN that recovers the values.

[0205] We tested CombPlex on the simulated compressed images described above. We used 33 random images to train the CNN and evaluated the results on 8 images. We found that CombPlex achieved excellent results, with median Fl scores of 0.98±0.11, dramatically outperforming the reconstruction using compressed sensing or deep learning alone; as demonstrated in Fig. 151 (Fl scores (y-axis) for each one of the 22 proteins (x-axis) recovered by CombPlex on the same eight FOVs (shown in Fig. 15H). CombPlex also achieved high correlation between the reconstructed values and the values measured in the ground truth images; Median Pearson R=0.99±0.03, shown in Fig. 15J (Pearson correlation coefficients (y-axis) between the ground truth images and the images recovered by CombPlex for the 22 proteins (x-axis) on the same eight FOVs shown in Fig. 15H). The accuracy of reconstruction was weakly related to the abundance and intensity of the signal (R= 0.24, p= 0.004), suggesting improved results for highly-expressed proteins. Performance was not affected by the identity or subcellular localization of the protein, suggesting that the approach is compatible with a large variety of proteins.

[0206] Next, we evaluated the sensitivity of CombPlex to the properties of the compression. In the simulations, 22 proteins were compressed to 5 channels, which is the minimal number of channels needed to encode 22 proteins. We tested whether relaxing the level of compression by increasing the number of channels affects CombPlex’ performance. To this end, we simulated compression of the same 22 proteins to either 5, 7, 9 or 11 channels. We found that increasing the number of channels generally increased the accuracy of the reconstruction. Overall, the mean Fl for five channels was 0.95±0.11 and increased to 0.97±0.10 for 11 channels as demonstrated in Fig. 15K (simulations were performed to compress the 22 proteins (x-axis) into 5, 7, 9 or 11 channels (y-axis) and models were trained accordingly. Colors denote the median F 1 scores across eight FOVs for the individual proteinimages recovered using CombPlex). We conclude that there is a tradeoff between the degree of compression and accuracy of reconstruction, and that the compression can be tailored to achieve a desired level of reconstruction accuracy.

[0207] There are many options to compress 22 proteins to 5 channels. We therefore generated 10 permutations of the compression matrix A, for each one simulated compressed data and then evaluated CombPlex’ performance in recovering the stains of the individual proteins. We found that CombPlex is relatively robust to the specific compression scheme used, as demonstrated in Fig. 15L (Ten models were trained using compression matrix A, while shuffling the columns to change the proteins’ location in the matrix. Boxplots show the distributions of median Fl scores across eight FOVs (y-axis) for each of the 22 proteins (x-axis); zoom shows the range between 0.8-1; green bars mark the 25th and 75th percentiles of the median Fl scores of 5 models trained with the same compression matrix). Standard errors for the median Fl scores of individual channels in different permutations of A ranged from 0.0004 to 0.008, similar to those obtained from retraining the network without changing A. We conclude that CombPlex is not very sensitive to the precise identity of A, allowing A to be largely determined by experimental considerations, such as availability of specific antibodies on specific channels etc.

[0208] Altogether, in silico simulations suggested that it is possible to compress the stains of many proteins to fewer measurement channels, and then recover the signal using deep learning.

[0209] TEST 4: CombPlex accurately reconstructs compressed cancer images measured by fluorescent microscopy:

[0210] Following the successful performance of CombPlex in simulations, we proceeded to test CombPlex experimentally. Most fluorescent imaging platforms easily allow measuring three channels, so we decided to test a compression scheme of seven proteins to three channels, which is the maximal level of compression. We chose seven target proteins with clinical relevance that are routinely used in immunostaining of tumors: Pan-Keratin, aSMA, Ki-67, p53, CD45, CD8 and HLADRDPDQ (HLA-II). These proteins constitute a variety of strong and weak, nuclear, cytoplasmic and membrane, and tumor and immune stains, and are a good representation of a plausible biologically-relevant immune-oncology panel. Importantly, this panel is expected to generate images with colocalization of proteins.For example, we expect to find pixels that are positive for CD45 and CD8 (cytotoxic T cells), Ki -67 and p53 (proliferating tumor cells), CD45 and HLA-II (myeloid cells) etc. Altogether, the selection of targets for the panel, and the compression scheme, utilizing all degrees of freedom in A, represent a realistic and difficult test case for the approach.

[0211] According to some embodiments of the invention, we designed a matrix A to compress the staining of the seven target proteins (P = 7), into three images (C = 3). Experimentally, this is achieved by labeling several antibodies with the same fluorophore. We then used CO-Detection by IndEXing (CODEX) to measure on a single tissue section both the ground truth images of individual targets and the compressed images, where the same targets are measured in a combinatorial manner. Fig. 16A demonstrates Experimental layout; CODEX cyclic imaging was used to image seven proteins on a single tissue section, both compressed to three measurement channels using multiple reporters per antibody (top matrix) and individually, using a single reporter per antibody (bottom matrix).

[0212] Altogether the experiment required ten channels: three for the compressed images and seven for the individual targets. We stained and imaged 132 breast cancer samples, from 66 different patients. We performed several analyses to gauge our protocol. First, we evaluated that the compressed images indeed contained signals overlapping with the expected proteins and did not contain signals of other proteins in the panel. For example, the ATTO-550 channel contained signal matching the single-channel images of Keratin, Ki-67, aSMA and HLA-II, but did not contain signal matching p53, CD8 or CD45. Fig. 16B top row of images: for a single FOV, shown are the three combinatorically-compressed images (red), which were decompressed by CombPlex to seven reconstructed single-protein images (not-shown); bottom row of images: overlay of four experimentally -measured single-protein images (ground truth, green) with the compressed image that contains all of their signals (Cl, red). Yellow pixels indicate overlap.

[0213] Moreover, pixel values in the compressed images were well -correlated with a simulation of a compressed channel, generated by overlaying the corresponding images in which the proteins were measured individually. Altogether, we concluded that it is experimentally possible to multiplex the stains of several antibodies on a single measurement channel.

[0214] We evaluated CombPlex’s performance in recovering the signal of the seven individual proteins from the compressed images. We trained the network on pairs of compressed images (y) and their corresponding individual -protein images (x). We used 100 paired images for training and evaluated performance on 32 held-out images. We found that CombPlex achieved highly accurate reconstruction, with median Fl scores on the test data of 0.97±0.07 across channels, and Pearson correlation coefficients of 0.98±0.02. Fig. 16C demonstrates CombPlex reconstruction (magenta) is overlaid with the ground truth singleprotein images (green); White pixels indicate overlap. Fig. 16D demonstrates Fl scores (y- axis) for each protein (x-axis) recovered by CombPlex on 32 test FOVs; the models were trained on experimentally-compressed images. Fig. 16E demonstration has similar layout as Fig. 16D, showing the Pearson correlation coefficients between the reconstructed and ground truth images.

[0215] Next, we wondered whether we could circumvent the need to obtain training data in the form of compressed measurements acquired on the same tissue as the ground-truth single protein measurements. Removing this requirement would be beneficial in making the approach more accessible, since pairs of compressed and single-protein images are not widely available and would have to be generated for each set of proteins and the specific compression matrix A. Simulating compressed images from widely-available sets of singleprotein measurements would negate the need to acquire such data, and would also allow flexibility by enabling to use the same training data for different compression schemes.

[0216] We therefore devised a scheme to simulate compressed images (j) from measurements of individual proteins (x) according to equation (y = A * x). For these simulations, we performed augmentations that are relevant to the domain measurement system. For example, we added noise to the compression matrix (.4) to account for experimental differences in intensities between fluorescent channels. We then trained CombPlex again, this time on pairs of individual protein images (x) and simulated compressed images (y = sim(x)). We then evaluated CombPlex’s performance on real, experimentally-compressed images. We found that even though it was trained on simulations, CombPlex achieved a highly accurate reconstruction, with median Fl scores on the test data of 0.95±0.1 across channels, and Pearson correlation coefficients of 0.97±0.05, largely comparable to the results achieved when training on real compressed images. Fig. 16F has similar layout as Fig 16D, where the models were trained on simulated compressedimages and evaluated on experimentally-compressed images. Fig. 16G. has similar layout as Fig. 16D, where the model was trained on simulated compressed images and evaluated on experimentally-compressed images. We conclude that it is possible to train CombPlex on simulations of compressed images generated from images in which the targets were measured individually, which is valuable for the utility and applicability of the approach.

[0217] TEST 5: CombPlex can support large-scale compressions of 10 antibodies per channel:

[0218] Following successful establishment of a small-scale compression scheme of seven antibodies in three channels, we evaluated the performance of the approach in a larger-scale compression, which necessitates to multiplex more proteins in each fluorescent channel. We chose 22 target proteins and designed an experiment to measure them in five channels. This scheme multiplexes 9-10 proteins per channel and represents a 4.5-fold compression. We utilized the CODEX system to obtain both the compressed and ground-truth images on the same breast cancer TMA. Fig. 17A demonstrates the experimental layout; CODEX cyclic imaging was used to image 22 proteins on a single tissue section, both compressed to five measurement channels using multiple reporters per antibody (top matrix) and individually, using a single reporter per antibody (bottom matrix). We validated that the compressed images indeed contained signals overlapping with the expected proteins and did not contain signals of other proteins in the panel. Fig. 17B top row of images: for a single FOV, shown are the five combinatorically-compressed images (red), decompressed by CombPlex to 22 reconstructed single-protein images (magenta); bottom 2 rows of images: overlay of nine experimentally -measured single-protein images (ground truth, green) with the compressed image that contains all of their signals (red); yellow pixels indicate overlap. Altogether, we concluded that this larger-scale compression is experimentally feasible.

[0219] We evaluated CombPlex’ performance in recovering the signal of the 22 individual proteins from the compressed images. We used 113 paired images for training and evaluated performance on 30 held-out images. We found that CombPlex achieved highly accurate reconstructions, both when training on real compressed images and on simulations. For the model trained on real images, median Fl scores on the test data were 0.95±0.13 and Pearson correlation coefficients were 0.98±0.08 across channels. Fig. 17C demonstrates CombPlex reconstruction (magenta) overlaid with the ground truth single-protein images(green); white pixels indicate overlap; for each protein image, the number in the bottom right comer indicates its crop location in the compressed image. Fig. 17D demonstrates Fl scores (y-axis) for each protein (x-axis) recovered by CombPlex on 30 test FOVs; the models were trained on experimentally-compressed images. Fig. 17E has a similar layout as in Fig. 17D, showing the Pearson correlation coefficients between the reconstructed and ground truth images.

[0220] Although CombPlex was overall highly accurate, we focused on the errors in prediction to better understand their properties and deduce the strengths and limitations of the approach. We separated the errors in prediction according to their type: False Negatives (FN) are pixels that contain staining for the protein in the ground truth images, but CombPlex fails to capture it, whereas False Positives (FP) are pixels in which CombPlex predicts signal that is absent in the ground truth images. Visual inspection of the images suggested that FNs and FPs tended to reside on the outskirts of real signal, correctly predicted by CombPlex (True Positives, TP); Fig. 17F. To test this more rigorously, for all channels across all test images we performed a 2-pixel dilation around TP pixels; Fig. 17F. We found that the majority of errors, both FN and FP, were located within a 2-pixel radius from accurately- detected signal (Fraction = 0.64±0.2) Fig. 17F and 17G demonstrate that for each protein in the 30 test FOVs, each pixel was compared between the ground truth images and the images reconstructed by CombPlex; pixels were classified as TP (True Positives), FN (False Negatives), FP (False Positives) and TN (True Negatives); to evaluate the fraction of FP and FN pixels in the vicinity of T P signal, areas of TP signal were dilated by two pixels; the number of FP and FN pixels inside these 2-pixel dilations was divided by the total number of FP and FN pixels in the image to calculate their fraction (y-axis) for each protein (x-axis). Moreover, FP and FN errors were more prevalent in pixels with lower intensities and higher multiplexing level. Taken together, we deduce that CombPlex’ errors mostly involve pixels with low intensity in the vicinity of a true signal.

[0221] Following our analysis of the errors made by CombPlex, we hypothesized that they would not materially change the expression patterns of the proteins in the cells and would have minimal downstream effects, if any, on commonly-performed tasks such as segmentation and cell classification 29,30. To test this explicitly, we segmented our images using Mesmer 31, and classified each protein in each cell as positive or negative. We then compared the cell classifications between the ground truth images and the imagesreconstructed by CombPlex (Methods). Overall, we found excellent agreement across all proteins; balanced Accuracy = 0.99±0.03. Fig. 17H demonstrate cells classified as positive (yellow) and negative (cyan) for Ki-67 using ground truth (left) or reconstructed (right) images. Fig. 171 has a similar layout as in Fig. 17H, for yH2AX. Fig. 17J demonstrate segmented cells were classified as positive or negative for each protein based on either the ground truth or the images reconstructed by CombPlex. Shown are the accuracy scores (y- axis) for each protein (x-axis). This suggests that even in cases in which the reconstruction is not perfect, it is robust enough to enable routine analyses. We conclude that CombPlex achieved accurate recovery of individual protein signals of twenty-two proteins measured using five fluorescent measurement channels.

[0222] TEST 6: CombPlex accurately reconstructs compressed images measured by mass-spec imaging:

[0223] Widely used multiplexed imaging approaches can be broadly grouped into two classes: fluorescence microscopy and mass-based imaging. We therefore tested CombPlex’ compatibility with Multiplexed Ion Beam Imaging by Time of Flight (MIBI-TOF), a massbased imaging modality. We devised a method for attaching multiple metals to one antibody by loading polymers with pre-defined metal ratios and then conjugating them to the desired antibodies. We measured on a single tissue section both the ground truth images of individual proteins and the compressed images, where the same targets were measured in a combinatorial manner. Fig. 18A demonstrates experimental layout; MIBI-TOF was used to image six proteins on a single tissue section, both compressed to three measurement channels using multiple metal isotopes per antibody (top matrix) and individually, using a single metal isotope per antibody (bottom matrix). We stained and imaged 19 samples from both breast cancer and lung cancer. We evaluated that the compressed images indeed contained signals overlapping with the expected proteins and did not contain signals of other proteins in the panel. For example, m / z 144 contained signal matching the single-channel images of aSMA, Keratin, Ki-67 and HLA-II, but did not contain signal matching CD8 or CD45. Fig. 18B Top row of images: for a single FOV, shown are the three combinatorically-compressed images (red), decompressed by CombPlex to six reconstructed single-protein images (not shown); Bottom row of images: overlay of four experimentally-measured single-protein images (ground truth, green) with the compressed image that contains all of their signals (red). Yellow pixels indicate overlap. Altogether, we concluded that it is experimentally possibleto multiplex the stain of several antibodies on a single measurement channel also in massbased imaging.

[0224] Encouraged by the results of using simulations to train the network in the fluorescence data, we further evaluated whether pre-existing data, not measured in the same experiment or in the same tissue of origin, could be used to train the network. We compiled a dataset from preexisting images in which these proteins were measured individually. We included 181 images from breast cancer, lung cancer and melanoma. Since the melanoma data did not contain staining for keratin, we interchanged the keratin images in this data for melan A, reasoning that both stain tumor cells and the stains are visually similar. The images for training were acquired in different experiments, conducted at different times, by different students, using different antibody batches and different MIBI-TOF instruments.

[0225] We evaluated CombPlex’ performance in recovering the signal of the individual proteins from the compressed images. We used the 181 simulated-compressed images for training and evaluated performance on the 19 experimentally-compressed images. We found that CombPlex achieved highly accurate reconstruction, with median Fl scores on the test data of 0.94±0.11 across channels and median Pearson correlation coefficients of 0.92±0.17. Fig. 18C demonstrates CombPlex reconstruction (magenta) is overlaid with the ground truth single-protein images (green); white pixels indicate overlap. Fig. 18D demonstrates Fl scores (y-axis) for each protein (x-axis) recovered by CombPlex on 19 test FOVs; the model was trained on compressed images generated by simulations on previously-available data. Fig. 18E has a similar layout as of Fig.l8D, showing the Pearson correlation coefficients between the reconstructed and ground truth images. Next, we evaluated the data generated by CombPlex in cell segmentation and classification. We segmented our images, and classified each protein in each cell as positive or negative. We compared the cell classifications between the ground truth images and the images reconstructed by CombPlex, and found high agreement across all proteins; Balanced Accuracy = 0.97±0.01. Fig. 18F demonstrates cells classified as positive (yellow) and negative (cyan) for Ki-67 using ground truth (left) or reconstructed (right) images. Fig. 18G demonstrates that segmented cells were classified as positive or negative for each protein based on either the ground truth or the images reconstructed by CombPlex; shown are the accuracy scores (y-axis) for each protein (x-axis). Similar to the results for the fluorescent data, this suggests that data generated byCombPlex yields cell-level classifications that are highly comparable to those obtained when proteins are measured individually.

[0226] Overall, our results indicate that CombPlex performs well also in mass-based imaging and that diverse pre-existing data can be used to train the algorithm. This greatly enhances the useability of the approach, negating the need to acquire data specifically for the purpose of training. Moreover, these results suggest that CombPlex is able to generalize between different tissues of origin and one model can be trained for a variety of pathologies.

[0227] Features contributing to CombPlex performance

[0228] In CombPlex, we use at least one neural network (NN) to constrain the solution space of an underdetermined equation system. We explored what the network learns to aid it in its classification by performing various perturbations to the input images and training scheme. First, we evaluated whether the network learns the compression matrix A, since it was not directly supplied to the network during the training process. We used the published CRC CODEX dataset and the network trained on the A matrix in Fig. 15 A. We then tested the network’s performance on simulated compressed images that were generated by a different compression matrix A*. Fig. 19A for the data from Schiirch et al., a model was trained on simulated compressed images using compression matrix A; inference was performed on simulated compressed images using compression matrices A and A*. We found that the network’s performance dropped by 98% compared to the original images (0.015±0.09 compared with 0.97±0.11; Fig. 19B demonstrates a comparison of Fl scores (y- axis) for each protein (x-axis) in the experiment illustrated in Fig. 19A, this indicates that CombPlex learns the compression matrix and that the network indeed learns the compression scheme encoded by the matrix used for training.

[0229] Next, we explored which features of the data aided the network in converging to the correct solution. First, we evaluated the effect of spatial continuity. We took our test images from the CRC dataset and shuffled the pixels along the x and y dimensions. This generates images that overall have the same pixel values in each channel, but the spatial continuity of stains is disrupted; Fig. 19C demonstrates CombPlex was trained on the data from Schiirch et al.; images of the test set were then shuffled with patches of varying sizes. We note that when the size of the shuffled block is large enough to maintain single-cell morphology (>162pixels) the network manages to produce accurate predictions (MedianFl=0.96±0.12). These drop dramatically when the size of the shuffled block decreases to one pixel (Median Fl=0.24±0.22), indicating that the network learns spatial continuity; in Fig. 19D shown are the median Fl scores for each protein (x-axis) on data shuffled with different patch sizes (y-axis). Next, we tested whether CombPlex learns priors on the staining properties of different proteins. To this end, we generated a toy example in which we generated a set of compressed images of two proteins, e.g., Ki-67 and aSMA; Fig. 19E, images of Ki-67 and SMA were used to simulate a naive compression of two proteins in one image. These images did not contain any additional information regarding co-expression of proteins, so to decompress them back to their single-protein images, the network must learn their corresponding staining patterns. We trained CombPlex on 33 images and tested performance on 8 images. CombPlex demonstrated high recovery for both proteins (Median Fl = 0.99±0.005, Fig. 19F,19G), indicating that it is able to learn to differentiate between the staining patterns of the different proteins. Fig. 19F: CombPlex was used to recover single-protein images from pairs of proteins compressed to one channel as shown in Fig. 19E; shown are the ground truth images (left) and CombPlex’ reconstruction (right). Fig. 19G: Shown are the Fl scores (y-axis) for three 2-to-l decompressions of pairs of proteins (x-axis) on a test set of 8 FOVs.

[0230] Since CombPlex is based on supervised learning, we evaluated how robust the approach is to unexpected changes in the test data. First, we simulated an experimentally- plausible situation in which one antibody doesn’t work. In current staining schemes, if one antibody doesn’t work, only its cognate stain is affected. We were concerned that in combinatorial staining, if one antibody doesn’t work then it may ruin the reconstruction of other antibodies. To this end, we used our CRC model, trained on a compression of 22 proteins to 5 channels. We then evaluated its performance on the simulated test data, each time setting the images of one protein to zero; Fig. 19H: CombPlex was trained on the data from Schurch et al. and then evaluated on a test set of 8 FOVs. In each run, one or more proteins were set as empty to simulate a scenario of a malfunctioning antibody or utilization of a subpanel. Reassuringly, the Fl scores for the other proteins were not affected by this perturbation (Differences in Fl were 0.001±0.003 Fig. 191). Fig. 191: the median Fl scores of different proteins (x-axis) are presented across experiments in which different proteins have been dropped out (y-axis). Encouraged by these results, we similarly simulated scenarios in which four or nine antibodies out of the 22 were missing, and found no reductionin the F 1 scores for the remaining proteins (Fig. 6H,I). These results highlight the robustness of the approach. Moreover, they indicate that a large model, trained on many proteins, could be used to decompress images that were combinatorically stained with only a subset of these proteins. This adds to the modularity of the approach, suggesting that a large pretrained foundation model could accommodate the decompression of many different staining schemes visualizing different subsets of proteins.

[0231] Finally, we evaluated whether the network learns rules of co-expression between proteins, and whether it can accommodate biological deviations from these expected patterns. To this end, we searched in our data for biological differences in protein coexpression patterns. One such example was the expression of HLA-II on malignant cells. HLA class II molecules are a class of major histocompatibility complex (MHC) proteins, normally found only on professional antigen-presenting cells 32. However, in tumors, they are sometimes expressed by malignant epithelial cells 33. In our data, we had images from patients in which the epithelial cells either expressed or did not express HLA-II; Fig. 19 J: Top row: shown are example images of tumor and non-tumor appearances of HLA-II (green) intersected with Pan-Keratin (marking tumor, magenta). Two models were trained on the CODEX data presented in figure (3). Model A was trained on images of HLA-II+ and HLA- II- tumors. Model B was trained only on images of HLA-II- tumors. Both models were evaluated on images of HLA-II+ tumors. To test whether the network learns to either expect or not expect this co-expression we trained two models. A baseline model A was trained on images of tumors both positive and negative for HLA-II whereas model B was trained only on images in which the tumor cells were negative for HLA-II. The performance of both models was then evaluated on 11 test images in which the tumor cells were positive for HLA- II (Fig. 19J). This scheme simulates a situation in which the test set contains novel biology, not exhibited during training. We found that both models were equally successful at reconstructing HLA-II on antigen presenting cells, but there was a 2-9% reduction in performance in reconstructing HLA-II expression on tumor cells when such cells were not presented at all in the training data; Fig. 6K: shown are the differences in Fl scores (y-axis) between the models in Fig. 19J; HLA-II reconstruction was evaluated in the entire image (left), tumor regions (middle) and non-tumor regions (right). We conclude that CombPlex can generalize to novel biology beyond the training set, but these capabilities are limited and,like in any supervised learning approach, ideally the training set should cover the full diversity of the domain.

[0232] Overall, we demonstrated that the network learns the compression matrix, spatial continuity of signals, the staining patterns of proteins and their co-expression, and leverages these properties in order to decompress combinatorically-multiplexed images. It can easily generalize to accommodate missing channels, but care should be taken with biological deviations that are not present in the training set.

[0233] More Methods

[0234] Generating the compression matrix

[0235] The compression scheme is determined by a binary matrix A with C rows and P columns, representing the measurement channels and individual proteins, respectively. In this matrix, a value of 1 at position (i, j) indicates that protein j is stained in measurement channel i. To generate matrix A we employ three principles, ordered by their priority:1. Each column vector must be unique such that different proteins are encoded differently.2. The sum of each column is minimized to facilitate the process of partitioning the antibodies.3. The sums of the rows are balanced, which aids in the computational task of distinguishing a higher number of protein signals within the same compressed image.

[0236] Simulations of combinatorically-compressed images

[0237] Given single-protein images x, we simulated compressed images y, as in Fig. 20A. We conducted a comparison between experimentally compressed images and those generated by summing intensities over corresponding single-protein images. While the comparisons revealed high correlations, there were still noticeable levels of noise and skewness. Therefore, our simulation procedure adheres to the fundamental concept of linearly adding the single-protein images, along with different noise sources and augmentations. Simulations for MIBI and CODEX were performed differently due to the distinct characteristics of the signal in each technology.

[0238] CODEX simulation process:1. Noise was applied on Matrix A, followed by convolution with the single-protein images x:This noise accounts for experimental differences between channels, resulting from either systematic differences in channel intensities or stochastic differences in antibody titers.2. Pixel-wise noise was applied on y to account for pixel-wise errors during imaging and accumulated background during simulation: y~A(y, n3• y) + 'V(? .4, 0.2 ■ n4),

[0239] MIBI-TOF simulation process:1. Noise applied on Matrix^ as described in the CODEX simulation process, creating A'.2. Convolving the single-protein images with Matrix A y = A' * x3. The compressed images were split to 2x2 patches, and each patch was randomly rotated.

[0240] Reconstructing individual protein images

[0241] During our explorations of the data, we found that from a biologist’s point of view, false predictions of signals are often more crucial than the precise reconstruction of abundant signals. As such, there is a need to reconcile between two incentives: accurately reconstructing the location of the signals and their magnitude. Therefore, according to some embodiments, and as an alternative to the optional optimization step (380, Fig. 3) CombPlex method (300) comprises a combination of two CNNs: a masking network, and a value reconstruction network, as demonstrated in Figs. 20B and 20C. The goal of the masking network is to determine where different proteins are expressed in the image, while the goal of the value reconstruction network is to determine how much each protein is expressed in each pixel of the image.

[0242] According to some embodiments, the input to the masking network is a compressed image with C channels (y), and the output is P images with probability valuesMB(y) = X’B. The probability value in a pixel in image p indicates the likelihood of the pixel to contain a signal of protein p. To train the network we perform supervised learning using pairs of compressed images (y) and their corresponding individual protein images (x). The goal is to minimize the binary cross-entropy between the binarized GT XB, and M (y). In parallel, we train the value reconstruction network. The input is again a compressed image with C channels (y), but the output is P images with predictions for the intensity values Mv(y) = x’v. We train it similarly in a supervised manner, to minimize the MSE between the GT single-protein images x and Mv(y).

[0243] To merge these outputs, a threshold of 0.5 is applied to the probability mask. For each pixel with a probability higher than 0.5, its predicted value is extracted from the output of the value reconstruction network using pixel-wise multiplication:

[0244] x' = (MB(y) > 0.5)OMv(y)

[0245] Training the decompression masking network My

[0246] According to some embodiments, the goal of M is to recover the binary support of the individual protein images, as in Fig. 20B. We perform pixel-wise classification for each pixel (i, j) in the image with respect to the presence or absence of each of the P proteins. Mathematically, we recover a function denoted as M (y), which takes a compressed image y G RHxWxCasinput and outputs a probability map x’v G RHXWXPsuch that:M}s y') ~~xs ’x xs where

[0247] The network is trained in a supervised manner using pairs {xk,yk}Nk=i of compressed images and their corresponding individual protein images. The compressed images can be either acquired experimentally or simulated.

[0248] The obj ective is to minimize the binary cross-entropy (BCE) loss between the real %B and the model output MB(J) in a set of N images. Formally, finding a set of parameters 0 to minimize:

[0249] Training the value reconstruction network My

[0250] According to some embodiments, the goal of M is to recover the intensities of the individual protein images, as in Fig. 20B. We perform value reconstruction for each pixel (i, j) in the image for each of the P proteins. Mathematically, we recover a function denoted as Mv(y), which takes a compressed image y G RHxWxCasinput and outputs a set of single-protein images x’v 6 RHxWxp

[0251] The training methodology for this network closely follows the approach detailed for the decompression masking network, except for two key modifications. Firstly, the sigmoid activation layer after the model architecture is omitted, enabling the prediction of values beyond one. Second, the training process adopts different loss functions for reconstructing the values according to the nature of the data. For CODEX data, we use MSE loss:

[0252] For MIBI-TOF data, which is a stochastic counting process, we employ the Poisson negative log likelihood loss:

[0253] While Zo^(Mv(ykijp)!) is approximated using the Stirling approximation term: log(n!) « n * loc;(fi) — n -4- 0.5 * log^iin)

[0254] Model ensembling

[0255] According to some embodiments, to enhance stability and performance, we utilize an ensemble of models for some of the experiments. Suppose we have trained decompression masking networks, denoted as MBI, MB2 MBU, and trained value reconstruction networks, denoted as Mvi, Mv2 ..., Mym. To create an ensemble, we establish a threshold T and combine the models by averaging their predictions.

[0256] Discussion

[0257] Current approaches to image multiple proteins in situ are inherently limited in throughput by their design: using a separate measurement channel for each individualprotein. Combinatorial staining holds great potential, because it exponentially increases the number of proteins that can be measured using any imaging platform from C to 2C- 1. For example, using three-color fluorescent microscopy one could measure seven proteins (2" - 1). Seven-color imagers, well established in research and in the clinic, could serve to image dozens of proteins, and a multiplexed imaging instrument could accommodate hundreds of targets. However, to date, combinatorial staining was not applied to proteins, presumably due to the high abundance of proteins in cells. High protein density makes it impossible to resolve the signals of individual proteins under standard magnification conditions and thus mathematically infeasible to reconstruct the individual protein expression images.

[0258] Here, we present CombPlex, a new experimental approach to perform combinatorial staining of proteins, coupled with an algorithmic framework to accurately decompress the combined signal back to individual protein images. We showed that CombPlex allows us to decouple the staining of individual proteins stained together with over 90% accuracy, even in low, 20X magnification, both in fluorescent and in mass-based imaging. We showed that although the problem of reconstructing individual protein images from compressed measurements is mathematically underdetermined, CombPlex manages to achieve highly accurate reconstructions, by leveraging various features of the data, including spatial continuity, individual protein expression patterns and protein co-expression. While it is infeasible to encode such features explicitly, a neural network trained on image examples is able to learn them and exploit them as strong priors for probable solutions for solving the reconstruction task.

[0259] Although supervised deep learning leads to improved performance, it also serves as the Achilles’ heel of the approach. CombPlex requires training data in the form of individual protein images, coupled with their combinatorically-compressed counterparts. In the above, we present several results that could reduce the entry barrier posed by the need to acquire such training data. We devised a method to simulate combinatorically-compressed images in silico from individual measurements of the corresponding proteins. We demonstrated that a network trained on simulated images achieves high performance, almost the same as a network that was trained on experimental combinatorically-compressed images. Using simulations to train the network holds great advantages. It negates the need to acquire both single-protein images and combinatorically-compressed images on the same tissue. Here, we showed that we were able to train CombPlex on diverse, pre-existing datafrom various tissue sources and use this network to generate accurate reconstructions of experimentally compressed data. Moreover, we showed in simulations that CombPlex could accommodate 30% missing proteins without a reduction in performance for the remaining proteins. Putting these results together, we envision that a community resource could be generated in which hundreds of proteins would be measured individually on various tissues using currently available multiplexed imaging approaches, such as cyclic fluorescence or mass-based imaging. This effort will only have to be done once, and could then serve as the basis to pre-train a large foundation model that accommodates the decompression of various staining schemes, visualizing different subsets of proteins. Subsequent studies could then use CombPlex to image multiple proteins on their respective samples, saving valuable time and money.

[0260] Protein imaging is a pillar of pathology. While recent technological developments have increased the number of proteins visualized in situ, these increases have been linear and thus far less than required for proteome-scale measurements. As such, this invention lays a foundation for a multitude of experimental, analytical and biological endeavors to undoubtedly follow.

[0261] Different embodiments are disclosed herein. Features of certain embodiments may be combined with features of other embodiments; thus certain embodiments may be combinations of features of multiple embodiments.

[0262] While certain features of the invention have been illustrated and described herein, many modifications, substitutions, changes, and equivalents will now occur to those of ordinary skill in the art. It is, therefore, to be understood that the appended claims are intended to cover all such modifications and changes as fall within the true spirit of the invention.

Claims

CLAIMS[0263] What is claimed is:

1. A method for decompressing a multiplexed-image (y), of C different channels, acquired to identify P molecules in a tissue sample; the method comprising steps of: combinatorically labeling (A) the molecules by at least C different labelingelements, which are configured to be detected by at least one of said C different channels, wherein P > C; and processor implemented step of: applying the multiplexed-image (y) as an input to at least one trained neural network (NN); the NN is configured to output a tensor B, having P binary-images each having ( / xj) binary -pixels; each binary -image p per a different molecule p of the P molecules (where p = 1,2, ...,Py, wherein a pixel (i,j; i = 1,2, j = 1,2, ..., J) in a binary-image p having a value of one (7) indicates that pixel (zj) is expected to contain a signal of its associated molecule p.

2. The method of claim 1, wherein the NN is a supervised NN.

3. The method of claim 2, wherein the supervised NN is CNN.

4. The method any one of claims 1 to 3, further comprising applying an optimization algorithm to the NN’s output tensor ; the optimization algorithm is configured to provide each of the pixels (zj) of a binary-image p with an intensity value; wherein the intensity value indicates the associated molecule p in pixel (ij).

5. The method any one of claims 1 to 4, further comprising combinatorically labeling the P molecules, via the C labeling-elements, wherein each molecule is uniquely labeled via one or several labeling-elements, and wherein at least one molecule is labeled via at least two labeling-elements.

6. The method of claim 5, wherein the labeling-elements comprise at least one of: fluorescent dyes, chromophores, heavy -metal isotopes, chemiluminescent substances, radioactive substances, fluorescent proteins, nanoparticles, any detectable moiety, and any combination thereof7. The method of claim 5 or 6, further comprising acquiring the multiplexed-image (y), of C channels, of the P labeled molecules, configured to be applied as the input to the trained NN at the decompression step.

8. The method of any one of claims 1 to 7, further comprising unsupervised learning of the NN.

9. The method of claim 2 or 3, further comprising supervised training of the NN, via:• at least one training multiplexed-image, to be applied as a training input-image to the NN; and• a known desired-tensor b, of the training multiplexed-image, having P binaryimages with binary-pixels, to be applied as a training’s desired-output for the NN.

10. The method of claim 9, further comprising acquiring the at least one training multiplexed-image.

11. The method of claim 9 or 10, further comprising:• acquiring P ground truth individual-images (x), of the P separately-labeled molecules, wherein each individual-image p is configured to depict a single molecule p of the P molecules; and• converting the P individual -images to P binary-images having (IXJ) binarypixels, configured to be applied as the known desired-tensor b.

12. The method of claim 11, further comprising simulating the at least one training multiplexed-image, from the P individual-images (x), based on the combinatorial labeling design.

13. The method of claim 12, further comprising convolving matrix A with the P individual-images (x).

14. A method of detecting targets in a sample comprising: a. combinatorically labeling P targets in a sample with at least C different labelingelements, which are configured to be detected by at least one of C different channels, wherein C < P b . imaging the sample to acquire at least one multiplexed-image of the sample; andc. decompressing the multiplexed-image, according to the method of any one of claims 1 to 13, to extract P different images each of a different target of the P targets.

15. The method of claim 14, wherein said targets comprise biological molecules.

16. The method of claim 14 or 15, wherein said biological molecules are selected from the group consisting of proteins, peptides, polypeptides, antibodies, antigens, enzymes, enzyme substrates, ligands, receptors, polynucleotides, RNA, DNA, metabolites, polysaccharides, lectins, sugars, glycoproteins, lipids, fats, vitamins, hormones, and cellular compartments.

17. The method of any one of claims 14 to 16, wherein the one or more biological molecules are biomarkers.

18. The method of any one of claims 14 to 17, wherein said labeling-elements are attached to said P targets by P reporters; and wherein said reporters are selected from the group consisting of antibodies, peptides, proteins, polynucleotides, oligonucleotides, polysaccharides, lipids, enzymes, enzyme substrates, ligands, receptors, antigens, haptens, and aptamers.

19. The method of any one of claims 14 to 18, wherein said labeling-element comprises at least one detectable moiety.

20. The method of claim 19, wherein said detectable moiety is selected from the group consisting of a dye, fluorescent dye, heavy-metal isotope, chromophore, chemiluminescent substance, radioactive substance, nanoparticle, surface plasmon resonant particle (PRP), and resonance light scattering particle (RLS).

21. The method of any one of claims 14 to 20, wherein the sample comprises a tissue sample.

22. The method of claim 21, wherein the tissue sample comprises a mammalian tissue.

23. The method of claim 21, wherein the tissue sample comprises a human tissue.

24. The method of claim 21 , wherein the tissue sample comprises a biopsy from a subj ect.

25. The method of any one of claims 14 to 24, wherein the sample comprises cells.

26. A device configured to execute the method steps according to any one of claims 1 to25, comprising:• at least one processor configured to execute the method steps according to any one of claims 1 to 25; and• at least one display -device, configured to display to a user data related to any one of claims 1 to 25.

27. The device of claim 26, further comprising at least one input-device, configured to collect and interpret data, according to method steps of any one of claims 1 to 13.

28. The device of claim 26 or 27, further comprising an apparatus selected from: a computer, a smart phone, a tablet and any combination thereof.

29. The device of any one of claims 26 to 28, further comprising at least one of: data storage, an input device, a computer mouse; and any combination thereof.

Citation Information

Cited By

  • High-sensitivity handheld multi-joint detection fluorescence immunoassay analyzer

    CN119916015A