Microscope image and spatial transcript data alignment using fiducial and feature-based alignment

Abstract

To align a microscope image with spatial transcriptomics data, a computing device obtains a microscope image of a tissue sample, and obtains a spatial transcript map for the tissue sample. Then the computing device registers the spatial transcript map and the microscope image to a common coordinate system by applying a transform based on detected locations of fiducials in the spatial transcript map and the microscope image. The computing device aligns at least a portion of the spatial transcript map with at least a portion of the microscope image within the common coordinate system by performing a feature-based alignment.

Description

33080 / IP-2888MICROSCOPE IMAGE AND SPATIAL TRANSCRIPT DATA ALIGNMENT USING FIDUCIAL AND FEATURE-BASED ALIGNMENTCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 735,519, filed December 18, 2024, entitled “Microscope Image and Spatial Transcript Data Alignment Using Fiducial and Feature-Based Alignment,” the entire disclosure of which is hereby expressly incorporated by reference herein.FIELD OF THE INVENTION

[0002] The present disclosure generally relates to techniques for aligning microscope images with spatial transcriptomics data for tissue samples, such as applying transforms based on detected locations of fiducials and performing feature-based alignments.BACKGROUND

[0003] The background description provided herein is for the purpose of generally presenting the context of the disclosure. Work of the presently named inventors, to the extent it is described in this background section, as well as aspects of the description that may not otherwise qualify as prior art at the time of filing, are neither expressly nor impliedly admitted as prior art against the present disclosure.

[0004] In the field of spatial transcriptomics, the precise localization of transcripts within tissue samples is fundamental for understanding cellular functions and interactions in a spatial context. Transcripts may be obtained from tissue samples placed on a substrate after tissue permeabilization and subsequent capture using target capture sites on the surface. The captured transcripts are then assigned locations to generate a spatial transcriptomics map.Prior to tissue permeabilization, the tissue can be stained and imaged using a microscope. The microscope image may then depict cells, nuclei, and / or other features of the tissue sample.However, the microscope image may not be aligned with the spatial transcriptomics map such that the transcripts cannot be accurately overlaid on the microscope image depicting the tissue. Microscopic images provide crucial morphological context but often do not directly correlate with the spatial positions of transcripts, leading to difficulties in accurately overlaying transcriptom ic data onto these images. Furthermore, discrepancies such as rotation, translation, skew, and distortion between the images can complicate the alignment process, potentially resulting in inaccuracies in data interpretation.33080 / IP-2888SUMMARY

[0005] Methods such as fiducial-based alignment can be employed to address these alignment challenges, using markers or fiducials at known locations to correct for image discrepancies. However, while fiducial-based approaches aid in achieving global alignment, they can fall short in correcting local or non-linear errors that may arise from stitching artifacts or diffusion processes. Alternatively, feature-based alignment techniques can be used to align images by matching or approximating similar features across them, like cell shapes or transcript density profiles. These methods can provide more detailed alignments in specific regions but often struggle with global alignment accuracy due to the aforementioned challenges of diffusion and due to the three-dimensional (3D) to two-dimensional (2D) transformation effects associated with capturing 2D microscope images from a 3D tissue sample. Consequently, both methods present limitations when used independently, suggesting a need for improved strategies that can leverage the strengths of both approaches to achieve both global and local alignment accuracy.

[0006] The disclosure provided herein focuses on a sophisticated image alignment system designed for spatial transcriptomics. The image alignment system addresses the challenges described above by employing a novel approach to image alignment, leveraging both fiducialbased and feature-based alignment methods to achieve precise overlay of spatial transcriptomics maps onto microscope images.

[0007] In some implementations, the image alignment system combines both the fiducialbased alignment with the feature-based alignment to align the microscope image with the spatial transcriptomics map (also referred to herein as a “spatial transcript map”) using a hybrid fiducial-based and feature-based alignment. The fiducial-based alignment and feature-based alignment may be combined in any suitable manner to align the microscope image with the spatial transcriptomics map. For example, the image alignment system may register the microscope image and the spatial transcriptomics map to a common coordinate system by detecting the locations of the fiducials within the microscope image and the spatial transcriptomics map. Then the image alignment system may apply respective transforms to the microscope image and to the spatial transcriptomics map so that the detected fiducials are located at their known, physical locations. More specifically, the image alignment system may generate affine transformation matrices that transform locations of fiducials detected within the microscope image and the spatial transcriptomics map to their known, physical locations. Then the image alignment system may apply the affine transformation matrices to the microscope33080 / IP-2888 image and the spatial transcriptomics map, respectively, to register the microscope image and the spatial transcriptomics map to a common coordinate system.

[0008] The image alignment system may identify portions of the aligned image with inaccuracies. For example, the image alignment system may determine correlation scores within a region of the aligned image and a correlation distribution around the peak correlation score to detect alignment accuracy. Then for regions with low correlation scores or noisy distributions, the image alignment system may perform a feature-based alignment in these regions. The image alignment system may then adjust the coordinates for the microscope image and / or the spatial transcriptomics map using the feature-based alignment.

[0009] In some implementations, the image alignment system determines whether or not to align the spatial transcriptomics map with the microscope image using a feature-based alignment. For example, the image alignment system may determine whether any of the regions include an accuracy metric which is below a threshold accuracy level. If none of the regions have an accuracy metric which is below the threshold accuracy level, the image alignment system does not align the spatial transcriptomics map with the microscope image using a feature-based alignment. The image alignment system processes the spatial transcriptomics map registered within the common coordinate system. If at least one of the regions has an accuracy metric which is at or above the threshold accuracy level, the image alignment system aligns the spatial transcriptomics map with the microscope image using a feature-based alignment. Then the image alignment system processes the spatial transcriptomics map registered within the common coordinate system and aligned within the microscope image.

[0010] In some implementations, the image alignment system may then adjust the coordinates for the spatial transcriptomics map by aligning the coordinates for a particular feature to match the coordinates of that feature in the microscope image (e.g., a transcript density profile in the spatial transcriptomics map having a similar shape with a cell in the microscope image). Alternatively, the image alignment system may adjust the coordinates for the microscope image by aligning the coordinates for a particular feature to match the coordinates of that feature in the spatial transcriptomics map. In other implementations, the image alignment system may determine a first set of coordinates for the feature in the microscope image using the fiducial-based alignment, and a second set of coordinates for the feature in the microscope image after it has been aligned using the feature-based alignment. Then the image alignment system may assign weights to the first and second sets of33080 / IP-2888 coordinates and calculate a weighted average to determine the coordinates for the feature. For example, if the feature in the spatial transcriptomics map is 6 pm from the matching feature in the microscope image after the fiducial-based alignment, the image alignment system may determine to align the spatial transcriptomics map by moving the feature 3 pm closer to the matching feature in the microscope image.

[0011] The image alignment system's approach to image alignment in spatial transcriptomics represents a significant advancement in the field. By addressing the challenges associated with accurately overlaying spatial transcriptomics maps onto microscope images, the system not only improves the reliability of spatial transcriptomics data but also enhances the efficiency and accuracy of the analysis. The combination of fiducial-based and feature-based alignment methods offers a comprehensive solution that leverages the strengths of both approaches, resulting in a more accurate and efficient alignment process. This, in turn, enables researchers to obtain a more detailed and accurate understanding of the spatial distribution of transcripts within tissue samples, contributing to advancements in the study of cellular functions and interactions. For example, the registered and / or aligned image can be used to determine which transcript corresponds to which cell, identify cells types for cells in the microscope image based on their corresponding transcripts, the transcript density in a particular cell, elevated expression of a certain gene in a cell, cell-to-cell interactions, etc.

[0012] In one aspect, a method for aligning a microscope image with spatial transcriptomics data for a tissue sample includes: (1 ) obtaining, by one or more processors, a microscope image of a tissue sample; (2) obtaining, by the one or more processors, a spatial transcript map for the tissue sample; (3) registering, by the one or more processors, the spatial transcript map and the microscope image to a common coordinate system by applying a transform based on detected locations of fiducials in the spatial transcript map and the microscope image; and (4) aligning, by the one or more processors, at least a portion of the spatial transcript map with at least a portion of the microscope image within the common coordinate system by performing a feature-based alignment.

[0013] In another aspect, a method for aligning a microscope image with spatial transcriptomics data for a tissue sample includes: (1 ) obtaining, by one or more processors, a microscope image of a tissue sample; (2) obtaining, by the one or more processors, a spatial transcript map for the tissue sample; (3) registering, by the one or more processors, the spatial transcript map and the microscope image to a common coordinate system by applying a transform based on detected locations of fiducials in the spatial transcript map and the33080 / IP-2888 microscope image; (4) comparing, by the one or more processors, a portion of the spatial transcript map to a portion of the microscope image within a region of the common coordinate system; and (5) determining, by the one or more processors, whether to align the portion of the spatial transcript map with the portion of the microscope image based on the comparison.

[0014] Advantages will become more apparent to those of ordinary skill in the art from the following description of the preferred embodiments which have been shown and described by way of illustration. As will be realized, the present embodiments may be capable of other and different embodiments, and their details are capable of modification in various respects.Accordingly, the drawings and description are to be regarded as illustrative in nature and not as restrictive.BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The figures described below depict various aspects of the system and methods disclosed herein. It should be understood that each figure depicts an embodiment of a particular aspect of the disclosed system and methods, and that each of the figures is intended to accord with a possible embodiment thereof.

[0016] There are shown in the drawings arrangements which are presently discussed, it being understood, however, that the present embodiments are not limited to the precise arrangements and instrumentalities shown, wherein:

[0017] Fig. 1 depicts a block diagram of an example computing environment for performing hybrid fiducial and feature-based alignment techniques to align microscope images of tissue samples with spatial transcriptomics maps, according to some aspects.

[0018] Fig. 2 depicts an example microscope image of a tissue sample with fiducials side-by- side with an example spatial transcriptomics map for the sample tissue with the same fiducials, demonstrating the alignment of physical structures with genomic data, according to some aspects.

[0019] Fig. 3 depicts an example field of view of a microscope image registered to a global coordinate system, according to some aspects.

[0020] Fig. 4 depicts a block diagram of an example fiducial-based alignment for a microscope image and a spatial transcriptomics map, according to some aspects.33080 / IP-2888

[0021] Fig. 5 depicts a combined block and flow diagram of an example fiducial-based alignment method for a microscope image and a spatial transcriptomics map, according to some aspects.

[0022] Fig. 6 illustrates an example portion of a microscope image of a tissue sample side- by-side with an example portion of a spatial transcriptomics map of the sample tissue registered to a common coordinate system, according to some aspects.

[0023] Fig. 7 illustrates an example portion of a spatial transcriptomics map aligned with an example portion of a microscope image of a tissue sample using a fiducial-based alignment side-by-side with the example portion of the spatial transcriptomics map aligned with the example portion of the microscope image aligned using the hybrid fiducial and feature-based alignment technique.

[0024] Fig. 8 is a flow diagram of an example method for aligning a microscope image with spatial transcriptomics data for a tissue sample, which may be implemented by a server device, according to some aspects.

[0025] Fig. 9 is a flow diagram of an example method for determining whether to align a microscope image with spatial transcriptomics data for a tissue sample, which may be implemented by a server device, according to some aspects.

[0026] The Figures depict preferred implementations for purposes of illustration only. Alternative implementations of the systems and methods illustrated herein may be employed without departing from the principles of the invention described herein.DETAILED DESCRIPTION

[0027] Although the following text discloses a detailed description of implementations of methods, apparatuses and / or articles of manufacture, it should be understood that the legal scope of the property right is defined by the words of the claims set forth at the end of this document. Accordingly, the following detailed description is to be construed as examples only and does not describe every possible implementation, as describing every possible implementation would be impractical, if not impossible. Numerous alternative implementations could be implemented, using either current technology or technology developed after the filing date of this patent. It is envisioned that such alternative implementations would still fall within the scope of the claims.33080 / IP-2888EXEMPLARY COMPUTING ENVIRONMENT

[0028] Fig. 1 is a block diagram of an example computing environment 100 for performing the hybrid fiducial and feature-based alignment techniques described herein. This computing environment 100 is designed to align microscope images of tissue samples with spatial transcriptomics maps, enabling precise overlay and analysis of genomic data with corresponding cellular structures. The example computing environment 100 may include a sequencing device 110, a server device 120, a client device 130, a microscope 140, a tissue sample 102a, 102b, and a network 150.

[0029] The tissue sample 102a may be sectioned and placed in proximity to a slide (e.g., a flow cell) having fiducials 104a, 106a or markers at known, physical locations. For example, the fiducials 104a, 106a may be placed in a specific pattern on the flow cell (e.g., a grid) where the spacing between the fiducials 104a, 106a is known and can be used to locate each fiducial. In other implementations, each fiducial 104a, 106a has associated physical coordinates (e.g., (5 pm, 10 pm)). The tissue sample 102a may be stained, for example with hematoxylin and eosin (H&E), and imaged via the microscope 140. In some implementations, the microscope 140 images different subsections or fields of view (FOVs) of the tissue sample 102a. Then each of the FOVs can be combined to generate a microscope image of the tissue sample 102a.

[0030] The microscope 140 may transmit the microscope image of the tissue sample 102a with fiducials 104a, 106a or FOVs of subsections of the tissue sample 102a with fiducials 104a, 106a to the server device 120 via the network 150. The microscope image may depict cells, nuclei, and / or other cellular and subcellular structures within the tissue sample.

[0031] Additionally, the tissue sample 102b may be sectioned and placed in proximity to a slide (e.g., a flow cell) with a very large set of (e.g. up to hundreds of millions) of barcoded locations, each containing a sequence of capture oligonucleotides constituting a spatial barcode unique to that location. As mentioned above, the flow cell may also include fiducials 104b, 106b or markers at known, physical locations. Prior to permeabilization, the sequencing device 110 may decode the spatial barcodes for creating a map of barcode sequences and coordinate locations. During permeabilization, the tissue sample 102b may release mRNA which binds to capture oligonucleotides from a proximal location on the tissue 102b. A reverse transcription reaction may occur while the tissue is still in place, generating a library that incorporates the spatial barcodes and preserves spatial information. For example, each library may include a combination of mRNA, a barcode, an index, and / or other known sequences. The barcoded sequences are mapped back to a specific location within the flow cell.33080 / IP-2888

[0032] In further implementations, capture oligonucleotides and spatially barcoded oligonucleotides having a spatial barcode may be separated but proximal to one another on the substrate. In such implementations, a mechanism may be used after molecule capture, where the captured molecules may bridge to a nearby spatially barcoded nucleotide, for example as described in U.S. Provisional Patent Application No. 63 / 615,558, which is incorporated by reference herein in its entirety. A reverse transcription reaction may occur resulting in a library having both mRNA and a spatial barcode, which the sequencing device 110 (and / or another sequencing device (not shown)) may sequence.

[0033] The tissue sample 102a and fiducials 104a, 106a imaged by the microscope image 140 may be the same tissue sample 102b and fiducials 104b, 106b imaged by the sequencing device 1 10.

[0034] Then the sequencing device 110 may image the flow cell having barcoded nucleotide fragments from the tissue sample 102b and having fiducials 104b, 106b. The sequencing device 1 10 may include a computing device, image sensors, and a sequencing application 112 for sequencing a genomic sample or other nucleic-acid polymer. In some versions, by executing the sequencing application 112 using a processor, the sequencing device 1 10 may analyze nucleotide fragments or oligonucleotides extracted from genomic samples to generate nucleotide reads or other data utilizing computer implemented methods and systems either directly or indirectly on the sequencing device 110.

[0035] More particularly, the sequencing device 1 10 may receive the flow cell with barcoded nucleotide fragments extracted from the tissue sample 102b and fiducials 104b, 106b, and the sequencing device may determine the nucleobase sequence of such extracted nucleotide fragments. The sequencing device 1 10 may be the sequencing system described in U.S.Patent Application No. 18 / 340,795, titled “Split-Read Alignment by Intelligently Identifying and Scoring Candidate Split Groups filed on July 23, 2023, which is hereby incorporated by reference in its entirety.

[0036] In some versions, the sequencing device 110 may utilize sequencing-by-synthesis (SBS) to sequence nucleotide fragments into nucleotide reads and determine nucleobase calls for the nucleotide reads. By executing the sequencing application 1 12, the sequencing device 110 may further store the nucleobase calls as part of base-call data that is formatted as a binary base call (BCL) file and send the BCL file to the server device 120. The sequencing device 110 may communicate the BCL file and / or other data to the sever device 120 via one or more network(s) 150 or directly (e.g., bypassing the one or more network(s) 150).33080 / IP-2888

[0037] The sequencing device 110 may image a sample within a flow cell in swaths which are regions of the sample. The swaths may overlap with each other. For example, adjacent swaths may have a 6% overlap. In some implementations, prior to permeabilization, the sequencing device 1 10 may decode the spatial barcodes for creating a map of barcode sequences and coordinate locations. Then after permeabilization, the cDNA along with barcode may be eluted and sequenced on a second flow cell. The sequencing device 1 10 or another sequencing device 1 10 may image the second flow cell in swaths.

[0038] As used herein, the term “cluster of oligonucleotides” (or simply “cluster(s)” or “DNA cluster(s)”) refers to a localized group or collection of DNA or RNA molecules on a nucleotide- sample slide, such as a flow cell, or other solid surface. In particular, a cluster includes tens, hundreds, thousands, or more copies of a cloned or the same DNA or RNA segment. For example, in one or more embodiments, a cluster includes a grouping of oligonucleotides immobilized in a section of a flow cell or other nucleotide-sample slide. In some embodiments, clusters are evenly spaced or organized in a systematic structure within a patterned flow cell. By contrast, in some cases, clusters are randomly organized within a non-patterned flow cell. A cluster of oligonucleotides can be imaged utilizing one or more light signals. For instance, an oligonucleotide-cluster image may be captured by a camera during a sequencing cycle of light emitted by irradiated fluorescent tags incorporated into oligonucleotides from one or more clusters on a flow cell.

[0039] The server device 120 receives the FOVs of the microscope image of the tissue sample 102a and receives the swaths from the sequencing device 1 10. The server device 120 may then divide the swaths into sequencing tiles. The server device 120 may include a memory and one or more processors (CPUs). The memory can be a non-transitory memory and can include one or several suitable memory modules, such as random access memory (RAM), readonly memory (ROM), flash memory, other types of persistent memory, etc.

[0040] The memory may store an operating system (OS), which can be any type of suitable mobile or general-purpose operating system. The memory also stores a hybrid alignment engine 122 that receives the FOVs of the microscope image and the sequencing tiles and registers them to a common coordinate system based on the locations of detected fiducials in the microscope image and the sequencing tiles. The hybrid alignment engine 122 may stitch the FOVs to together to generate the microscope image and may stitch the sequencing tiles to generate the spatial transcriptomics map. In some implementations, the hybrid alignment engine 122 also may align portions of the spatial transcript map with corresponding portions of the microscope image within the common coordinate system.33080 / IP-2888

[0041] To register the microscope image and the spatial transcriptomics map to a common coordinate system, the server device 120 first detects fiducials within both the microscope image and the spatial transcriptomics map. Fiducials are markers or features at known physical locations used as reference points for alignment. The process typically involves image processing techniques such as thresholding, edge detection, and pattern recognition to accurately locate the fiducials within each image. The hybrid alignment engine 122 transforms the locations of these detected fiducials to their known physical locations, effectively correcting for any discrepancies due to rotation, translation, scaling, skew, or distortion. Additionally, the hybrid alignment engine 122 aligns the spatial transcriptomics map to the microscope image by identifying similar features in both images, further refining the alignment process (also referred to herein as the “feature-based alignment”).

[0042] To transform the locations of the detected fiducials, the hybrid alignment engine 122 generates affine transformation matrices for the microscope image and the spatial transcriptomics map. To calculate a transform and generate a first affine transformation matrix for the microscope image, the hybrid alignment engine 122 establishes correspondences between a first set of detected fiducial locations in the microscope image and their known physical locations. Using these correspondences, the hybrid alignment engine 122 solves for the parameters of the affine transformation, which includes translation, rotation, scaling, and shearing factors. The resulting first affine transformation matrix encapsulates these parameters, enabling the conversion of locations of fiducials within the microscope image to their known physical locations.

[0043] To calculate a transform and generate a second affine transformation matrix for the spatial transcriptomics map, the hybrid alignment engine 122 first establishes correspondences between a second set of detected fiducial locations in the spatial transcriptomics map and their known physical locations. Using these correspondences, the hybrid alignment engine 122 solves for the parameters of the affine transformation, which includes translation, rotation, scaling, and shearing factors. The resulting second affine transformation matrix encapsulates these parameters, enabling the conversion of locations of fiducials within the spatial transcriptomics map to their known physical locations.

[0044] Then the hybrid alignment engine 122 applies the first affine transformation matrix to the microscope image. Applying the first affine transformation matrix to the microscope image involves manipulating the microscope image according to the parameters defined in the first affine transformation matrix. This process adjusts the position, orientation, and scale of the33080 / IP-2888 microscope image to align it with the common coordinate system. For example, the microscope image may span from locations (0, 0) to (1000, 1000) whereas the common coordinate system may include physical locations (0 pm, 0 pm) to (500 pm, 450 pm). The transformation ensures that the fiducials in the microscope image are positioned at their correct physical locations, facilitating accurate alignment with the spatial transcriptomics map.

[0045] The hybrid alignment engine 122 also applies the second affine transformation matrix to the spatial transcriptomics map. Applying the second affine transformation matrix to the spatial transcriptomics map involves manipulating the spatial transcriptomics map according to the parameters defined in the second affine transformation matrix. This process adjusts the position, orientation, and scale of the spatial transcriptomics map to align it with the common coordinate system. For example, the spatial transcriptomics map may span from locations (0, 0) to (800, 900) whereas the common coordinate system may include physical locations (0 pm, 0 pm) to (500 pm, 450 pm). The transformation ensures that the fiducials in the spatial transcriptomics map are positioned at their correct physical locations, facilitating accurate alignment with the microscope image.

[0046] The server device 120 may then transmit the spatial transcript map registered within the common coordinate system and aligned with the microscope image to the client device 130. The client device 130 may include a memory and one or more processors (CPUs). The memory can be a non-transitory memory and can include one or several suitable memory modules, such as random access memory (RAM), read-only memory (ROM), flash memory, other types of persistent memory, etc.

[0047] The memory may store an operating system (OS), which can be any type of suitable mobile or general-purpose operating system. The memory also stores a spatial transcriptomics application 132 which may present the spatial transcriptomics map overlaid on the microscope image via a user interface. For example, the spatial transcriptomics application 132 may present a heatmap indicating the number of transcripts at various locations overlaid on the microscope image of the tissue. Additionally or alternatively, the spatial transcriptomics application 132 may indicate the cell type (e.g., by color coding the cell types) of cells within the microscope image along with the spatial transcripts heatmap.

[0048] In other implementations, the spatial transcriptomics application 132 may present a user control, such as a slider bar which allows a user to select a transparency level indicating the extent to which they can see the spatial transcripts heatmap overlaid on the microscope image. The spatial transcripts heatmap may cover various portions of the microscope image33080 / IP-2888 and by adjusting the transparency, the user can see the entire microscope image when the transparency level is set to a maximum, can see the spatial transcripts heatmap entirely when the transparency level is set to a minimum, and can see both spatial transcripts heatmap features and the microscope image features they cover when the transparency level is in between the minimum and maximum. In yet other implementations, the spatial transcriptomics application 132 may present estimated cell contours of the tissue from the microscope image on the spatial transcriptomics heatmap. For example, the spatial transcriptomics application 132 may estimate cell borders by identifying cell nuclei and expanding the contours of the cell nuclei by a threshold distance to estimate the cell borders. In another example, the spatial transcriptomics application 132 may identify the cell borders directly from the microscope image.

[0049] In this manner, users can identify which transcript corresponds to which cell, determine cell types based on corresponding transcripts, assess transcript density within cells, identify elevated gene expression, and analyze cell-to-cell interactions. The client device 130 may also perform this analysis and provide an indication of the transcript density within cells, elevated gene expression in particular cells, cell-to-cell interactions, etc. for display to the user.

[0050] While the hybrid alignment engine 122 is shown as part of the server device 120, the functionality of the hybrid alignment engine 122 may be included in the client device 130, another computing device, or any suitable combination of these. For example, some portions of the hybrid alignment engine 122 may be included in a client device 130 or local computing device while other portions of the hybrid alignment engine 122 may be included in a server device 120 or remote computing device.

[0051] More specifically, a local computing device communicatively coupled to the microscope 140 (e.g., a laptop, a desktop computer, a tablet computer, etc.) may stitch FOVs of the microscope image together and register the stitched FOVs to a common coordinate system using detected fiducials. A local computing device communicatively coupled to the sequencing device 1 10 (e.g., a laptop, a desktop computer, a tablet computer, etc.) may stitch sequencing tiles for the spatial transcriptomics map together and register the stitched sequencing tiles to the common coordinate system using detected fiducials. Then the local computing devices may provide the stitched microscope image and the stitched spatial transcriptomics map to the server device 120. The server device 120 may then perform the feature-based alignment.

[0052] Fig. 2 depicts an example microscope image 200 of a tissue sample 202 with fiducials 204 side-by-side with an example spatial transcriptomics map 250 for the sample tissue 252 with the same fiducials 254. The microscope image 200 is a 2D image of a 3D tissue structure33080 / IP-2888 from a top-down perspective. This snapshot flattens the 3D structure of the tissue into a 2D plane for visualization and analysis. The depth and complexity of the tissue's structure may be inferred from the arrangement and appearance of cells and other components within the plane of the microscope image 200.

[0053] The spatial transcriptomics map 250 may include a heat map of the transcripts - the clusters of RNA molecules produced from DNA within the cells - at various locations within the tissue sample 254. The spatial transcriptomics map 250 is generated by analyzing the tissue 254 for the presence of RNA sequences and then displaying the density of these sequences across the sample 254. The spatial transcriptomics heat map 250 may use colors to indicate the concentration of transcripts, with warmer colors typically representing higher concentrations.

[0054] In some implementations, the microscope image 200 and spatial transcriptomics map 250 are received by the server device 120 from the microscope 140 and the sequencing device 110, respectively. The server device 120 may also obtain indications of the known, physical locations of the fiducials 204, 254. For example, the server device 120 may receive an indication of the spacing between the fiducials (e.g., 5 pm horizontally and vertically). In other implementations, the server device 120 may receive physical coordinate locations of each fiducial (e.g., 10 pm, 35 pm). In other implementations, the server device 120 may receive FOVs of the microscope image 200 and / or sequencing tiles for the spatial transcriptomics map 250.

[0055] Fig. 3 depicts an example FOV 302 of the microscope image 200 registered to a global coordinate system 300. The global coordinate system 300 is depicted as a grid with the fiducials 204 from the microscope image 200 at their known, physical locations.

[0056] In some implementations, the hybrid alignment engine 122 registers the FOV 302 to the global coordinate system 300 by transforming the locations in the FOV 302 into physical locations and stitching the FOV 302 to adjacent FOVs 302 within the microscope image using detected fiducials 204 which overlap in adjacent FOVs 302. As mentioned above, the hybrid alignment engine 122 may transform the locations in the FOV 302 into physical locations by calculating a transform and generating an affine transformation matrix for the FOV 302 using correspondences between the detected fiducial locations in the FOV 302 and their known physical locations. Using these correspondences, the hybrid alignment engine 122 solves for the parameters of the affine transformation, which includes translation, rotation, scaling, and shearing factors. Then the hybrid alignment engine 122 applies the affine transformation matrix to the FOV 302 to adjust the position, orientation, and / or scale of the FOV 302. By correlating33080 / IP-2888 the pixel locations of these fiducials to their known physical locations, the hybrid alignment engine 122 can accurately map the FOV 302 from its pixel-based coordinate system to a physical coordinate system that aligns with the global coordinate system 300.

[0057] In some implementations, each fiducial has a known, global location within the microscope image 200. In this manner, the FOV 302 can be mapped to a particular cell 304 (e.g., a row and column) within a grid in the global coordinate system 300 based on the known, global locations of the fiducials within the FOV 302.

[0058] In other implementations, the fiducials have known, local locations within the FOV 302. In yet other implementations, the hybrid alignment engine 122 obtains a known spacing between the fiducials which can be used to derive local locations of the fiducials within the FOV 302. The FOV 302 may also have an identifier such as an FOV ID which can be used to obtain information regarding the spatial relationship of the FOV 302 within the microscope image 200. For example, the FOV ID may indicate whether the FOV 302 is situated in the top left cell of the global coordinate system 300, directly below the top left cell, directly to the right of the top left cell, or in another specific location.

[0059] The hybrid alignment engine 122 may then stitch the FOV 302 to adjacent FOVs to the cell 304 where the FOV 302 is located. As mentioned above, the FOV 302 includes at least some overlap with adjacent FOVs. In this manner, the hybrid alignment engine 122 may identify common features within the adjacent FOVs and assign them the same or a similar physical location in the global coordinate system 300. For example, the hybrid alignment engine 122 may identify the same fiducials on the right side of one FOV and on the left side of an adjacent FOV. The hybrid alignment engine 122 may assign these fiducials the same physical location in the global coordinate system 300. More specifically, if the FOV 302 includes fiducials having known, local locations, and a first fiducial has local location (2 pm, 50 pm) which matches with or is similar to a second fiducial in an adjacent FOV having global location (300 pm, 320 pm) after stitching, the FOV 302 may be stitched to the adjacent FOV such that the first fiducial has global location (300 pm, 320 pm) or a similar location in the global coordinate system 300. In another example, the hybrid alignment engine 122 may stitch the FOVs based on common image features, such as cells, nuclei, etc. within the adjacent FOVs.

[0060] In any event, the hybrid alignment engine 122 may stitch each of the FOVs 302 to each other to register the microscope image 200 to the global coordinate system 300. The hybrid alignment engine 122 may perform a similar process for registering the spatial transcript map 250 to the global coordinate system 300. For example, the hybrid alignment engine 12233080 / IP-2888 may transform the locations in each sequencing tile into physical locations and stitch the sequencing tiles to adjacent sequencing tiles within the spatial transcriptomics map using detected fiducials which overlap in adjacent sequencing tiles.

[0061] The fiducials may have known, local locations within the sequencing tile. In yet other implementations, the hybrid alignment engine 122 obtains a known spacing between the fiducials which can be used to derive local locations of the fiducials within the sequencing tile. The sequencing tile may also have an identifier such as a tile ID which can be used to obtain information regarding the spatial relationship of the sequencing tile within the spatial transcriptomics map 250. For example, the tile ID may indicate whether the sequencing tile is the top left sequencing tile in the global coordinate system 300, directly below the top left sequencing tile, directly to the right of the top left sequencing tile, or in another specific location.

[0062] The hybrid alignment engine 122 may then stitch the sequencing tile to adjacent sequencing tiles. As mentioned above, the sequencing tile includes at least some overlap with adjacent sequencing tiles. In this manner, the hybrid alignment engine 122 may identify common features within the adjacent sequencing tiles and assign them the same physical location in the global coordinate system 300. For example, the hybrid alignment engine 122 may identify the same fiducials on the right side of one sequencing tile and on the left side of an adjacent sequencing tile. The hybrid alignment engine 122 may assign these fiducials the same or a similar physical location in the global coordinate system 300.

[0063] Fig. 4 is a block diagram 400 of an example fiducial-based alignment for a microscope image 200 and a spatial transcriptomics map 250 for tissue sample 3 of multiple tissue samples sequenced by the sequencing device 1 10. At block 402, the microscope 140 images a flow cell including tissue sample 3 and fiducials to generate a microscope image 200. At block 404, the sequencing device 110 images the flow cell having the same fiducials after nucleotide fragments from tissue sample 3 have bound to oligonucleotide barcode sequences to generate a spatial transcriptomics map 250 for tissue sample 3. Then at block 406, the server device 120 receives the microscope image 200 and the spatial transcriptomics map 250 and registers them to a common coordinate system based on the known locations of detected fiducials in the microscope image 200 and the spatial transcriptomics map 250. As shown in Fig. 4, the spatial transcriptomics map 250 is overlaid on the microscope image 200 at block 406.

[0064] Fig. 5 is a combined block and flow diagram of an example fiducial-based alignment method 500 for a microscope image 200 and a spatial transcriptomics map 250. The method33080 / IP-2888500 may be executed by the server device 120, and more specifically, the hybrid alignment engine 122.

[0065] At block 502, the server device 120 obtains a microscope image 200 of a tissue sample, which includes fiducials. Then the server device 120 detects the fiducials within the microscope image 200 (e.g., using image processing techniques such as thresholding, edge detection, and pattern recognition) (block 504) and includes the fiducial locations within the microscope image (block 506). Next, the server device 120 calculates and applies an affine transformation to the microscope image 200 based on the known physical locations of the detected fiducials to register the microscope image to a common coordinate system (block 508). This involves calculating and applying an affine transformation matrix to transform the locations of the microscope image 200 to physical locations. The affine transformation adjusts the scale, rotation, and position of the microscope image 200 to match the real-world coordinates, ensuring that the image aligns with the spatial transcriptomics map 250.

[0066] Additionally, the server device 120 obtains sequencing image tiles from the flow cell for generating the spatial transcriptomics map 250 (block 510). These tiles are small, segmented portions of the overall spatial transcriptomics map 250. Then the server device 120 detects the fiducials within the sequencing tiles (e.g., using image processing techniques such as thresholding, edge detection, and pattern recognition) (block 512) and includes the fiducial locations within the spatial transcriptomics map (block 514).

[0067] After detecting the fiducial locations in each tile, the server device 120 calculates and applies an affine transformation to the spatial transcriptomics map 250 based on the known physical locations of the detected fiducials to register the spatial transcriptomics map 250 to the common coordinate system (block 508). This involves calculating and applying an affine transformation matrix for each tile, and transforming the locations in each tile to physical locations based on the known, physical locations of the fiducials. This transformation aligns the tiles with the real-world coordinates, ensuring that they can be accurately overlaid on the microscope image 200. In some implementations, the server device 120 registers both the microscope image 200 and the spatial transcriptomics map 250 to a common coordinate system. In other implementations, the server device 120 registers the microscope image 200 to a coordinate system for the spatial transcriptomics map 250. For example, the server device 120 may calculate and apply an affine transformation to the microscope image 200 based on the difference between the fiducial locations in the microscope image 200 and the spatial transcriptomics map 250. In yet other implementations, the server device 120 registers the33080 / IP-2888 spatial transcriptomics map 250 to a coordinate system for the microscope image 200. For example, the server device 120 may calculate and apply an affine transformation to the spatial transcriptomics map 250 based on the difference between the fiducial locations in the microscope image 200 and the spatial transcriptomics map 250.

[0068] At block 516, the server device 120 overlays the transformed spatial transcriptomics map 250 on the transformed microscope image 200 according to their registration within the common coordinate system.

[0069] As mentioned above, fiducial-based alignment may result in local or non-linear errors that may arise from various processes including for example, stitching artifacts or diffusion processes. To address these issues, the hybrid alignment engine 122 may combine the fiducial-based alignment with a feature-based alignment. More specifically, after performing the fiducial-based alignment to register the microscope image 200 and the spatial transcriptomics map 250 to a common coordinate system, the hybrid alignment engine 122 may determine whether to align at least a portion of the spatial transcriptomics map 250 with at least a portion of the microscope image 200 using a feature-based alignment. As used herein, aligning the spatial transcriptomics map 250 with the microscope image 200 may including shifting, rotating, translating, scaling, or skewing features in the spatial transcriptomics map 250 toward features in the microscope image 200 or vice versa.

[0070] To determine whether to align at least a portion of the spatial transcriptomics map 250 with at least a portion of the microscope image 200, the hybrid alignment engine 122 may identify candidate regions within the common coordinate system (e.g., quadrants of the common coordinate system, 10 pm x 10 pm regions, etc.). Then within each candidate region, the hybrid alignment engine 122 may compare the microscope image 200 to the spatial transcriptomics map 250 to identify inaccuracies. As one method of comparison, the hybrid alignment engine 122 may perform a correlation between the microscope image 200 and the spatial transcriptomics map 250. For example, the hybrid alignment engine 122 may determine correlation scores between the portion of the microscope image 200 and spatial transcriptomics map 250 within the region and a correlation distribution around the peak correlation score to detect alignment accuracy. Then for regions with low correlation scores or noisy distributions, the hybrid alignment engine 122 may determine that the candidate region is misaligned. The hybrid alignment engine 122 may perform a feature-based alignment in these misaligned candidate regions. The hybrid alignment engine 122 may then adjust the coordinates for the microscope image 200 and / or the spatial transcriptomics map 250 using the feature-based33080 / IP-2888 alignment. In some implementations, the hybrid alignment engine 122 may adjust the coordinates for the microscope image 200 and / or the spatial transcriptomics map 250 using the correlation scores. For example, the hybrid alignment engine 122 may shift the coordinates for the microscope image 200 and / or the spatial transcriptomics map 250 based on a peak shift in the correlation.

[0071] In another example, the hybrid alignment engine 122 may apply Fast Fourier Transforms (FFTs) to the microscope image 200 and the spatial transcriptomics map 250 to filter frequency components from the images 200, 250 and to maximize the correlation signal between the images 200, 250 while minimizing noise. More specifically, the hybrid alignment engine 122 may apply an FFT-based correlation between the portions of the microscope image 200 and the spatial transcriptomics map 250. The hybrid alignment engine 122 may determine an FFT peak shift between the portions of the images 200, 250 to determine that the candidate region is misaligned.

[0072] In a first example method, the hybrid alignment engine 122 may separate the fiducial aligned microscope image 200 and the fiducial aligned spatial transcriptomics map 250 into 500 pm by 500 pm windows. The hybrid alignment engine 122 may increase the window for the microscope image 200 by 20 pm on each side. Then the hybrid alignment engine 122 may analyze the spatial transcriptomics map 250 including each of the transcripts to determine whether the spatial transcriptomics map 250 includes an average of at least one transcript per pixel. If the spatial transcriptomics map 250 includes an average of at least one transcript per pixel, the hybrid alignment engine 122 performs an FFT-based correlation between the microscope image 200 and the spatial transcriptomics map 250 in the 500 pm by 500 pm windows. Then the hybrid alignment engine 122 may determine the FFT peak shift for each window within a -20 pm to +20 pm range in the x and y axes. In an example experiment applying this method to a tissue sample, most of the FFT peaks have an x-axis shift of 2~4 pm while the y-axis shift is more broadly distributed. In some windows, there is no clear FFT peak while in others the FFT peak is clear, but the distribution is large.

[0073] In a second example method, the hybrid alignment engine 122 may separate the fiducial aligned microscope image 200 and the fiducial aligned spatial transcriptomics map 250 into 500 pm by 500 pm windows. The hybrid alignment engine 122 may increase the window for the microscope image 200 by 20 pm on each side. However, unlike the first example method, the hybrid alignment engine 122 may only analyze nuclear transcripts in the spatial transcriptomics map 250 and filter out transcripts which are not likely to have originated in a cell33080 / IP-2888 nucleus. Then the hybrid alignment engine 122 may analyze the filtered spatial transcriptomics map 250 to determine whether the filtered spatial transcriptomics map 250 includes an average of at least one 200 nuclear transcripts per window. If the filtered spatial transcriptomics map 250 includes an average of at least 200 nuclear transcripts per window, the hybrid alignment engine 122 performs an FFT-based correlation between the microscope image 200 and the spatial transcriptomics map 250 in the 500 pm by 500 pm windows. Then the hybrid alignment engine 122 may determine the FFT peak shift for each window within a -20 pm to +20 pm range in the x and y axes. In an example experiment applying this method to a tissue sample, the FFT peak shifts for most of the windows are small. Also, the FFT peaks appear to be sharper with smaller distributions than the FFT peaks using all of the transcripts in the spatial transcriptomics map 250.

[0074] In a third example method, the hybrid alignment engine 122 may separate the fiducial aligned microscope image 200 and the fiducial aligned spatial transcriptomics map 250 into 500 pm by 500 pm windows. The hybrid alignment engine 122 may increase the window for the microscope image 200 by 20 pm on each side. However, unlike the first example method, the hybrid alignment engine 122 may only analyze a subset of selected transcripts (e.g., transcripts corresponding to a particular gene or genes) in the spatial transcriptomics map 250 and filter out transcripts which are not in the selected subset. Then the hybrid alignment engine 122 may analyze the filtered spatial transcriptomics map 250 to determine whether the filtered spatial transcriptomics map 250 includes an average of at least one 1000 selected transcripts per window. If the filtered spatial transcriptomics map 250 includes an average of at least 1000 selected transcripts per window, the hybrid alignment engine 122 performs an FFT-based correlation between the microscope image 200 and the spatial transcriptomics map 250 in the 500 pm by 500 pm windows. Then the hybrid alignment engine 122 may determine the FFT peak shift for each window within a -20 pm to +20 pm range in the x and y axes. In an example experiment applying this method to a tissue sample, the FFT peak shifts for most of the windows are large. Additionally, the FFT peaks are not as sharp as the FFT peaks using nuclear transcripts.

[0075] In the above example methods, the sharpness of the FFT peaks can be used as an approximate measure of the accuracy of the alignment. For instance, if a sharp peak is detected at some <Ax, Ay>, then a corresponding shift can be performed to align those subsections of images to improve the alignment. Alternatively, if a shift of <0,0> is considered as the fiducial-based alignment and <Ax, Ay> is considered the optimum feature-based33080 / IP-2888 alignment, a weighted average of the two estimates (based on the respective correlation scores of the two methods) could be used to compute the final alignment.

[0076] However, these are merely examples for ease of illustration only. The hybrid alignment engine 122 may compare the microscope image 200 to the spatial transcriptomics map 250 to identify inaccuracies in any suitable manner.

[0077] More specifically, for each candidate region, the hybrid alignment engine 122 may analyze the corresponding portion of the microscope image 200 and the corresponding portion of the spatial transcriptomics map 250 to identify artifacts such as a gradient, multiple reflections, double vision, etc. If the hybrid alignment engine 122 identifies an artifact in a candidate region or more than a threshold number of artifacts, the hybrid alignment engine 122 may determine to align the corresponding portion of the spatial transcriptomics map 250 with the corresponding portion of the microscope image 200 in that candidate region. In another example, for each candidate region, the hybrid alignment engine 122 may cross-correlate the corresponding portion of the microscope image 200 with the corresponding portion of the spatial transcriptomics map 250 to generate a peak correlation score and / or a correlation score standard deviation. If the peak correlation score exceeds a threshold score or the correlation score standard deviation exceeds a threshold standard deviation, the hybrid alignment engine 122 may determine to align the corresponding portion of the spatial transcriptomics map 250 with the corresponding portion of the microscope image 200 in that candidate region.

[0078] In any event, to perform a feature-based alignment for a misaligned candidate region of the common coordinate system, the hybrid alignment engine 122 may identify image features in the corresponding portion of the spatial transcriptomics map 250 and image features in the corresponding portion of the microscope image 200. The hybrid alignment engine 122 may compare the image features in the corresponding portion of the spatial transcriptomics map 250 to the image features in the corresponding portion of the microscope image 200 to identify similar image features (e.g., image features having a similar size or shape). Then the hybrid alignment engine 122 may align the corresponding portion of the spatial transcriptomics map 250 with the corresponding portion of the microscope image 200 by shifting rotating, translating, scaling, or skewing the coordinates of the portion of the spatial transcriptomics map or the microscope image so that the similar image features are at the same or similar locations. For example, the hybrid alignment engine 122 may identify a cell within the microscope image 200. More specifically, the hybrid alignment engine 122 may identify a cell nucleus and expand the contours of the cell nucleus by a threshold distance to estimate the cell borders of the cell. The33080 / IP-2888 hybrid alignment engine 122 may also identify a dense transcript region within the spatial transcriptomics map 250 having a matching or similar shape to the cell. Then the hybrid alignment engine 122 may align the cell within the microscope image with the dense transcript region within the spatial transcript map.

[0079] For example, the hybrid alignment engine 122 may align a cell within the microscope image with a dense transcript region within the spatial transcript map by determining the centroid of the cell and the centroid of the dense transcript region (e.g., using Gaussian distribution fitting). Then the hybrid alignment engine 122 may align the centroid of the cell within the microscope image with the centroid of the dense transcript region within the spatial transcript map.

[0080] In another example, the hybrid alignment engine 122 may align a cell within the microscope image with a dense transcript region within the spatial transcript map by identifying a cell nucleus of a cell within the microscope image and expanding the contours of the cell nucleus by a threshold distance. The hybrid alignment engine 122 may also identify the contours of the dense transcript region within the spatial transcript map. Then the hybrid alignment engine 122 may align the contours of the cell within the microscope image with the contours of the dense transcript region within the spatial transcript map. In yet another example, the hybrid alignment engine 122 may generate a pixel mask around the cell and the dense transcript region and align the pixel masks.

[0081] In another example, the hybrid alignment engine 122 may use a neural network such as a graph neural network to generate the boundaries around cells and dense transcript regions. For example, the hybrid alignment engine 122 may train the graph neural network using microscope images, labeled boundaries of cells in the microscope images, spatial transcript images, and labeled boundaries of dense transcript regions in the spatial transcript images. Then for a particular microscope image and spatial transcript image, the hybrid alignment engine 122 may apply image features of the microscope image and spatial transcript image to the graph neural network to identify the boundaries around cells and the boundaries around the dense transcript regions in the microscope image and the spatial transcript image, respectively. The hybrid alignment engine 122 may align the boundaries of the cells within the microscope image with the boundaries of the dense transcript regions within the spatial transcript map.

[0082] In some implementations, the hybrid alignment engine 122 may filter the spatial transcript map to remove transcripts which are not likely to have originated in a cell nucleus.33080 / IP-2888Then the hybrid alignment engine 122 may identify a cell nucleus within the microscope image 200 and a dense transcript region within the filtered spatial transcriptomics map 250 having a matching or similar shape to the cell nucleus. Then the hybrid alignment engine 122 may align the cell nucleus within the microscope image with the dense transcript region within the filtered spatial transcript map.

[0083] Also in some implementations, the hybrid alignment engine 122 may identify a marker gene associated with certain clusters or cell types. Then the hybrid alignment engine 122 may filter the spatial transcript map 250 to remove transcripts which are not associated with the marker gene. The hybrid alignment engine 122 may align nuclei within the microscope image with transcripts associated with the marker gene in the filtered spatial transcript map.

[0084] In some implementations, the hybrid alignment engine 122 may also mask the microscope image 200 to remove nuclei which are not associated with the marker gene. Then the hybrid alignment engine 122 may align nuclei associated with the marker gene within the masked microscope image with transcripts associated with the marker gene in the filtered spatial transcript map.

[0085] Fig. 6 illustrates an example portion of a microscope image 600 of a tissue sample side-by-side with an example portion of a spatial transcriptomics map 650 of the sample tissue registered to a common coordinate system. As shown in Fig. 6, both portions 600, 650 correspond to the same region of the common coordinate system spanning from coordinates (400 pm, 650 pm) to coordinates (600 pm, 850 pm). The spatial transcriptomics map 650 includes a heat map of transcripts, where regions with a high density of transcripts are depicted in darker colors, indicating areas of high gene expression, while regions with sparse transcript presence are shown in brighter colors, indicating lower gene expression levels.

[0086] The hybrid alignment engine 122 may outline dense transcript regions 652, 654 in the spatial transcriptomics map 650, for example by applying image processing techniques that detect areas of contrast, which correspond to the boundaries of high transcript density. Then the hybrid alignment engine 122 may identify image features 602, 604 in the microscope image 600 having similar shapes, sizes, patterns, etc., to the dense transcript regions 652, 654 in the spatial transcriptomics map 650. For example, the hybrid alignment engine 122 may determine a similarity metric between an image feature 652 in the spatial transcriptomics map 650 and an image feature in the microscope image 600. If the similarity metric exceeds a similarity threshold, the hybrid alignment engine 122 may determine that the image features are similar33080 / IP-2888 and should be aligned. As shown in Fig. 6, features 602 and 652 may be identified as similar and features 604 and 654 may be identified as similar due to their similar shapes and sizes.

[0087] To align the similar image features 602, 652, the hybrid alignment engine 122 may determine a distance between the similar image features 602, 652 in the horizontal (x) and vertical (y) directions. Then the hybrid alignment engine 122 may shift the coordinates of the portion of the microscope image 600 or the portion of the spatial transcriptomics map 650 based on the distances so that the similar image features are at the same location in the common coordinate system. In other implementations, the hybrid alignment engine 122 may shift the coordinates of the portion of the microscope image 600 toward the portion of the spatial transcriptomics map 650 by half of the distance between them and may shift the coordinates of the portion of the spatial transcriptomics map 650 toward the portion of the microscope image 600 by half of the distance between them so that they meet each other halfway.

[0088] In yet other implementations, the hybrid alignment engine 122 may assign weights to the coordinates of the portion of the microscope image 600 and the portion of the spatial transcriptomics map 650. Then the hybrid alignment engine 122 may weight the distance according to the assigned weights. The hybrid alignment engine 122 may shift the coordinates of the portion of the microscope image 600 or the portion of the spatial transcriptomics map 650 based on the weighted distances. For example, if the image feature in the spatial transcriptomics map 650 is 6 pm from the similar image feature feature in the microscope image 600, the hybrid alignment engine 122 may determine to align the spatial transcriptomics map 650 by moving the image feature 3 pm closer to the similar image feature in the microscope image 600.

[0089] The hybrid alignment engine 122 may also determine the distance between each pair of similar image features in the portion of the microscope image 600 and the spatial transcriptomics map 650. Then the hybrid alignment engine 122 may calculate an average of the distances in the x and y directions as the distance for the shift. In some implementations, the hybrid alignment engine 122 may assign different weights to different pairs of image features based on the similarity metric for the pair, such that pairs having higher similarity metrics are weighted more heavily. Then the hybrid alignment engine 122 may calculate a weighted average of the distances in the x and y directions as the distance for the shift.

[0090] In any event, the hybrid alignment engine 122 may then align the portion of the spatial transcriptomics map 650 with the portion of the microscope image 600 by shifting the coordinates of the portion of the spatial transcriptomics map 650 or the microscope image 60033080 / IP-2888 according to the determined distance for the shift. The hybrid alignment engine 122 may shift the coordinates of the portion of the spatial transcriptomics map 650 or the microscope image 600 by the determined distance or some distance which is less than the determined distance.

[0091] More generally, the hybrid alignment engine 122 may shift, rotate, translate, scale, or skew the coordinates of the portion of the microscope image 600 or the portion of the spatial transcriptomics map 650 based on a difference in location between the similar image features so that the similar image features are at the same or a similar location in the common coordinate system.

[0092] Fig. 7 illustrates an example portion of a spatial transcriptomics map overlaid on an example portion of a microscope image of a tissue sample using a fiducial-based alignment 700 side-by-side with the example portion of the spatial transcriptomics map overlaid on the example portion of the microscope image using the hybrid fiducial and feature-based alignment technique 750. As shown in Fig. 7, both portions 700, 750 correspond to the same region of the common coordinate system spanning from coordinates (180 pm, 20 pm) to coordinates (320 pm, 180 pm). The portions 700, 750 include heat maps of transcripts overlaid on the same microscope image of the tissue sample, where regions with a high density of transcripts are depicted in darker colors, indicating areas of high gene expression, while regions with sparse transcript presence are shown in brighter colors, indicating lower gene expression levels. The microscope image includes darker regions which depict cell nuclei.

[0093] The hybrid fiducial and feature-based alignment portion 750 includes a higher percentage of transcripts within cell nuclei than the fiducial-based alignment portion 700. For example, regions 702-706 in the fiducial-based alignment portion 700 correspond to regions 752-756 in the hybrid fiducial and feature-based alignment portion 750. As shown in Fig. 7, region 752 includes a higher spatial transcript density in the cell nuclei than region 702, region 754 includes a higher spatial transcript density in the cell nuclei than region 704, and region 756 includes a higher spatial transcript density in the cell nuclei than region 706. Accordingly, the hybrid fiducial and feature-based alignment appears to be a more accurate alignment as cell nuclei and immediate surrounding region (e.g. 5 urn expansion around nuclei) reflecting the area of the corresponding cell typically have higher spatial transcript densities than acellular regions where there are no nuclei detected.

[0094] Fig. 8 is a flow diagram of an example method 800 for aligning a microscope image with spatial transcriptomics data for a tissue sample. The method 800 may be implemented by a server device 120, and more specifically, a hybrid alignment engine 122.33080 / IP-2888

[0095] At block 802, the server device 120 obtains a microscope image of a tissue sample, which may include intricate details of the tissue sample such as cells, nuclei, and other cellular and subcellular structures. The server device 120 also obtains a spatial transcript map for the tissue sample, which includes a heat map of transcript density at various regions within the tissue sample (block 804).

[0096] At block 806, the server device 120 registers the spatial transcript map and the microscope image to a common coordinate system. This registration may include applying a transform based on detected locations of fiducials in both the spatial transcript map and the microscope image. For example, the server device 120 may generate affine transformation matrices which transform locations of fiducials detected within the microscope image and the spatial transcript map to their known, physical locations, ensuring that both sets of data are accurately aligned to the common coordinate system.

[0097] Following registration, the server device 120 aligns at least a portion of the spatial transcript map with at least a portion of the microscope image within the common coordinate system by performing a feature-based alignment (block 808). This involves identifying similar image features in both the microscope image and the spatial transcript map and aligning these features to be at the same or a similar location within the common coordinate system.

[0098] In some implementations, the server device 120 may also process the spatial transcript map registered within the common coordinate system and aligned with the microscope image. This processing can involve overlaying the spatial transcript map on the microscope image according to the registration and alignment and presenting the overlaid map for display to a user, for example on the client device 130. Further processing steps may include determining which transcripts in the spatial transcript map correspond to which cells in the microscope image, identifying cell types based on corresponding transcripts, identifying interactions between cells based on common gene expression, etc.

[0099] In some implementations, the server device 120 adjusts the coordinates for the spatial transcript map by aligning coordinates for a particular feature in the spatial transcript map to match or approximate the coordinates of that feature in the microscope image. This adjustment can be based on a weighted average of coordinates determined through both fiducial-based and feature-based alignments, ensuring that features such as dense transcript regions are accurately aligned with corresponding cells or nuclei in the microscope image.33080 / IP-2888

[0100] By combining fiducial-based alignment with feature-based alignment, the hybrid alignment engine effectively aligns the microscope image with the spatial transcriptomics map, leveraging the complementary properties of both methods. This alignment enables the accurate overlay of transcript data on the microscope image, providing valuable insights into the spatial distribution of gene expression within the tissue sample.

[0101] Fig. 9 is a flow diagram of an example method 900 for determining whether to align a microscope image with spatial transcriptomics data for a tissue sample. The method 900 may be implemented by a server device 120, and more specifically, a hybrid alignment engine 122.

[0102] At block 902, the server device 120 obtains a microscope image of a tissue sample, which may include intricate details of the tissue sample such as cells, nuclei, and other cellular and subcellular structures. The server device 120 also obtains a spatial transcript map for the tissue sample, which includes a heat map of transcript density at various regions within the tissue sample (block 904).

[0103] At block 906, the server device 120 registers the spatial transcript map and the microscope image to a common coordinate system. This registration may include applying a transform based on detected locations of fiducials in both the spatial transcript map and the microscope image. For example, the server device 120 may generate affine transformation matrices which transform locations of fiducials detected within the microscope image and the spatial transcript map to their known, physical locations, ensuring that both sets of data are accurately aligned to the common coordinate system.

[0104] After registration, the server device 120 compares a portion of the spatial transcript map to a portion of the microscope image within a region of the common coordinate system (block 908). For example, the hybrid alignment engine 122 may identify candidate regions within the common coordinate system (e.g., quadrants of the common coordinate system, 10 pm x 10 pm regions, etc.). Then within each candidate region, the hybrid alignment engine 122 may compare the microscope image to the spatial transcriptomics map to identify inaccuracies. For example, the hybrid alignment engine 122 may determine correlation scores between the portion of the microscope image and spatial transcriptomics map within the region and a correlation distribution around the peak correlation score to detect alignment accuracy. More specifically, for each candidate region, the hybrid alignment engine 122 may analyze the corresponding portion of the microscope image and the corresponding portion of the spatial transcriptomics map to identify artifacts such as a gradient, multiple reflections, double vision, etc. In another example, for each candidate region, the hybrid alignment engine 122 may cross-33080 / IP-2888 correlate the corresponding portion of the microscope image 200 with the corresponding portion of the spatial transcriptom ics map 250 to generate a peak correlation score and / or a correlation score standard deviation.

[0105] Based on this comparison, the server device 120 determines whether to align the portion of the spatial transcript map with the portion of the microscope image (block 910). This decision is made by analyzing the alignment accuracy and identifying any discrepancies that may require further alignment. For example, if the hybrid alignment engine 122 identifies an artifact in a candidate region or more than a threshold number of artifacts, the hybrid alignment engine 122 may determine to align the corresponding portion of the spatial transcriptomics map 250 with the corresponding portion of the microscope image 200 in that candidate region. Additionally or alternatively, if the peak correlation score exceeds a threshold score or the correlation score standard deviation exceeds a threshold standard deviation, the hybrid alignment engine 122 may determine to align the corresponding portion of the spatial transcriptomics map 250 with the corresponding portion of the microscope image 200 in that candidate region.

[0106] If the comparison indicates that the alignment is accurate and no further alignment is necessary, the server device 120 processes the spatial transcript map registered within the common coordinate system. This processing may include overlaying the spatial transcript map on the microscope image according to the registration and presenting the overlaid map for display to a user, for example via the client device 130.

[0107] In the event that the comparison reveals discrepancies indicating the need for further alignment, the server device 120 aligns the portion of the spatial transcript map with the portion of the microscope image. This alignment may be performed by identifying similar image features in both the microscope image and the spatial transcript map and aligning these features to be at the same location within the common coordinate system. The alignment process may involve adjusting the coordinates for the spatial transcript map by aligning coordinates for a particular feature to match or approximate the coordinates of that feature in the microscope image.

[0108] Aspects of the techniques described in the present disclosure may include any of the following aspects, either alone or in combination:

[0109] 1 . A method for aligning a microscope image with spatial transcriptomics data for a tissue sample, the method comprising: obtaining, by one or more processors, a microscope33080 / IP-2888 image of a tissue sample; obtaining, by the one or more processors, a spatial transcript map for the tissue sample; registering, by the one or more processors, the spatial transcript map and the microscope image to a common coordinate system by applying a transform based on detected locations of fiducials in the spatial transcript map and the microscope image; and aligning, by the one or more processors, at least a portion of the spatial transcript map with at least a portion of the microscope image within the common coordinate system by performing a feature-based alignment.

[0110] 2. The method of aspect 1 , further comprising: processing, by the one or more processors, the spatial transcript map registered within the common coordinate system and aligned with the microscope image.

[0111] 3. The method of any of aspects 1 -2, wherein processing the spatial transcript map includes: overlaying, by the one or more processors, the spatial transcript map on the microscope image according to the registration and alignment; and presenting, by the one or more processors, the spatial transcript map overlaid on the microscope image for display to a user.

[0112] 4. The method of any of aspects 1 -2, wherein processing the spatial transcript map includes: determining, by the one or more processors, which transcripts in the spatial transcript map correspond to which cells in the microscope image.

[0113] 5. The method of any of aspects 1 -4, further comprising: identifying, by the one or more processors, a cell type for a cell in the microscope image based on one or more corresponding transcripts within the cell.

[0114] 6. The method of any of aspects 1 -4, further comprising: identifying, by the one or more processors, interaction between two or more cells based on common gene expression for the two or more cells according to the transcripts which correspond to the two or more cells

[0115] 7. The method of any of aspects 1 -6, wherein applying a transform based on detected locations of fiducials in the spatial transcript map and the microscope image includes: assigning a first set of known locations to a first set of fiducials within the microscope image; applying a first transform for the microscope image so that the first set of fiducials are located at the first set of known locations; assigning a second set of known locations to a second set of fiducials within the spatial transcript map; and applying a second transform for the spatial transcript map so that the second set of fiducials are located at the second set of known locations.33080 / IP-2888

[0116] 8. The method of any of aspects 1 -7, wherein aligning at least a portion of the spatial transcript map with at least a portion of the microscope image includes: identifying, by the one or more processors, an image feature within the portion of the microscope image; identifying, by the one or more processors, a similar image feature within the portion of the spatial transcript map matching the image feature within the portion of the microscope image; and aligning, by the one or more processors, the similar image features in the microscope image and spatial transcript map.

[0117] 9. The method of any of aspects 1 -8, wherein aligning similar image features includes: determining, by the one or more processors, a difference in location between the similar image features prior to alignment; and aligning, by the one or more processors, the similar image features by shifting, rotating, translating, scaling, or skewing the portion of the spatial transcript map or the portion of the microscope image within the common coordinate system based on the difference in location.

[0118] 10. The method of any of aspects 1 -9, wherein aligning similar image features includes: determining, by the one or more processors, a distance between the similar image features prior to alignment; assigning, by the one or more processors, a weight to the distance; and shifting, by the one or more processors, a location of the image feature within the portion of the spatial transcript map by the weighted distance within the common coordinate system.

[0119] 11. The method of any of aspects 1 -10, wherein aligning at least a portion of the spatial transcript map with at least a portion of the microscope image includes: after registering the spatial transcript map and the microscope image to the common coordinate system, analyzing, by the one or more processors, candidate regions within the common coordinate system to identify at least one candidate region which is misaligned; for the at least one misaligned candidate region: identifying, by the one or more processors, an image feature within the portion of the microscope image corresponding to the misaligned candidate region; identifying, by the one or more processors, a similar image feature within the portion of the spatial transcript map corresponding to the misaligned candidate region; and aligning, by the one or more processors, the misaligned candidate region by shifting, rotating, translating, scaling, or skewing the portion of the spatial transcript map or the portion of the microscope image within the common coordinate system based on a difference in location between the similar image features.

[0120] 12. The method of any of aspects 1 -11 , wherein analyzing candidate regions to identify at least one candidate region which is misaligned includes: determining, by the one or33080 / IP-2888 more processors, a peak correlation score for the at least one candidate region; determining, by the one or more processors, a correlation distribution around the peak correlation score for the at least one candidate region; and determining, by the one or more processors, that the at least one candidate region is misaligned based on the peak correlation score or the correlation distribution.

[0121] 13. The method of any of aspects 1 -12, wherein aligning at least a portion of the spatial transcript map with at least a portion of the microscope image includes: identifying, by the one or more processors, a cell within the microscope image; identifying, by the one or more processors, a dense transcript region within the spatial transcript map that matches a shape of the identified cell; and aligning, by the one or more processors, the cell within the microscope image with the dense transcript region within the spatial transcript map.

[0122] 14. The method of any of aspects 1 -13, wherein aligning at least a portion of the spatial transcript map with at least a portion of the microscope image includes: filtering, by the one or more processors, the spatial transcript map to remove transcripts which are not likely to have originated in a cell nucleus; identifying, by the one or more processors, a cell nucleus within the microscope image; identifying, by the one or more processors, a dense transcript region within the filtered spatial transcript map that matches a shape of the identified cell nucleus; and aligning, by the one or more processors, the cell nucleus within the microscope image with the dense transcript region within the filtered spatial transcript map.

[0123] 15. The method of any of aspects 1 -14, wherein aligning at least a portion of the spatial transcript map with at least a portion of the microscope image includes: identifying, by the one or more processors, a marker gene; filtering, by the one or more processors, the spatial transcript map to remove transcripts which are not associated with the marker gene; and aligning, by the one or more processors, nuclei within the microscope image with transcripts associated with the marker gene in the filtered spatial transcript map.

[0124] 16. The method of aspect 15, further comprising: masking, by the one or more processors, the microscope image to remove nuclei which are not associated with the marker gene; and aligning, by the one or more processors, nuclei associated with the marker gene within the masked microscope image with transcripts associated with the marker gene in the filtered spatial transcript map.33080 / IP-2888

[0125] 17. The method of any of aspects 1 -16, wherein the spatial transcript map for the tissue sample includes a heat map of transcript density at a plurality of regions within the tissue sample.

[0126] 18. A method for aligning a microscope image with spatial transcriptomics data for a tissue sample, the method comprising: obtaining, by one or more processors, a microscope image of a tissue sample; obtaining, by the one or more processors, a spatial transcript map for the tissue sample; registering, by the one or more processors, the spatial transcript map and the microscope image to a common coordinate system by applying a transform based on detected locations of fiducials in the spatial transcript map and the microscope image; comparing, by the one or more processors, a portion of the spatial transcript map to a portion of the microscope image within a region of the common coordinate system; and determining, by the one or more processors, whether to align the portion of the spatial transcript map with the portion of the microscope image based on the comparison.

[0127] 19. The method of aspect 18, further comprising: in response to determining not to align the portion of the spatial transcript map with the portion of the microscope image, processing, by the one or more processors, the spatial transcript map registered within the common coordinate system.

[0128] 20. The method of any of aspects 18-19, wherein processing the spatial transcript map includes: overlaying, by the one or more processors, the spatial transcript map on the microscope image according to the registration; and presenting, by the one or more processors, the spatial transcript map overlaid on the microscope image for display to a user.

[0129] 21 . The method of any of aspects 18-19, wherein processing the spatial transcript map includes: determining, by the one or more processors, which transcripts in the spatial transcript map correspond to which cells in the microscope image.

[0130] 22. The method of any of aspects 18-21 , further comprising: identifying, by the one or more processors, a cell type for a cell in the microscope image based on one or more corresponding transcripts within the cell.ADDITIONAL CONSIDERATIONS

[0131] Although the disclosure herein sets forth a detailed description of numerous different implementations, it should be understood that the legal scope of the description is defined by the words of the claims set forth at the end of this patent and equivalents. The detailed description is to be construed as exemplary only and does not describe every possible33080 / IP-2888 implementation since describing every possible implementation would be impractical.Numerous alternative implementations may be implemented, using either current technology or technology developed after the filing date of this patent, which would still fall within the scope of the claims.

[0132] The following additional considerations apply to the foregoing discussion. Throughout this specification, plural instances may implement components, operations, or structures described as a single instance. Although individual operations of one or more methods are illustrated and described as separate operations, one or more of the individual operations may be performed concurrently, and nothing requires that the operations be performed in the order illustrated. Structures and functionality presented as separate components in example configurations may be implemented as a combined structure or component. Similarly, structures and functionality presented as a single component may be implemented as separate components. These and other variations, modifications, additions, and improvements fall within the scope of the subject matter herein.

[0133] Additionally, certain implementations are described herein as including logic or a number of routines, subroutines, applications, or instructions. These may constitute either software (e.g., code embodied on a machine-readable medium or in a transmission signal) or hardware. In hardware, the routines, etc., are tangible units capable of performing certain operations and may be configured or arranged in a certain manner. In example implementations, one or more computer systems (e.g., a standalone, client or server computer system) or one or more hardware modules of a computer system (e.g., a processor or a group of processors) may be configured by software (e.g., an application or application portion) as a hardware module that operates to perform certain operations as described herein.

[0134] The various operations of example methods described herein may be performed, at least partially, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors may constitute processor-implemented modules that operate to perform one or more operations or functions. The modules referred to herein may, in some example implementations, comprise processor-implemented modules.

[0135] Similarly, the methods or routines described herein may be at least partially processor implemented. For example, at least some of the operations of a method may be performed by one or more processors or processor-implemented hardware modules. The performance of certain of the operations may be distributed among the one or more processors, not only33080 / IP-2888 residing within a single machine, but deployed across a number of machines. In some example implementations, the processor or processors may be located in a single location, while in other implementations the processors may be distributed across a number of locations.

[0136] The performance of certain of the operations may be distributed among the one or more processors, not only residing within a single machine, but deployed across a number of machines. In some example implementations, the one or more processors or processor- implemented modules may be located in a single geographic location (e.g., within a home environment, an office environment, or a server farm). In other implementations, the one or more processors or processor-implemented modules may be distributed across a number of geographic locations.

[0137] This detailed description is to be construed as exemplary only and does not describe every possible implementation, as describing every possible implementation would be impractical, if not impossible. A person of ordinary skill in the art may implement numerous alternate implementations, using either current technology or technology developed after the filing date of this application.

[0138] Those of ordinary skill in the art will recognize that a wide variety of modifications, alterations, and combinations may be made with respect to the above described implementations without departing from the scope of the invention, and that such modifications, alterations, and combinations are to be viewed as being within the ambit of the inventive concept.

[0139] The patent claims at the end of this patent application are not intended to be construed under 35 U.S.C. § 1 12(f) unless traditional means-plus-function language is expressly recited, such as “means for” or “step for” language being explicitly recited in the claim(s). The systems and methods described herein are directed to an improvement to computer functionality and improve the functioning of conventional computers.

Claims

33080 / IP-2888CLAIMSWhat is claimed is:1 . A method for aligning a microscope image with spatial transcriptomics data for a tissue sample, the method comprising: obtaining, by one or more processors, a microscope image of a tissue sample; obtaining, by the one or more processors, a spatial transcript map for the tissue sample; registering, by the one or more processors, the spatial transcript map and the microscope image to a common coordinate system by applying a transform based on detected locations of fiducials in the spatial transcript map and the microscope image; and aligning, by the one or more processors, at least a portion of the spatial transcript map with at least a portion of the microscope image within the common coordinate system by performing a feature-based alignment.

2. The method of claim 1 , further comprising: processing, by the one or more processors, the spatial transcript map registered within the common coordinate system and aligned with the microscope image.

3. The method of claim 2, wherein processing the spatial transcript map includes: overlaying, by the one or more processors, the spatial transcript map on the microscope image according to the registration and alignment; and presenting, by the one or more processors, the spatial transcript map overlaid on the microscope image for display to a user.

4. The method of claim 2, wherein processing the spatial transcript map includes: determining, by the one or more processors, which transcripts in the spatial transcript map correspond to which cells in the microscope image.

5. The method of claim 4, further comprising: identifying, by the one or more processors, a cell type for a cell in the microscope image based on one or more corresponding transcripts within the cell.33080 / IP-28886. The method of claim 4, further comprising: identifying, by the one or more processors, interaction between two or more cells based on common gene expression for the two or more cells according to the transcripts which correspond to the two or more cells.

7. The method of claim 1 , wherein applying a transform based on detected locations of fiducials in the spatial transcript map and the microscope image includes: assigning a first set of known locations to a first set of fiducials within the microscope image; applying a first transform for the microscope image so that the first set of fiducials are located at the first set of known locations; assigning a second set of known locations to a second set of fiducials within the spatial transcript map; and applying a second transform for the spatial transcript map so that the second set of fiducials are located at the second set of known locations.

8. The method of claim 1 , wherein aligning at least a portion of the spatial transcript map with at least a portion of the microscope image includes: identifying, by the one or more processors, an image feature within the portion of the microscope image; identifying, by the one or more processors, a similar image feature within the portion of the spatial transcript map approximating the image feature within the portion of the microscope image; and aligning, by the one or more processors, the similar image features in the microscope image and spatial transcript map.

9. The method of claim 8, wherein aligning similar image features includes: determining, by the one or more processors, a difference in location between the similar image features prior to alignment; and aligning, by the one or more processors, the similar image features by shifting, rotating, translating, scaling, or skewing the portion of the spatial transcript map or the portion of the microscope image within the common coordinate system based on the difference in location.33080 / IP-288810. The method of claim 8, wherein aligning similar image features includes: determining, by the one or more processors, a distance between the similar image features prior to alignment; assigning, by the one or more processors, a weight to the distance; and shifting, by the one or more processors, the portion of the spatial transcript map or the portion of the microscope image within the common coordinate system by the weighted distance.11 . The method of claim 1 , wherein aligning at least a portion of the spatial transcript map with at least a portion of the microscope image includes: after registering the spatial transcript map and the microscope image to the common coordinate system, analyzing, by the one or more processors, candidate regions within the common coordinate system to identify at least one candidate region which is misaligned; for the at least one misaligned candidate region: identifying, by the one or more processors, an image feature within the portion of the microscope image corresponding to the misaligned candidate region; identifying, by the one or more processors, a similar image feature within the portion of the spatial transcript map corresponding to the misaligned candidate region; and aligning, by the one or more processors, the misaligned candidate region by shifting, rotating, translating, scaling, or skewing the portion of the spatial transcript map or the portion of the microscope image within the common coordinate system based on a difference in location between the similar image features.

12. The method of claim 11 , wherein analyzing candidate regions to identify at least one candidate region which is misaligned includes: determining, by the one or more processors, a peak correlation score for the at least one candidate region; determining, by the one or more processors, a correlation distribution around the peak correlation score for the at least one candidate region; and determining, by the one or more processors, that the at least one candidate region is misaligned based on the peak correlation score or the correlation distribution.33080 / IP-288813. The method of claim 1 , wherein aligning at least a portion of the spatial transcript map with at least a portion of the microscope image includes: identifying, by the one or more processors, a cell within the microscope image; identifying, by the one or more processors, a dense transcript region within the spatial transcript map that matches a shape of the identified cell; and aligning, by the one or more processors, the cell within the microscope image with the dense transcript region within the spatial transcript map.

14. The method of claim 1 , wherein aligning at least a portion of the spatial transcript map with at least a portion of the microscope image includes: filtering, by the one or more processors, the spatial transcript map to remove transcripts which are not likely to have originated in a cell nucleus; identifying, by the one or more processors, a cell nucleus within the microscope image; identifying, by the one or more processors, a dense transcript region within the filtered spatial transcript map having a similar shape as the identified cell nucleus; and aligning, by the one or more processors, the cell nucleus within the microscope image with the dense transcript region within the filtered spatial transcript map.

15. The method of claim 1 , wherein aligning at least a portion of the spatial transcript map with at least a portion of the microscope image includes: identifying, by the one or more processors, a marker gene; filtering, by the one or more processors, the spatial transcript map to remove transcripts which are not associated with the marker gene; and aligning, by the one or more processors, nuclei within the microscope image with transcripts associated with the marker gene in the filtered spatial transcript map.

16. The method of claim 15, further comprising: masking, by the one or more processors, the microscope image to remove nuclei which are not associated with the marker gene; and aligning, by the one or more processors, nuclei associated with the marker gene within the masked microscope image with transcripts associated with the marker gene in the filtered spatial transcript map.33080 / IP-288817. The method of claim 1 , wherein the spatial transcript map for the tissue sample includes a heat map of transcript density at a plurality of regions within the tissue sample.

18. A method for aligning a microscope image with spatial transcriptomics data for a tissue sample, the method comprising: obtaining, by one or more processors, a microscope image of a tissue sample; obtaining, by the one or more processors, a spatial transcript map for the tissue sample; registering, by the one or more processors, the spatial transcript map and the microscope image to a common coordinate system by applying a transform based on detected locations of fiducials in the spatial transcript map and the microscope image; comparing, by the one or more processors, a portion of the spatial transcript map to a portion of the microscope image within a region of the common coordinate system; and determining, by the one or more processors, whether to align the portion of the spatial transcript map with the portion of the microscope image based on the comparison.

19. The method of claim 18, further comprising: in response to determining not to align the portion of the spatial transcript map with the portion of the microscope image, processing, by the one or more processors, the spatial transcript map registered within the common coordinate system.

20. The method of claim 19, wherein processing the spatial transcript map includes: overlaying, by the one or more processors, the spatial transcript map on the microscope image according to the registration; and presenting, by the one or more processors, the spatial transcript map overlaid on the microscope image for display to a user.21 . The method of claim 19, wherein processing the spatial transcript map includes: determining, by the one or more processors, which transcripts in the spatial transcript map correspond to which cells in the microscope image.

22. The method of claim 21 , further comprising: identifying, by the one or more processors, a cell type for a cell in the microscope image based on one or more corresponding transcripts within the cell.

Images