Single-cell spatial annotation method fusing images and sequencing data

Abstract

This invention discloses a single-cell spatial annotation method that integrates image and sequencing data. It obtains candidate cue point locations, cell nucleus instance segmentation masks, and cell nucleus type probability vectors by processing full-field H&E stained tissue images. It also obtains a preprocessed spatial transcriptome raw sequencing matrix and corresponding cell type labels by processing the raw spatial transcriptome sequencing matrix. Each capture site and its corresponding integrated data structure are constructed. Based on rigid constraints on the number of cells allocated to each type within each capture site and main type quota constraints, the main type annotation is written. For cells in the intercapture zone, one or more neighboring capture sites with existing annotation results are found. The main type and cell subtype of the intercapture zone cells are inferred through morphological feature similarity calculation, thus obtaining the annotation result. This solves the problem that existing technologies cannot determine the exact type of each cell in ST-sequencing tissue.

Description

Technical Field

[0001] This invention belongs to the field of intelligent analysis technology that integrates digital pathology and spatial omics. Specifically, it involves a single-cell spatial annotation method that integrates image and sequencing data. It is a method for cell nuclear segmentation and single-cell resolution cell annotation based on an image segmentation model, which is applied to digital pathological image and spatial transcriptomics analysis. Background Technology

[0002] Spatial transcriptomics (ST) technology can record gene expression in the spatial context of tissues. However, most current ST sequencing platforms (such as 10xVisium, DBiT-seq, and Decoder-seq) are still multi-cell resolution, meaning that each spatial sequencing capture site (hereinafter referred to as "spot") often contains multiple cells simultaneously, and the measured transcriptional expression signal is a mixture of multiple cells. This multi-cell resolution characteristic masks the expression heterogeneity between single cells to some extent, limiting the accurate analysis of cell type composition, intercellular interactions, and fine spatial structures in complex tissues, and making it difficult to meet the research needs of single-cell spatial transcriptomics analysis.

[0003] Spatial transcriptomics can usually provide tissue images and sequencing information simultaneously. In recent years, some new methods have attempted to combine tissue images with sequencing information and use tissue slice images to estimate the number of cells at each capture site. However, these methods do not make full use of the morphological features of cell nuclei in the images to distinguish cell types, and still cannot determine the exact type of each cell within the capture site, which is prone to large errors.

[0004] The following lists some existing technologies, including cell nucleus segmentation techniques for tissue images and deconvolution techniques based on sequencing data:

[0005] 1. H&E staining image kernel segmentation technique (1) MaskR-CNN: Detection first, segmentation later, relying on a large number of pixel-level annotations; insufficient splitting of overlapping kernels.

[0006] (2) HoVer-Net: Simultaneously predicts kernel segmentation and classification, using horizontal / vertical offset fields to separate adjacent kernels; weak generalization ability across staining patterns.

[0007] (3) Cellpose3: Predicts flow field through cell contours, relies on edge gradients, and is prone to missed detection in weakly stained or low-contrast areas.

[0008] (4) SegmentAnythingModel(SAM): Zero-sample segmentation of any object, but automatic mask generation requires hundreds or even thousands of cue points, which is inefficient and has many false detections.

[0009] 2. Deconvolution techniques based on sequencing data (1) Cell2location (Kleshchevnikov et al., Nat Biotechnol, 2022): Using a Bayesian hierarchical model, with scRNA-seq reference data as a priori, the proportion of cell types at each capture site was inferred.

[0010] (2) RCTD (Cable et al., Nat Biotechnol, 2022): Based on Poisson-Gamma regression, the abundance of each cell type within the capture site is estimated.

[0011] 3. Super-resolution interpolation representation (1) BayesSpace (Zhao et al., Nat Biotechnol, 2021): Decomposes the capture site into a smaller grid and expresses it by Gaussian process or Markov random field interpolation.

[0012] (2) XFuse (Bryan Heetal., Nat Biotechnol, 2022): Uses a deep generative model to perform fine-grained segmentation of expression within the capture site.

[0013] Limitations: Virtual pixels do not correspond to real kernel coordinates, the results require thresholding and post-processing, and morphological verification is lacking.

[0014] 4. Image and Sequencing Data Fusion Strategies (1) SpatialScope: The number of cells obtained based on H&E images is added as a regularization term to the RCTD algorithm to obtain the number of cell types at each capture site, and then allocated in a random manner; (3) Spotiphy: Combines generative models to infer the proportion of cell types at capture sites from the transcriptome and segments cell nuclei in H&E based on Stardist, but the allocation of cell types to individual cell nuclei is still random; (4) Tangram / CytoSPACE: Maps scRNA data onto the ST space and assigns single cells according to distance or random rules; Limitations: The allocation strategy is mostly random or locally greedy, which cannot guarantee global optimality; there is a lack of automatic conflict correction; and there is no completion for capture intervals.

[0015] Analysis of the defects and causes of existing technologies: First, commonly used nuclear segmentation models (MaskR-CNN, HoVer-Net, Cellpose3, etc.) often make segmentation errors or miss detections in complex H&E images (because the models are poorly adapted to changes in staining color and morphology of pathological sections). Many nuclear segmentation models cannot achieve accurate nuclear segmentation and classification at the same time. Secondly, traditional deconvolution algorithms (such as Cell2location, RCTD, etc.) can only output the proportion of cell types at the capture site level, but cannot map these types to real single cells, resulting in the inability to obtain single-cell localization and annotation (because they only use sequencing data and do not combine morphological information). Third, there is no interaction between image-based classification results and sequencing-based deconvolution results, and there is a lack of excellent spatial cell annotation methods that integrate the two types of information. Fourth, the capture site region of common multi-cell resolution spatial transcriptome sequencing platforms only accounts for about 50%-60% of the complete slice. Existing methods have difficulty annotating cells outside the sequencing capture site (hereinafter referred to as "intercapture region"), resulting in missing or erroneous single-cell atlases. Summary of the Invention

[0016] To address the technical problems existing in the background art, the present invention provides a single-cell spatial annotation method that fuses images and sequencing data, comprising the following steps: Acquire full-field H&E stained tissue images, and based on the full-field H&E stained tissue images, obtain the corresponding spatial transcriptome raw sequencing matrix, single-cell transcriptome data, and corresponding annotated cell subtype files in single cells; The full-view H&E stained tissue image was preprocessed, and the preprocessed full-view H&E stained tissue image was input into the trained image encoder to obtain the candidate cue point location and cell main class probability distribution map. Input the candidate cue point locations and cell main class probability distribution map into the fine-tuned SAM model to obtain the cell nucleus instance segmentation mask with instance ID and the cell nucleus main type probability vector corresponding to each cue point; Low-quality sequencing capture points in the original spatial transcriptome sequencing matrix were filtered out, and abnormal or noisy genes were removed. Then, normalization and transformation were performed to obtain the preprocessed spatial transcriptome sequencing matrix. Using the instance ID in the cell nucleus instance segmentation mask as an index, write the candidate prompt point position and cell nucleus main type probability vector to obtain the integrated data structure. Generate the initial main type label based on the principle of maximum probability. Under the same coordinate scale, read the shape of the capture site and determine the coverage area. Continue to determine the cell affiliation within the capture site to obtain each cell instance and corresponding integrated data structure within the capture site. The preprocessed spatial transcriptome sequencing matrix and single-cell transcriptome data are input into the deconvolution tool, which outputs the cell subtypes in the captured sites to form an abundance or composition ratio matrix. Based on the total number of cell instances within each capture site and the abundance or composition ratio matrix of each cell subtype within each capture site, a rigid constraint is calculated on the number of each cell subtype to be allocated within each capture site. Based on the rigid constraint of the number of each cell subtype to be allocated within each capture site, combined with the preset initial main type label and the mapping relationship of each cell subtype, the number of cells is summarized according to the initial main type label to obtain the main type quota constraint; Based on the primary type quota constraint, with the goal of maximizing the sum of the primary type allocation probability values ​​of all cells within the capture site, a primary type is selected for each cell and a primary type annotation is written. Based on the rigid constraint of the number of cells to be allocated for each cell subtype within each capture site, the primary type allocation results and primary type annotations are refined into subtype annotations and written to each capture site and the corresponding integrated data structure. For cells in the capture interval, one or more capture sites with existing annotation results in their spatial neighborhood are found. The main type and cell subtype of the cells in the capture interval are inferred by calculating the morphological feature similarity. The annotation results are obtained by combining the corresponding annotation cell subtype files in the single cell.

[0017] Preferably, the step of inputting the preprocessed full-view H&E stained tissue image into the trained image encoder to obtain the cell nucleus instance segmentation mask with instance ID and the cell nucleus principal type probability vector corresponding to each cue point includes the following steps: Multi-scale feature vectors are obtained by pathological multi-scale feature extraction on the preprocessed data. Based on the labeled cell nucleus mask, the centroid coordinates are obtained by calculating the geometric center. The data is then upsampled step by step by a convolutional decoder. The upsampled features are directly concatenated. The Gaussian heatmap value of each pixel is predicted by the convolutional classification head. A two-dimensional Gaussian distribution heatmap is generated at the centroid coordinate position. The two-dimensional Gaussian distribution heatmap has a value that decays from the centroid to the surrounding area. Local extreme points above the threshold are selected from the two-dimensional Gaussian distribution heatmap and regarded as candidate cue points for the cell nucleus center. The position of the candidate cue point is obtained. Simultaneously, based on multi-scale feature vectors, separate convolutional decoding and softmax calculation are performed to output a pixel-level cell principal type probability map with the same size as the image.

[0018] This invention provides a single-cell spatial annotation method that integrates image and sequencing data. It obtains candidate cue point locations, cell nucleus instance segmentation masks, and cell nucleus type probability vectors by processing full-field H&E stained tissue images. It also obtains a preprocessed spatial transcriptome raw sequencing matrix and corresponding cell type labels by processing the raw spatial transcriptome sequencing matrix. Each capture site and its corresponding integrated data structure are constructed. Based on rigid constraints on the number of cells allocated to each type within each capture site and main type quota constraints, the main type annotation is written. For cells in the intercapture zone, one or more neighboring capture sites with existing annotation results are found. The main type and cell subtype of the intercapture zone cells are inferred through morphological feature similarity calculation, thus obtaining the annotation results. This solves the problem that existing technologies cannot determine the exact type of each cell in ST-sequencing tissue. Attached Figure Description

[0019] Figure 1 A schematic diagram of the data processing flow and module architecture of the single-cell spatial annotation method that fuses image and sequencing data provided in an embodiment of the present invention; Figure 2 A detailed data processing flowchart of the single-cell spatial annotation method for fused image and sequencing data provided in the embodiments of the present invention is shown below; Figure 3 The calculation process of the H&E image analysis module and the encoder structure of the automatic prompt generator; Figure 4 The performance of cell nucleus segmentation and classification is benchmarked on multiple datasets for cell nucleus segmentation and classification. Figure 5 A schematic diagram comparing the performance of spatial cell annotation at single-cell resolution; Figure 6 This diagram illustrates the performance comparison of deconvolution results using CBI adaptive correction. Detailed Implementation

[0020] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0021] like Figure 1 and Figure 2 As shown, this embodiment of the invention provides a single-cell spatial annotation method that fuses image and sequencing data. The method includes... Figure 1 I. Cell nucleus segmentation and analysis, the specific implementation process is as follows: Figure 2 The high-precision H&E image analysis module (Module I) is shown below: Procedure 1: Precise kernel segmentation and analysis of H&E stained images: A. The image preprocessing process is as follows: Step 1: H&E staining full-field image segmentation: The entire H&E stained slide image is segmented using an overlapping patch method. A sliding window is used to select blocks across the entire field of view, cropping the image into several uniformly sized blocks for subsequent processing (in this embodiment, the block size is 256×256 pixels; the step size can be set according to image size and memory availability to ensure seamless coverage of the entire slide without missing any cells; overlapping or filling can be used for edge areas if necessary). Each block is then sequentially input into the subsequent cell nucleus segmentation model to ensure that the entire field of view of the slide is processed. This sliding window-based segmentation avoids the memory consumption issues associated with directly processing very large images and ensures that all cells in the entire slide are analyzed.

[0022] Step 2: Data Augmentation Then, random data augmentation is performed on all the obtained patches, including translation in any direction, rotation at multiple angles, and scaling and stretching to simulate different microscopic image framing and staining variations, thereby expanding the diversity of training data. These augmentation operations can increase the data diversity for model training and improve the robustness of the model to variations in different tissue sections.

[0023] Step 3: Feature Normalization: Color normalization is performed on the enhanced image patches: Addressing the differences in the three-channel color distribution of H&E stained images, color normalization methods such as the Macenko algorithm are used to adjust the color distribution of each patch to match that of the reference standard slice (adjusting the hue and intensity distribution of H&E stained images). This reduces color deviations caused by different batches of stained images and scans, thus standardizing the model input. After this preprocessing, the normalized image patch data is output as input to the H&E image analysis module, reducing the impact of staining differences between different slices and batches on subsequent analysis, thereby improving the model's generalization ability to different data sources.

[0024] B. The Prompter auto-completion generation process is as follows: The preprocessed full-view H&E stained tissue image was input into the image encoder to obtain the candidate cue point locations and cell principal type probability maps. The steps are as follows: Step 4: Pathological multi-scale feature extraction: This invention employs a deep neural network integrating a multi-scale visual Transformer structure as an image encoder to extract multi-scale features from pathological images. Specifically, using SegFormer-B3 (i.e., MiT-B3 hybrid Transformer backbone) as the backbone network, image features are extracted layer by layer: feature maps are extracted at the original size and at scaled sizes of 1 / 2, 1 / 4, and 1 / 8, obtaining representations at four different scales. Each level of feature map uses a Transformer module (multi-head self-attention + feedforward network) to extract local and global salient features, and performs spatial downsampling through hierarchical feature map merging, gradually expanding the receptive field from low to high layers. The encoder ultimately outputs feature maps at four scales, which are then fused together after linear projection adjustment to form a rich multi-scale feature set for use by the subsequent decoder.

[0025] Step 5: Generation of nuclear cue points: Compared to other methods that directly classify the type of each pixel in the cell nucleus mask to obtain the cell nucleus mask, this invention innovatively proposes to use a multi-level ViT model to train and identify the coordinates of the cell nucleus center position as the cell nucleus cue point, thereby realizing the automatic generation of cue points.

[0026] 1) Determining the location of the cell nucleus center point: During the training phase, the centroid coordinates of each labeled cell nucleus mask are obtained by computationally calculating the geometric center, and a two-dimensional Gaussian distribution heatmap (with values ​​decaying outwards from the centroid) is generated at this coordinate location. The Gaussian heatmaps of all cell nuclei are accumulated on the corresponding cell type channels to obtain a multi-channel heatmap with shape [C,G,G] (where C is the number of cell types and G is the heatmap size).

[0027] 2) Model Inference: The pathological multi-scale feature extraction module in step 4 acts as an image encoder. The preprocessed H&E image is input into the encoder to extract multi-scale features. The multi-scale feature vectors are upsampled step-by-step by a convolutional decoder, and the upsampled features are directly concatenated. The Gaussian heatmap value of each pixel is predicted by the convolutional classification head, and local extrema points above a threshold (e.g., 0.7) are selected as candidate "cue points" for the cell nucleus center. The coordinates of these cue points then serve as input clues to the SAM model, achieving cell nucleus target detection without human intervention.

[0028] Step 6: Semantic Feature Extraction: In parallel with cue point generation, the cue point generation module in this invention also includes semantic segmentation functionality, which can learn semantic features related to cell types and perform coarse cell classification. The encoder uses a Transformer to implement overlapping block embedding. The classification branch and the cell nucleus cue point generation branch share the same multi-scale feature map, performing separate convolutional decoding and softmax calculations to output a pixel-level cell type probability map with the same image size, i.e., predicting the probability distribution of each pixel belonging to each cell type. Through this branch, the model initially obtains spatial distribution information of cell types (e.g., which regions might be lymphocytes, epithelial cells, etc.), providing a basis for subsequent extraction of type probabilities by cell nucleus.

[0029] Step 7: Input the cell nucleus instance segmentation mask and cell nucleus principal type probability vector into the trained image decoder to obtain the cell nucleus instance segmentation mask and cell nucleus principal type probability vector with instance ID corresponding to each cue point.

[0030] C.SAM fine-tuning and kernel instance splitting: Step 8: Input the cell nucleus center coordinates (cue points) and cell main category probability spatial distribution information generated in Step 5 into the fine-tuned SAM model. Summarize the cell category spatial distribution information within each instance mask (e.g., take the mean / weight) to obtain the "instance-level" cell nucleus instance segmentation mask corresponding to each "each cell nucleus" cue point. Obtain the semantic segmentation mask based on the boundary of the cell nucleus instance segmentation mask. Then obtain the cell main type probability vector based on the semantic segmentation mask (mask post-processing and morphological probability calculation).

[0031] D. Post-mask processing and morphological probability calculation: Step 9: Candidate Nucleus Instance Generation and Deduplication Screening. The candidate cue point locations are input into the pre-trained SAM model. The SAM model, combined with the original pathological image, outputs multiple candidate nucleus instance masks and corresponding prediction scores. The candidate instance masks are initially screened to remove low-quality candidate instances with excessively small or large areas, abnormal morphology, incomplete boundaries, or prediction scores below a threshold. Subsequently, non-maximum suppression (NMS) is performed on the screened candidate instances, sorting them from highest to lowest prediction score. When the overlap between two candidate instances exceeds a preset threshold, the candidate instance with the higher score is retained, while the candidate instance with the lower score is suppressed, thereby removing duplicate segmentation of the same nucleus.

[0032] Step 10: Instance Mask Correction and Classification Result Fusion Output. The deduplicated instance masks are corrected to improve the integrity and continuity of the cell nucleus outline. The corrected cell nucleus instance masks are spatially mapped to the cell principal type probability map. The probability values ​​of each cell principal type within each instance mask region are statistically analyzed, and the principal type with the highest probability or the highest weighted score is taken as the classification result for that cell nucleus instance. Finally, the instance mask, center coordinates, and cell principal type annotation results for each cell nucleus are output.

[0033] Step 2: Idle Data Preprocessing: The cell nucleus instance segmentation mask and cell nucleus principal type probability vector obtained from the image analysis module are combined with the preprocessed spatial transcriptome sequencing matrix and single-cell transcriptome data obtained from spatial transcriptome sequencing to perform refined cell type annotation. The cell nucleus center coordinates, cell nucleus instance segmentation mask, and cell nucleus principal type probability vector obtained in process 1 provide the foundation for cell annotation of spatial transcriptome data. Next, this invention fuses these image extraction information with spatial transcriptome sequencing information by solving a constrained optimization problem to annotate cell subtypes at the single-cell level. The input of this module includes two aspects: (1) the cell nucleus instance segmentation mask and cell nucleus principal type probability vector from the image analysis module; (2) the preprocessed spatial transcriptome sequencing matrix and single-cell transcriptome data from the spatial transcriptome sequencing data. The output of the module is the cell type (including principal type and subtype) annotation result for each specific cell nucleus. See Figure 1 II. Transcriptome information section of cell annotation, the implementation steps are as follows: Figure 2 The single-cell resolution spatial transcriptome cell annotation module (Module II) is shown below: E.ST data deconvolution: Step 1: Spatial Transcriptome Data Preprocessing: The raw spatial transcriptome sequencing matrix undergoes quality control and standardization. First, low-quality sequencing capture points (e.g., points with excessively low total transcript read counts or excessively high mitochondrial gene proportions) are filtered out, and abnormal or noisy genes (such as mitochondrial genes and erythrocyte genes are typically filtered out to reduce background interference) are removed. The remaining data is normalized and transformed (e.g., using total count normalization and logarithmic transformation) to eliminate sequencing depth differences. Second, single-cell transcriptome data of the same tissue type as the spatial transcriptome sequencing tissue is obtained from publicly published literature or databases as a reference (usually with cell type labels for each cell obtained through clustering or manual annotation). This reference data undergoes the same gene filtering and normalization as the spatial data, ensuring that the spatial data and reference data use consistent gene feature sets and annotation categories. Thus, the processed spatial transcriptome expression matrix and reference single-cell data can be used as input for subsequent deconvolution algorithms.

[0034] Step 2: Based on the preprocessed spatial transcriptome sequencing matrix, single-cell transcriptome data, and the corresponding annotated cell subtype files in single cells, the composition abundance or composition ratio matrix of each cell subtype in the sequencing site is obtained by using the deconvolution method (ST data deconvolution), thereby obtaining the rigid constraint of the number of cells to be allocated to each cell subtype in each capture site.

[0035] The cell subtypes obtained from deconvolution include, for example, colon cancer epithelial cells, colonic epithelial cells, vascular endothelial cells and fibroblasts, as well as T cells, B cells, neutrophils and macrophages.

[0036] Existing cell type deconvolution methods typically utilize single-cell transcriptome data from the same tissue with cell subtype annotations as a reference. They estimate the proportion of each cell subtype within each site by fitting the gene expression profile of the spatial transcriptome capture site. This invention does not limit the specific deconvolution algorithm in this step, allowing the use of any publicly available and validated deconvolution method, such as Cell2location, RCTD, SPOTlight, and Tangram. The validated deconvolution tool Cell2location from published articles is used to integrate the aforementioned spatial data with reference scRNA-seq data to infer the cell composition (proportion or number of each cell type) at each spatial capture site. This invention does not limit the specific deconvolution algorithm, as long as its output includes the proportion or estimated number of each cell type within each capture site.

[0037] Spatial transcriptome sequencing data, single-cell sequencing data from the same tissue type, and corresponding annotated cell subtype files in the single cells are input according to the requirements of various publicly available deconvolution methods. The output is an abundance or composition ratio matrix of each cell subtype in the sequencing site, which will be used to calculate the cell number limit in the cell subtype in subsequent steps.

[0038] F. Spatial matching of the cell nucleus and the capture site: Step 3: Cell location allocation, obtaining each capture point within the capture interval and its corresponding integrated data structure.

[0039] 1) Cell Instance Standardization and Information Structure Construction: This step receives the set of cell nucleus instances (including the instance number and centroid coordinates of each cell nucleus) output by the H&E image analysis module, as well as the cell nucleus principal type probability vector (i.e., the probability distribution of the cell belonging to each principal type, which is divided into tumor epithelial cells, normal epithelial cells, connective tissue cells, and inflammatory / immune cells). This information is integrated into a unified data structure, cell_info: using the cell instance ID as an index, its centroid coordinates (centroid) and type probability vector (type_prob) are written into it, and an initial principal type label is generated based on the principle of maximizing probability as a reference for subsequent constraint allocation.

[0040] 2) Spatial Coordinate System Alignment and Geometric Parameter Resolution: The module receives capture site coordinate information (spatial_positions) from the spatial transcriptomics platform, typically including the center pixel coordinates of each spot and array row and column identifiers. This information is then aligned with the cell centroid coordinates in the cell_info module (if image scaling or coordinate transformation is involved, this module performs the unification). Based on this, the shape of the capture site is read, and the coverage area is determined: for example, the radius is used as the inclusion threshold in a circular spot scenario, and the boundary range is used as the inclusion threshold in a square spot scenario. Furthermore, the coverage area of ​​the unsequencing region (intercapture zone) is deduced based on the coordinate differences between adjacent spots.

[0041] 3) Cell Attribution Module within Capture Sites: Based on the cell centroid coordinates and spatial coverage of the capture sites obtained in the previous steps, this module performs a spatial attribution determination to determine whether a cell belongs to a specific spot, and generates a set structure for subsequent allocation. For circular spots, this module calculates the distance from the cell centroid to the spot center for each spot and filters cell instances whose distance is less than the radius r; for square spots, it constructs a rectangular area based on the center and filters cell instances falling within the area. Finally, a list of cell_ids containing all cells within each spot and the image module output information are generated. This mapping structure is directly used in subsequent step 4 to determine the set of cell instances to be allocated within the spot.

[0042] 4) Interval Construction and Attribution Module: In addition to the capture region, this module uses a similar method to determine the cell_id and image module information of all cells within each capture interval. (Dynamic Programming) G. Preliminary numerical estimates (see...) Figure 1 II. Dynamic Programming in Cell Annotation): Step 4: Based on each capture point within the capture interval and the corresponding integrated data structure, as well as the abundance or composition ratio matrix of each cell subtype in each capture site, estimate the number of cell types within the capture site (cell nucleus and spot spatial matching and preliminary data estimation to obtain subtype ratio and cell count).

[0043] The total number of cells (N) is calculated from the location and area of ​​each sequencing capture site. spot Deconvolution yields the proportion of cell types within the capture site. The total number of cells of each cell type (typei) within the capture site. It can be estimated as follows:

[0044] For total cell count For rounding, first round down and record the difference between the original number and the rounded-down integer. Sort the cells according to the difference from largest to smallest, and fill the total number of cells to N. spot .

[0045] Step 3: Precisely annotate cell subtypes: H.DP allocation and CBI correction: Step 5: Cell type assignment (DP assignment) within the sequencing capture region.

[0046] 1) Cell type composition constraints within a spot: This module uses the spot-level cell type composition `cell_compose`, output by deconvolution, as the constraint source based on each capture point within the capture interval and its corresponding integrated data structure. For each spot, it first reads the abundance or composition ratio matrix of each cell subtype composition at each capture site estimated in step 3 and normalizes it into a "quota constraint" that can be used for allocation. This step outputs a rigid constraint on the number of cells that should be allocated to each type within each spot, providing boundary conditions for subsequent dynamic programming solutions.

[0047] 2) Main type-subtype hierarchy consistency module (constraint transformation + expansion preparation): The H&E image analysis module outputs the probability vector of cell "major category / main type", while cell annotation often requires a more detailed "subtype" composition.

[0048] This step converts the number of cell subclasses within the spot output from the previous step into the number of cell major classes, thereby aligning the constraints with the type probabilities of the image analysis results to ensure allocability. The module utilizes a pre-established "cell subtype → main type" mapping (e.g., colon cancer epithelial cells → tumor epithelial cells, colon epithelial cells → normal epithelial cells, vascular endothelial cells and fibroblasts → connective tissue cells, T cells, B cells, neutrophils and macrophages → inflammatory / immune cells) to aggregate the subtype quotas within the spot to the main type level, obtaining the main type quota constraints required for dynamic programming; at the same time, it retains the set of optional subtypes and the list of subtype quotas under each main type. Through this hierarchical strategy of "first performing global optimization at the main type level, and then expanding subtypes within the main type", this invention can simultaneously utilize the morphological discrimination capability of the image (main type probability) and the fine constraints of sequencing (subtype quotas), avoiding the state explosion caused by directly solving in the high-dimensional subtype space.

[0049] 3) In-spot Assignment Problem Instantiation (Cell Set and Probability Matrix Assembly): This module reads the type probability distribution output by the image module for each cell instance list contained in each spot, forming a probability matrix of "cell × main type" (the probability value serves as the reward score for subsequent optimization). Simultaneously, it reads the main type quota constraints generated in the previous step, constructing a "must-assign quota vector". At this point, the cell identity assignment problem within a spot is fully instantiated: on one side is the set of cells to be labeled and their type probability evidence, and on the other side are the quota constraints that must be satisfied. After this module is completed, the core solution steps will begin.

[0050] 4) Constrained Global Optimal Allocation Solution (Dynamic Programming for Primary Type Allocation): This core step, under the hard constraint that "the number of primary type slots must be satisfied," selects a primary type for each cell within the spot, maximizing the sum of the probabilities of all cells being assigned a type. To avoid local optima caused by random allocation or point-by-point greedy approaches, this module uses dynamic programming to recursively search the state space of "the nth cell processed + the combination of remaining slots for each primary type": When processing each new cell, all primary types with remaining slots are traversed as candidate decisions, and the probability increment brought by "selecting this primary type" is accumulated into the existing optimal path; if different paths lead to the same state, only the path with the higher cumulative probability is retained, and the worse paths are discarded, thus ensuring global optima while compressing the search space. The module finally obtains the primary type allocation result and the corresponding optimal cumulative score for each cell within the spot, and uses this as the basis for subsequent subtype allocation.

[0051] 5) Subtype Placement and Cell-Level Annotation: This step refines the main type allocation results to the subtype level and writes the final annotations back to `cell_info`. Specifically, based on the cell type quantity constraints within the spot obtained in Step 1, an allocable subtype quota pool is constructed. Then, for each main type, a corresponding number of subtypes are extracted from the quota pool within the subtype set covered by that main type, and these subtypes are assigned one by one to cell instances identified as that main type, thus ensuring that the total number of subtypes strictly conforms to the sequencing composition constraints. Finally, based on the corresponding annotated cell subtype file in a single cell, each cell instance obtains a unique subtype annotation, achieving single-cell resolution type labeling within the spot capture region.

[0052] 6) Whole-slice Spot Batch Processing and Parallel Acceleration Module: Considering the large number of spots in the entire slice and the considerable number of cells within each spot, this module encapsulates the inference process within each spot into an independently executable unit and processes all spots in parallel using a multi-process approach. Each process returns the cell-level annotation increment for that spot, and the main process writes it back to cell_info. Through parallel design, this invention can significantly shorten the annotation time for the entire slice while ensuring the determinism and reproducibility of the algorithm.

[0053] Step 6: Inferring cell type in the capture zone This invention defines cell sets within each capture interval based on the spatial coordinates of the cell nucleus. For cells not located within any sequencing capture site (i.e., cells in the "capture interval"), this invention infers their cell type using spatial proximity information and morphological features. The process involves: using the cell set within each capture interval (typically a polygonal region enclosed by adjacent capture sites) as a unit, identifying one or more adjacent capture sites with existing annotations, and comparing the similarity of cell type probability distributions between these adjacent sequencing regions and the cells in the interval. Similarity is measured using Kullback-Leibler divergence or other probability distribution distances. For each capture interval, we calculate the major cell type probability distribution p(x) (based on image analysis classification results) and the type probability distribution q(x) of annotated cells within each adjacent capture site. Then, we select the top-k neighboring cells most similar to the interval (k=5 in this embodiment). Among these most similar cells, the cell type with the highest frequency is statistically analyzed and used as the annotation result (major cell type) for all cells in the interval. In simple terms, it involves identifying the cell types and morphological similarities surrounding unsequencing regions, and then determining the predominant cell type in that region. This method allows for the completion of cell type annotation in tissue regions not covered by spatial transcriptomics, resulting in a continuous cell type atlas with ST spots across the entire slide (i.e.,...). Figure 1 The expression profiles in the text are shown below. The similarity calculation method is as follows:

[0054] Where p(x) and q(x) represent the cell principal type probability distributions in the intercapture zone and adjacent capture sites, respectively.

[0055] Deconvolution result correction mechanism (CBI correction).

[0056] This invention proposes a method for dynamically adjusting the cell type composition within a capture site by combining H&E image information. This method introduces a Competition Balance Index (CBI) to correct the cell type distribution results obtained from deconvolution of sequencing data. The CBI quantifies the relative difference between image-based cell nucleus classification results and deconvolution inference results within the same capture site.

[0057] Specifically, after cell annotation is completed, the total probability of cell types within the capture site is calculated. The sum of the largest class probabilities in the cell nucleus Let the predicted class probability vector of the nth cell be... ,in This indicates that the cell belongs to the first... The probability of the class, the total number of cells within the capture site is .

[0058]

[0059] i represents the assigned cell type.

[0060] And define CBI as follows:

[0061] 10 of them 4 As a smoothing factor, it prevents the CBI value from becoming too large.

[0062] Once the CBI of a capture site exceeds the user-defined threshold (the default threshold in this invention is 10, which can be adjusted according to the level of confidence in the image results; if the image is trusted, the threshold should be set lower), it indicates that the sequencing-based annotation results at that site may be biased and need to be corrected. This invention employs an iterative update strategy to automatically correct these discrepancies: First, the maximum dominant type and its probability value in the H&E image classification probability for each cell within the capture site are statistically analyzed, and all cells are sorted from highest to lowest according to this maximum probability value. Then, starting with the highest-ranked cell, it is checked whether its currently assigned dominant type is consistent with the previously assigned most probable type; for those inconsistent cells, their previously assigned dominant type is replaced with the type predicted based on the H&E image. Each time a cell type is replaced, the CBI is recalculated. If the CBI is still higher than the threshold, the next inconsistent cell type is replaced, and this process is iterated until the CBI drops below the threshold. This adjustment is equivalent to allowing the strong confidence information provided by the image to correct the deconvolution results. For example, if the H&E image clearly shows that a cell belongs to an inflammatory / immune cell, but the deconvolution classifies it as a fibroblast (belonging to connective tissue cells), correction is performed. After this round of category-based correction, for cells whose types have been modified, we further infer their cell subtypes: the method is to find the cell with the most similar morphological characteristics among unmodified cells of the same type within the same capture site, and then assign the subtype of the unmodified cell to the modified major cell category. This allows for annotation at the subtype level while maintaining macro-type consistency, resulting in expression profile cell subtype ratios and annotations. Through the CBI feedback mechanism, this invention achieves dynamic adjustment and error correction of sequencing deconvolution results using image information: this is a bidirectional fusion that utilizes both the quantitative composition calculated from transcriptome data deconvolution and the visual evidence of cell morphology to correct potential biases in deconvolution, thereby outputting more reliable cell annotation results. It is worth emphasizing that this type of interactive fusion correction is lacking in existing technologies; traditional methods either perform separate operations or simply superimpose them, while this invention provides a mechanism for image and sequencing results to "compete and balance" each other, significantly improving integration accuracy.

[0063] Step 4: Generate high-resolution single-cell spatial maps: I: Cell nucleus segmentation and final output: After the above processing steps, we finally obtained detailed annotation results for each cell nucleus in the entire tissue slice (including the major cell class and specific subtype). Compared with the prior art, the method of this invention organically integrates cell morphology information with molecular expression information: it fully utilizes the spatial cell location and morphological features provided by image segmentation and classification, and combines the molecular type composition provided by sequencing data. It achieves globally optimized cell type allocation through dynamic programming, supplemented by a competitive balance index for iterative correction. Therefore, this invention can reconstruct the spatial gene expression map of tissue at the single-cell level, accurately labeling the identity of each cell, rather than providing only a vague proportion at the spot level as in traditional deconvolution. This innovative deep fusion strategy effectively overcomes the various difficulties mentioned in the background art, providing a higher resolution and higher accuracy solution for the analysis of spatial omics data.

[0064] Model structural details like Figure 3 China A and Figure 1 As shown in Figure I, the H&E image analysis module automatically and accurately extracts cell nucleus location, morphology, and possible cell type information at single-cell resolution from preprocessed H&E-stained pathological images using deep learning methods. It performs cell nucleus instance segmentation on the input preprocessed H&E-stained tissue image and outputs a major category (primary type) probability vector for each cell nucleus.

[0065] 1. Prompter structure (used to achieve accurate semantic segmentation): The encoder of the prompt generator uses the Segformer hybrid transformer-b3 (mit-b3) backbone network. Figure 3 (B) The H&E image undergoes a multi-scale feature extraction process using a "self-attention layer - feedforward neural network - feature map merging" workflow. During feature extraction, feature maps are constructed from the input image at four scales: 1×, 1 / 2×, 1 / 4×, and 1 / 8× of the original size. (1) Hint point branch: The cue point branch generates a 2D Gaussian heatmap using the center point of all cell nucleus instances as anchor points. For each cell nucleus instance in the training image, the geometric center point or the ground truth center point is first calculated based on its mask. Then, a 2D Gaussian heatmap is generated at that location; this is a continuous probability distribution map centered at a certain point and gradually decaying around it, representing the probability of the cell nucleus center point existing in the image space. All Gaussian heatmaps are superimposed on their respective category channels to form a multi-channel heatmap of shape [C, G, G] (where C is the number of cell types and G is the heatmap size).

[0066] During the inference phase, high-value points in the heatmap are identified as candidate cue points. A high-value point is defined as a heatmap value greater than a user-preset threshold (e.g., 0.7). The final selected cue points serve as input prompts for the SAM model, enabling automatic localization of cell nuclei and instance segmentation.

[0067] (2) Classification branches: This branch is used to predict the probability distribution of cell type to which each pixel belongs. The multi-scale fused features are compressed by the number of channels in the convolutional layer and then input into the softmax layer for normalization, outputting the cell type probability vector corresponding to each pixel, forming a pixel-level semantic segmentation map.

[0068] This cue generator module provides SAM with automatically located cue points (nucleus center coordinates) and also provides structured semantic distribution information for subsequent cell type annotation.

[0069] 2. SAM structure (used for accurate boundary delineation): In this invention, we fine-tuned MetaAI's SegmentAnythingModel (SAM) for pathological cell nucleus segmentation tasks, enabling it to segment cell nucleus boundaries more precisely. Specifically, SAM comprises three parts: an image encoder, a cue encoder, and a mask decoder. The image encoder (ViT architecture) extracts global visual features of the image patches; the cue encoder encodes the input cell nucleus centroid cue points and fuses them with image features through a bidirectional cross-attention mechanism; finally, the mask decoder uses an improved Transformer structure and a multilayer perceptron (MLP) to decode the fused features into pixel-level instance segmentation masks. Combined with the semantic segmentation mask output by the semantic feature extraction module, the cell nucleus principal type probability vector is obtained by summarizing the spatial distribution information of cell categories within each instance mask. After receiving the automatically generated cue point set and image features, the fine-tuned SAM simultaneously outputs the cell nucleus instance segmentation mask and the cell nucleus type probability vector (obtained by combining the semantic features of the region where the cue point falls within the mask) for each cue point. In other words, for each detected cell nucleus, this invention can simultaneously obtain the precise segmentation contour of the cell nucleus (cell nucleus instance segmentation mask) and the probability distribution of its belonging to each cell class (cell nucleus type probability vector). These results will serve as input for the subsequent spatial cell annotation module.

[0070] Through the image analysis modules described above, this invention achieves accurate localization, segmentation, and preliminary classification of each cell nucleus in H&E images without human intervention. Tests on multiple publicly available pathological datasets (such as Kumar, CPM17, PanNuke, CoNSeP, MoNuSAC, and CellBinDB) demonstrate that the cell nucleus segmentation and classification performance of this invention outperforms traditional algorithms (such as HoVer-Net, Cellpose, and MaskR-CNN), achieving optimal results in both segmentation accuracy and classification accuracy. This proves that our SegFormer-based prompt generator + SAM scheme effectively overcomes the segmentation errors and missed detections in H&E images found in existing models, significantly improving the accuracy of cell nucleus segmentation and classification.

[0071] Instance support like Figure 4 As shown, Figure 4 In the figure, A represents the benchmark test of the H&E image analysis module of the present invention on the public datasets CPM17 (left), Kumar (middle), and CoNSeP (right). Each data point represents an evaluation index calculated on a single sample image in the test dataset. The SpatioCell of the present invention is compared with five algorithms: Hover-Net, Cellpose, Stardist, Mask-RCNN, and CellProfiler. The present invention achieves the best performance in all aspects.

[0072] Figure 4 In the figure, B represents the segmentation performance of various methods on the PanNuke dataset using three-fold cross-validation. Error bars represent the standard error.

[0073] Figure 4 The C in the figure represents a visualization example of cell nucleus segmentation on the PanNuke dataset, and is labeled with AJI and PQ scores.

[0074] Figure 4 In the figure, D represents the performance of different methods on H&E-stained images of FF samples in the CellBinDB dataset for cell nuclear segmentation.

[0075] Figure 4 In the figure, E represents the detection rate of different methods at different IoU thresholds in the CellBinDB dataset.

[0076] Figure 4 In the figure, F represents the mPQ score distribution obtained by SpatioCell, Hover-Net, and MaskR-CNN on the PanNuke dataset through three-fold cross-validation. SpatioCell's cell nucleus type classification is based on the category corresponding to the highest probability in the classification probability spectrum.

[0077] Figure 4 G represents a visualization example of SpatioCell's nucleus classification on the PanNuke dataset.

[0078] like Figure 5 As shown, to evaluate the spatial cell annotation performance of this invention (labeled SpatioCell in the figure) at single-cell resolution, single-cell resolution spatial transcriptome data acquired using the 10×Xenium platform were used to simulate multi-cell resolution spatial transcriptome sequencing results for four tumors: breast cancer, ovarian cancer, pancreatic cancer, and lung cancer. This invention was compared with two baseline methods: one based solely on H&E images for cell nuclear classification, and the other based solely on cell composition allocation for annotation.

[0079] Figure 5 The diagram in A represents the SpatioCell annotation process, which transforms the cell type composition information obtained from deconvolution into a single-cell spatial map.

[0080] Figure 5 In the figure, B represents the comparison of the performance of SpatioCell, deconvolution-based annotation methods, and H&E classification methods in supertype cell annotation accuracy at three spot resolutions of 35μm, 55μm, and 100μm in a simulated dataset (including BRCA, OV, and PDAC tissues).

[0081] Figure 5 The "C" indicates a comparison of the performance of SpatioCell, deconvolution-based methods, and H&E classification on supertype annotation accuracy (measured by F1-score) across three simulated datasets. The dots represent the median, and the error bars represent the interquartile range (IQR). The data shown here is for a 55μm mpot size.

[0082] Figure 5 In the example, D represents the comparison of SpatioCell and deconvolution-based methods in terms of subtype annotation accuracy in a simulated dataset with three spot sizes of 35μm, 55μm, and 100μm, using the actual cell type composition as a constraint.

[0083] Figure 5 E represents the comparison of SpatioCell and deconvolution-based methods in terms of supertype (top) and subtype (bottom) annotation accuracy within the spot on three datasets and at different spot sizes, when interval regions are present.

[0084] Figure 5The diagram in Figure F illustrates the interpolation strategy of SpatioCell for the interval regions between speckles. Gray squares represent query cells, and squares within black boxes represent labeled adjacent speckles. The heatmap, based on the probability distribution of Module I, shows the KL divergence between query cells and labeled cells, with the darkest red square representing the lowest divergence value.

[0085] Figure 5 G represents the performance of SpatioCell and deconvolution-based methods in supertype annotation accuracy across three datasets at a 55μm mpot size, and shows the accuracy within the spot interval region (blue) and the entire slice range (green), respectively.

[0086] In both main cell type and subtype annotation tasks, this invention demonstrates significantly higher annotation accuracy than the aforementioned baseline methods. Furthermore, in regions lacking sequencing coverage, this invention can infer cell type probability distributions through morphological features and, combined with the similarity of cells in neighboring capture sites, achieve inter-capture region type imputation, resulting in continuous spatial annotation. Its overall performance surpasses traditional deconvolution methods.

[0087] like Figure 6 As shown, this invention implements adaptive correction of existing deconvolution methods through CBI, corrects traditional deconvolution results under a suitable threshold, and restores the organizational structure.

[0088] Figure 6 In the image, A represents the proportion of tumor cells estimated by Cell2location in a 10xVisium lung squamous cell carcinoma sample (left), and the corresponding magnified H&E image (right); the magnified image shows the tumor area (red) and necrotic area (blue) annotated by a pathologist.

[0089] Figure 6 In the figure, B represents the distribution of mean absolute error (MAE, left) and mean squared error (MSE, right) of the deconvolution results after correction at different CBI thresholds in the spatial transcriptome data of breast cancer tissue samples. The curves of different colors represent the correction results at different perturbation levels (0.2, 0.15, 0.1, 0.05). The red horizontal line represents the MAE and MSE values ​​corresponding to each perturbation level without the use of CBI.

[0090] Figure 6 In the text, C represents the MAE (left) and MSE (right) distributions of the deconvolution results after correction at different CBI thresholds in the spatial transcriptome data of ovarian cancer tissue samples. The rest of the explanation is the same as in (B).

[0091] Figure 6In this context, D represents a comparison of the spatial transcriptome cell annotation results and deconvolution results of ovarian cancer tissue samples before and after CBI correction.

[0092] The spatial transcriptome deconvolution used in this invention can be interfaced with any deconvolution method that provides the proportion of cell types within a spot.

[0093] This invention provides a single-cell spatial annotation method that fuses images and sequencing data. It is an automatic, accurate, and high-resolution spatial transcriptome cell annotation method based on deep fusion of H&E images and sequencing information. The overall scheme includes the following aspects: 1. Automated and precise segmentation and classification of cell nuclei in high-density tissue sections; 2. Precise localization and fine-grained type annotation of each cell within the capture site by multi-cell resolution sequencing; 3. Inferring and annotating cell types outside the capture area, enabling continuous single-cell annotation of the entire slice; 4. Based on the cell morphology features of H&E images, adaptive correction is performed on the deconvolution inference results to improve annotation accuracy.

[0094] This solution can simultaneously perform: automatic segmentation and coarse classification of cell nuclei in H&E digital pathology images, and fine annotation of cell types in spatial transcriptome data.

[0095] The beneficial effects of the embodiments of the present invention are as follows: 1. Improved accuracy in cell nucleus segmentation and classification; Based on SegFormer The B3 cue generator automatically detects the nucleus centroid as the SAM cue point, and simultaneously performs nucleus type classification. After fine-tuning, the SAM receives the cue point and performs accurate nucleus segmentation.

[0096] It achieves accurate cell nucleus segmentation and classification, with optimal performance on multiple publicly available H&E staining pathology datasets.

[0097] 2. Higher resolution spatial transcriptome cellular annotation; By modeling the "cell type assignment within the capture site" as a multidimensional knapsack problem and using dynamic programming to solve for the global optimum, we can obtain single-cell resolution spatial transcriptome cell annotation results and reconstruct the details of the tissue microenvironment.

[0098] 3. Complete slice cell annotation covering the capture zone; For cells in the capture intercellular region, their morphological probabilities are calculated and their similarity to the major cell types of already annotated cells to infer their subtypes. This yields complete cell annotations for the entire slice, which can be used to create single-cell resolution spatial maps.

[0099] 4. Adaptive correction of deconvolution results; By combining cell classification results from H&E images and calculating the degree of difference in cell composition based on images and deconvolution, we adaptively correct deconvolution results based solely on sequencing, thereby achieving more accurate cell annotation.

[0100] The key technical points and areas to be protected in this application are as follows: 1. Automatic cell nucleus segmentation and classification using Prompter-SegFormer + fine-tuning SAM; Prompter automatically generates nucleus centroid prompts, outputting high-precision cell nucleus segmentation masks and tissue type probability distributions on H&E-stained tissue sections without requiring manual prompts.

[0101] 2. Single-cell resolution spatial cell annotation method; By integrating cell morphology information from H&E images with cell type composition information from sequencing data, the cell annotation problem is modeled as a variant of a knapsack problem. A dynamic programming algorithm is used to recursively search all feasible label combinations (i.e., states) to obtain the globally optimal annotation result.

[0102] Furthermore, for cells within the capture interval, their cell subtypes are inferred and annotated by comparing the similarity between their tissue type probability distribution and the tissue type probability distribution of annotated cells in adjacent capture regions, thus obtaining complete spatial cell annotation for the entire slice.

[0103] 3. Adaptive deconvolution correction mechanism; This invention proposes a method to dynamically adjust and optimize deconvolution of cell type composition within a capture site by combining H&E image information. It introduces CBI to quantify the relative difference between image-based and sequencing-based cell type annotations within the same capture site. When CBI exceeds a threshold, the cell nuclear tissue type based on H&E images is adaptively used to correct the sequencing-based deconvolution results.

Claims

1. A single-cell spatial annotation method that integrates image and sequencing data, characterized in that, Includes the following steps: Based on the preprocessed full-view H&E stained tissue image, a cell nucleus instance segmentation mask with instance ID and a cell nucleus principal type probability vector corresponding to each cue point are obtained. Using the instance ID as an index, the capture site is processed according to the principle of maximum probability to obtain each cell instance and corresponding integrated data structure within the capture site. Based on the preprocessed spatial transcriptome sequencing matrix and single-cell transcriptome data, the cell subtypes in the capture sites were obtained and formed an abundance or composition ratio matrix. Based on the total number of cell instances within each capture site, a rigid constraint is constructed on the number of cells to be allocated for each cell subtype within each capture site. Combining the preset initial main type label and the mapping relationship of each cell subtype, the number of cells is summarized according to the initial main type label to obtain the main type quota constraint. Based on the primary type quota constraint, a primary type is selected for each cell and a primary type annotation is written. Based on the rigid constraint and the corresponding annotation cell subtype file in a single cell, subtype annotations are obtained. The primary type allocation results and primary type annotations are refined to subtype annotations and written to each capture site and the corresponding integrated data structure to obtain the annotation results.

2. The single-cell spatial annotation method for fusing image and sequencing data as described in claim 1, characterized in that, Based on the preprocessed full-view H&E stained tissue image, obtain the corresponding spatial transcriptome raw sequencing matrix, single-cell transcriptome data, and corresponding annotated cell subtype files in the single cell.

3. The single-cell spatial annotation method for fusing image and sequencing data as described in claim 2, characterized in that, Low-quality sequencing capture points in the original spatial transcriptome sequencing matrix are filtered out, and abnormal or noisy genes are removed. Then, normalization and transformation are performed to obtain the preprocessed spatial transcriptome sequencing matrix.

4. The single-cell spatial annotation method for fusing image and sequencing data as described in claim 1, characterized in that, Based on the preprocessed full-view H&E stained tissue image, the candidate cue point locations and cell principal class probability distribution maps are obtained, and then input into the fine-tuned SAM model to obtain the cell nucleus instance segmentation mask with instance ID and the cell nucleus principal type probability vector corresponding to each cue point.

5. The single-cell spatial annotation method for fusing image and sequencing data as described in claim 4, characterized in that, Based on the preprocessed full-view H&E stained tissue image, candidate cue point locations and cell major class probability distribution maps are obtained, including the following steps: Multi-scale feature vectors are obtained by pathological multi-scale feature extraction on the preprocessed data. Based on the labeled cell nucleus mask, the centroid coordinates are obtained by calculating the geometric center. The data is then upsampled step by step by a convolutional decoder. The upsampled features are directly concatenated. The Gaussian heatmap value of each pixel is predicted by the convolutional classification head. A two-dimensional Gaussian distribution heatmap is generated at the centroid coordinate position. The two-dimensional Gaussian distribution heatmap has a value that decays from the centroid to the surrounding area. Local extreme points above the threshold are selected from the two-dimensional Gaussian distribution heatmap and regarded as candidate cue points for the cell nucleus center. The position of the candidate cue point is obtained. Simultaneously, based on multi-scale feature vectors, separate convolutional decoding and softmax calculation are performed to output a pixel-level cell main type probability map with the same size as the image, thus initially obtaining the cell main category probability distribution map.

6. The single-cell spatial annotation method for fusing image and sequencing data as described in claim 1, characterized in that, Initial main type labels are generated based on the principle of maximizing probability. The shape of the capture site is read and the coverage is determined under the same coordinate scale. Cell affiliation within the capture site is then determined to obtain each cell instance and its corresponding integrated data structure within the capture site.

7. The single-cell spatial annotation method for fusing image and sequencing data as described in claim 1, characterized in that, The method further includes selecting a primary type for each cell and writing a primary type annotation based on the primary type quota constraint and aiming to maximize the sum of the primary type assignment probability values ​​of all cells within the capture site.

8. The single-cell spatial annotation method for fusing image and sequencing data as described in claim 1, characterized in that, The method further includes, for cells in the capture inter-cell region, finding one or more capture sites in the spatial neighborhood of existing annotation results, inferring the main type and cell subtype of the cells in the capture inter-cell region by calculating morphological feature similarity, and combining the corresponding annotation cell subtype file in the single cell to obtain the annotation result.

Images