A spatial multi-omics sequencing method based on cross-modal generation

Abstract

This invention discloses a spatial multi-omics sequencing method based on cross-modal generation, comprising the following steps: acquiring H&E staining images of tissue sample slices; automatically identifying and locating regions of interest (ROIs); manually cutting the tissue sample slices based on the ROIs and transferring them to a detection chip; acquiring a multi-omics expression matrix at single-cell resolution; extracting spatial feature anchor points that match the H&E staining images and the multi-omics expression matrix; performing coordinate transformation based on the coordinate mapping relationship of the spatial feature anchor points to achieve physical spatial dimension registration; and utilizing a cross-modal prediction model to perform spatial inference and full-sample information completion on discrete detection signals through spatial expression reconstruction methods. This invention effectively integrates high-precision local physical detection with global algorithm inference, ultimately outputting high-fidelity multi-omics expression information containing the entire tissue sample, completing cross-scale completion from local targeted detection to full-sample multi-omics.

Description

Technical Field

[0001] This invention relates to the technical field of biomedical image analysis, and in particular to a spatial multi-omics sequencing method based on cross-modal generation. Background Technology

[0002] Spatial omics technology provides a crucial means for in-situ analysis of cellular molecular characteristics and their spatial distribution patterns by acquiring gene and protein expression information while preserving the original spatial structure of tissues. However, current technologies still face bottlenecks in simultaneously conducting whole transcriptome and proteome detection, where it is difficult to balance spatial resolution, detection throughput, and experimental cost.

[0003] Traditional physical detection methods generally suffer from significant scale limitations. For example, while microdissection-based platforms can achieve high-precision local observations, they are limited by the size of the dissection and cannot reconstruct the entire tissue morphology. In situ hybridization-based platforms, due to physical limitations in fluorescence channels or isotope types, cannot handle large-sized samples. Even in the widely used chip-based capture sequencing technology, when capturing multiple modal signals in the same step, intermolecular signal interference and sensitivity attenuation remain pressing problems. Researchers often have to compromise between sacrificing the overall field of view or reducing resolution.

[0004] To overcome the drawbacks of high cost and limited coverage of physical testing, existing technologies attempt to infer molecular expression from pathological images using artificial intelligence. However, this inference method has significant limitations. Due to the lack of real physical sampling data as a reference, the model is prone to deviating from reality when faced with sparsely sampled areas, resulting in unreliable data. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and propose a spatial multi-omics sequencing method based on cross-modal generation. Through automatic region identification and cross-modal information completion, high-precision sequencing of key regions and reliable reconstruction of molecular information of the whole sample are achieved in large-scale tissue samples.

[0006] The objective of this invention is achieved through the following technical solution: a spatial multi-omics sequencing method based on cross-modal generation, comprising the following steps:

[0007] S1. Obtain H&E staining images of tissue sample sections;

[0008] S2. Automatically identify and locate the region of interest on the H&E staining image;

[0009] S3. Based on the located region of interest, the tissue sample slice is cut to obtain the target tissue and transferred to the detection chip;

[0010] S4. Perform co-localization in situ detection of transcriptome and proteome on the target tissue on the detection chip to obtain a multi-omics expression matrix;

[0011] S5. In the acquired H&E staining image and multi-omics expression matrix, extract the matching spatial feature anchor points, perform coordinate transformation based on the coordinate mapping relationship of the spatial feature anchor points, and output spatial multi-omics data registered to the same physical coordinate system; the spatial multi-omics data includes two-dimensional physical spatial coordinates and ribonucleic acid and protein expression level matrices corresponding to the two-dimensional physical spatial coordinates;

[0012] S6. Based on spatial multi-omics data and external tissue sample pathological images, a cross-modal large model is used to generate multi-omics expression profiles of complete tissue sample slices.

[0013] Furthermore, step S2 includes the following steps:

[0014] S21. Construct the H&E staining image obtained in step S1 into a tissue spatial domain. Morphological representation functions on:

[0015] ;

[0016] in, Represents the function for expressing pathological images. This represents the two-dimensional spatial region where the tissue sample slice is located. This represents the spatial coordinates of a pathological image; by employing a sliding window sampling strategy, the pathological image is divided into multiple local image blocks, forming a set of local tissue structure units.

[0017] ;

[0018] in, A set of local image patches Represents the set of elements. A local image patch, The sample number for the image patch. , Indicates the total number of local image patches;

[0019] S22. Using feature extraction mapping functions For local image patch set Encode each sample by numbering it. The local image patches are transformed into representation vectors in the feature space, that is, the tissue morphology is mapped to a feature space of a unified dimension:

[0020] ;

[0021] in, Indicates the first Feature representation vectors corresponding to each image patch Represents the dimension of the feature space;

[0022] S23. Perform structural pattern clustering based on feature space distance; in the feature space, let the set of cluster centers be... ; by calculating the feature representation vector With cluster center The Euclidean distance between them determines each local image patch. Category label :

[0023] ;

[0024] in, For cluster category numbers, , Indicates the first Cluster centers, This represents the total number of cluster categories. For the first Category labels are assigned to local image patches;

[0025] S24. Based on prior knowledge of the morphological characteristics of the target organizational structure, class labels are... The process involves filtering and retaining local image patches of the target category; then, connectivity analysis is performed on the filtered local image patches in the feature space to obtain a set of regions of interest for the target tissue structure.

[0026] ;

[0027] in, Represents the set of regions of interest. Represents the set of elements. Areas of interest The region of interest is numbered. , This represents the total number of regions of interest.

[0028] S25. For each numbered... Region of interest Boundary fitting enables precise localization of target tissue structures in pathological images and outputs the spatial coordinate matrix of the region of interest and the corresponding region mask.

[0029] Furthermore, step S3 includes the following steps:

[0030] S31. Based on the spatial coordinate matrix and region mask of the region of interest output in step S2, manually mark the target range of the region of interest on the sample tissue slice.

[0031] S32. By manually cutting and peeling, remove excess tissue outside the target area from the sample tissue slice to obtain the target tissue;

[0032] S33. Move the cut target tissue and attach it to the predetermined position of the detection chip.

[0033] Furthermore, step S4 includes:

[0034] Based on in situ sequencing technology, multi-cycle fluorescence imaging and codebook decoding are performed on the target tissue. Combined with cell contouring, single-cell mapping of target molecules is completed, and finally a multi-omics expression matrix containing spatial coordinates, ribonucleic acid, and protein expression levels is generated.

[0035] Furthermore, step S5 includes:

[0036] S51. In the H&E staining image obtained in step S1, several characteristic cell clusters are manually selected, and the spatial coordinates of the several characteristic cell clusters in the first coordinate system are extracted as image anchor points.

[0037] S52. In the multi-omics expression matrix and its corresponding microscopic field of view obtained in step S4, a feature region that matches the image anchor point is manually selected, and the spatial coordinates of the feature region in the second coordinate system are extracted as the omics anchor point.

[0038] S53. Based on the coordinate pairing relationship between the image anchor point and the omics anchor point, calculate the affine mapping matrix, and transform the two-dimensional physical space coordinates in the spatial multi-omics data through the affine mapping matrix to complete the accurate registration of the physical space dimension, and output the spatial multi-omics data registered to the same physical coordinate system; wherein, the spatial multi-omics data includes two-dimensional physical space coordinates and the ribonucleic acid and protein expression level matrix corresponding to the two-dimensional physical space coordinates.

[0039] Furthermore, step S6 includes the following steps:

[0040] S61. Obtain externally known H&E-stained single-cell images and their corresponding real multi-omics sequencing matrices, and construct a training set including single-cell images. and the corresponding multi-omics sequencing matrix Simultaneously, a cross-modal large model is constructed, consisting of the feature extraction module Transformer and the hybrid expert network MoE. ;

[0041] During the training phase, single-cell images from the training set are used. The high-dimensional features are obtained by inputting the feature extraction module Transformer. The expression matrix is ​​then input into the hybrid expert network MoE to generate the prediction. Its basic computational logic is expressed as follows:

[0042] ;

[0043] in, For the number of expert networks, For the first Features of the output of an expert network The weights assigned to the gating network are compared with the predicted expression matrix. With the multi-omics sequencing matrix in the training set By analyzing the differences between them and updating the parameters, a cross-modal large model can be completed. Pre-training is used to establish a universal mapping relationship from H&E pathological morphology to multi-omics data;

[0044] S62. Perform single-cell morphology segmentation on the H&E staining images obtained in step S1 to obtain independent single-cell image patches. Simultaneously, the multi-omics expression matrix of the region of interest actually measured in step S4 is aggregated according to the cellular physical boundary to construct the real sequencing data of the region of interest of the current tissue sample slice. ;

[0045] S63. Perform weakly supervised fine-tuning, using real sequencing data. As physical anchors and weak supervision signals, single-cell images of the region of interest are input into a pre-trained cross-modal large model. To obtain the inferential omics data for this region of the current sample. Computational inference of omics data Compared with real sequencing data Error loss between modes, by minimizing the error loss for cross-modal large models Make adjustments;

[0046] S64. Utilize the calibrated cross-modal large model Input a single-cell image of a region of non-interest in the current tissue sample slice. Calculate and output the inference omics data for the corresponding non-interest regions. The calculation process is as follows:

[0047] ;

[0048] Finally, the actual sequencing data of the region of interest will be used. Inferential omics data from regions of non-interest The images are stitched together in physical space to generate a continuous multi-omics expression profile covering complete tissue sample slices:

[0049] ;

[0050] Among them, the continuous multi-omics expression profiles of the output complete tissue sample slices It contains high-fidelity multi-omics expression information of the entire tissue sample slice space, and completes cross-scale completion from local target detection to global map reconstruction.

[0051] Furthermore, step S63 includes the following steps:

[0052] Mean squared error is used as the loss function to calculate the inference omics data. Compared with real sequencing data Error loss between :

[0053] .

[0054] Furthermore, step S63 includes the following steps:

[0055] Cosine similarity loss is used as the loss function to calculate the inference omics data. Compared with real sequencing data Error loss between :

[0056] ;

[0057] D represents the total number of feature dimensions of the ROI region multi-omics sequencing matrix, and d represents the index of the dimension.

[0058] A spatial multi-omics sequencing system based on cross-modal generation, used to implement the aforementioned spatial multi-omics sequencing method based on cross-modal generation, includes:

[0059] The H&E staining image acquisition module is used to acquire H&E staining images of tissue sample sections.

[0060] The target tissue acquisition module is used to automatically identify and locate the region of interest on the H&E staining image. Based on the located region of interest, the tissue sample slice is cut to obtain the target tissue, and the target tissue is transferred to the detection chip.

[0061] The multi-omics expression matrix acquisition module is used to perform in situ co-localization detection of transcriptome and proteome of target tissue on the detection chip to obtain the multi-omics expression matrix.

[0062] The spatial multi-omics data registration module extracts matching spatial feature anchor points from the acquired H&E staining images and multi-omics expression matrices, performs coordinate transformation based on the coordinate mapping relationship of the spatial feature anchor points, and outputs spatial multi-omics data registered to the same physical coordinate system; the spatial multi-omics data includes two-dimensional physical spatial coordinates and ribonucleic acid and protein expression level matrices corresponding to the two-dimensional physical spatial coordinates.

[0063] The cross-modal large model training module constructs a training set based on externally known H&E-stained single-cell images and their corresponding real multi-omics sequencing matrices, and trains a cross-modal large model composed of the feature extraction module Transformer and the hybrid expert network MoE based on the training set.

[0064] The full sample completion module generates multi-omics expression profiles of complete tissue sample slices based on spatial multi-omics data and external tissue sample pathological images, using a cross-modal large model.

[0065] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0066] 1. This invention significantly reduces experimental costs by automatically identifying and sequencing only regions of interest, and solves the problem of not being able to sequence large areas of tissue in their entirety.

[0067] 2. This invention obtains real reference data through on-site testing. Compared with methods that rely entirely on computer calculations, the generated results are more reliable and have higher data credibility.

[0068] 3. This invention utilizes spatial reconstruction technology to connect scattered sampling points into a complete distribution surface, realizing information completion from local detection to the entire sample range. Attached Figure Description

[0069] Figure 1 This is a flowchart of a spatial multi-omics sequencing method based on cross-modal generation.

[0070] Figure 2 This is a flowchart of the automatic identification process in step S2 of the present invention.

[0071] Figure 3 This is a flowchart of the full sample information completion process in step S6 of the present invention. Detailed Implementation

[0072] The present invention will be further described below with reference to specific embodiments.

[0073] Example 1

[0074] See Figure 1As shown, the spatial multi-omics sequencing method based on cross-modal generation provided in this embodiment, which identifies tertiary lymphoid structures in tissue sections as target regions, is described in detail, including the following steps:

[0075] S1. Obtain fresh frozen tumor tissue sample sections, perform H&E staining on the tumor tissue sample sections, and use a high-resolution microscopy scanning device to scan the entire sample to obtain H&E staining images of the tumor tissue sample sections.

[0076] S2, see also Figure 2 As shown, an unsupervised clustering model is used to automatically identify and locate regions of interest (ROIs) in H&E stained images, including:

[0077] S21. Construct the H&E staining image obtained in step S1 into a tissue spatial domain. Morphological representation functions on:

[0078] ;

[0079] in, The expression function representing the pathological image, specifically the expression function of the H&E staining image. This represents the two-dimensional spatial region where the tissue sample slice is located. This represents the spatial coordinates of a pathological image; by employing a sliding window sampling strategy, the pathological image is divided into multiple local image blocks, forming a set of local tissue structure units.

[0080] ;

[0081] in, A set of local image patches Represents the set of elements. A local image patch, The sample number for the image patch. , This indicates the total number of local image patches.

[0082] S22. Using feature extraction mapping functions For local image patch set Encode each sample by numbering it. The local image patches are transformed into representation vectors in the feature space, that is, the tissue morphology is mapped to a feature space of a unified dimension:

[0083] ;

[0084] in, Indicates the first Feature representation vectors corresponding to each image patch The dimension of the feature space is represented in this embodiment. =512.

[0085] S23. Perform structural pattern clustering based on feature space distance; in the feature space, let the set of cluster centers be... ; by calculating the feature representation vector With cluster center The Euclidean distance between them determines each local image patch. Category label :

[0086] ;

[0087] in, For cluster category numbers, , Indicates the first Cluster centers, This represents the total number of cluster categories. For the first The category labels are assigned to local image patches; through this unsupervised clustering operation, local image patches that all contain high-density lymphocytes will be automatically assigned to the same category label.

[0088] S24. Based on prior knowledge of the morphological characteristics of tertiary lymphoid structures, namely, the presence of high-density lymphocyte aggregation, regarding the category label... The process involves filtering and retaining local image patches that represent the target category of high-density lymphocyte aggregation. Then, in the original two-dimensional tissue physical space, connected component analysis is performed on the filtered local image patches, merging and stitching spatially adjacent patches to obtain a candidate region set for tertiary lymphoid structures.

[0089] ;

[0090] in, Represents the set of regions of interest. Represents the set of elements. Areas of interest The region of interest is numbered. , This represents the total number of regions of interest.

[0091] S25. For each numbered... Region of interest Boundary fitting enables precise localization of tertiary lymphoid structures in pathological images, and outputs the spatial coordinate matrix of the region of interest and the corresponding region mask.

[0092] S3. Based on the located region of interest, the tumor tissue sample slice is dissected to obtain the target tissue, and the target tissue is transferred to the detection chip, including the following steps:

[0093] S31. Based on the spatial coordinate matrix and region mask of the region of interest output in step S2, the target physical range of the tertiary lymphoid structure is manually marked on the solid tumor tissue sample slice using a high-power microscope and a fine marking pen.

[0094] S32. Use tools such as microdissecting scalpels and surgical blades to manually cut and peel away excess tissue outside the target area on the sample tissue section to obtain the target tissue.

[0095] S33. Using precision tools such as fine tweezers or flexible micro-manipulation brushes, the cut target tissue is moved and attached to the predetermined position of the detection chip.

[0096] S4. Based on in situ sequencing, co-localization in situ detection of transcriptome and proteome of target tissue on the detection chip is performed to obtain a multi-omics expression matrix with single-cell resolution.

[0097] S5. In the acquired H&E staining image and multi-omics expression matrix, extract the matching spatial feature anchor points. Based on the coordinate mapping relationship of the spatial feature anchor points, perform coordinate transformation and output spatial multi-omics data registered to the same physical coordinate system. The spatial multi-omics data includes two-dimensional physical spatial coordinates and ribonucleic acid and protein expression level matrices corresponding to the two-dimensional physical spatial coordinates. This includes the following steps:

[0098] S51. In the H&E staining image obtained in step S1, manually select... Cell clusters with significant local morphological features are used as image anchors, and the number of anchors must meet the following requirements. Then, extract this The set of spatial coordinates of each anchor point in the first coordinate system, i.e., the two-dimensional pixel coordinate system of the pathological image:

[0099] ;

[0100] Let M be the coordinates of the Mth image anchor point in the first coordinate system.

[0101] S52. In the multi-omics expression matrix obtained in step S4 and its corresponding microscopic field of view, feature regions that strictly match the image anchor points in anatomical structure are manually selected and used as omics anchor points. Simultaneously, these feature regions are extracted. The set of spatial coordinates corresponding to each omics anchor point in the second coordinate system, i.e., the physical coordinate system of the sequencing instrument:

[0102] ;

[0103] The coordinates of the corresponding Mth omics anchor point in the second coordinate system.

[0104] S53. Based on the coordinate pairing relationship between image anchor points and omics anchor points, calculate the affine mapping matrix, and transform the coordinate system of the multi-omics representation matrix using the affine mapping matrix to complete the spatial dimension registration, including:

[0105] The spatial alignment process of cross-modal data is modeled as a two-dimensional affine transformation involving translation, scaling, and rotation, and the following homogeneous coordinate mapping equation is constructed:

[0106] ;

[0107] in, For the first The coordinates of each image anchor point in the first coordinate system. For the corresponding number The coordinates of each omics anchor point in the second coordinate system. In the matrix parameters, and Parameters for controlling the spatial translation of data in two dimensions. To control the parameters of data space rotation and scaling, the obtained affine mapping matrix is ​​used to perform a global batch linear transformation on the spatial coordinates of all single cells in the multi-omics expression matrix generated in step S4, transforming the entire matrix and aligning it with the first coordinate system of the H&E staining image. Finally, spatial multi-omics data registered to the same physical coordinate system is output; wherein, the spatial multi-omics data includes two-dimensional physical spatial coordinates and the corresponding ribonucleic acid and protein expression level matrices.

[0108] S6, see also Figure 3 As shown, based on spatial multi-omics data and external tissue sample pathological images, a cross-modal large model is used to generate multi-omics expression profiles of complete tissue sample slices, including the following steps:

[0109] S61. Obtain externally known H&E-stained single-cell images and their corresponding real multi-omics sequencing matrices, and construct a training set including single-cell images. and the corresponding multi-omics sequencing matrix Simultaneously, a cross-modal large model is constructed, consisting of the feature extraction module Transformer and the hybrid expert network MoE. .

[0110] During the training phase, single-cell images from the training set are used. The high-dimensional features are obtained by inputting the feature extraction module Transformer. The expression matrix is ​​then input into the hybrid expert network MoE to generate the prediction. In this embodiment, the hybrid expert network has more than 8 parallel expert branches, and its basic computational logic is represented as follows:

[0111] ;

[0112] in, For the number of expert networks, For the first Features of the output of an expert network The weights assigned to the gating network are compared with the predicted expression matrix. With the multi-omics sequencing matrix in the training set The differences between them were analyzed, and the backpropagation algorithm was used to update the network parameters to complete the cross-modal large model. Pre-training is used to establish a universal mapping relationship from H&E pathological morphology to multi-omics data.

[0113] S62. Perform single-cell morphology segmentation on the H&E staining images obtained in step S1 to obtain independent single-cell image patches. Simultaneously, the multi-omics expression matrices of the tertiary lymphoid structural regions actually measured in step S4 are aggregated according to the cellular physical boundaries to construct the true sequencing data of the tertiary lymphoid structural regions of the current tumor tissue sample slice. .

[0114] S63. Perform weakly supervised fine-tuning to extract the true sequencing data from the tertiary lymphoid structure regions. As physical anchors and weak supervision signals, single-cell images of the tertiary lymphoid structure region are input into a pre-trained cross-modal large model. To obtain the inferential omics data for this region of the current sample. ; Computational inference of omics data Compared with real sequencing data For the error loss between the two, the mean squared error is used as the loss function in this embodiment:

[0115] ;

[0116] This is the error loss. Minimizing this error loss is used for cross-modal large models. Make minor adjustments.

[0117] S64. Utilize the calibrated cross-modal large model Input a single-cell image of a non-interest region (i.e., a region that has not undergone omics sequencing) from the current tumor tissue sample slice. Calculate and output the inference omics data for the corresponding non-interest regions. The calculation process is as follows:

[0118] ;

[0119] Finally, the actual sequencing data of the tertiary lymphoid structures were used. Inferential omics data from regions of non-interest By stitching the data together in the same physical space, a continuous multi-omics expression profile covering a complete tumor tissue sample slice is generated:

[0120] ;

[0121] Among them, the continuous multi-omics expression profile of the complete tumor tissue sample slices was output. It contains high-fidelity multi-omics expression information of the entire tumor tissue sample slice, and completes cross-scale completion from local target detection to global atlas reconstruction.

[0122] Example 2

[0123] The spatial multi-omics sequencing method based on cross-modal generation provided in this embodiment differs from that in Embodiment 1 in that cosine similarity loss is used as the loss function in step S63:

[0124] ;

[0125] D represents the total number of feature dimensions of the ROI region multi-omics sequencing matrix, and d represents the index of the dimension.

[0126] Example 2

[0127] This embodiment discloses a spatial multi-omics sequencing system based on cross-modal generation, used to implement the spatial multi-omics sequencing method based on cross-modal generation described in Embodiment 1, including:

[0128] The H&E staining image acquisition module is used to acquire H&E staining images of tissue sample sections.

[0129] The target tissue acquisition module is used to automatically identify and locate the region of interest on the H&E staining image. Based on the located region of interest, the tissue sample slice is cut to obtain the target tissue, and the target tissue is transferred to the detection chip.

[0130] The multi-omics expression matrix acquisition module is used to perform in situ co-localization detection of transcriptome and proteome of target tissue on the detection chip to obtain the multi-omics expression matrix.

[0131] The spatial multi-omics data registration module extracts matching spatial feature anchor points from the acquired H&E staining images and multi-omics expression matrices, performs coordinate transformation based on the coordinate mapping relationship of the spatial feature anchor points, and outputs spatial multi-omics data registered to the same physical coordinate system; the spatial multi-omics data includes two-dimensional physical spatial coordinates and ribonucleic acid and protein expression level matrices corresponding to the two-dimensional physical spatial coordinates.

[0132] The cross-modal large model training module constructs a training set based on externally known H&E-stained single-cell images and their corresponding real multi-omics sequencing matrices, and trains a cross-modal large model composed of the feature extraction module Transformer and the hybrid expert network MoE based on the training set.

[0133] The full sample completion module generates multi-omics expression profiles of complete tissue sample slices based on spatial multi-omics data and external tissue sample pathological images, using a cross-modal large model.

[0134] Example 4

[0135] This embodiment discloses a non-transitory computer-readable medium storing instructions that, when executed by a processor, perform the steps of the spatial multi-omics sequencing method based on cross-modal generation as described in Embodiment 1.

[0136] In this embodiment, the non-transitory computer-readable medium can be a disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), USB flash drive, portable hard drive, etc.

[0137] Example 5

[0138] This embodiment discloses a computing device, including a processor and a memory for storing processor-executable programs. When the processor executes the program stored in the memory, it implements the spatial multi-omics sequencing method based on cross-modal generation described in Embodiment 1.

[0139] The computing device described in this embodiment may be a desktop computer, laptop computer, smartphone, PDA handheld terminal, tablet computer, programmable logic controller (PLC), or other terminal device with processor functionality.

[0140] The above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Therefore, any changes made in accordance with the shape and principle of the present invention should be covered within the protection scope of the present invention.

Claims

1. A spatial multi-omics sequencing method based on cross-modal generation, characterized in that, Includes the following steps: S1. Obtain H&E staining images of tissue sample sections; S2. Automatically identify and locate the region of interest on the H&E staining image; S3. Based on the located region of interest, the tissue sample slice is cut to obtain the target tissue and transferred to the detection chip; S4. Perform co-localization in situ detection of transcriptome and proteome on the target tissue on the detection chip to obtain a multi-omics expression matrix; S5. In the acquired H&E staining image and multi-omics expression matrix, extract the matching spatial feature anchor points, perform coordinate transformation based on the coordinate mapping relationship of the spatial feature anchor points, and output spatial multi-omics data registered to the same physical coordinate system; the spatial multi-omics data includes two-dimensional physical spatial coordinates and ribonucleic acid and protein expression level matrices corresponding to the two-dimensional physical spatial coordinates; S6. Based on spatial multi-omics data and external tissue sample pathological images, a cross-modal large model is used to generate multi-omics expression profiles of complete tissue sample slices.

2. The spatial multi-omics sequencing method based on cross-modal generation according to claim 1, characterized in that, Step S2 includes the following steps: S21. Construct the H&E staining image obtained in step S1 into a tissue spatial domain. Morphological representation functions on: ； in, Represents the function for expressing pathological images. This represents the two-dimensional spatial region where the tissue sample slice is located. This represents the spatial coordinates of a pathological image; by employing a sliding window sampling strategy, the pathological image is divided into multiple local image blocks, forming a set of local tissue structure units. ； in, A set of local image patches Represents the set of elements. A local image patch, The sample number for the image patch. , Indicates the total number of local image patches; S22. Using feature extraction mapping functions For local image patch set Encode each sample by numbering it. The local image patches are transformed into representation vectors in the feature space, that is, the tissue morphology is mapped to a feature space of a unified dimension: ； in, Indicates the first Feature representation vectors corresponding to each image patch Represents the dimension of the feature space; S23. Perform structural pattern clustering based on feature space distance; in the feature space, let the set of cluster centers be... ; by calculating the feature representation vector With cluster center The Euclidean distance between them determines each local image patch. Category label : ； in, For cluster category numbers, , Indicates the first Cluster centers, This represents the total number of cluster categories. For the first Category labels are assigned to local image patches; S24. Based on prior knowledge of the morphological characteristics of the target organizational structure, class labels are... The process involves filtering and retaining local image patches of the target category; then, connectivity analysis is performed on the filtered local image patches in the feature space to obtain a set of regions of interest for the target tissue structure. ； in, Represents the set of regions of interest. Represents the set of elements. Areas of interest The region of interest is numbered. , This represents the total number of regions of interest. S25. For each numbered... Region of interest Boundary fitting enables precise localization of target tissue structures in pathological images and outputs the spatial coordinate matrix of the region of interest and the corresponding region mask.

3. The spatial multi-omics sequencing method based on cross-modal generation according to claim 1, characterized in that, Step S3 includes the following steps: S31. Based on the spatial coordinate matrix and region mask of the region of interest output in step S2, manually mark the target range of the region of interest on the sample tissue slice. S32. By manually cutting and peeling, remove excess tissue outside the target area from the sample tissue slice to obtain the target tissue; S33. Move the cut target tissue and attach it to the predetermined position of the detection chip.

4. The spatial multi-omics sequencing method based on cross-modal generation according to claim 1, characterized in that, Step S4 includes: Based on in situ sequencing technology, multi-cycle fluorescence imaging and codebook decoding are performed on the target tissue. Combined with cell contouring, single-cell mapping of target molecules is completed, and finally a multi-omics expression matrix with single-cell resolution containing spatial coordinates, ribonucleic acid, and protein expression levels is generated.

5. The spatial multi-omics sequencing method based on cross-modal generation according to claim 1, characterized in that, Step S5 includes the following steps: S51. In the H&E staining image obtained in step S1, several characteristic cell clusters are manually selected, and the spatial coordinates of the several characteristic cell clusters in the first coordinate system are extracted as image anchor points. S52. In the multi-omics expression matrix and its corresponding microscopic field of view obtained in step S4, a feature region that matches the image anchor point is manually selected, and the spatial coordinates of the feature region in the second coordinate system are extracted as the omics anchor point. S53. Based on the coordinate pairing relationship between the image anchor point and the omics anchor point, calculate the affine mapping matrix, and transform the two-dimensional physical space coordinates in the spatial multi-omics data through the affine mapping matrix to complete the accurate registration of the physical space dimension, and output the spatial multi-omics data registered to the same physical coordinate system; wherein, the spatial multi-omics data includes two-dimensional physical space coordinates and the ribonucleic acid and protein expression level matrix corresponding to the two-dimensional physical space coordinates.

6. The spatial multi-omics sequencing method based on cross-modal generation according to claim 1, characterized in that, Step S6 includes the following steps: S61. Obtain externally known H&E-stained single-cell images and their corresponding real multi-omics sequencing matrices, and construct a training set including single-cell images. and the corresponding multi-omics sequencing matrix Simultaneously, a cross-modal large model is constructed, consisting of the feature extraction module Transformer and the hybrid expert network MoE. ; During the training phase, single-cell images from the training set are used. The high-dimensional features are obtained by inputting the feature extraction module Transformer. The expression matrix is ​​then input into the hybrid expert network MoE to generate the prediction. Its basic computational logic is expressed as follows: ； in, For the number of expert networks, For the first Features of the output of an expert network The weights assigned to the gating network are determined by the predicted expression matrix. With the multi-omics sequencing matrix in the training set By analyzing the differences between them and updating the parameters, a cross-modal large model can be completed. Pre-training is used to establish a universal mapping relationship from H&E pathological morphology to multi-omics data; S62. Perform single-cell morphology segmentation on the H&E staining images obtained in step S1 to obtain independent single-cell image patches. Simultaneously, the multi-omics expression matrix of the region of interest actually measured in step S4 is aggregated according to the cellular physical boundary to construct the real sequencing data of the region of interest of the current tissue sample slice. ; S63. Perform weakly supervised fine-tuning, using real sequencing data. As physical anchors and weak supervision signals, single-cell images of the region of interest are input into a pre-trained cross-modal large model. To obtain the inferential omics data for this region of the current sample. ; Computational inference of omics data Compared with real sequencing data Error loss between modes, by minimizing the error loss for cross-modal large models Make adjustments; S64. Utilize the calibrated cross-modal large model Input a single-cell image of a region of non-interest in the current tissue sample slice. Calculate and output the inference omics data for the corresponding non-interest regions. The calculation process is as follows: ； Finally, the actual sequencing data of the region of interest will be used. Inferential omics data from regions of non-interest The images are stitched together in physical space to generate a continuous multi-omics expression profile covering complete tissue sample slices: ； Among them, the continuous multi-omics expression profiles of the output complete tissue sample slices It contains high-fidelity multi-omics expression information of the entire tissue sample slice space, and completes cross-scale completion from local target detection to global map reconstruction.

7. A spatial multi-omics sequencing method based on cross-modal generation according to claim 6, characterized in that, Step S63 includes the following steps: Mean squared error is used as the loss function to calculate the inference omics data. Compared with real sequencing data Error loss between : 。 8. A spatial multi-omics sequencing method based on cross-modal generation according to claim 6, characterized in that, Step S63 includes the following steps: Cosine similarity loss is used as the loss function to calculate the inference omics data. Compared with real sequencing data Error loss between : ； D represents the total number of feature dimensions of the ROI region multi-omics sequencing matrix, and d represents the index of the dimension.

9. A spatial multi-omics sequencing system based on cross-modal generation, characterized in that, To implement the spatial multi-omics sequencing method based on cross-modal generation as described in any one of claims 1-8, comprising: The H&E staining image acquisition module is used to acquire H&E staining images of tissue sample sections. The target tissue acquisition module is used to automatically identify and locate the region of interest on the H&E staining image. Based on the located region of interest, the tissue sample slice is cut to obtain the target tissue, and the target tissue is transferred to the detection chip. The multi-omics expression matrix acquisition module is used to perform in situ co-localization detection of transcriptome and proteome of target tissue on the detection chip to obtain the multi-omics expression matrix. The spatial multi-omics data registration module extracts matching spatial feature anchor points from the acquired H&E staining images and multi-omics expression matrices, performs coordinate transformation based on the coordinate mapping relationship of the spatial feature anchor points, and outputs spatial multi-omics data registered to the same physical coordinate system; the spatial multi-omics data includes two-dimensional physical spatial coordinates and ribonucleic acid and protein expression level matrices corresponding to the two-dimensional physical spatial coordinates. The cross-modal large model training module constructs a training set based on externally known H&E-stained single-cell images and their corresponding real multi-omics sequencing matrices, and trains a cross-modal large model composed of the feature extraction module Transformer and the hybrid expert network MoE based on the training set. The full sample completion module generates multi-omics expression profiles of complete tissue sample slices based on spatial multi-omics data and external tissue sample pathological images, using a cross-modal large model.

Images