A histological image-based three-dimensional spatial multi-omics reconstruction method

Abstract

This invention discloses a three-dimensional spatial multi-omics reconstruction method based on histological images, comprising: acquiring continuous sections of biological tissue; selecting a portion of the sections, performing spatial single-omics sequencing on each section to obtain corresponding spatial single-omics measured data, and acquiring its morphological image, wherein the molecular information of each spatial single-omics measured data is different; acquiring only morphological images for the remaining sections; mapping the morphological images of all sections to a three-dimensional coordinate system; extracting the morphological features of single cells in each section; training a spatial multi-omics prediction model using all spatial single-omics measured data and their corresponding morphological images; inputting all morphological features into the trained model, performing encoding fusion, association calculation with a learnable prototype set, and decoding to output corresponding multi-omics prediction data; thereby generating three-dimensional spatial multi-omics expression data. This invention overcomes the cost and throughput bottlenecks of slice-by-slice sequencing, achieving high-resolution three-dimensional multi-omics reconstruction of large-volume tissues.

Description

Technical Field

[0001] This invention relates to the field of interdisciplinary technology of bioinformatics and computational pathology, and in particular to a three-dimensional spatial multi-omics reconstruction method based on histological images. Background Technology

[0002] Three-dimensional multi-omics technology aims to analyze the original distribution and interactions of biomolecules (such as RNA and proteins) in the three-dimensional space of tissues. It is a key link connecting tissue anatomy and complex molecular functions, and is of great significance for understanding the mechanisms of life processes and the occurrence and development of diseases.

[0003] Currently, the mainstream technical solutions for achieving three-dimensional spatial multi-omics sequencing are mainly divided into two categories: tissue slice stacking and reconstruction and in situ imaging technology.

[0004] Tissue slice stacking reconstruction schemes require continuous, dense slicing of tissue blocks, followed by high-resolution spatial multi-omics sequencing (such as spatial transcriptome sequencing) of each slice. Finally, image registration algorithms are used to reconstruct the three-dimensional structure from the two-dimensional data. However, this scheme has a significant drawback: its sequencing cost increases exponentially with tissue volume. Because it requires sequencing massive amounts of tissue slices, this technology becomes economically and operationally infeasible for large-volume tissue samples (such as entire organs or large tumors), severely limiting its potential for large-scale application.

[0005] In situ imaging techniques (such as multiplex fluorescence imaging) attempt to avoid physical tissue sections and perform imaging and sequencing directly within three-dimensional tissues. However, this approach faces three severe technical challenges: (1) Tissue thickness limitation: There is a physical trade-off between the low penetration efficiency of chemical reagents and the limitation of molecular diffusion within thick tissues, resulting in signal attenuation and insufficient imaging depth; (2) Sequencing resolution limitation: Maintaining molecular activity while ensuring tissue permeability is extremely difficult, making it difficult for the technology to simultaneously achieve single-cell-level resolution and high sequencing throughput; (3) Sequencing feature dimensional limitation: Due to the limited number of imaging channels and technical complexity, the number of omics features (such as the number of genes or proteins) that can be detected in a single experiment is limited, making it difficult to achieve true high-dimensional multi-omics coverage. Therefore, researchers have to make a difficult trade-off between operational complexity, spatial resolution, and multi-omics coverage, and often cannot achieve both simultaneously.

[0006] In summary, existing technologies, limited by high costs or inherent technical bottlenecks, struggle to economically and efficiently obtain high-resolution three-dimensional spatial multi-omics data of large-volume tissues. Therefore, there is an urgent need in this field for a new technological solution that can overcome these limitations, enabling the three-dimensional spatial multi-omics reconstruction of large-volume biological tissues at a lower cost and with less complexity.

[0007] It should be noted that the information disclosed in the background section above is only for understanding the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0008] The technical problem this application aims to solve is "how to overcome the fundamental limitations of existing three-dimensional spatial multi-omics technologies in terms of cost, throughput and dimensionality, so as to achieve economical, efficient and high-fidelity three-dimensional spatial multi-omics reconstruction of large-volume biological tissues".

[0009] The technical solution adopted in this application to solve the above-mentioned technical problems is as follows.

[0010] This application provides a three-dimensional spatial multi-omics reconstruction method based on histological images, including the following steps: Multiple slices of biological tissue that have undergone serial slicing were obtained; a portion of the slices were selected from the multiple slices, and spatial single-omics sequencing was performed on each slice in the selected portion to obtain the corresponding spatial single-omics measured expression data; and their tissue morphology images were obtained, wherein the molecular information of each spatial single-omics measured expression data is different; for the remaining slices, only their tissue morphology images were obtained. All tissue morphology images were registered, and all slices were mapped to a unified three-dimensional spatial coordinate system. A three-dimensional structural framework of the tissue was established along the slice depth direction, and the morphological features of single cells were extracted from the tissue morphology images of each slice along the slice depth direction. All spatial single-omics measured expression data were used as supervision signals to train a spatial multi-omics prediction model based on tissue morphology features; The morphological features of all extracted slices are input into the trained spatial multi-omics prediction model, which then performs the following operations: (a) Encode and fuse the input morphological features to generate a fused feature representation that includes spatial neighborhood context information; (b) The fused feature representation is correlated with a set of learnable prototypes to generate a feature representation that implies the mapping relationship between morphological features and omics modalities; (c) Decode the feature representation and output the multi-omics predicted expression data for all slices; Based on the multi-omics predicted expression data of all slices output by the spatial multi-omics prediction model, three-dimensional spatial multi-omics expression data of biological tissues are generated.

[0011] In some embodiments, the spatial multi-omics prediction model uses a domain adaptation module to eliminate domain shifts caused by differences in slice staining.

[0012] In some embodiments, the domain adaptation module includes a domain discriminator and a gradient inversion layer. The domain discriminator is used to determine the identity of the slice from which it originates based on the input morphological features. The gradient inversion layer is connected before the domain discriminator and is used to invert the data gradient pointing to the morphological feature extraction process so that the model can eliminate the staining batch differences between slices, thereby obtaining a domain-invariant feature representation.

[0013] In some embodiments, step (b) involves associating the fused feature representation with a learnable prototype set to generate a feature representation that implies the mapping relationship between morphological features and omics modalities. This is achieved through a cross-attention mechanism, where the fused feature representation is used as the source of the query vector, and the learnable prototype set is used as the source of the key vector and value vector.

[0014] In some embodiments, after generating a feature representation that implies the mapping relationship between morphological features and omics modalities, a self-attention calculation step is also included: using the feature representation as a query vector, key vector, and value vector to calculate and establish a global feature dependency relationship.

[0015] In some embodiments, the feature representation is decoded in step (c) by: recalibrating the feature representation into an omics embedding using a nonlinear module and dividing it into blocks along the omics layer dimension to obtain multiple independent blocks; and performing matrix multiplication on each block and the transpose of its corresponding learnable prototype matrix to output multi-omics prediction expression data.

[0016] In some embodiments, the morphological features are multi-scale morphological features extracted by a pre-trained computational pathology model. The extraction process includes: cropping image patches centered on single cells from a tissue morphology image; inputting the image patches into the pre-trained computational pathology model to obtain the corresponding image feature tensor; performing pooling operations on the image feature tensor in different ways to obtain local feature components and global feature components; concatenating the local feature components and global feature components to generate multi-scale morphological features; wherein, the local feature components are obtained by average pooling the central grid region of the image feature tensor; and the global feature components are obtained by global average pooling of the image feature tensor.

[0017] In some embodiments, when performing spatial single-omics sequencing, the sequencing panels or sequencing modalities corresponding to different slices are different, so as to obtain multiple sets of spatial single-omics measured expression data with different molecular information.

[0018] In some embodiments, the training and execution process of the spatial multi-omics prediction model is based on the principle of spatial diagonal integration. Spatial diagonal integration refers to using all spatial single-omics measured expression data of a portion of the slice as supervision signals, and learning the cross-slice and cross-modal mapping relationship between morphological features and omics expression levels through the spatial multi-omics prediction model, thereby achieving inference of multi-omics prediction expression data of all slices.

[0019] In some embodiments, an apparatus for reconstructing three-dimensional spatial multi-omics data based on histological images is also provided, comprising: The data acquisition module is used to: acquire multiple slices of biological tissue that have undergone serial slicing; select a portion of the slices from the multiple slices, perform spatial single-omics sequencing on each slice in the portion of the slices to obtain the corresponding spatial single-omics measured expression data, and acquire its tissue morphology image, wherein the molecular information of each spatial single-omics measured expression data is different; for the remaining slices, only their tissue morphology images are acquired. The data processing and model training module is used to: register all tissue morphology images, map all slices to a unified three-dimensional spatial coordinate system, and establish a three-dimensional structural framework of the tissue along the slice depth direction; extract the morphological features of single cells from the tissue morphology images of each slice along the slice depth direction; and train the spatial multi-omics prediction model based on tissue morphology features using all spatial single-omics measured expression data as supervision signals. The data prediction and reconstruction module is used to: input the morphological features of all extracted slices into a trained spatial multi-omics prediction model; encode and fuse the input morphological features to generate a fused feature representation containing spatial neighborhood context information; associate the fused feature representation with a learnable prototype set to generate a feature representation that contains the mapping relationship between morphological features and omics modalities; decode the feature representation and output the multi-omics prediction expression data of all slices; and generate three-dimensional spatial multi-omics expression data of biological tissue based on the multi-omics prediction expression data of all slices output by the spatial multi-omics prediction model.

[0020] The present invention has the following beneficial effects: This invention first utilizes the technical feature of "obtaining multiple slices of biological tissue after serial slicing; selecting a portion of the slices and performing spatial single-omics sequencing on each slice within that portion to obtain corresponding spatial single-omics measured expression data; and acquiring their tissue morphology images, wherein the molecular information of each spatial single-omics measured expression data is different; for the remaining slices, only their tissue morphology images are acquired." This principle leverages low-cost tissue morphology images to significantly reduce the reliance on high-cost spatial omics sequencing, thereby directly reducing the economic cost of three-dimensional spatial multi-omics reconstruction. Secondly, through the technical feature of "registering all tissue morphology images and mapping all slices to a unified three-dimensional spatial coordinate system, establishing a three-dimensional structural framework of the tissue along the slice depth direction," a unified and accurate spatial framework is provided for the data in all tissue slices, serving as the geometric basis for subsequent high-fidelity three-dimensional reconstruction. Thirdly, through the technical feature of "extracting the morphological features of single cells from the tissue morphology images of each slice along the slice depth direction," image information is transformed into feature data that can be processed by the model, providing a unified input for deep learning-based prediction. Fourth, by using the technical feature of “training a spatial multi-omics prediction model based on tissue morphology features by using all spatial single-omics measured expression data as supervision signals,” a learnable mapping relationship across modalities and slices from morphology to multi-omics expression has been established. This is the computational basis for realizing the transition from sparse measurement to dense prediction. Fifth, by using the technical feature of "inputting the morphological features of all extracted slices into the trained spatial multi-omics prediction model", the model can uniformly process all slices, thus bridging the gap in physical measurement. Obviously, the model's operation of "(a) encoding and fusing the input morphological features to generate a fused feature representation containing spatial neighborhood context information" can fully explore and utilize the rich spatial and depth information contained in the morphological features; its operation of "(b) associating the fused feature representation with a learnable prototype set to generate a feature representation containing the mapping relationship between morphological features and omics modalities" establishes a bridge between morphology and multi-omics, realizing cross-modal knowledge transfer and association; its operation of "(c) decoding the feature representation and outputting the multi-omics prediction expression data of all slices" ultimately achieves unified and collaborative prediction of all location omics data within the entire three-dimensional volume, ensuring data consistency and integrity. Ultimately, by leveraging the technical feature of "generating three-dimensional spatial multi-omics expression data of biological tissues based on the multi-omics prediction expression data of all slices output by the spatial multi-omics prediction model," sparse real data and dense prediction data are seamlessly integrated to form a complete and unified three-dimensional multi-omics dataset.

[0021] In summary, through the synergistic effect of the aforementioned technical features, this invention ultimately achieves a paradigm shift, transforming the reconstruction of three-dimensional spatial multi-omics from a traditional model relying on high-intensity physical measurements to a new model of "sparse sequencing-dense imaging-computational inference." This not only brings direct benefits such as significantly reduced economic costs and operational complexity, but more importantly, it systematically solves the core bottleneck in this field: In terms of cost and throughput: By designing a method of "selecting a portion of tissue slices for sequencing while imaging the remaining tissue slices," this invention transforms the sequencing cost from a linear growth model proportional to tissue volume / number of tissue slices into a fixed-cost model. This makes three-dimensional multi-omics analysis of large-volume biological tissues (such as entire organs or tumors) "economically infeasible" to "economically feasible," breaking down the barriers to scaling.

[0022] Regarding data dimensionality, completeness, and consistency: Through the "spatial multi-omics prediction model" and its internal (a) encoding fusion, (b) association calculation, and (c) decoding operations, this invention can uniformly learn and infer a complete, high-dimensional, and internally consistent three-dimensional multi-omics map from limited experimental data. This cleverly avoids the "sequencing feature dimensionality limitation" caused by physical constraints in techniques such as in-situ imaging, and achieves seamless reconstruction of multi-omics modalities (such as transcriptomics and proteomics) in three-dimensional space.

[0023] In terms of technical feasibility and robustness: This invention bypasses the limitations of "tissue thickness" and "sequencing resolution" that cannot be simultaneously addressed in in-situ imaging techniques. By utilizing mature two-dimensional tissue section imaging and sequencing technologies as reliable data inputs, and by leveraging the powerful unified inference capabilities of the computational model to compensate for the lack of physical measurements, it provides a more stable and easier-to-implement high-resolution three-dimensional reconstruction scheme.

[0024] Therefore, the beneficial effects of this invention are multi-layered and systematic: it provides a three-dimensional spatial multi-omics reconstruction scheme for large-volume biological tissues that significantly reduces economic costs, greatly improves operational feasibility, ensures reliable technical implementation paths, and guarantees the integrity and consistency of three-dimensional data, providing new and effective technical means for life science research and clinical precision medicine.

[0025] Other beneficial effects of the present invention will be further described below. Attached Figure Description

[0026] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart of the overall process of the three-dimensional spatial multi-omics reconstruction method, which includes the experimental design process, the model inference process, and the output three-dimensional spatial multi-omics data. Figure 2 This is a schematic diagram of morphological feature extraction and network architecture (Spatial Omics Reconstruction Network, SPONGE Network); where a is a schematic diagram of single-cell morphological feature encoding, b is a schematic diagram of SPONGE network prediction model architecture, and c is a detailed diagram of attention mechanism and feedforward network module. Figure 3 This is a benchmark comparison chart of the present invention and existing mainstream spatial inference methods in the embodiments; Figure 4 This refers to the number of molecular features predicted by the present invention at different performance thresholds in the embodiments; Figure 5 In this embodiment, the present invention retrieves cell type tags passed to unsequencing slices using a majority voting strategy, and compares and verifies the obtained tag results with standard results obtained based on real measurement space multi-omics data inference. Figure 6 The present invention provides the crossover ratio analysis of cell types along the z-axis depth variation in the present invention, as well as the analysis results of the interaction between cell types in three-dimensional space. Detailed Implementation

[0027] The embodiments of the present invention will be described in detail below. It should be emphasized that the following description is merely exemplary and not intended to limit the scope and application of the present invention.

[0028] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0029] This invention recognizes the problems of high cost, limited throughput, and difficulty in handling large amounts of multi-omics data in existing three-dimensional spatial multi-omics sequencing technologies. Limited by the physical barriers of existing technologies, they fail to recognize the learnable mapping relationship between morphological features and multi-omics expression levels across tissue sections and modalities. This relationship can be effectively mined and established from sparse experimental data using deep learning models, thereby enabling low-cost construction of three-dimensional spatial multi-omics data. Therefore, this invention, by discovering and based on a deep understanding of this "morphology-omics" cross-modal association and its continuity in three-dimensional space, proposes an innovative paradigm of "sparse sequencing-dense imaging-computational inference." This involves sequencing only a small number of tissue sections, while imaging all tissue sections, and predicting omics expression data from morphological features using a specially designed spatial multi-omics prediction model. This addresses the economic cost and technical feasibility issues of three-dimensional spatial multi-omics reconstruction of large-volume biological tissues.

[0030] In some embodiments, this invention proposes a three-dimensional spatial multi-omics construction scheme based on histological images and a spatial diagonal integration algorithm. "Spatial diagonal integration" is a cross-omics feature integration and prediction mechanism based on a two-dimensional spatial-tissue slice approach. Since different single-omics sequencing (such as transcriptomics and proteomics) are physically distributed across continuous tissue slices at different depths along the Z-axis, a sparse data distribution of "sequencing slices and omics modalities interleaved" is formed. This invention extracts high-resolution morphological features of single cells in the XY plane from all tissue slices as a shared information bridge, enabling deep learning networks to perform cross-inference across slice depths along the Z-axis and different omics modalities (e.g., using real proteomic data from the 100th layer tissue slice to supervise and infer protein expression maps from the 1st layer and other intermediate unsequencing slices). This strategy of simultaneously utilizing spatial neighborhood morphological information (XY axis) and continuous slice depth information (Z-axis) for multi-modal completion is referred to in this paper as spatial diagonal integration.

[0031] In some embodiments, refer to Figure 1 , 2 This application allows for three-dimensional spatial multi-omics reconstruction using only sparse spatial omics data and dense tissue images in conjunction with "spatial diagonal integration." This approach reduces sequencing costs and complexity, overcomes the limitations of traditional three-dimensional sequencing in terms of sequencing panels and sequencing depth, and lays the foundation for biological research using three-dimensional multi-omics data. Specifically, it includes the following steps: Step S1: Obtain continuous tissue slices and perform multimodal data acquisition: (1) The complete tissue block was continuously sliced ​​along the z-axis to obtain... A continuous tissue section with a specific thickness (e.g., 5~10μm); (2) From Representative tissue sections were selected. Spatial single-omics sequencing was performed on sparse tissue slices. To maximize volumetric feature representation and reduce costs, boundary slices are preferentially selected as sequencing slices, and different slices have different omics sequencing panels (e.g., spatial transcriptomics, spatial proteomics, spatial metabolomics, etc.). Furthermore, Each sparse tissue section requires histological staining and imaging (e.g., hematoxylin and eosin staining), ultimately constructing a sequence of morphological images and discrete... Comprehensive data of single-omics measurements in space; (3) For the remaining Each tissue section was subjected to the same histological staining and imaging.

[0032] Step S2: Tissue slice image registration and 3D volume reconstruction: (1) Perform quality control on morphological images, exclude tissue sections with artifacts such as tissue folds, tears or uneven staining, and arrange them in the order of tissue sections; (2) Extract and confirm consistent morphological landmarks (such as vascular branches, structural boundaries, etc.) between adjacent tissue sections. (3) Based on the confirmed landmarks, estimate the two-dimensional (2D) affine transformation matrix (including translation, rotation, scaling and shearing) for each pair of tissue slices. (4) An iterative registration method is used to align each tissue slice with its previous stable tissue slice, and the transformation matrix is ​​accumulated to construct a global registration framework; Step S3: Single-cell instance segmentation based on deep learning: For those with only morphological images For each tissue slice, a deep learning-based segmentation framework (such as Cellpose) is applied to perform automated single-cell segmentation, and the centroid of each single cell is returned as its spatial coordinates; for tissue images undergoing omics sequencing, the single-cell location coordinates provided during sequencing analysis can be used.

[0033] Step S4: Cell modeling and hierarchical feature extraction: (1) Based on the obtained single-cell spatial coordinates, extract image patches containing the target cell and its immediate spatial neighborhood from the tissue slice image. The initial pixel size of the cropped patch is 128×128 pixels (physical range approximately 30). μm The image was then scaled to 256×256 pixels using interpolation to fit the input requirements of the pathology pre-trained model and to preserve key micro-environment context information. (2) Use pre-trained computational pathology models (such as UNI, Conch, etc.) to extract features from image patches, and extract local feature representations respectively. (Average pooling is used to obtain the central grid) and global feature representation ( (derived from global feature average pooling), where , The two are concatenated along the feature dimension to generate a unified multi-scale morphological feature representation. The final feature vector dimension of the spliced ​​single cell is During model training, for batch sizes of... The cell set whose input feature tensor dimension is .

[0034] Step S5: Construct and train the Spatial Omics Reconstruction Network (SPONGE Network): (1) Input the morphological features of n sequencing slices and the morphological features of the adjacent n unsequencing slices into the primary feature extractor; for batch size of Input features (dimension 1) Each global feature component is subjected to an independent nonlinear module. and local feature components Recalibrated and concatenated, each nonlinear module consists of two linear layers connected in series, with each linear layer followed by a LeakyReLU activation function layer, resulting in a multi-scale morphological feature representation: ; in .

[0035] Subsequently, the multi-scale morphological features were represented. Input to the corresponding different omics layers Feature extraction is performed in a series of joint nonlinear modules. Each joint nonlinear module consists of a single linear layer and a LeakyReLU activation function layer. The outputs of each module are stacked to generate a multi-omics feature representation. , in .

[0036] (2) ProtoEncoder and Cross-Attention Calculation: Constructing a set of learnable prototypes The prototype encoder, the prototype set It is initialized as a set of trainable parameter matrices, where For the number of omics layers, each prototype Corresponding to the Each omics layer serves as a potential anchor point. Representing the feature dimensions of this omics; representing multi-omics features With learnable prototype set The input is fed into the Multi-Head Cross-Attention (MHCA) mechanism, using multi-omics features. For query vectors, Cross-attention is performed on the key and value vectors. Specifically, the input is mapped to Q, K, and V vectors respectively through a learnable projection matrix, and scaled dot product attention is then performed. By combining the pre-layer normalization (Pre-LN) strategy with residual connections, a size-invariant refined feature representation is calculated: .

[0037] in .

[0038] (3) Self-attention calculation: After the cross-attention stage, the refined features It performs self-attention (MHSA) computation as Q, K, and V to establish global feature dependencies, capture complex intra-modal and inter-modal dependencies, and generates the final latent embedding through multiple iterations. .

[0039] (4) Omics Decoder: Utilizes non-linear modules to decode features Recalibrated for omics embedding By dividing the data into blocks along the omics layer, we obtain... A separate block Then, the various blocks... The transpose of its corresponding prototype matrix ( Perform matrix multiplication operations, and finally directly output the batch size as... The number of features is The predictive omics expression matrix.

[0040] (5) Domain Adaptation Based on Gradient Reversal Layer: In actual pathological slide preparation and imaging, even continuous tissue slices from the same tissue block will produce color and contrast differences (i.e., staining variations) in the final histological images due to slight differences in H&E staining time, reagent batches, tissue slice thickness (e.g., fluctuations of 5-10 μm), and different scanner calibrations. This non-biological systematic technical bias will form a significant domain shift in the high-dimensional feature space. If the model trained on a few sequenced slices is directly generalized to a large number of unsequencing pure imaging tissue slices without intervention, it will lead to severe overfitting and prediction distortion. In order to eliminate this domain shift caused by H&E staining, this application innovatively introduces an adaptive branch containing a gradient reversal layer (GRL) and a domain discriminator into the model. The domain discriminator consists of two linear layers, where the first linear layer is followed by a LeakyReLU activation function layer to identify which specific tissue slice the input features come from (i.e., identify the staining domain label); while the GRL reverses and scales the gradient through negative hyperparameters during backpropagation. This process forces the network's morphological feature encoder to ignore the staining domain noise specific to tissue sections and extract only robust features that are highly correlated with the underlying biological morphological structure, thereby ensuring the stability of multi-omics diagonal prediction and the ability to generalize across tissue sections.

[0041] (6) Calculate the loss function and iteratively optimize it: Select the loss function according to the characteristics of the omics data: For spatial transcriptomics data with high sparsity (a large number of zero values), use the weighted mean squared error loss (WMSE); for spatial proteomics data, use the standard mean squared error loss (MSE). At the same time, combine the cross-entropy loss function of the domain discriminator to jointly train the network until the model converges.

[0042] To further clarify the appendix Figure 2 The network unit structure shown in Figure c, including the multi-head self-attention module, the multi-head cross-attention module, and the feedforward neural network, correspond to the self-attention (MHSA) module, the multi-head cross-attention mechanism (MHCA) module, and the nonlinear module in the omics decoding stage of the network of this invention, respectively. Figure 2 The Chinese meanings of the various English symbols and signs in the Chinese character "c" are as follows: Let m represent the query vector, key vector, and value vector of the i-th attention head in the multi-head attention mechanism, respectively, where the superscript m indicates the number of attention heads; Linear represents a linear layer; Concat represents the feature concatenation operation; Layer Norm represents layer normalization; Scaled Dot-Product Attention represents scaled dot product attention calculation; and GEGLU represents the gated linear unit activation function.

[0043] Step S6: Multi-omics data inference and generation: Complete, unsequencing, pure imaging tissue slices are input into a trained multi-omics prediction network model, which outputs high-resolution 3D spatial multi-omics data.

[0044] Step S7: Experimental Validation and Model Performance Evaluation To verify the reliability and practicality of the reconstructed 3D multi-omics data predicted by this invention, this embodiment employs a hold-out method (using a portion of tissue slices with real sequencing data as a test set) for cross-validation. Specifically, we divided a complete hepatocellular carcinoma (HCC) tissue block into 100 continuous slices (N=100), and selected 3 slices (e.g., ID1, ID10, ID100) as anchor slices for spatial multi-omics sequencing. To simulate a sparse 3D spatial multi-omics scenario, we assigned non-overlapping spatial transcriptomics panels (Panel A and Panel B) and spatial proteomics panels (Panel C) to these 3 slices for model training. The model was then used to predict all panels of these 3 slices, and the results were compared with real sequencing data. Multiple quantitative evaluation metrics were introduced to comprehensively assess the predictive performance and 3D reconstruction effect of the SPONGE network.

[0045] (1) Multi-dimensional quantitative evaluation indicators: For the accuracy of regression prediction tasks, this application adopts the following two core evaluation indicators: Pearson correlation coefficient (PCC): This is the primary evaluation metric used to assess the linear relationship between the absolute magnitude of predicted molecular expression levels and actual observed expression levels.

[0046] Correlation Matrix Distance (CMD): Used to calculate the distance between the true correlation matrix and the predicted correlation matrix (value range 0 to 1, 0 indicates complete consistency), thereby globally assessing the degree to which the model preserves gene / protein co-expression structures and regulatory networks.

[0047] (2) Comparison of multi-omics prediction results and technological advantages: Refer to Figure 3The model of this invention was benchmarked against existing mainstream omics inference methods (OmiCLIP, STNet, STMCL, DeepPT, iSTAR, EGN, SpatialEx, mclSTExp, etc.). The results show that the SPONGE network achieved the highest average PCC and the lowest CMD across all evaluated omics levels. Specifically, in the cross-validation of multi-omics feature prediction, compared with existing suboptimal baseline methods (such as mclSTExp, etc.), the model achieved significant performance improvements in average prediction accuracy of up to 66.3%, 46.5%, and 23.5% (PCC Omics1: 0.3421 vs 0.2057, Omics2: 0.3258 vs 0.2223, Omics3: 0.6088 vs 0.4930). Meanwhile, referring to... Figure 4 This model generated the largest number of highly reliable (with high PCC values) predictions, establishing its leading advantage in spatial diagonal inference.

[0048] (3) Reconstruction and verification of 3D cell types: Refer to Figure 5 In this embodiment, the present invention retrieves and transmits cell type tags to unsequencing slices using a multi-data-ticket strategy, and compares and verifies the obtained tag results with standard results obtained based on real measurement space multi-omics data inference; in the figure, spatial 1 and spatial 2 represent two-dimensional spatial coordinates; the Chinese meanings of the cell types in the legend are as follows: CD8 Temra represents CD8+ effector memory T cells, Cholangiocyte represents bile duct cells, Endothelial represents endothelial cells, Fibroblasts represent fibroblasts, Hepaticstellate cells represent hepatic stellate cells, Hepatocytes represent hepatocytes, Macrophage represents macrophages, Malignant represents malignant cells, Monocyte represents monocytes, Malignant proliferating represents proliferating malignant cells, and Plasma represents plasma cells.

[0049] Regarding cell type labeling, the cell labels obtained by this system through retrieval and majority voting strategies were compared with the results of labeling based on multi-omics information. The results showed that the transferred labels highly matched the actual biological situation (label accuracy exceeded 70%). Furthermore, referring to... Figure 6This example illustrates the analysis of the Intersection over Union (IoU) ratios of various cell types along the z-axis depth, as well as the analysis results of interactions between cell types in three-dimensional space. In the figure, IoU Value represents the numerical value of the IoU ratio, and Vol. represents the voxel or volume level. In the legend of the left-hand line graph, Malignant vs Monocyte, Malignant Pro. vs Monocyte, Malignant vs Plasma, and Malignant Pro. vs Plasma represent the spatial co-location relationships between malignant cells and monocytes, proliferative malignant cells and monocytes, malignant cells and plasma cells, and proliferative malignant cells and plasma cells, respectively. In the right-hand heatmap, lg enrichment represents the logarithmic enrichment value, and the Chinese meaning of the cell type labels is... Figure 5 Consistent.

[0050] In the interaction analysis of the three-dimensional microenvironment, this system successfully quantified and captured complex three-dimensional biological dynamic changes using Intersection over Union (IoU) analysis and permutation test, such as the spatial co-location attenuation phenomenon of monocytes and malignant cells along the Z-axis depth. The above verification fully demonstrates that this invention not only achieves high-fidelity multi-omics inference in mathematical indicators, but also effectively supports precise 3D tissue microenvironment analysis and biological discovery, possessing both high practicality and reliability.

[0051] The background section of this invention may include background information about the problems or environment in which the invention is being developed, and is not necessarily a description of prior art. Therefore, the content included in the background section does not constitute an admission of prior art by the applicant.

[0052] The above description provides a further detailed explanation of the present invention in conjunction with specific / preferred embodiments, and it should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various substitutions or modifications can be made to these described embodiments without departing from the concept of the present invention, and all such substitutions or modifications should be considered within the scope of protection of the present invention. In the description of this specification, the reference to terms such as "an embodiment," "some embodiments," "preferred embodiment," "example," "specific example," or "some examples," etc., indicates that the specific features, structures, materials, or characteristics described in connection with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples. Although the embodiments of the present invention and their advantages have been described in detail, it should be understood that various changes, substitutions, and modifications can be made herein without departing from the scope of protection of the patent application.

Claims

1. A histological image-based three-dimensional spatial multi-omics reconstruction method, characterized in that, Includes the following steps: Multiple slices of biological tissue that have undergone serial slicing are obtained; a portion of the slices are selected from the multiple slices, and spatial single-omics sequencing is performed on each slice in the portion of the slices to obtain the corresponding spatial single-omics measured expression data; and their tissue morphology images are obtained, wherein the molecular information of each spatial single-omics measured expression data is different; for the remaining slices, only their tissue morphology images are obtained. All the tissue morphology images are registered, and all slices are mapped to a unified three-dimensional spatial coordinate system. A three-dimensional structural framework of the tissue is established along the slice depth direction, and the morphological features of single cells are extracted from the tissue morphology images of each slice along the slice depth direction. All spatial single-omics measured expression data were used as supervision signals to train a spatial multi-omics prediction model based on tissue morphology features; The morphological features of all extracted slices are input into the trained spatial multi-omics prediction model, which then performs the following operations: (a) Encode and fuse the input morphological features to generate a fused feature representation that includes spatial neighborhood context information; (b) The fused feature representation is correlated with a set of learnable prototypes to generate a feature representation that implies the mapping relationship between morphological features and omics modalities; (c) Decode the feature representation and output the multi-omics prediction expression data for all slices; Based on the multi-omics predicted expression data of all slices output by the spatial multi-omics prediction model, the three-dimensional spatial multi-omics expression data of the biological tissue is generated.

2. The histological image-based three-dimensional spatial multi-omics reconstruction method of claim 1, wherein, The spatial multi-omics prediction model uses a domain adaptive module to eliminate domain shifts caused by differences in slice staining.

3. The three-dimensional spatial multi-omics reconstruction method based on histological images according to claim 2, characterized in that, The domain adaptation module includes a domain discriminator and a gradient inversion layer. The domain discriminator is used to determine the identity of the slice from which it originates based on the input morphological features. The gradient inversion layer is connected before the domain discriminator and is used to invert the data gradient pointing to the morphological feature extraction process so that the model can eliminate the coloring batch differences between slices, thereby obtaining a domain-invariant feature representation.

4. The three-dimensional spatial multi-omics reconstruction method based on histological images according to claim 1, characterized in that, In step (b), the fused feature representation is associated with a learnable prototype set to generate a feature representation that contains the mapping relationship between morphological features and omics modalities. This is achieved through a cross-attention mechanism, where the fused feature representation is used as the source of the query vector, and the learnable prototype set is used as the source of the key vector and value vector.

5. The three-dimensional spatial multi-omics reconstruction method based on histological images according to claim 4, characterized in that, After generating a feature representation that contains the mapping relationship between morphological features and omics modalities, the method also includes a self-attention calculation step: using the feature representation as a query vector, key vector and value vector to calculate and establish a global feature dependency relationship.

6. The three-dimensional spatial multi-omics reconstruction method based on histological images according to claim 1, characterized in that, In step (c), the feature representation is decoded, which specifically includes: recalibrating the feature representation into an omics embedding using a nonlinear module, and dividing it into blocks along the omics layer dimension to obtain multiple independent blocks; performing matrix multiplication on each block and the transpose of its corresponding learnable prototype matrix to output the multi-omics prediction expression data.

7. The three-dimensional spatial multi-omics reconstruction method based on histological images according to claim 1, characterized in that, The morphological features are multi-scale morphological features extracted by a pre-trained computational pathology model. The extraction process includes: cropping image patches centered on single cells from the tissue morphology image; inputting the image patches into the pre-trained computational pathology model to obtain corresponding image feature tensors; performing pooling operations on the image feature tensors in different ways to obtain local feature components and global feature components; concatenating the local feature components and the global feature components to generate the multi-scale morphological features; wherein the local feature components are obtained by average pooling the central grid region of the image feature tensor; and the global feature components are obtained by global average pooling of the image feature tensor.

8. The three-dimensional spatial multi-omics reconstruction method based on histological images according to claim 1, characterized in that, When performing spatial single-omics sequencing, different slices correspond to different sequencing panels or sequencing modalities to obtain spatial single-omics measured expression data with different molecular information.

9. The three-dimensional spatial multi-omics reconstruction method based on histological images according to claim 1, characterized in that, The training and execution process of the spatial multi-omics prediction model is based on the principle of spatial diagonal integration. Spatial diagonal integration refers to using all spatial single-omics measured expression data of a portion of the slice as supervision signals, and learning the cross-slice and cross-modal mapping relationship between morphological features and omics expression levels through the spatial multi-omics prediction model, thereby realizing the inference of multi-omics prediction expression data of all slices.

10. An apparatus for reconstructing three-dimensional spatial multi-omics data based on histological images, characterized in that, include: The data acquisition module is used to acquire multiple slices of biological tissue that have undergone serial slicing. Select a portion of the slices from the plurality of slices, and perform spatial single-omics sequencing on each slice in the selected portion to obtain the corresponding spatial single-omics measured expression data and obtain its tissue morphology image. The molecular information of each spatial single-omics measured expression data is different. For the remaining slices, only their tissue morphology images are obtained. The data processing and model training module is used to: register all the tissue morphology images, map all slices to a unified three-dimensional spatial coordinate system, and establish a three-dimensional structural framework of the tissue along the slice depth direction; extract the morphological features of single cells from the tissue morphology images of each slice along the slice depth direction; and train the spatial multi-omics prediction model based on tissue morphology features using all spatial single-omics measured expression data as supervision signals. The data prediction and reconstruction module is used for: inputting the morphological features of all extracted slices into a trained spatial multi-omics prediction model; encoding and fusing the input morphological features by the model to generate a fused feature representation containing spatial neighborhood context information; associating the fused feature representation with a learnable prototype set by the model to generate a feature representation containing the mapping relationship between morphological features and omics modalities; decoding the feature representation by the model to output multi-omics prediction expression data of all slices; and generating three-dimensional spatial multi-omics expression data of the biological tissue based on the multi-omics prediction expression data of all slices output by the spatial multi-omics prediction model.

Images