Multimodal data processing method and apparatus, electronic device, and storage medium

Abstract

The present disclosure provides a multi-modal data processing method and device, an electronic device and a storage medium, and relates to the technical field of biological information. The method comprises the following steps: acquiring a multi-modal data matrix after spatial multi-omics data of a target slice is registered, wherein the multi-modal data matrix is used to describe the expression level of genes and the relative abundance of metabolites of each spatial site in the target slice; the spatial multi-omics at least includes spatial transcriptomics and spatial metabolomics; performing cross-modal analysis according to the multi-modal data matrix, wherein the cross-modal analysis includes spatial functional domain identification and / or spatial multi-omics correlation analysis. The embodiment of the present application can realize the joint processing of spatial transcriptome and spatial metabolome.

Description

Technical Field

[0001] This disclosure relates to the field of bioinformatics, specifically to a multimodal data processing method and apparatus, electronic device and storage medium. Background Technology

[0002] In recent years, with the continuous development of high-throughput sequencing technology and space omics research, a variety of space omics technologies have emerged, such as space transcriptomics and space metabolomics.

[0003] Single spatial transcriptomics or spatial metabolomics cannot fully understand the complexity and diversity of biological systems. Therefore, how to provide a method for joint processing of spatial multimodal data from spatial metabolomics and spatial transcriptomics has become an urgent technical problem to be solved. Summary of the Invention

[0004] The main objective of this application is to propose a multimodal data processing method, apparatus, electronic device, and storage medium, which aims to achieve joint processing of spatial transcriptomics and spatial metabolomics.

[0005] To achieve the above objectives, a first aspect of this application proposes a multimodal data processing method, the method comprising: A multimodal data matrix is ​​obtained after spatial multi-omics data registration of the target slice, wherein the multimodal data matrix is ​​used to describe the gene expression level and relative abundance of metabolites at each spatial site in the target slice; the spatial multi-omics includes at least spatial transcriptomics and spatial metabolomics; Cross-modal analysis is performed based on the multimodal data matrix, including spatial functional domain identification and / or spatial multi-omics association analysis. The spatial functional domain identification includes: constructing a spatial smoothing constraint based on the spatial adjacency strength between any two spatial points in the multimodal data matrix, and decomposing the multimodal data matrix under the spatial smoothing constraint to obtain the target factor score matrix and the target factor loading matrix; Based on the target factor score matrix, all spatial sites are divided into multiple spatial domains, and the functional modules of each spatial domain are determined based on genes and metabolites according to the target factor loading matrix. The spatial multi-omics association analysis includes: determining the local association strength between each gene and each metabolite in the local neighborhood of the spatial site based on the gene expression level and relative abundance of each neighboring site in the multimodal data matrix, and constructing a spatial local regulatory network based on the local association strength, wherein the spatial local regulatory network is used to describe the molecular regulatory relationship between each gene and each metabolite.

[0006] In some embodiments, the step of constructing a spatial smoothing constraint based on the spatial adjacency strength between any two spatial points in the multimodal data matrix, and decomposing the multimodal data matrix under the spatial smoothing constraint to obtain a target factor score matrix and a target factor loading matrix, includes: A spatial weight matrix is ​​constructed based on the spatial adjacency strength between any two spatial points in the multimodal data matrix; The multimodal data matrix is ​​initially decomposed into the product of an initial factor score matrix and an initial factor loading matrix. The initial factor score matrix is ​​used to describe the matching score of each factor corresponding to each spatial site, and the initial factor loading matrix is ​​used to describe the weight of each gene and metabolite in each factor. The factors represent spatial domains with biological functions. With the objective function convergence as the goal, the initial factor score matrix and the initial factor loading matrix are iteratively updated multiple times to obtain the target factor score matrix and the target factor loading matrix. The objective function is to minimize the sum of the reconstruction error and the spatial smoothing regularization term. The reconstruction error is determined based on the multimodal data matrix, the initial factor score matrix and the initial factor loading matrix after the previous iteration update, and the spatial smoothing regularization term is determined based on the spatial weight matrix and the initial factor score matrix after the previous iteration update.

[0007] In some embodiments, a method for registering spatial multi-omics data of the target slice includes: Spatial metabolomics data are used to generate spatial metabolomics-assisted registration images, which include metabolite intensity imaging maps or metabolite cluster spatial distribution maps. Acquire staining images of the spatial transcriptome, or generate oscillation-assisted registration images based on the spatial transcriptome data, wherein the oscillation-assisted registration images include gene expression heatmaps or spatial distribution maps of gene clusters. The empty-cycle-assisted registration image or the staining image of the spatial transcriptome is registered with the empty-cycle-assisted registration image.

[0008] In some embodiments, dividing all the spatial sites into spatial domains based on the target factor score matrix includes: For each spatial location, the factor corresponding to the highest score of the spatial location in the target factor score matrix is ​​used as the spatial domain label of the target spatial location; All spatial sites are spatially divided into spatial domains based on the spatial domain label corresponding to each spatial site.

[0009] In some embodiments, determining the functional modules of each spatial domain based on genes and metabolites according to the target factor loading matrix includes: For each factor, the top N genes and metabolites by weight in the target factor loading matrix are extracted to form the element set of the factor; where N is a positive integer greater than 1. The functional modules representing the spatial domain of the factor are determined based on the set of elements corresponding to the factor.

[0010] In some embodiments, determining the local association strength between each gene and each metabolite within the local neighborhood of the spatial site, based on the gene expression levels and relative abundance of each neighboring site in the multimodal data matrix, includes: Determine the number of sites within the local neighborhood of each spatial site; If the number of sites is greater than a preset threshold, extract the expression submatrix and abundance submatrix of all neighboring sites corresponding to the spatial site from the multimodal data matrix; For any gene in the expression sub-matrix and any metabolite in the abundance sub-matrix, the local association strength between the gene and the metabolite is calculated based on the expression vector of the gene in the expression sub-matrix and the abundance vector of the metabolite in the abundance sub-matrix.

[0011] In some embodiments, constructing a spatial local regulation network based on the local correlation strength includes: Multiple associated regulatory pairs were constructed based on each gene and each metabolite, and a local association strength matrix was constructed based on all local association strengths corresponding to each spatial site. For each correlation control pair, the global correlation strength of the correlation control pair is obtained by averaging the local correlation strengths of the correlation control pair across all local correlation strength matrices. Genes and metabolites indicated by association regulatory pairs with global association strength greater than a preset coefficient threshold are used as target nodes, and the global association strength corresponding to the association regulatory pairs is used as the edge weight of the connecting edges, so as to construct a spatial local regulatory network based on the target nodes, the connecting edges, and the edge weights.

[0012] To achieve the above objectives, a second aspect of this application provides a multimodal data processing apparatus, the apparatus comprising: The data acquisition unit is used to acquire a multimodal data matrix after spatial multi-omics data registration of the target slice, wherein the multimodal data matrix is ​​used to describe the gene expression level and relative abundance of metabolites at each spatial site in the target slice; the spatial multi-omics includes at least spatial transcriptomics and spatial metabolomics. A cross-modal analysis unit is used to perform cross-modal analysis based on the multimodal data matrix. The cross-modal analysis includes spatial functional domain identification and / or spatial multi-omics association analysis. The spatial functional domain identification includes: constructing a spatial smoothing constraint based on the spatial adjacency strength between any two spatial points in the multimodal data matrix, and decomposing the multimodal data matrix under the spatial smoothing constraint to obtain the target factor score matrix and the target factor loading matrix; Based on the target factor score matrix, all spatial sites are divided into multiple spatial domains, and the functional modules of each spatial domain are determined based on genes and metabolites according to the target factor loading matrix. The spatial multi-omics association analysis includes: determining the local association strength between each gene and each metabolite in the local neighborhood of the spatial site based on the gene expression level and relative abundance of each neighboring site in the multimodal data matrix, and constructing a spatial local regulatory network based on the local association strength, wherein the spatial local regulatory network is used to describe the molecular regulatory relationship between each gene and each metabolite.

[0013] To achieve the above objectives, a third aspect of this application provides an electronic device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0014] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0015] To achieve the above objectives, a fifth aspect of this application provides a computer program product comprising a computer program that is read and executed by a processor of a computer device, causing the computer device to perform the method described in the first aspect.

[0016] The multimodal data processing method, apparatus, electronic device, and storage medium proposed in this application first acquire a multimodal data matrix after spatial multi-omics data registration of target slices. Then, the multimodal data matrix is ​​decomposed to obtain a target factor score matrix and a target factor loading matrix. Based on the target factor score matrix, spatial domain division can be achieved for the registered spatial sites, and based on the target factor loading matrix, the functional modules of the spatial domain can be determined. And / or, the local association strength between each gene and each metabolite in the neighborhood sites of the registered spatial sites is determined, and a spatial local regulatory network is constructed based on the local association strength. Therefore, this application not only achieves the registration of spatial transcriptome data and spatial metabolome data, but also realizes the joint cross-modal analysis of spatial transcriptome and spatial metabolome. Attached Figure Description

[0017] The accompanying drawings are provided to further understand the technical solutions of this disclosure and constitute a part of the specification. They are used together with the embodiments of this disclosure to explain the technical solutions of this disclosure and do not constitute a limitation on the technical solutions of this disclosure.

[0018] Figure 1 This is a flowchart of the multimodal data processing method provided in the embodiments of this application; Figure 2 This is a flowchart of a specific embodiment of the multimodal data processing method provided in this application; Figure 3 This is a schematic diagram of the metabolite intensity imaging provided in the embodiments of this application; Figure 4 This is a schematic diagram of the spatial distribution of metabolite clusters provided in the embodiments of this application; Figure 5 This is a schematic diagram of the gene expression heatmap provided in the embodiments of this application; Figure 6 This is a schematic diagram of the spatial distribution map of gene clustering provided in the embodiments of this application; Figure 7 This is a flowchart illustrating an embodiment of decomposing a multimodal data matrix provided in this application. Figure 8 This is a flowchart of an iterative update method for the factor score matrix and factor loading matrix provided in an embodiment of this application; Figure 9 This is a flowchart of an embodiment of spatial domain partitioning provided in this application; Figure 10 This is a flowchart illustrating an embodiment of the determining functional module provided in this application. Figure 11 This is a flowchart of an embodiment of calculating the local association strength between genes and metabolites provided in this application; Figure 12 This is a flowchart of an embodiment of constructing a spatial local control network provided in this application; Figure 13 This is a schematic diagram of spatial metabolomics data provided in an embodiment of this application; Figure 14 This is a schematic diagram of the spot information table provided in an embodiment of this application; Figure 15 This is a schematic diagram of spatial transcriptome data provided in an embodiment of this application; Figure 16 This is a schematic diagram of an ssDNA staining image provided in an embodiment of this application; Figure 17 This is a schematic diagram illustrating the registration operation using the feature point-based affine transformation method provided in this application embodiment; Figure 18 This is a schematic diagram of the spatial point distribution of registration-after empty substitution and idling provided in the embodiments of this application; Figure 19 This is a schematic diagram of a registration-aided image of spatial transcriptome data provided in an embodiment of this application; Figure 20 This is a schematic diagram of the registration operation interface provided in an embodiment of this application; Figure 21 This is a schematic diagram showing the registration of the spatial metabolomics auxiliary image and the spatial transcriptomics auxiliary image provided in the embodiments of this application; Figure 22 This is a schematic diagram of the spatial weight matrix provided in an embodiment of this application; Figure 23 This is a schematic diagram of the target factor score matrix provided in the embodiments of this application; Figure 24 This is a schematic diagram of the target factor loading matrix provided in an embodiment of this application; Figure 25 This is a schematic diagram of the spatial domain distribution of cross-modal recognition provided in an embodiment of this application; Figure 26 This is a schematic diagram of the individual spatial distribution of the eight spatial domains provided in the embodiments of this application; Figure 27 This is a schematic diagram of the functional module distribution provided in the embodiments of this application; Figure 28 This is a schematic diagram of the average local correlation strength matrix provided in the embodiments of this application; Figure 29 This is a schematic diagram of the spatial local regulatory network of genes and metabolites provided in the embodiments of this application; Figure 30 This is a schematic diagram of the multimodal data processing device provided in the embodiments of this application; Figure 31This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this disclosure.

[0020] In recent years, the continuous development of high-throughput sequencing technology and spatial omics research has given rise to various spatial omics technologies. Spatial omics is an emerging molecular analysis technique that integrates spatial location information with multidimensional biomolecular information. Its core is to systematically analyze the types, abundances, spatial distribution patterns, and interactions of biomolecules such as genes, RNA, proteins, and metabolites in biological tissues while preserving their original spatial structure. This reveals the spatial heterogeneity of biomolecules in tissues, the functional synergy between cells (or regions), and the molecular mechanisms of overall tissue function. Based on the types of molecules detected, it can be divided into spatial transcriptomics, spatial metabolomics, and spatial proteomics, among others.

[0021] Spatial transcriptomics (STRA) refers to the in-situ detection of RNA (mainly mRNA, but also including non-coding RNA) expression profiles while preserving the spatial location of tissues, revealing the spatial distribution patterns of gene expression at the tissue region, cellular, or subcellular levels. Its core is to precisely correlate gene expression information with the spatial coordinates of tissue sections, revealing the heterogeneity of gene expression in different cells (or regions), and the regulatory relationship between this space-specific expression and tissue function, cell interactions, or growth and development. Spatial transcriptomics technologies can be broadly classified into two categories based on their principles: imaging-based and sequencing-based. Imaging-based technologies mainly include in situ hybridization techniques, such as fluorescence in situ hybridization (FISH) and its derivatives, including single-molecule fluorescence in situ hybridization (smFISH) and multiplex fluorescence hybridization techniques (SeqFISH, MERFISH). Sequencing-based technologies mainly include microdissection, in-situ sequencing, and in-situ spatial barcoding technologies, including laser capture microdissection (LCM) sequencing, FISSEQ, 10×Genomics' Visium, and Stereo-seq technology, respectively. Among them, sequencing-based spatial transcriptomics has the advantages of high-throughput and high-resolution spatial information localization, which can simultaneously detect the expression of tens of thousands of genes with precision down to the single-cell or even subcellular level. It can determine whether a gene is expressed and also pinpoint its specific location in the tissue.

[0022] Spatial metabolomics (STM) refers to the technique of qualitative and quantitative analysis of the types, amounts, and regionally specific distribution of small molecule metabolites (such as amino acids, lipids, carbohydrates, neurotransmitters, and secondary metabolites) in the in-situ spatial dimension of tissue sections. It is primarily based on mass spectrometry (MSI) technology, integrating metabolomics with spatial localization techniques. It allows for direct spatial localization and quantitative analysis of metabolites on tissue sections, revealing their spatial distribution within the tissue. Depending on the ion source and mass spectrometer type, the mainstream techniques are currently matrix-assisted laser desorption / ionization mass spectrometry (MALDI-MSI) and desorption / electrospray ionization mass spectrometry (DESI-MSI). The core MSI technology ionizes compounds on the sample surface through in-situ ionization, inputs the ions into a mass spectrometer, obtains the mass-to-charge ratio (m / z) and ion intensity, and then converts this into an ion distribution image using mass spectrometry imaging software, visualizing the spatial distribution characteristics of the analytes. Spatial metabolomics can not only accurately locate the spatial distribution of metabolites, but also analyze multiple metabolites simultaneously. It has the characteristics of high throughput and high resolution, can detect low-abundance metabolites, and provide comprehensive metabolic information.

[0023] Single spatial transcriptomics or spatial metabolomics methods cannot fully understand the complexity and diversity of biological systems; therefore, joint analysis using multiple spatial omics is essential. Joint analysis using spatial metabolomics and spatial transcriptomics technologies mainly includes image registration, data fusion, multi-omics spatial domain partitioning, and gene-metabolite correlation analysis. However, there is currently no mature joint analysis technique for spatial generation and rotation.

[0024] Based on this, embodiments of this application provide a multimodal data processing method and apparatus, electronic device and storage medium, which can jointly process spatial transcriptome data and spatial metabolome data.

[0025] The multimodal data processing method provided in this application relates to the field of bioinformatics. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the multimodal data processing method, but is not limited to the above forms.

[0026] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0027] The multimodal data processing method provided in the embodiments of this application is described below.

[0028] Reference Figure 1 and Figure 2 In some embodiments, the multimodal data processing method provided in this application includes, but is not limited to, steps S101 to S102.

[0029] Step S101: Obtain the multimodal data matrix after spatial multi-omics data registration of the target slice, wherein the multimodal data matrix is ​​used to describe the gene expression level and relative abundance of metabolites at each spatial site in the target slice; spatial multi-omics includes at least spatial transcriptomics and spatial metabolomics. Step S102: Perform cross-modal analysis based on the multimodal data matrix. Cross-modal analysis includes spatial functional domain identification and / or spatial multi-omics association analysis. The spatial functional domain identification includes: step S1021, constructing a spatial smoothing constraint based on the spatial adjacency strength between any two spatial points in the multimodal data matrix, and decomposing the multimodal data matrix under the spatial smoothing constraint to obtain the target factor score matrix and the target factor loading matrix; Step S1022: Based on the target factor score matrix, divide all spatial sites into multiple spatial domains, and determine the functional modules of each spatial domain based on the common definition of genes and metabolites according to the target factor loading matrix. The spatial multi-omics association analysis includes: step S1023, determining the local association strength between each gene and each metabolite in the local neighborhood of each spatial site based on the gene expression level and metabolite relative abundance of each neighborhood site in the multimodal data matrix, and constructing a spatial local regulatory network based on the local association strength, wherein the spatial local regulatory network is used to describe the molecular regulatory relationship between each gene and each metabolite.

[0030] In step S101 of some embodiments, the target slice may refer to a tissue slice of the target sample. Multi-omics data of the target slice is acquired, such as spatial transcriptomic data and spatial metabolomic data. Spatial transcriptomic data refers to the dataset obtained by performing transcriptomic measurements on multiple spatial sites (for ease of distinction, spatial sites in spatial transcriptomic data are defined as first spatial sites, which are detection points defined in spatial transcriptomic technology, such as spots or bins in Stereo-seq, each with unique two-dimensional coordinates) on the target tissue slice while preserving the tissue's spatial structure. This dataset is used to represent the expression level of each gene in the target tissue slice at each first spatial site. Spatial transcriptomic data is usually represented in the form of a gene expression matrix, where rows correspond to first spatial sites, columns correspond to genes, and matrix elements are quantitative values ​​of gene expression (such as UMI counts). Spatial metabolomics data refers to the dataset obtained by detecting metabolites at multiple spatial sites (for ease of distinction, these spatial sites are defined as second spatial sites, which are the scanning positions corresponding to each pixel in mass spectrometry imaging, and each second spatial site has unique two-dimensional coordinates) on a target tissue section while preserving the tissue's spatial structure. This dataset represents the relative abundance of each metabolite at each second spatial site in the target tissue section. Spatial metabolomics is typically represented as a metabolite abundance matrix, where rows correspond to second spatial sites, columns correspond to the mass-to-charge ratio (m / z) characteristics of metabolites, and matrix elements are the corresponding ionic intensity values ​​(representing relative abundance).

[0031] In practice, spatial metabolomics and spatial transcriptomics experiments are performed sequentially on the same target slice (or on two adjacent target slices, respectively). For example, using MALDI-IMS technology for spatial metabolomics experiments, the experimental sample (i.e., the target sample) is first acquired, and then sample preparation is performed. The sample is then cryo-sectioned using a slicer to obtain the target slice. The target slice is mounted onto a Stereo-seq chip, and after experimental processing such as matrix coating, the target slice is scanned using a MALDI mass spectrometer at a certain spatial resolution (e.g., pixel size of 50×50μm or 100×100μm) to obtain metabolite molecule information at different spatial sites (i.e., the second spatial site). The charge-to-weight ratio (m / z) signal of the metabolites is recorded using high-resolution mass spectrometry to generate a spatial ion mass spectrum. Further, peak alignment and extraction, normalization, and ion imaging visualization are performed using mass spectrometry imaging data analysis software such as SCiLS. Finally, the intensity information of each metabolite (mass-to-charge ratio m / z) at the second spatial site was obtained, and the metabolite abundance matrix was constructed.

[0032] Spatial transcriptomics experiments are conducted using Stereo-seq as an example. First, the target slide is stained (fluorescent or H&E) and photographed to obtain stained images. Then, permeabilization, RNA reverse transcription, tissue removal, cDNA release, recovery and amplification, library construction, and sequencing are performed to obtain spatial transcriptomics sequencing data. Further, standard analysis is performed using SAW software. Based on spatial barcodes and molecular barcodes (UMIs), the sequencing reads are assigned to their respective spatial locations and deduplicated, generating a gene expression matrix where rows correspond to spatial coordinates, columns correspond to genes, and elements are UMI counts. Simultaneously, the SAW software registers the stained images with the expression matrix, ensuring that each pixel corresponds to a spatial detection point (spot).

[0033] The obtained spatial transcriptome data and spatial metabolome data are registered to obtain a multimodal data matrix. Registration refers to aligning spatial sites (i.e., the first spatial site and the second spatial site) belonging to the same tissue slice (or adjacent tissue slices) but from different technology platforms through spatial coordinate transformation. This ensures that the aligned target spatial sites include both gene expression information and metabolite abundance information at the corresponding locations. The resulting multimodal data matrix is ​​a two-dimensional matrix with the number of rows equal to the number of spatial sites (e.g., n; for ease of distinction, the spatial sites in the multimodal data matrix are defined as target spatial sites) and the number of columns equal to the sum of the number of genes (e.g., p) and the number of metabolites (e.g., q). The multimodal data matrix is ​​used to represent the expression level and relative abundance of each gene and metabolite (or each selected gene and metabolite) in the target slice at the target spatial site after registration with the first and second spatial sites. That is, each row corresponds to a registered target spatial site (with a unified coordinate system), the first few columns store the expression level of each gene, and the last few columns store the relative abundance of each metabolite.

[0034] In some embodiments, the method for registering spatial multi-omics data of a target slice may include, but is not limited to, the following steps: Spatial metabolomics data are used to generate spatial metabolomics-assisted registration images, which include metabolite intensity imaging maps or metabolite cluster spatial distribution maps. Acquire stained images of the spatial transcriptome, or generate oscillation-assisted registration images based on the spatial transcriptome data. These oscillation-assisted registration images include gene expression heatmaps or spatial distribution maps of gene clusters. Registration was performed with the empty-generation-assisted registration image based on the chromatographic image of the spatial transcriptome or the empty-generation-assisted registration image.

[0035] In this embodiment, before registering spatial transcriptome data and spatial metabolome data, it is necessary to address the issue that these two types of data may lack common tissue section staining images. Specifically, the normal spatial transcriptome process generally produces HE or ssDNA staining images, so they are usually not lacking. However, in some experiments, these staining images may not be produced or may be lost, and spatial metabolome experiments themselves usually do not produce staining images, thus lacking staining images for registration. To address this situation, before registration, this embodiment first generates auxiliary images for spatial registration (i.e., empty-cycle auxiliary registration images and empty-generation auxiliary registration images): For spatial metabolome, a metabolite intensity imaging map is generated based on the spatial metabolome data (e.g., directly generated using mass spectrometry imaging software such as SCiLS), such as... Figure 3 As shown, Figure 3This image shows the expression intensity of a metabolite with m / z 174.023 on the current slice. The color gradient from blue to green to red represents the metabolite intensity from low to high. Alternatively, a spatial distribution map of metabolite clusters can be generated (first cluster the metabolite intensity data using Seurat or Scanpy, then plot the spatial distribution map), such as... Figure 4 As shown, Figure 4 The spatial distribution of eight clusters on the target slice is shown. For spatial transcriptomics, gene expression heatmaps are generated based on spatial transcriptomics data (e.g., first generating the expression matrix using Stereo-seq's SAW or 10×Visium's Space Ranger, then generating the heatmap using plotting programs such as ggplot2), as shown below. Figure 5 As shown, Figure 5 The colors in the middle, from blue to red, represent gene expression from low to high; or, generate a spatial distribution map of gene clusters (similarly plotting gene expression clusters), such as... Figure 6 As shown, Figure 6 The spatial distribution of nine cell clusters on the target slice is shown. Furthermore, if the original spatial detection point density is low, upsampling can be used to generate a higher resolution image proportionally, meaning each original spatial coordinate corresponds to multiple pixels, thus forming a continuous and clear spatial distribution map.

[0036] Based on this, this application provides two registration methods. The first method is affine transformation registration based on feature point recognition: First, using spatial metabolomics image processing software such as SCiLS, a structurally clear spatial metabolomics-assisted registration image (i.e., a metabolite intensity imaging map or a metabolite cluster spatial distribution map) is selected, and feature points on three different lines (denoted as spot1, spot2, and spot3) are marked. The coordinates (x1, y1), (x2, y2), and (x3, y3) of these three feature points in the spatial metabolomics coordinate system (such as spatial metabolomics data) are obtained. Then, ImageJ is used to align the spatial metabolomics image onto the stained image or spatial transcriptomics-assisted registration image, and the corresponding coordinates (X1, Y1), (X2, Y2), and (X3, Y3) of these three feature points in the spatial transcriptomics coordinate system are recorded. Thus, the affine transformation matrix shown in Equation 1 can be constructed, where a, b, c, d, e, and f are the transformation parameters to be solved, and the corresponding transformation equation is shown in Equation 2.

[0037] (Formula 1), (Formula 2), After obtaining the transformation parameters by solving the transformation equations, the original coordinates of each first spatial locus in the spatial transcriptome data can be transformed according to these parameters, thereby aligning the two omics coordinate systems. Then, all the transformed coordinates are compiled to generate a coordinate correspondence table between the spatial transcriptome data and the spatial metabolome data, thus obtaining a unified coordinate system. The coordinates in this unified coordinate system are defined as the coordinates of the target spatial locus. It is understandable that, depending on the actual analytical needs, the spatial transcriptome data can also be used as a reference coordinate system to transform the coordinates of the spatial metabolome data accordingly; this is not limited.

[0038] The second method is registration based on ImageJ software: First, auxiliary images for spatial transcriptomics and spatial metabolomics (i.e., idling-assisted registration images and idling-assisted metabolomics registration images) are generated as described above. Then, the TrakEM2 plugin is loaded into ImageJ, and the idling-assisted registration image (or the stained image of spatial transcriptomics) is subjected to linear and nonlinear transformations using the idling-assisted metabolomics image as a reference image. After registration, the project log is exported to an XML file. The displacement matrix [[rx1,rx2], [ry1,ry2], [tx,ty]] is extracted from the log. Here, rx1 and rx2 represent the rotation information of the idling-assisted registration image along the x-axis, ry1 and ry2 represent the rotation information of the idling-assisted registration image along the y-axis, and tx and ty represent the translation information of the idling-assisted registration image along the x-axis and y-axis, respectively. This displacement matrix is ​​multiplied by the coordinate matrix of the transposed first spatial site to obtain the registered coordinate matrix (i.e., the coordinate matrix corresponding to the target spatial site). Finally, the registered matrix can also be used to draw an image to verify the registration accuracy.

[0039] After registration using any of the methods described above, the registered gene expression matrix (n target spatial sites × p genes) and metabolite abundance matrix (n target spatial sites × q metabolites) can first be non-negatively normalized to ensure that the values ​​of all genes and metabolites fall within the relevant scale range. Then, the non-negatively normalized gene expression matrix and metabolite abundance matrix are concatenated to construct the multimodal data matrix. It can be understood that when using spatial metabolomics data as the reference coordinate system, the coordinates of the registered target spatial sites are consistent with the original coordinates of the second spatial site in the spatial metabolomics data (i.e., the metabolomics coordinates remain unchanged), while the coordinates of the first spatial site in the spatial transcriptomics data are transformed and mapped to this coordinate system. Similarly, when using spatial transcriptomics data as the reference coordinate system, the registered target spatial sites are consistent with the original coordinates of the first spatial site in the spatial transcriptomics data (i.e., the transcriptomics coordinates remain unchanged), while the coordinates of the second spatial site in the spatial metabolomics data are transformed and mapped to this coordinate system.

[0040] In step S102 of some embodiments, cross-modal analysis can be performed based on the multimodal data obtained after registration. Cross-modal analysis may include spatial functional domain identification and / or spatial multi-omics association analysis. These two types of analysis will be described below.

[0041] First, the identification of spatial functional domains will be explained.

[0042] In step S1021 of some embodiments, the spatial adjacency strength between any two target spatial points in the multimodal data matrix is ​​determined, and a spatial smoothing constraint is constructed based on the spatial adjacency strength. The spatial adjacency strength can refer to the degree of spatial proximity calculated based on the spatial coordinates of the target spatial points, as shown in Formula 3 below. Typically, a Gaussian kernel function is used to convert the Euclidean distance between two points into a weight value between 0 and 1, with the closer the distance, the higher the adjacency strength.

[0043] (Formula 3), in, Indicates the target spatial location With the target spatial location The Euclidean distance between them This refers to the Gaussian kernel bandwidth parameter. In practical implementation, a spatial weight matrix W can be constructed. This matrix is ​​an n×n square matrix (where n is the number of target spatial sites), and the matrix elements are the spatial adjacency strengths. And set the diagonal elements to zero.

[0044] Assume the multimodal data matrix is , where n represents the number of target spatial locations. , Indicates the number of genes. This represents the number of metabolites. After constructing the spatial weight matrix W, under spatial smoothing constraints, the multimodal data matrix can be decomposed using the spatial weight matrix W to obtain the target factor score matrix and the target factor loading matrix. That is, the multimodal data matrix is ​​decomposed using nonnegative matrix factorization. The approximate decomposition is the product of two nonnegative matrices (target factor score matrix U and target factor loading matrix V): X ≈ UV. Wherein, the size of the target factor score matrix U is... , For a pre-defined number of factors, each row of the target factor score matrix U represents the score of a target spatial locus on z factors. A higher score indicates a stronger correlation between the target spatial locus and the biological function represented by that factor. The size of the target factor loading matrix V is... Each row of the target factor loading matrix V represents the loading weight of a factor on each gene and metabolite. The higher the weight, the greater the contribution of the gene or metabolite to the factor. Spatial smoothing constraints are used to force adjacent target spatial sites to tend to be similar in factor scores, thereby ensuring that the decomposed spatial domains and functional modules have a continuous and smooth transition on the target slice.

[0045] Reference Figure 7 In some embodiments, step S1021 may include, but is not limited to, steps S701 to S703.

[0046] Step S701: Construct a spatial weight matrix based on the spatial adjacency strength between any two spatial points in the multimodal data matrix; Step S702: The multimodal data matrix is ​​initially decomposed into the product of an initial factor score matrix and an initial factor loading matrix. The initial factor score matrix is ​​used to describe the matching score of each factor corresponding to each spatial site, and the initial factor loading matrix is ​​used to describe the weight of each gene and metabolite in each factor. A factor represents a spatial domain with biological function. Step S703: With the objective function convergence as the goal, the initial factor score matrix and the initial factor loading matrix are iteratively updated multiple times to obtain the target factor score matrix and the target factor loading matrix. The objective function is to minimize the sum of the reconstruction error and the spatial smoothing regularization term. The reconstruction error is determined based on the multimodal data matrix, the initial factor score matrix and the initial factor loading matrix after the previous iteration update, and the spatial smoothing regularization term is determined based on the spatial weight matrix and the initial factor score matrix after the previous iteration update.

[0047] In steps S701 to S702 of some embodiments, firstly, based on any two target spatial locations in the multimodal data matrix... With the target spatial location Spatial adjacency strength between First, a spatial weight matrix W is constructed. Then, the multimodal data matrix X is decomposed. The decomposition can be divided into two stages. The first stage is to randomly initialize two non-negative matrices (i.e., the initial factor score matrix U and the initial factor loading matrix V) such that the product of these two non-negative matrices is numerically approximately equal to the multimodal data matrix X. The initialization method can use random positive number assignment or deterministic initialization strategies such as non-negative bisingular value decomposition to obtain better convergence results. No specific restrictions are placed on the initialization method.

[0048] In step S703 of some embodiments, the second stage involves iteratively updating the initial factor score matrix U and the initial factor loading matrix V multiple times, with the objective function convergence as the goal, while keeping the product relationship unchanged. Here, "with the objective function convergence as the goal" means that after each iteration, it is determined whether the change in the value of the objective function relative to the previous round is less than a preset threshold (or no longer significantly decreasing). If so, the iteration stops and the optimal solution is considered to have been found. That is, the factor score matrix and factor loading matrix determined in the current iteration are used as the final target factor score matrix and target factor loading matrix. Specifically, as shown in Equation 4 below, the objective function can be defined as minimizing the sum of the reconstruction error and the spatial smoothing regularization term.

[0049] (Formula 4), In Formula 4, the first term is the reconstruction error, and the second term is the spatial smoothing regularization term (also the spatial smoothing error). The regularization parameter controls the intensity of spatial smoothing. The reconstruction error is determined based on the multimodal data matrix X, the factor score matrix U updated in the previous iteration, and the factor loading matrix V updated in the previous iteration. The reconstruction error measures the ability of the decomposition result to restore the original data; a smaller reconstruction error indicates that the decomposition retains more original information. The spatial smoothing regularization term is determined based on the spatial weight matrix W and the factor score U updated in the previous iteration. The spatial weight matrix W is used to calculate the graphical Laplacian matrix L = IW, where I represents the identity matrix. The spatial smoothing regularization term penalizes spatially adjacent locations with large differences in factor scores, making adjacent target spatial locations tend to have similar factor scores. In practice, a multiplicative update algorithm can be used for iterative solving. This algorithm ensures that the factor score matrix and factor loading matrix remain non-negative throughout the update process and monotonically decreases the objective function value.

[0050] This application embodiment quantifies the spatial adjacency strength between target spatial sites by constructing a spatial weight matrix, and initially decomposes the multimodal data matrix into the product of a factor score matrix and a factor loading matrix based on a nonnegative matrix factorization framework. Then, iterative updates are performed with the goal of minimizing the sum of reconstruction error and spatial smoothing regularization terms, ultimately obtaining the target factor score matrix and target factor loading matrix. This approach, on the one hand, preserves the high-dimensional information of the original data and extracts latent factors through dimensionality reduction; on the other hand, the spatial smoothing regularization term smooths the factor scores of adjacent spatial sites, thereby automatically identifying spatial domains with continuity and biological significance in tissue sections. Compared with related technologies, this application embodiment effectively overcomes the deficiency of existing methods in identifying cross-modal collaborative change functional modules. It can simultaneously utilize both gene and metabolite molecular information to define the biological significance of each factor, providing a systematic, spatially aware data fusion scheme for the joint analysis of spatial transcriptome data and spatial metabolome data, significantly improving the accuracy and interpretability of multimodal spatial domain partitioning and functional module identification.

[0051] Reference Figure 8 In some embodiments, the method of iteratively updating the initial factor score matrix and the initial factor loading matrix in each round may include, but is not limited to, steps S801 to S802.

[0052] Step S801: Fix the initial factor score matrix after the previous iteration update, and update the initial factor load matrix after the previous iteration update using the multimodal data matrix and the initial factor load matrix after the previous iteration update through the multiplication update algorithm to obtain the initial factor load matrix after the current iteration update. Step S802: Based on the initial factor loading matrix, multimodal data matrix, spatial weight matrix, and initial factor score matrix updated in this iteration, the initial factor score matrix updated in the previous iteration is updated using a multiplicative update algorithm with a spatial smoothing term to obtain the initial factor score matrix updated in this iteration.

[0053] In step S801 of some embodiments, in the current iteration, the initial factor score matrix is ​​fixed and the initial factor loading matrix is ​​updated. That is, the initial factor score matrix U updated in the previous iteration is treated as a known constant and remains unchanged, and the initial factor loading matrix V updated in the previous iteration is updated. Specifically, a multiplicative update algorithm can be used. This algorithm updates the matrix by multiplying and dividing element-wise, ensuring that the updated matrix elements remain non-negative and that the objective function value is monotonically delivered. Specifically, the update of the initial factor loading matrix V is shown in Formula 5 below: (Formula 5), in, This represents element-wise multiplication; division is also element-wise division. It is a very small positive number (e.g.) ), used to prevent division by zero errors. The physical meaning of Formula 5 is that the molecule ( This reflects the portion of the multimodal data matrix X that should be explained by the initial factor loading matrix V under the current initial factor score matrix U; the denominator ( The ratio reflects the reconstruction signal generated by the model itself under the current initial factor score matrix U (i.e., the initial factor score matrix after the previous iteration update) and the current initial factor loading matrix V (i.e., the initial factor loading matrix after the previous iteration update). A ratio greater than 1 indicates that the current initial factor loading matrix V is too small and needs to be increased, while a ratio less than 1 indicates that the current initial factor loading matrix V is too large and needs to be decreased. Adaptive adjustment is achieved through element-wise multiplication and division. In specific implementation, the initial factor score matrix U and the initial factor loading matrix V after the previous iteration update are substituted into Formula 5 to calculate the new V value, which is the initial factor loading matrix after the current iteration update.

[0054] In step S802 of some embodiments, in the current iteration, after updating the initial factor loading matrix V, the initial factor loading matrix V can be fixed and the initial factor score matrix U updated. Specifically, as shown in Equation 6 below, a multiplicative update algorithm with a spatial smoothing term can be used to update the initial factor score matrix U: (Formula 6), Among them, "multiplicative update algorithm with spatial smoothing term" can refer to the addition of a product term between the spatial weight matrix (i.e., W) and the current initial factor score matrix (i.e., the initial factor score matrix after the previous iteration update) to the multiplicative update algorithm of non-negative matrix factorization shown in Formula 5, so that the factor scores of adjacent spatial sites influence each other and tend to be similar during the update process. The numerator of Formula 6 contains two terms: the first term ( ) represents the product of the multimodal data matrix and the current initial factor loading matrix (i.e., the initial factor loading matrix updated through Equation 5 in this round); the second term ( ) represents the spatial smoothing term, where λ (>0) is the regularization parameter, controlling the strength of spatial smoothing. It adds the factor scores of neighboring sites to the current site after spatial weighting, causing the scores of adjacent sites to converge. The denominator also contains two terms: the first term ( The first term represents the current model's own reconstruction contribution, and the second term ( The term denoted by represents the self-regulating term resulting from spatial regularization. Through element-wise multiplication and division, this update rule maintains non-negativity while ensuring that the updated initial factor score matrix U can both effectively reconstruct the original multimodal data matrix and ensure a smooth and continuous distribution of factor scores among spatially adjacent sites. In practice, the initial factor score matrix U updated in the previous iteration, the initial factor loading matrix V obtained through step S801, and the spatial weight matrix W are substituted into Formula 6 above to calculate the new U value, which is the updated initial factor score matrix for this iteration.

[0055] Then, the initial factor score matrix and the initial factor loading matrix updated in this iteration can be used as the initial values ​​for the next iteration. Steps S801 and S802 are repeated until the objective function converges. When the iteration stops, the final initial factor score matrix U and initial factor loading matrix V are determined as the target factor score matrix and target factor loading matrix.

[0056] The iterative update method provided in this application realizes non-negative matrix decomposition of multimodal data matrices with spatial regularization. Specifically, step S801, with a fixed initial factor score matrix, uses a multiplicative update algorithm without spatial smoothing to update the initial factor loading matrix, ensuring the reconstruction of the original multi-omics data from the decomposition result. Step S802, with a fixed initial factor loading matrix, uses a multiplicative update algorithm with a spatial smoothing term to update the initial factor score matrix, introducing the spatial weight matrix as a regularization term into the update process, making the factor scores of adjacent target spatial sites mutually coupled and tending towards smoothness during iteration. The two steps are executed alternately and optimized collaboratively, jointly minimizing the sum of reconstruction error and spatial smoothing regularization term in the objective function.

[0057] In step S1022 of some embodiments, after obtaining the target factor score matrix U based on the method of the above embodiments, all target spatial sites can be divided into z different spatial domains according to the target factor score matrix U. Each spatial domain corresponds to a functional region in the target slice that has synergistic variation characteristics in gene expression and metabolite abundance. Simultaneously, for each factor (i.e., each spatial domain), the biological significance of the factor is jointly defined based on the high-load genes and high-load metabolites corresponding to that factor in the target factor loading matrix V. This biological significance constitutes the cross-modal functional module of the factor. For example, if the top N high-load genes and high-load metabolites in factor 1 are significantly enriched in functional pathways related to intestinal epithelial stem cells, then factor 1 may be defined as a "stem cell metabolic module." In this way, joint multimodal spatial domain identification and automatic mining of functional modules based on spatial transcriptome data and spatial metabolome data are achieved.

[0058] Reference Figure 9 In some embodiments, step S1022, "dividing all spatial sites into spatial domains according to the target factor score matrix", may include, but is not limited to, steps S901 to S902.

[0059] Step S901: For each spatial location, the factor corresponding to the highest score of the spatial location in the target factor score matrix is ​​used as the spatial domain label of the spatial location. Step S902: Divide all spatial sites into spatial domains according to the spatial domain label corresponding to each spatial site.

[0060] In step S901 of some embodiments, after the aforementioned iterative update convergence, the target score matrix is ​​finally obtained. For each target spatial site, the highest score of that target spatial site in the target score matrix is ​​determined, as well as the factor corresponding to the highest score (i.e., the factor with the largest value among the z factor scores corresponding to that target spatial site). In the target score matrix, a higher score indicates that the gene expression and metabolite abundance characteristics of that target spatial site are more relevant to the biological function represented by that factor. The factor with the highest score is used as the spatial domain label of that target spatial site. The spatial domain label is an identifier used to identify the category of a spatial domain, which assigns each target spatial site a category, indicating that the site belongs to a spatial functional region dominated by a certain factor. In specific implementation, firstly, the factor score vector (i.e., the row vector corresponding to that target spatial site) of each target spatial site can be extracted from the target factor score matrix U. Then, the factor score vector is traversed to find the index of the position where the maximum value is located (such as the factor identifier), and this index value is used as the spatial domain label of that target spatial site. Through this operation, each target spatial site is assigned a unique spatial domain label, providing a direct classification basis for subsequent spatial domain division.

[0061] In step S902 of some embodiments, after determining the spatial domain label of each target spatial site, spatial domain division can be performed on all target spatial sites according to the spatial domain label. Spatial domain division can refer to grouping target spatial sites with the same spatial domain label into the same spatial domain, thereby forming a set of spatial functional regions composed of continuous or non-contiguous regions on the target slice. Each spatial domain corresponds to a factor, representing a cross-modal functional module in which gene expression and metabolite abundance show synergistic changes. For example, all target spatial sites with a spatial domain label of 1 constitute the first spatial domain, all target spatial sites with a spatial domain label of 2 constitute the second spatial domain, and so on, until the target spatial sites with a spatial domain label of z constitute the z-th spatial domain. After grouping, the target spatial sites in each spatial domain can be mapped back to the geometric position of the target slice according to their spatial coordinates (i.e., the registered x and y coordinates), and plotting software (such as the ggplot2 package in R, the Seurat toolkit, etc.) can be used to draw each spatial domain on the target slice outline map with different colors. For example, a unique color can be assigned to each spatial domain, and then target spatial locations belonging to that domain can be marked with that color, thereby generating a visual image of the spatial domain distribution. This visual image can intuitively show the boundaries, positions, and relative sizes of different spatial domains in the target slice.

[0062] Understandably, due to the spatial smoothing constraints introduced in steps S703 and S802, the factor scores of adjacent target spatial sites are similar. Therefore, the actual spatial domains are usually presented as relatively continuous regional clusters with fewer isolated sites, which conforms to the real situation of continuous distribution of functional regions in biological tissues. In addition, if a small number of isolated sites are assigned different labels from the surrounding areas, post-processing smoothing can be performed based on spatial continuity to further improve the biological rationality of spatial domain division.

[0063] Reference Figure 10 In some embodiments, step S1022, "determining the functional modules of each spatial domain based on the common definition of genes and metabolites according to the target factor loading matrix", may include, but is not limited to, steps S1001 to S1002.

[0064] In step S1001, for each factor, the genes and metabolites with the top N weights in the target factor loading matrix are extracted to form the element set of the factor; where N is a positive integer greater than 1. In step S1002, the functional module of the spatial domain represented by the factor is determined according to the set of elements corresponding to the factor.

[0065] In step S1001 of some embodiments, a factor refers to a latent variable obtained through nonnegative matrix factorization, with each factor corresponding to a possible functional module. After iterative update convergence, each factor is represented by a row in the target factor loading matrix V. In the target factor loading matrix, a larger weight indicates a greater contribution of the corresponding gene (or metabolite) to the factor, meaning the corresponding gene (or metabolite) plays an important role in the biological process represented by the factor. In specific implementation, the row vector of the factor in the target factor loading matrix is ​​determined, and the weights of all genes and all metabolites in that row are sorted from largest to smallest. Multiple genes and metabolites with the highest weights are selected (the number of genes and metabolites selected can be the same or different). These selected genes and metabolites together constitute the set of elements most closely related to the factor. It can be understood that each factor is defined by a high-weight set of specific genes and metabolites (corresponding to a functional module), and its spatial distribution is characterized by the continuity of high-score regions, representing a spatial domain with biological function.

[0066] In step S1002 of some embodiments, after determining the set of elements corresponding to each factor, the functional module of the factor can be determined based on the genes and metabolites included in the set of elements. For example, functional enrichment analysis can be performed on the set of elements. Functional enrichment analysis is a bioinformatics method used to determine whether a given set of genes or metabolites is significantly over-occurring in a known biological pathway, molecular function, or cellular component. It typically uses statistical tests (such as hypergeometric tests or Fisher's exact tests) to calculate the significance of enrichment (P-value), and then performs multiple hypothesis testing corrections (such as FDR correction) to obtain the corrected Q-value. Commonly used functional enrichment analysis databases include Gene Ontology (GO, covering three dimensions: biological processes, cellular components, and molecular functions), Kyoto Encyclopedia of Genes and Genomes (KEGG, covering metabolic pathways, signal transduction pathways, etc.), the Reactome Pathway Database, and the Human Metabolome Database (HMDB), among others.

[0067] Furthermore, since each factor's element set contains both genes and metabolites, two strategies can be employed for functional enrichment analysis: the first is to perform independent enrichment analyses on the gene and metabolite portions separately, and then interpret them collectively; the second is to use a database integrating gene-metabolite pathways (such as KEGG, which contains pathways for both genes and metabolites) for joint enrichment analysis. In one implementation, firstly, all genes are extracted from the element set to form a gene subset, and all metabolites are extracted to form a metabolite subset. Then, GO enrichment analysis is performed on the gene subset, and KEGG pathway enrichment analysis is performed on the metabolite subset. For the gene subset, a p-value threshold (e.g., 0.05) and a minimum number of genes (e.g., 3) are set, and significantly enriched biological processes or pathways are output. For the metabolite subset, the Metabolite Set Enrichment Analysis (MSEA) method is used to map metabolite names to KEGG metabolic pathways or Smiles strings for pathway enrichment. Finally, the enrichment results from both parts are integrated to obtain the functional modules of the corresponding factors. For example, if a factor's corresponding gene subset is enriched in the "fatty acid oxidation" pathway, and its metabolite subset is simultaneously enriched in both "long-chain fatty acids" and "acylcarnitine" metabolites, then that factor can be collectively defined as a "fatty acid metabolism module." Through the above operations, each factor (representing a spatial domain) is given a clear biological function annotation, thereby linking spatial location information with cross-modal molecular regulatory mechanisms and providing interpretable modular knowledge for understanding the synergistic effects of genes and metabolites in the tissue microenvironment.

[0068] Secondly, the spatial multi-omics association analysis is explained.

[0069] In step S1023 of some embodiments, after registering the spatial transcriptome data and spatial metabolome data, in addition to performing the "multimodal spatial domain and functional module identification based on spatial weighted nonnegative matrix factorization" described in steps S1021 and S1022, considering spatial heterogeneity, a gene and metabolite may be highly covariant in a local region but not significant globally. Therefore, local spatial autocorrelation can also be introduced to perform spatial multi-omics association analysis to identify gene-metabolite interaction relationships occurring in specific tissue regions, construct spatial local regulatory networks, and identify key regulatory pairs in the tissue microenvironment.

[0070] Specifically, the first step is to determine the neighboring sites of the target spatial site. Neighboring sites can refer to multiple sites that are spatially adjacent to the target spatial site. For example, for the already registered [site name]... There are 1 target spatial location, whose spatial coordinates are: Based on the spatial coordinates and the k-nearest neighbor algorithm, find the nearest neighbor with the k-th nearest neighbor. The k nearest target spatial locations (e.g., using Euclidean distance) to the nth target spatial location (the value of k is not specifically limited) are defined as the kth target spatial locations. The k neighborhood sites of a target spatial location. These k neighborhood sites can form a neighborhood. Then, in each local neighborhood (i.e. Within a specific microenvironment, the local association strength is calculated. Local association strength refers to the statistical correlation between gene expression levels and the relative abundance of metabolites within that local neighborhood. It reflects the covariation trend between a gene and a metabolite in that specific microenvironment: a positive local association strength indicates that both increase or decrease synchronously within the local neighborhood; a negative strength indicates that their changes are in opposite directions; the larger the absolute value, the stronger the covariation relationship. Finally, a spatial local regulatory network is constructed based on the calculated local association strength. This network can be an undirected weighted network with genes as the first type of nodes, metabolites as the second type of nodes, and gene-metabolite regulatory pairs satisfying preset conditions as edges. The weights of the edges can be determined based on the local association strength. This spatial local regulatory network can characterize the synergistic regulatory relationships between genes and metabolites in different spatial local regions of the target slice.

[0071] Reference Figure 11 In some embodiments, step S1023, "determining the local association strength between each gene and each metabolite in the local neighborhood based on the gene expression level and metabolite relative abundance of each neighborhood site in the multimodal data matrix", may include, but is not limited to, steps S1101 to S1103.

[0072] Step S1101: Determine the number of sites in the local neighborhood of each spatial site. Step S1102: If the number of sites is greater than a preset threshold, extract the expression submatrix and abundance submatrix of all neighborhood sites corresponding to the spatial sites from the multimodal data matrix. Step S1103: For any gene in the expression sub-matrix and any metabolite in the abundance sub-matrix, calculate the local association strength between the gene and the metabolite based on the expression vector of the gene in the expression sub-matrix and the abundance vector of the metabolite in the abundance sub-matrix.

[0073] In some embodiments, steps S1101 to S1102, the number of sites may refer to the total number of neighboring sites determined according to a preset neighborhood definition (such as the k-nearest neighbor algorithm). The number of sites is compared with a preset number threshold. The number threshold is a preset integer used to determine the local neighborhood of the target spatial site. Whether it has sufficient statistical reliability, the quantity threshold can be set to 2 or other values, with no specific limitation on the value. When the number of sites exceeds this quantity threshold, the local neighborhood of the target spatial site... Perform subsequent calculations of local association strength; otherwise, remove the target spatial site (because a small sample size will lead to unstable estimation of local association strength).

[0074] Determining the target spatial location Once the number of sites exceeds a threshold, local neighborhoods can be extracted from the multimodal data matrix X. Corresponding expression submatrix abundance submatrix Expression submatrix It can refer to data extracted from a multimodal data matrix that only contains local neighborhoods. A submatrix of expression levels of each neighboring locus on each gene (which could be all genes in the multimodal data matrix; or, pre-screened hypervariable genes, such as the top 100 hypervariable genes; or, genes selected according to analysis requirements). The number of rows is equal to the local neighborhood. The number of neighboring sites (i.e., the number of sites) is equal to the number of genes analyzed. Abundance submatrix It refers to the data extracted from a multimodal data matrix that contains only local neighborhoods. An abundance submatrix showing the relative abundance of each neighborhood site on all metabolites (which could be all metabolites in the multimodal data matrix; or, pre-screened highly variable metabolites, such as the top 100 highly variable metabolites; or, metabolites selected according to analytical requirements). The number of rows and the expression submatrix The number of rows is equal, and the number of columns is equal to the number of metabolites being analyzed.

[0075] In step S1103 of some embodiments, as shown in Formula 7 below, according to the expression submatrix Any gene in this expression submatrix The corresponding expression vector (i.e., the expression vector of the gene in the expression submatrix) (corresponding column vectors in the matrix), and abundance submatrix Any metabolite in this abundance submatrix The corresponding abundance vector (i.e., the abundance vector of the metabolite in the abundance submatrix) The local association strength between the gene and the metabolite is calculated by using the corresponding column vector in the graph.

[0076] (Formula 7), in, This represents the strength of the local association between the g-th gene and the m-th metabolite. Represents the expression vector. Represents an abundance vector. and The lengths of all neighborhoods The number of neighboring sites included. This represents the Spearman rank correlation coefficient. It's understandable that the calculated local correlation strength ranges from -1 to 1, with positive values ​​indicating a strong local correlation. Intracellular genes and metabolites change synchronously (i.e., the relative abundance of metabolites also increases when gene expression increases), with negative values ​​indicating inverse changes, and the closer the absolute value is to 1, the stronger the correlation. Therefore, the embodiments of this application enable the quantitative assessment of each local neighborhood. The intensity of synergistic changes in endogenous genes and metabolites provides a quantitative basis for subsequent network construction.

[0077] Reference Figure 12 In some embodiments, the step S1023 of “constructing a spatial local control network based on the local correlation strength” may include, but is not limited to, steps S1201 to S1203.

[0078] Step S1201: Construct multiple associated regulatory pairs based on each gene and each metabolite, and construct a local association strength matrix based on all local association strengths corresponding to each spatial site; Step S1202: For each correlation control pair, calculate the mean of the local correlation strength of the correlation control pair in all local correlation strength matrices to obtain the global correlation strength of the correlation control pair. Step S1203: The genes and metabolites indicated by the association regulatory pairs with global association strength greater than a preset coefficient threshold are taken as target nodes, and the global association strength corresponding to the association regulatory pairs is taken as the edge weight of the connection edge, so as to construct a spatial local regulatory network based on the target nodes, connection edges and edge weights.

[0079] In step S1201 of some embodiments, an associated regulatory pair refers to a binary pair consisting of a gene and a metabolite (e.g., gene g and metabolite m constitute an associated regulatory pair), used to characterize the potential correlation between the gene and the metabolite. Pairwise combinations of all genes and all metabolites constitute multiple associated regulatory pairs, and the total number of associated regulatory pairs equals the number of genes analyzed multiplied by the number of metabolites analyzed. Here, "number of genes analyzed" and "number of metabolites analyzed" are determined in step S1102, in the expression submatrix. Column number and abundance submatrix The number of columns. Local correlation strength matrix. It is a two-dimensional matrix constructed for a valid target spatial site (i.e., a target spatial site whose number of sites is greater than the number threshold in step S1102), and is a local correlation strength matrix. The rows correspond to the analyzed genes (or metabolites), and the columns correspond to the analyzed metabolites (or genes). Each element in the matrix represents the local association strength between the gene and the metabolite within the neighborhood of the target spatial site, as calculated in step S1103. Through the above operations, multiple local correlation strength matrices can be constructed based on multiple valid target spatial locations.

[0080] In step S1202 of some embodiments, for each gene-metabolite association regulatory pair, the mean value of the corresponding element values ​​in the local association strength matrix of all effective target spatial sites is calculated to obtain the global association strength of the association regulatory pair. Specifically, the global association strength of each gene g-metabolite m association regulatory pair can be calculated according to the following formula 8. .

[0081] (Formula 8), Where n represents the number of effective target spatial locations. Global correlation strength It reflects the average covariance intensity of gene g and metabolite m across all local neighborhoods throughout the target slice.

[0082] In step S1203 of some embodiments, after determining the global association strength of all associated regulatory pairs, all associated regulatory pairs can be screened according to a preset coefficient threshold (the value is not limited). Specifically, the genes and metabolites indicated by the associated regulatory pairs whose absolute values ​​of global association strength are greater than the coefficient threshold are designated as target nodes. Simultaneously, the edges connecting the gene target node and the metabolite target node are designated as target edges (i.e., connection edges), and the absolute value of the global association strength of the associated regulatory pair is designated as the edge weight of the target edge. Thus, a spatial local regulatory network can be constructed based on all target nodes, target edges, and the edge weights of the target edges. Therefore, the spatial local regulatory network can be used to represent the regulatory relationship between each gene and each metabolite, and the weight of each edge reflects the average covariation strength between the gene and the metabolite at a spatial local scale.

[0083] Furthermore, after constructing the spatial local regulatory network G, community detection and visualization analysis can be performed on G to reveal the synergistic modules of genes and metabolites in the tissue microenvironment. Specifically, the Louvain algorithm can be used with edge weights as input to perform community detection on the spatial local regulatory network G, dividing G into multiple communities (modules), and assigning a module label to each target node, thereby constructing the core network G_main. Subsequently, a layout algorithm (such as the Fruchterman-Reingold layout) is used to visualize the core network G_main, where node types (genes or metabolites) are distinguished by different shapes, and nodes are classified and colored according to community categories, while the thickness of edges is adjusted according to weights. Through the above processing, genes and metabolites within the same community are considered to constitute a "cell-metabolic niche," representing molecular modules that synergistically change within that spatial region. The identification of this niche can be used for subsequent biological interpretation, such as determining the metabolic pathways or signal transduction pathways involved in each niche through enrichment analysis, and then inferring key gene-metabolite regulatory axes in specific tissue regions, providing direct evidence for revealing pathological mechanisms driven by spatial heterogeneity, discovering region-specific biomarkers or drug intervention targets.

[0084] The multimodal data processing method provided in this application firstly offers multiple registration methods based on feature point affine transformation and the TrakEM2 plugin of ImageJ software. It can also generate auxiliary registration images using omics data itself when tissue slide staining images are lacking, making it applicable to most spatial metabolomics and spatial transcriptomics techniques, significantly improving the method's versatility and platform compatibility. Secondly, by decomposing the multimodal data matrix, this application can automatically identify spatially continuous multimodal domains and functional modules jointly defined by genes and metabolites, achieving deep integration of spatial transcriptomics and spatial metabolomics data. This facilitates the analysis of tissue functional regions from the perspective of cross-modal molecular synergistic changes. Finally, by calculating gene-metabolite correlation coefficients within each local neighborhood and constructing a spatial local regulatory network, this application can discover multi-gene-multi-metabolite interaction relationships existing only in specific tissue microenvironments. This effectively compensates for the shortcomings of global correlation analysis in masking local regulatory signals, providing a powerful analytical method for revealing key gene-metabolite regulatory networks in tissue microenvironments, discovering region-specific biomarkers, and identifying drug intervention targets. In summary, this application has achieved significant technical results in three aspects: registration universality, multimodal spatial domain and functional module identification, and spatial local control network construction.

[0085] It is understood that the multimodal data processing method provided in the embodiments of this application is not limited to spatial transcriptome data and spatial metabolome data, but can also be applied to the registration between other omics data such as spatial proteome and spatial epigenome.

[0086] In a specific embodiment, such as Figure 2 As shown, taking a mouse colon sample as an example, after OCT embedding, the intestinal cross-section was cryosectioned using a microtome, and spatial transcriptomics and spatial metabolomics experiments were performed on the same slice. The multimodal data processing method provided in this application includes the following steps: 1. Obtain spatial metabolomics data.

[0087] Mass spectrometry imaging was used to scan mouse colon slices to obtain metabolite molecular information at different spatial sites. High-resolution mass spectrometry was then used to record the m / z signals of metabolites, generating spatial ion mass spectra. Subsequently, peak alignment and extraction, normalization, ion imaging visualization, and spatial feature extraction were performed using the SCiLS software. The final results include a spatial distribution matrix of metabolites (i.e., spatial metabolomics data, such as...). Figure 13 (as shown) and the spot information table (containing the spatial coordinates of the second spatial location, such as...) Figure 14 (As shown).

[0088] 2. Obtain spatial transcriptome data.

[0089] After completing the spatial metabolomics experiment, mouse colon sections were used for spatial transcriptomics experiments (or adjacent mouse colon sections were used). This process included tissue staining (fluorescence or H&E), photography, permeabilization, reverse RNA transcription, tissue removal, cDNA release and recovery, amplification, and library construction and sequencing to obtain sequencing data. Standard analysis was then performed in the SAW workflow, yielding results including an expression matrix in Gem format (i.e., spatial transcriptomics data, such as...). Figure 15 As shown, the matrix includes five columns: gene name, x-coordinate of gene expression, y-coordinate of gene expression, number of recognized molecules, and number of MIDs falling in exons; and ssDNA staining map of mouse colon slices registered with the expression matrix (e.g., ...). Figure 16 (As shown).

[0090] 3. Register the spatial metabolomics data with the spatial transcriptomics data.

[0091] The first method uses a feature-point-based affine transformation method for registration, and its process is as follows: a. Using SCiLS Lab software, mark three non-collinear points, namely spot1, spot2, and spot3, on the staining image corresponding to the spatial metabolome, according to the tissue outline (e.g., ...). Figure 17As shown, the white cross indicates the feature point.

[0092] b. Using ImageJ, align the spatial metabolomics mass spectrum onto the stained image (ssDNA map) of the spatial transcriptome, and record the coordinates of the three feature points in the spatial transcriptome coordinate system at this time.

[0093] c. The three feature points of the spatial metabolome are labeled (x1, y1), (x2, y2), and (x3, y3), and the corresponding points in the spatial transcriptome coordinate system are labeled (X1, Y1), (X2, Y2), and (X3, Y3). Using these two sets of coordinates, the affine transformation matrix for spatial metabolome registration is calculated (i.e., Equation 1). d. Use affine transformations to align the coordinate system of gene distribution in the spatial transcriptome with the coordinate system of metabolic distribution in the spatial metabolome. For example... Figure 18 The figure shows the distribution of spatial points after registration, including those for idling and idling. In the figure, black (sp) represents idling spatial points, and gray (sm) represents idling spatial points.

[0094] The second method uses the MageJ software for registration, and the process is as follows: a. Generate a corresponding TIF image (i.e., auxiliary image) for the spatial metabolomics data and spatial transcriptomics data. Each pixel in the TIF image corresponds to a square bin or cell in the spatial transcriptomics data, and the pixel's grayscale is associated with the clustering or gene expression of the spatial metabolomics data. For example... Figure 19 As shown, this is a registration aid image for spatial transcriptome data, with each pixel corresponding to 50×50 detection points in the spatial transcriptome.

[0095] b. Using the TrakEM2 plugin in ImageJ software, with the spatial metabolomics-aided image as a reference, perform linear and nonlinear transformations on the spatial transcriptomics-aided image until the two images completely overlap, and save the registration log file regist.xml. Figure 20 and Figure 21 As shown, Figure 20 This is a schematic diagram of the registration operation interface. The toolbar at the top left is used to perform operations such as image translation and rotation. The image list in the middle left is used to select the image to be operated on and the reference image. The thumbnails at the bottom left are shown; the image content currently displayed is within the red box. The operation interface on the right allows for image translation, selection, and scaling. The blue and green images are the spatial metabolomics auxiliary image and the spatial transcriptomics auxiliary image to be registered, respectively. Figure 21 This is a schematic diagram after registration, where the spatial metabolomics auxiliary image and the spatial transcriptomics auxiliary image are now completely overlapped.

[0096] c. Extract image displacement information from the regist.xml file, typically a numerical matrix. Multiply the two-dimensional coordinates of each spatial locus in the spatial transcriptome data with the extracted matrix to obtain the registered two-dimensional coordinates of each spatial locus.

[0097] 4. Identification of cross-modal spatial domains and functional modules based on spatial weighted nonnegative matrix factorization.

[0098] After performing SCTransform nonnegative normalization on the registered gene expression matrix and metabolite abundance matrix, highly variable genes and the top 200 highly variable metabolites were extracted and merged into a joint matrix X. Then, based on spatial coordinates, for each target spatial site in the joint matrix X, the weights of its other target spatial sites were calculated using a Gaussian kernel based on Euclidean distance, constructing a spatial weight matrix W (e.g., ...). Figure 22 As shown, the rows and columns are the IDs of the target spatial locations, and the matrix elements are the weights.

[0099] Based on the spatial weight matrix W, the graph Laplacian matrix L (regularization term) is calculated, and this graph Laplacian matrix L is used to perform spatially regularized nonnegative matrix decomposition analysis on the joint matrix X: Setting the number of factors (spatial domain) z=8, the joint matrix is ​​approximately decomposed into the product of two nonnegative matrices: X≈UV, where U is the factor score matrix (representing the score of each target spatial location on z factors), and V is the factor loading matrix (representing the weight of each factor on each feature). The factor score matrix U(0,1) and the factor loading matrix V(0,1) are randomly initialized, and then a multiplicative update algorithm is used to iteratively update the factor score matrix U and the factor loading matrix V, iterating 200 times to obtain the final result. Figure 23 The image shows the final target factor score matrix, where rows represent target spatial location IDs and columns represent factor identifiers. Figure 24 As shown, this is the final target factor loading matrix, where behavioral factors are identified, and columns represent gene and metabolite identifiers.

[0100] Finally, based on the target factor score matrix U, the highest-scoring factor for each target spatial location is extracted, and the highest-scoring factor is used as the label for that target spatial location. Based on this, cross-modal spatial domain partitioning is performed on the target spatial locations. For example... Figure 25 The diagram shows the spatial domain distribution of cross-modal recognition, which is divided into 8 spatial domains. Figure 26 As shown, this is a separate spatial distribution map of eight spatial domains, with black representing the target spatial locations contained in each spatial domain.

[0101] Furthermore, based on the target factor loading matrix V, it is column-wise split into gene loading and metabolite loading components. For each spatial domain, the top 50 genes and top 20 metabolites with the highest weights are identified, collectively defining a cross-modal functional module. Then, using a gene set scoring method, the score of each functional module at the target spatial site is calculated for this set of genes and metabolites. Finally, the scoring results are visualized using a plotting function. Figure 27 As shown, this is a functional module distribution diagram. For each spatial domain, the top 50 genes and top 20 metabolites with the highest weights are selected. Each element set is scored as a module, and then the distribution of each module score on the target slice is visualized.

[0102] 5. Analysis of multimodal molecular regulatory networks based on spatial local correlation networks.

[0103] Gene expression matrices and metabolite abundance matrices were extracted from the registered spatial transcriptome and spatial metabolome data Seurat objects, respectively. For each target spatial locus, the 20 nearest neighbors of each target spatial locus were calculated using Euclidean distance, and the k-nearest neighbor algorithm was used to find the 20 closest spatial loci, forming the neighborhood set of each target spatial locus. In addition, neighborhoods with less than 2 spatial loci were removed, and the remaining ones were considered as the effective neighborhood.

[0104] The top 200 highly variable genes and metabolites were selected, and genes or metabolites with a global variance of 0 were filtered out. For each effective neighborhood, corresponding submatrices were extracted from the gene expression matrix and metabolite abundance matrix to calculate the local Spearman correlation coefficient matrix of the effective neighborhood. Finally, the local correlation strength matrices of all effective neighborhoods were summed and averaged to obtain the average local correlation strength matrix. Thus, the average covariance strength of the top 200 genes and metabolites across all local neighborhoods was obtained, where positive values ​​indicate a positive correlation at the local scale, and negative values ​​indicate a negative correlation (e.g., ...). Figure 28 As shown, behavioral genes are listed as metabolites, and the element value is the average local association strength between genes and metabolites.

[0105] Elements with correlation coefficients > 0.1 are selected to obtain reliable gene-metabolite pairs, resulting in a set P of significantly associated regulatory pairs and a candidate edge set E. Then, an undirected weighted network (i.e., a spatially local regulatory network) is constructed based on the association set P, where genes and metabolites are nodes, significantly associated regulatory pairs are edges, and the absolute value of the correlation coefficient is used as the edge weight. The Louvain algorithm (with edge weights as input) is then used to perform community detection on the network, dividing it into multiple communities (modules), and each node is labeled with a module label, resulting in the final gene-metabolite cross-modal regulatory network. Finally, the Fruchterman-Reingold layout is used to visualize the network graph, with nodes (genes / metabolites) distinguished by different shapes, edge thickness displayed according to weight, and different modules represented by different colors. This results in multiple modules, each containing different genes and metabolites. Genes and metabolites within the same module can be defined as a "cell-metabolite niche," and the edge relationships reflect the gene-metabolite regulatory relationships. Figure 29 The diagram shows a spatial local regulatory network of genes and metabolites. Circles represent genes, boxes represent metabolites, and numbers are community numbers. Four communities were identified in the diagram.

[0106] Reference Figure 30 This application also provides a multimodal data processing apparatus, which includes: The data acquisition unit 3001 is used to acquire a multimodal data matrix after spatial multi-omics data registration of the target slice. The multimodal data matrix is ​​used to describe the gene expression level and relative abundance of metabolites at each spatial site in the target slice. Spatial multi-omics includes at least spatial transcriptomics and spatial metabolomics. The cross-modal analysis unit 3002 is used to perform cross-modal analysis based on a multimodal data matrix. Cross-modal analysis includes spatial functional domain identification and / or spatial multi-omics association analysis. Spatial functional domain identification includes: constructing spatial smoothing constraints based on the spatial adjacency strength between any two spatial points in the multimodal data matrix, and decomposing the multimodal data matrix under the spatial smoothing constraints to obtain the target factor score matrix and the target factor loading matrix; Based on the target factor score matrix, all spatial sites are divided into multiple spatial domains, and the functional modules of each spatial domain are determined based on the target factor loading matrix and the common definition of genes and metabolites. Spatial multi-omics association analysis includes: determining the local association strength between each gene and each metabolite in the local neighborhood of a spatial site based on the gene expression level and relative abundance of each neighboring site in the multimodal data matrix, and constructing a spatial local regulatory network based on the local association strength. The spatial local regulatory network is used to describe the molecular regulatory relationship between each gene and each metabolite.

[0107] It is evident that the content of the above-described multimodal data processing method embodiments is applicable to the embodiments of this multimodal data processing device. The specific functions implemented by this multimodal data processing device embodiment are the same as those of the above-described multimodal data processing method embodiments, and the beneficial effects achieved are also the same as those achieved by the above-described multimodal data processing method embodiments.

[0108] Reference Figure 31 , Figure 31 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 3101 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 3102 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 3102 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 3102 and is called and executed by the processor 3101 using the multimodal data processing method of the embodiments of this application. The input / output interface 3103 is used to implement information input and output; The communication interface 3104 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 3105 transmits information between various components of the device (e.g., processor 3101, memory 3102, input / output interface 3103, and communication interface 3104); The processor 3101, memory 3102, input / output interface 3103 and communication interface 3104 are connected to each other within the device via bus 3105.

[0109] This application also provides a computer program product, which includes a computer program. A processor of a computer device reads and executes the computer program, causing the computer device to perform the multimodal data processing method described above.

[0110] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described single-cell multimodal data processing method.

[0111] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0112] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0113] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0114] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0115] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0116] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0117] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0118] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0119] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0120] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0121] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0122] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A multimodal data processing method, characterized in that, The method includes: A multimodal data matrix is ​​obtained after spatial multi-omics data registration of the target slice, wherein the multimodal data matrix is ​​used to describe the gene expression level and relative abundance of metabolites at each spatial site in the target slice; the spatial multi-omics includes at least spatial transcriptomics and spatial metabolomics; Cross-modal analysis is performed based on the multimodal data matrix, including spatial functional domain identification and / or spatial multi-omics association analysis. The spatial functional domain identification includes: constructing a spatial smoothing constraint based on the spatial adjacency strength between any two spatial points in the multimodal data matrix, and decomposing the multimodal data matrix under the spatial smoothing constraint to obtain the target factor score matrix and the target factor loading matrix; Based on the target factor score matrix, all spatial sites are divided into multiple spatial domains, and the functional modules of each spatial domain are determined based on genes and metabolites according to the target factor loading matrix. The spatial multi-omics association analysis includes: determining the local association strength between each gene and each metabolite in the local neighborhood of the spatial site based on the gene expression level and relative abundance of each neighboring site in the multimodal data matrix, and constructing a spatial local regulatory network based on the local association strength, wherein the spatial local regulatory network is used to describe the molecular regulatory relationship between each gene and each metabolite.

2. The method according to claim 1, characterized in that, The step of constructing a spatial smoothing constraint based on the spatial adjacency strength between any two spatial points in the multimodal data matrix, and decomposing the multimodal data matrix under the spatial smoothing constraint to obtain the target factor score matrix and the target factor loading matrix, includes: A spatial weight matrix is ​​constructed based on the spatial adjacency strength between any two spatial points in the multimodal data matrix; The multimodal data matrix is ​​initially decomposed into the product of an initial factor score matrix and an initial factor loading matrix. The initial factor score matrix is ​​used to describe the matching score of each factor corresponding to each spatial site, and the initial factor loading matrix is ​​used to describe the weight of each gene and metabolite in each factor. The factors represent spatial domains with biological functions. With the objective function convergence as the goal, the initial factor score matrix and the initial factor loading matrix are iteratively updated multiple times to obtain the target factor score matrix and the target factor loading matrix. The objective function is to minimize the sum of the reconstruction error and the spatial smoothing regularization term. The reconstruction error is determined based on the multimodal data matrix, the initial factor score matrix and the initial factor loading matrix after the previous iteration update, and the spatial smoothing regularization term is determined based on the spatial weight matrix and the initial factor score matrix after the previous iteration update.

3. The method according to claim 1, characterized in that, A method for registering spatial multi-omics data of the target slice includes: Spatial metabolomics data are used to generate spatial metabolomics-assisted registration images, which include metabolite intensity imaging maps or metabolite cluster spatial distribution maps. Acquire staining images of the spatial transcriptome, or generate oscillation-assisted registration images based on the spatial transcriptome data, wherein the oscillation-assisted registration images include gene expression heatmaps or spatial distribution maps of gene clusters. The empty-cycle-assisted registration image or the staining image of the spatial transcriptome is registered with the empty-cycle-assisted registration image.

4. The method according to claim 2, characterized in that, The step of dividing all the spatial sites into spatial domains based on the target factor score matrix includes: For each spatial location, the factor corresponding to the highest score of the spatial location in the target factor score matrix is ​​used as the spatial domain label of the spatial location. All spatial sites are spatially divided into spatial domains based on the spatial domain label corresponding to each spatial site.

5. The method according to claim 2, characterized in that, The step of determining the functional modules of each spatial domain based on the target factor loading matrix, defined jointly by genes and metabolites, includes: For each factor, the genes and metabolites with the top N weights of the factor in the target factor loading matrix are extracted to form the element set of the factor; where N is a positive integer greater than 1. The functional modules representing the spatial domain of the factor are determined based on the set of elements corresponding to the factor.

6. The method according to claim 1, characterized in that, The step of determining the local association strength between each gene and each metabolite within the local neighborhood of the spatial site, based on the gene expression levels and relative abundance of each neighboring site in the multimodal data matrix, includes: Determine the number of sites within the local neighborhood of each spatial site; If the number of sites is greater than a preset threshold, extract the expression submatrix and abundance submatrix of all neighboring sites corresponding to the spatial site from the multimodal data matrix; For any gene in the expression sub-matrix and any metabolite in the abundance sub-matrix, the local association strength between the gene and the metabolite is calculated based on the expression vector of the gene in the expression sub-matrix and the abundance vector of the metabolite in the abundance sub-matrix.

7. The method according to claim 1, characterized in that, The construction of a spatial local regulation network based on the local correlation strength includes: Multiple associated regulatory pairs were constructed based on each gene and each metabolite, and a local association strength matrix was constructed based on all local association strengths corresponding to each spatial site. For each correlation control pair, the global correlation strength of the correlation control pair is obtained by averaging the local correlation strengths of the correlation control pair across all local correlation strength matrices. Genes and metabolites indicated by association regulatory pairs with global association strength greater than a preset coefficient threshold are used as target nodes, and the global association strength corresponding to the association regulatory pairs is used as the edge weight of the connecting edges, so as to construct a spatial local regulatory network based on the target nodes, the connecting edges, and the edge weights.

8. A multimodal data processing device, characterized in that, The device includes: The data acquisition unit is used to acquire a multimodal data matrix after spatial multi-omics data registration of the target slice, wherein the multimodal data matrix is ​​used to describe the gene expression level and relative abundance of metabolites at each spatial site in the target slice; the spatial multi-omics includes at least spatial transcriptomics and spatial metabolomics. A cross-modal analysis unit is used to perform cross-modal analysis based on the multimodal data matrix. The cross-modal analysis includes spatial functional domain identification and / or spatial multi-omics association analysis. The spatial functional domain identification includes: constructing a spatial smoothing constraint based on the spatial adjacency strength between any two spatial points in the multimodal data matrix, and decomposing the multimodal data matrix under the spatial smoothing constraint to obtain the target factor score matrix and the target factor loading matrix; Based on the target factor score matrix, all spatial sites are divided into multiple spatial domains, and the functional modules of each spatial domain are determined based on genes and metabolites according to the target factor loading matrix. The spatial multi-omics association analysis includes: determining the local association strength between each gene and each metabolite in the local neighborhood of the spatial site based on the gene expression level and relative abundance of each neighboring site in the multimodal data matrix, and constructing a spatial local regulatory network based on the local association strength, wherein the spatial local regulatory network is used to describe the molecular regulatory relationship between each gene and each metabolite.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 7.

Images