Multi-slice data processing method and device, electronic equipment and storage medium

Abstract

This disclosure provides a multi-slice data processing method, apparatus, electronic device, and storage medium. The multi-slice data processing method includes: acquiring multimodal data of multiple target slices; clustering the multimodal data based on an analysis unit for each target slice to determine multimodal labeled data, whereby the multimodal labeled data describes the association between the target slices and target label data; extracting features from the multimodal data using an initial feature extractor based on a preset neural network model to obtain a multimodal feature matrix; performing label prediction on the multimodal feature matrix using a label prediction unit based on the preset neural network model to obtain predicted label data; training the preset neural network model based on the target label data and the predicted label data; and obtaining a target feature extractor for multi-slice feature extraction of multiple target slices based on the trained preset neural network model. The embodiments of this application provide a method suitable for multi-slice data processing.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of slice data processing, in particular to a multi-slice data processing method and device, electronic equipment and storage medium. BACKGROUND

[0002] Spatial transcriptomics is an emerging technology that combines imaging technology and gene expression analysis, which can accurately locate gene expression in the spatial structure of cells or tissues. When spatial transcriptomics is associated and combined with multi-modal information, the behavior and function of cells in a specific biological structure can be revealed.

[0003] In related technologies, although the processing and analysis of spatial transcriptomics are supported to some extent, when multi-slice spatial transcriptomic data is jointly analyzed, there are great difficulties in integrating processing these data due to the fact that multi-slice data often comes from different experimental conditions and time points, thereby affecting the accuracy of the analysis results. Therefore, how to provide a multi-slice data processing method that can be applied has become a technical problem to be solved. SUMMARY

[0004] The main purpose of the embodiments of the present application is to provide a multi-slice data processing method and device, electronic equipment and storage medium, which can be applied to a multi-slice data processing method.

[0005] To achieve the above-mentioned purpose, the first aspect of the embodiments of the present application provides a multi-slice data processing method, which comprises:

[0006] Obtaining multi-modal data of a plurality of target slices;

[0007] Clustering the multi-modal data based on an analysis unit of each target slice to determine multi-modal annotation data; wherein the multi-modal annotation data is used to describe the association between the target slice and target label data, and the target label data is used to describe the label information of each analysis unit after clustering in the target slice;

[0008] Performing feature extraction on the multi-modal data based on an initial feature extractor of a preset neural network model to obtain a multi-modal feature matrix;

[0009] Performing label prediction on the multi-modal feature matrix based on a label prediction unit of the preset neural network model to obtain predicted label data, wherein the predicted label data is used to describe the label information of each analysis unit in the target slice.

[0010] The preset neural network model is subjected to adversarial training based on the target label data and the predicted label data, and a target feature extractor is obtained based on the trained preset neural network model. The target feature extractor is used to extract multi-slice features from multiple target slices.

[0011] In some embodiments, the label prediction unit includes a clustering classifier and a domain classifier. The label prediction unit based on the preset neural network model performs label prediction on the multimodal feature matrix to obtain predicted label data, including:

[0012] Based on the clustering classifier, clustering label prediction is performed on the multimodal feature matrix to obtain the clustering prediction label of each analysis unit in each target slice;

[0013] Based on the domain classifier, the multimodal feature matrix is ​​used to predict unit labels to obtain the predicted unit labels for each analysis unit in each target slice; the predicted unit labels are used to describe the label information of the target slice to which each analysis unit belongs;

[0014] The predicted label data for each target slice is constructed based on the cluster prediction label and the unit prediction label.

[0015] In some embodiments, the step of predicting unit labels for the multimodal feature matrix based on the domain classifier to obtain the predicted unit labels for each analysis unit in each target slice includes:

[0016] Obtain the initial gradient data passed from the domain classifier to the initial feature extractor;

[0017] The initial gradient data is inverted to obtain inverted gradient data, and the parameters of the initial feature extractor are adjusted based on the inverted gradient data.

[0018] Based on the parameter-adjusted initial feature extractor, features are extracted from the multimodal data to update the multimodal feature matrix;

[0019] Based on the domain classifier, the updated multimodal feature matrix is ​​used to predict unit labels, thereby obtaining the predicted unit labels for each analysis unit in each target slice.

[0020] In some embodiments, training the preset neural network model based on the target label data and the predicted label data includes:

[0021] Based on the target label data, determine the first unit cluster label and target unit label of each analysis unit in each target slice;

[0022] A first loss is constructed based on the first unit clustering label and the clustering prediction label;

[0023] A second loss is constructed based on the target unit label and the predicted unit label;

[0024] Construct a target loss based on the first loss and the second loss;

[0025] The preset neural network model is trained based on the target loss.

[0026] In some embodiments, acquiring multimodal data of multiple target slices includes:

[0027] Acquire spatial transcriptome data and staining images of the target slice;

[0028] An analysis unit is constructed based on the preset merged data and the spatial coordinate system of the target slice;

[0029] The spatial location data of the target slice are determined based on the spatial transcriptome data and the analysis unit.

[0030] Data fusion is performed on the spatial transcriptome data, the staining images, and the spatial location data of multiple target slices to obtain the multimodal data of multiple target slices.

[0031] In some embodiments, the analysis unit clustering the multimodal data based on each target slice to determine multimodal labeled data includes:

[0032] Based on the coordinate information of the analysis unit, the unit representation data of each analysis unit is determined from the multimodal data;

[0033] Cluster the expressed data of multiple units to determine the first cluster label of each analysis unit;

[0034] The slice representation characters of the target slice are converted into slice representation tags by character tag conversion;

[0035] The target unit label for each analysis unit is determined based on the slice representation label;

[0036] The multimodal annotation data is constructed based on the first unit clustering label, the slice representation character, and the target unit label.

[0037] In some embodiments, the method further includes:

[0038] Based on the target feature extractor, a multi-slice feature matrix is ​​determined for multiple target slices;

[0039] Clustering is performed based on the multi-slice feature matrix to determine the second unit clustering label for each analysis unit;

[0040] Generate a multi-slice projection image based on multiple second-unit clustering labels;

[0041] The de-batch status of multiple target slices is determined based on the multi-slice projection image.

[0042] To achieve the above objectives, a second aspect of this application provides a multi-slice data processing apparatus, the apparatus comprising:

[0043] The data acquisition module is used to acquire multimodal data from multiple target slices;

[0044] The clustering and labeling module is used to cluster the multimodal data based on the analysis units of each target slice to determine multimodal labeled data; wherein, the multimodal labeled data is used to describe the association between the target slice and the target label data, and the target label data is used to describe the label information of each analysis unit in the target slice after clustering;

[0045] The feature extraction module is used to extract features from the multimodal data based on the initial feature extractor of the preset neural network model to obtain a multimodal feature matrix;

[0046] The label prediction module is used to perform label prediction on the multimodal feature matrix based on the label prediction unit of the preset neural network model to obtain predicted label data. The predicted label data is used to describe the label information of each analysis unit in the predicted target slice.

[0047] The training module is used to train the preset neural network model based on the target label data and the predicted label data, and to obtain a target feature extractor based on the trained preset neural network model. The target feature extractor is used to perform multi-slice feature extraction on multiple target slices.

[0048] To achieve the above objectives, a third aspect of this application provides an electronic device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0049] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0050] To achieve the above objectives, a fifth aspect of the present application provides a computer program product comprising a computer program that is read and executed by a processor of a computer device, causing the computer device to perform the method described in the first aspect.

[0051] The multi-slice data processing method, apparatus, electronic device, and storage medium proposed in this application acquire multimodal data of multiple target slices and cluster the multimodal data based on the analysis units of each target slice to determine multimodal labeled data. The multimodal labeled data describes the association between the target slices and target label data, while the target label data describes the label information of each analysis unit after clustering in the target slice. Features are extracted from the multimodal data using an initial feature extractor based on a preset neural network model to obtain a multimodal feature matrix. Label prediction is performed on the multimodal feature matrix using a label prediction unit based on the preset neural network model to obtain predicted label data, which describes the predicted label information of each analysis unit in the target slice. The preset neural network model is then subjected to adversarial training based on the target label data and the predicted label data, and a target feature extractor is obtained based on the trained preset neural network model. The target feature extractor is used for multi-slice feature extraction of multiple target slices. Therefore, the embodiments of this application can determine target label data based on multimodal labeled data obtained by clustering multimodal data, and train a preset neural network model based on the target label data and predicted label data. This allows the target feature extractor to be better used for multi-slice feature extraction, reducing the situation in related technologies where it is impossible to effectively integrate relevant data from multiple target slices collected under different experimental conditions and at different time points, thus affecting the accuracy of the analysis results. Therefore, when performing joint analysis based on the extracted multi-slice features, the embodiments of this application can better improve the accuracy of the multi-slice analysis results. Attached Figure Description

[0052] The accompanying drawings are provided to further understand the technical solutions of this disclosure and constitute a part of the specification. They are used together with the embodiments of this disclosure to explain the technical solutions of this disclosure and do not constitute a limitation on the technical solutions of this disclosure.

[0053] Figure 1 This is a flowchart of a multi-slice data processing method provided in an embodiment of this application;

[0054] Figure 2 yes Figure 1 A flowchart of an embodiment of step S101;

[0055] Figure 3 This is a schematic diagram of the structure of multimodal data provided in an embodiment of this application;

[0056] Figure 4 yes Figure 1 A flowchart of an embodiment of step S102;

[0057] Figure 5 This is a schematic diagram of the structure of multimodal labeled data provided in an embodiment of this application;

[0058] Figure 6 yes Figure 1 A flowchart of step S104 in the process;

[0059] Figure 7 yes Figure 6 A flowchart of step S602 in the process;

[0060] Figure 8 yes Figure 1 A flowchart of step S105 in the process;

[0061] Figure 9 This is a schematic diagram of a training preset neural network model provided in an embodiment of this application;

[0062] Figure 10 This is another flowchart of the multi-slice data processing method provided in the embodiments of this application;

[0063] Figure 11 This is a schematic diagram of a UMAP graph generated based on multiple second unit clustering labels provided in an embodiment of this application;

[0064] Figure 12 This is a flowchart of a specific embodiment of the multi-slice data processing method provided in this application;

[0065] Figure 13 This is a schematic diagram of a multi-slice data processing apparatus provided in an embodiment of this application;

[0066] Figure 14 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this disclosure.

[0068] Before providing a further detailed description of the embodiments of this disclosure, the terms and concepts used in these embodiments are explained, and they are subject to the following interpretations:

[0069] Spatial transcriptomics (ST) sequencing is a technique for analyzing RNA at a spatial level, used to analyze all RNA in a single tissue slice. Stereo-seq (Spatial Enhanced Resolution Omics-sequencing) is an advanced spatial transcriptomics technique. Developed based on DNA nanoballs (DNB), Stereo-seq is a high-throughput, ultra-high-resolution, and wide-field-of-view in-situ panoramic technology. Stereo-seq can simultaneously perform spatial transcriptome analysis of the same sample at four scales: tissue, cell, subcellular, and molecular. Stereo-seq captures mRNA in tissues using spatiotemporal chips and reconstructs its spatial location using coordinate ID (CID), enabling the detection of gene spatial expression in tissues and establishing a strong research foundation for a deeper understanding of the relationship between cellular gene expression and morphology and the local environment.

[0070] Pathological staining images: Specific staining agents are used to enhance the contrast of specific structures or components in tissue samples, allowing for clearer observation and analysis of cellular morphology and pathological changes under a microscope. Commonly used staining techniques include hematoxylin and eosin (HE) staining and special staining. HE staining clearly displays the different structures of the cell nucleus and cytoplasm. Special staining refers to the use of specialized staining techniques for specific purposes. Special staining can include Masson's trichrome staining, PAS staining, silver staining, etc. These techniques can highlight specific tissue structures or components, such as collagen fibers, muscle tissue, and nerve tissue. Furthermore, artificial intelligence (AI) technology can be used for virtual staining, such as AI staining. AI staining refers to the virtual staining of microscopic images of tissue sample sections in pathology.

[0071] Artificial intelligence (AI) is a new branch of computer science that studies, develops, and applies theories, methods, technologies, and systems to simulate, extend, and expand human intelligence. It aims to understand the essence of intelligence and produce intelligent machines that can react in a way similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thought. Furthermore, AI utilizes digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceiving the environment, acquiring knowledge, and using that knowledge to achieve optimal results.

[0072] Spatial transcriptomics is an emerging technology that combines imaging techniques with gene expression analysis to precisely locate gene expression within the spatial structure of cells or tissues. When spatial transcriptomics is correlated and combined with multimodal information, it can reveal the behavior and function of cells in specific biological structures, thus providing a research foundation for understanding complex biological processes such as disease development, tissue formation, and immune responses. Through spatial transcriptomics, the distribution of different cell types in tissues, as well as the functional or characteristic interactions between different cell types, can be observed.

[0073] The challenges of processing multi-slice data primarily lie in batch effects, the difficulty of data integration, and the preservation of biological differences. This is because systematic biases between different experimental batches can lead to significant differences in the data, affecting the comparability and consistency of the results. This effect is particularly pronounced in multi-slice, multi-condition experiments, making it impossible to effectively integrate multi-slice data and limiting the depth and breadth of multi-slice data analysis, thus impacting the accuracy and reliability of the analytical results. Therefore, preserving the unique biological differences between each classification during the process of removing batch effects and integrating data is crucial to ensuring that the biological significance of the data is not diminished.

[0074] Currently, relevant technologies typically employ model-based or anchor-based methods to remove batch effects. Model-based methods can pre-assume the distribution of data (such as raw cell data or data after normalization) or cell type clustering. For example, the Combining Batches (ComBat) method relies on an empirical Bayesian framework and assumes that the normalized gene expression of a given slice in a given batch follows a normal distribution with a pre-specified prior distribution. However, ComBat is sensitive to outliers in the data and requires a large number of slices to accurately estimate batch effects. Another example is the limma method, which uses a linear model to fit the input data. However, limma may encounter computational efficiency and memory usage issues when dealing with high-dimensional data. Furthermore, limma requires the user to provide a design matrix, which can increase the difficulty of use for complex experimental designs. Yet another approach is the novel soft K-means method for cell clustering, which implicitly assumes that cell clusters follow a linear mixture model. However, this assumption may not hold in some cases, affecting the effectiveness of batch effect removal. In addition, the batch removal method based on anchor points can specifically use the Mutual Nearest Neighbor (MNN) method to remove batch effects. However, this method requires first identifying cell pairs with similar expression patterns across batches, so it is very sensitive to the selection of cell pairs, and the selection of incorrect cell pairs can lead to huge downstream bias.

[0075] While related technologies support the processing and analysis of spatial transcriptomics to some extent, researchers face significant challenges when conducting joint analysis of multi-slice spatial transcriptomics data. This is because differences in experimental batches, conditions, and time points introduce batch effects, leading to inconsistencies in the obtained multi-slice data. Consequently, it is impossible to effectively integrate multi-slice data, limiting the depth and breadth of multi-slice data analysis, and thus affecting the accuracy and reliability of the analytical results.

[0076] Based on this, embodiments of this application propose a multi-slice data processing method, apparatus, electronic device, and storage medium, which are better suited for multi-slice data processing, effectively remove batch effects between multi-slice data, and improve the consistency between multi-slice data.

[0077] The multi-slice data processing method provided in the embodiments of this application is described below.

[0078] Reference Figure 1 In some embodiments, the multi-slice data processing method provided in this application includes, but is not limited to, steps S101 to S105.

[0079] Step S101: Obtain multimodal data of multiple target slices;

[0080] Step S102: Cluster the multimodal data based on the analysis units of each target slice to determine the multimodal labeled data;

[0081] Step S103: Based on the initial feature extractor of the preset neural network model, feature extraction is performed on the multimodal data to obtain the multimodal feature matrix;

[0082] Step S104: The label prediction unit based on the preset neural network model performs label prediction on the multimodal feature matrix to obtain predicted label data.

[0083] Step S105: Train the preset neural network model based on the target label data and the predicted label data, and obtain the target feature extractor based on the trained preset neural network model. The target feature extractor is used to extract features from multiple target slices.

[0084] Steps S101 to S105, as illustrated in this embodiment, involve acquiring multimodal data from multiple target slices and clustering the multimodal data based on the analysis unit of each target slice to determine multimodal labeled data. The multimodal labeled data describes the association between the target slices and the target label data. A multimodal feature matrix is ​​obtained by extracting features from the multimodal data using an initial feature extractor based on a preset neural network model. Predicted labels are obtained by predicting labels on the multimodal feature matrix using a label prediction unit based on the preset neural network model. The preset neural network model is trained based on the target label data and the predicted label data, and a target feature extractor is obtained based on the trained preset neural network model. The target feature extractor is used for multi-slice feature extraction from multiple target slices. Therefore, the embodiments of this application can determine target label data based on multimodal labeled data obtained by clustering multimodal data, and train a preset neural network model based on the target label data and predicted label data. This allows the target feature extractor to be better used for multi-slice feature extraction, reducing the situation in related technologies where it is impossible to effectively integrate relevant data from multiple target slices collected under different experimental conditions and at different time points, thus affecting the accuracy of the analysis results. Thus, when the embodiments of this application perform joint analysis based on the extracted multi-slice features, they can better solve the batch effect problem in multi-slice data analysis, optimize the data integration process, improve data consistency and reliability, and thereby improve the accuracy of multi-slice analysis results.

[0085] In step S101 of some embodiments, the target slice may refer to a tissue slice to be processed for spatial transcriptome data. The target slice is a biological slice obtained in accordance with relevant legal regulations; biological slices include plant slices and animal slices. Multimodal data may refer to a multimodal feature representation composed of relevant information from multiple target slices, achieving information fusion of relevant data from multiple target slices from multiple dimensions.

[0086] Reference Figure 2 In some embodiments, step S101 may include, but is not limited to, steps S201 to S204.

[0087] Step S201: Obtain spatial transcriptome data and staining images of the target slice;

[0088] Step S202: Construct an analysis unit based on the preset spatial coordinate system of the merged data and the target slice;

[0089] Step S203: Determine the spatial location data of the target slice based on the spatial transcriptome data and analysis unit;

[0090] Step S204: Data fusion is performed on the spatial transcriptome data, staining images and spatial location data of multiple target slices to obtain multimodal data of multiple target slices.

[0091] In step S201 of some embodiments, when constructing multimodal data, spatial transcriptome data of a single target slice is first acquired. This spatial transcriptome data is used to describe the location of cells and gene expression information in the target slice, and includes gene expression sequencing data for each independent region (i.e., spot). This application can obtain spatial transcriptome data by performing ST processing on the target slice, or by obtaining spatial transcriptome data based on Stereo-seq.

[0092] Spatial transcriptome data can be in JSON file format, and these gene expression sequencing data can be in triplet or h5 format, etc. A staining image refers to a pathological staining image of a single target section, providing visual information about the tissue structure of that single target section. Staining images can be acquired using specialized equipment and techniques, such as processing the target section image using specific staining techniques (e.g., HE staining or IHC staining).

[0093] In step S202 of some embodiments, merging data can refer to a preset merging rule. The merging rule can be determined based on cell size, gene quantity, and the needs of subsequent analysis operations, etc., and this application embodiment does not specifically limit this. An analysis unit can refer to a bin. In the field of spatial transcription technology, a bin can refer to the process of merging spatially adjacent or similar cells into a single unit for analysis. Constructing bins can reduce data dimensionality and improve computational efficiency. Simultaneously, it can also reduce noise to a certain extent, making the data pattern clearer. Specifically, a bin is the basic unit for statistical analysis of data. One bin represents a fixed-size region where the expression levels of DNBs (DNB can refer to DNA nanospheres) are accumulated, and regions do not overlap. On a spatiotemporal chip, each DNB is represented as a pixel on the gene expression heatmap. In this case, the analysis unit is bin1, meaning that one pixel contains only the data of one DNB. Merging N×N DNB data within a neighborhood and displaying them as a single pixel on the gene expression heatmap is called bin N. For example, bin 100 represents an analysis unit containing data from 100×100 DNB regions. In this way, the specific value of N in the analysis unit bin N can be determined based on the merging rules and the spatial coordinate system of the target slice.

[0094] In step S203 of some embodiments, the spatial coordinate system of the target slice can be divided into multiple analysis units through the above steps. Further, the spatial location data of the target slice can be determined based on the spatial transcriptome data and the analysis units. This spatial location data can be used to describe the overall spatial location of the target slice, as well as the spatial location of each analysis unit within the target slice. For example, the spatial location of an analysis unit bin is 191_135, which represents the position coordinates of the analysis unit bin in two-dimensional space. Here, 191 represents the value corresponding to the horizontal coordinate (x-coordinate) of the analysis unit bin in the spatial coordinate system of the target slice, and 135 represents the value corresponding to the vertical coordinate (y-coordinate) of the analysis unit bin in the spatial coordinate system of the target slice. This representation method can be used to identify the location of specific pixels or regions in the slice image. Therefore, based on the values ​​corresponding to the horizontal coordinate (x-coordinate) and vertical coordinate (y-coordinate) of the target slice's spatial coordinate system, an analysis unit bin or a region of the target slice can be determined.

[0095] In step S204 of some embodiments, the multimodal data of multiple target slices refers to the data obtained by fusing spatial transcriptome data, staining images, and spatial location data of multiple target slices, through learning multiple modal features and integrating information. This multimodal data can be in matrix form, such as X1∈R b×dLet X1 represent a b×d dimensional matrix belonging to the real number set R, where b is the total number of analysis units contained in the multiple target slices, and n is the number of target slices, and d is the preset multimodal feature length (such as 50, 100, etc., which can be defined by yourself, or the dimensional length can be determined according to the amount of information between modalities to achieve better feature representation for downstream tasks).

[0096] Understandably, please refer to Figure 3 , Figure 3 This illustration shows a structural diagram of multimodal data provided in an embodiment of this application. The multimodal data can be in matrix form, specifically comprising 41,108 rows × 100 columns. Each column, combined with column markers (e.g., 0-99), indicates a preset modal feature, thus the multimodal data can have 100 columns. Each row of data indicates the data obtained by an analysis unit after data fusion. For example, Figure 3 The first column indicates the target slice and spatial location of the analysis unit. An analysis unit is labeled 191_135-0, where 191 represents the x-coordinate of analysis unit bin in the spatial coordinate system of target slice 0, 135 represents the y-coordinate of analysis unit bin in the spatial coordinate system of target slice 0, and 0 represents the target slice number. Similarly, another analysis unit might be labeled 111_84-11, where 111 represents the x-coordinate of analysis unit bin in the spatial coordinate system of target slice 11, and 84 represents the y-coordinate of analysis unit bin in the spatial coordinate system of target slice 11. This means that the current multimodal data incorporates 12 (e.g., 0-11) target slices. Furthermore, the data where rows and columns intersect represents the data obtained after data fusion of the corresponding analysis unit in the row and the modal features in the column.

[0097] It should be noted that the data fusion process of spatial transcriptome data, staining images and spatial location data of multiple target slices in this application can be achieved by various methods, such as graph fusion models (StereoMM) that integrate spatial transcriptome data and pathological images, spatial graph convolutional networks (SpaGCN), graph structured time series (GraphST), spatial language pre-training benchmarks (SpatialGLUE), deep spatio-temporal networks (DeepST), and other deep learning technologies, to learn the features of each modality and integrate the data, without making specific limitations.

[0098] Based on steps S201 to S204 above, by fusing spatial transcriptome data, staining images and spatial location data of different target slices, information from different sources and different types of data can be integrated to obtain a more comprehensive data view, which can better realize the joint analysis of multiple slices.

[0099] In step S102 of some embodiments, multimodal annotation data refers to data obtained after annotating multimodal information, and the multimodal annotation data is used to describe the association between target slices and target label data. Target label data is used to describe the label information of each analysis unit in the target slice after clustering. The annotation information contained in the multimodal annotation data can be generated based on automatic annotation operations or manual annotation operations, and this application embodiment does not specifically limit this.

[0100] Reference Figure 4 In some specific embodiments, step S102 may include, but is not limited to, steps S401 to S405.

[0101] Step S401: Determine the unit representation data of each analysis unit from the multimodal data based on the coordinate information of the analysis unit;

[0102] Step S402: Cluster the data expressed by multiple units and determine the first unit cluster label for each analysis unit;

[0103] Step S403: Perform character label conversion on the slice representation characters of the target slice to obtain the slice representation label;

[0104] Step S404: Determine the target cell label for each analysis cell based on the slice representation label;

[0105] Step S405: Construct multimodal labeled data based on the first unit cluster label, slice representation character, and target unit label.

[0106] In step S401 of some embodiments, as illustrated in the examples above, the coordinate information of the analysis unit can be the physical position coordinates of the analysis unit bin in the spatial coordinate system of the target slice. The unit expression data of each analysis unit is used to characterize the feature information formed after highly summarizing the multimodal information (such as gene expression, image information, protein data, etc.) composed of multiple target slices. Therefore, the unit expression data of each analysis unit on the multimodal features can be determined from the multimodal data based on the coordinate information of the analysis unit.

[0107] In step S402 of some embodiments, this application can use a clustering algorithm to cluster the unit expression data from multiple analysis units to determine the clustering result of each analysis unit in each target slice, that is, to determine the first unit clustering label of each analysis unit in each target slice. The obtained first unit clustering label represents the distinction between different clustering groups. For example, after clustering the unit expression data of multiple analysis units, there are 116 clustering groups, that is, there are 116 first unit clustering labels, which can be labeled as first unit clustering label 0 to first unit clustering label 115 respectively. At this time, analysis unit 191_135-0 in target slice 0 belongs to the clustering group corresponding to first unit clustering label 4 after clustering, while analysis unit 212_131-0 in target slice 0 belongs to the clustering group corresponding to first unit clustering label 7 after clustering. In this way, multiple analysis units can be divided into different clustering groups, and the influence of slice batches is not considered in the grouping process, which can initially achieve the removal of batch effects.

[0108] It should be noted that the method used in this application for clustering multimodal data based on analysis units can include Leiden clustering, K-means clustering, etc., without specific limitations. Among them, Leiden clustering is a graph-based clustering algorithm that creates clusters by considering the ratio between the number of connections between cells in a cluster and the expected total number of connections in the dataset.

[0109] In step S403 of some embodiments, the slice representation character refers to the descriptive text or characters used to describe the target slice, equivalent to the original slice name of the target slice. The slice representation label refers to a numerical label that a computer or model can recognize to indicate the target slice. Thus, character label conversion refers to converting the descriptive text or characters of the target slice into a recognizable slice representation label, with each slice representation label corresponding to a different target slice. For example, if there are 12 target slices, namely target slice 0 to target slice 11, then the corresponding slice representation labels can be the values ​​0 to 11. In this case, if the slice representation character of target slice 0 is "C04139F3X", then after character label conversion, this slice representation character can correspond to the value 0. If the slice representation character of target slice 11 is "C04139F3X", then after character label conversion, this slice representation character can correspond to the value 11.

[0110] In step S404 of some embodiments, the target unit label refers to a numerical label that can be recognized by a computer or model and is used to indicate an analysis unit. Since a target slice includes multiple analysis units, the target unit labels corresponding to analysis units belonging to the same target slice are the same, which are all slice representation labels of the target slice to which they belong. Therefore, the target unit label of each analysis unit can be determined according to the slice representation label. For example, the slice representation character of target slice 0 is "C04139F3X", and the slice representation label of target slice 0 is the value 0. Since target slice 0 contains multiple analysis unit labels including 191_135-0, 212_131-0, 186_44-0, etc., the target unit labels corresponding to these analysis units are all the value 0.

[0111] In step S405 of some embodiments, after obtaining the first unit clustering label, slice representation character, and target unit label corresponding to each analysis unit, annotation merging can be performed based on these data to construct multimodal annotation data. This multimodal annotation data can be in matrix form, such as X2∈R b×3 Let X2 represent a b×3 matrix belonging to the set of real numbers R, where b is the total number of analysis units contained in the multiple target slices, and n represents the number of target slices. The three columns are as follows: the first column, "cluster," is the cluster label of the first unit determined after clustering the data from multiple units; the second column, "slice_name," is the original slice name of the target slice, i.e., the slice representation character; the third column, "slice_label," contains the target unit label since each row in the multimodal representation data corresponds to an analysis unit.

[0112] Understandably, please refer to Figure 5, Figure 5 The diagram shows a structural schematic of a multimodal annotation data provided in an embodiment of this application. The multimodal annotation data can be in matrix form, and specifically can include data with 41,108 rows × 3 columns. Figure 5 The first column is used to indicate the target slice and spatial location to which the analysis unit belongs. For example, an analysis unit is labeled as 191_135-0 (and its specific meaning has been explained in the above embodiments and will not be repeated here). The content of the first column corresponding to the analysis unit includes: [4, C04139F3X, 0], which means that the analysis unit belongs to the cluster group corresponding to the first unit cluster label 4 after clustering. The slice representation character of the target slice to which the analysis unit belongs is "C04139F3X", and the target unit label of the analysis unit is the value 0.

[0113] In steps S401 to S405 above, this application integrates the first unit clustering label, slice representation character, and target unit label related to the analysis unit to construct multimodal labeled data containing label information for each analysis unit with multiple target slices. This provides true labels for each analysis unit for subsequent model training, thereby providing a standardized representation for further data analysis and comparison, and thus improving the accuracy of multi-slice analysis results. Furthermore, this application improves labeling efficiency through a series of automated labeling methods such as clustering and character label conversion.

[0114] In step S103 of some embodiments, the multimodal feature matrix is ​​used to characterize the matrix that represents the data features extracted from the multimodal data by the initial feature extractor during the training phase of the preset neural network model. Furthermore, this feature matrix can be used for subsequent label prediction and model training. The initial feature extractor in this application can be constructed based on structures such as Multilayer Perceptron (MLP), Convolutional Neural Networks (CNN), Transformer, and autoencoder, and is not specifically limited thereto.

[0115] In step S104 of some embodiments, the preset neural network model of this application may include an initial feature extractor and a label prediction unit, and the label prediction unit is used to predict the label of each analysis unit. After obtaining the multimodal feature matrix, the multimodal feature matrix can be input into the label prediction unit for label prediction to obtain predicted label data. The predicted label data is the label data obtained after each analysis unit in each target slice has been predicted by the label prediction unit.

[0116] It should be noted that the label prediction unit of this application includes a clustering classifier and a domain classifier. The clustering classifier is used to predict the clustering label of each analysis unit to ensure correct spatial domain identification. The domain classifier is used to predict the target slice to which each analysis unit belongs to, to distinguish data from different slices. For example, Figure 5 As shown, the clustering classifier is used to predict the content of the first column "cluster" in the multimodal labeled data. The clustering classifier can be any classifier built based on structures such as CNNs and feedforward neural networks, without specific limitations. The spatial domain is an expression along the spatial transcriptome. For example, spatial visualization can be achieved by applying a uniform manifold approximation and projection (UMAP) to multiple analysis units based on the predicted labels from the clustering classifier. Figure 5 As shown, the domain classifier is used to predict the content of the third column "slice_label" in the multimodal labeled data, and the domain classifier can be constructed based on structures such as fully connected layers, without specific limitations. Therefore, this application can simultaneously input the multimodal feature matrix into the cluster classifier and the domain classifier for label prediction to obtain predicted label data.

[0117] Reference Figure 6 In some embodiments, step S104 may include, but is not limited to, steps S601 to S603.

[0118] Step S601: Based on the cluster classifier, perform cluster label prediction on the multimodal feature matrix to obtain the cluster prediction label of each analysis unit in each target slice;

[0119] Step S602: Based on the domain classifier, predict the unit labels of the multimodal feature matrix to obtain the predicted unit labels of each analysis unit in each target slice;

[0120] Step S603: Construct prediction label data for each target slice based on cluster prediction labels and unit prediction labels.

[0121] In step S601 of some embodiments, the cluster prediction label is used to characterize the cluster group to which each predicted analysis unit belongs. The cluster prediction label can be used to compare with the first unit cluster label already identified for the analysis unit to determine the accuracy of the prediction of the analysis unit's clustering.

[0122] In step S602 of some embodiments, the unit prediction tag is used to characterize the target slice to which each predicted analysis unit belongs. The unit prediction tag describes the tag information of the target slice to which each predicted analysis unit belongs. The unit prediction tag can be used to compare with the target unit tags already identified by the analysis unit to determine the prediction accuracy for the target slice to which the analysis unit belongs.

[0123] In step S603 of some embodiments, this application may use the cluster prediction labels and unit prediction labels obtained from each target slice as a set of data to construct the prediction label data for each target slice.

[0124] In steps S601 to S603 above, related technologies first generate a reconstructed expression matrix and then determine batch labels based on a discriminator. However, the original gene expression itself may exhibit batch effects, which may lead to incomplete batch effect removal if used as a guide. Furthermore, this method is computationally intensive, and directly reconstructing the original expression matrix may be affected by noise and technical variations in high-dimensional data, resulting in insufficient feature optimization capabilities and impacting batch effect removal. In contrast, this application's embodiment can simultaneously predict different types of labels using a parallel-designed clustering classifier and domain classifier. By predicting two types of labels, features from different analysis units can be mixed, ensuring the accuracy of label prediction and improving the accuracy of analysis unit prediction. The multimodal data obtained in this application does not consider batch information corresponding to the target slice, thus avoiding the introduction of prior knowledge of batch information that could affect subsequent model training. Moreover, this application, by reconstructing the clustering results of the prediction analysis units, can better reflect the biological similarities and differences between multiple slices, reducing noise interference and thus more effectively and implicitly removing batch effects from multiple slices.

[0125] Reference Figure 7 In some embodiments, step S602 may include, but is not limited to, steps S701 to S704.

[0126] Step S701: Obtain the initial gradient data passed from the domain classifier to the initial feature extractor;

[0127] Step S702: Invert the gradient of the initial gradient data to obtain inverted gradient data, and adjust the parameters of the initial feature extractor based on the inverted gradient data.

[0128] Step S703: Based on the initial feature extractor with adjusted parameters, feature extraction is performed on the multimodal data to update the multimodal feature matrix;

[0129] Step S704: Based on the domain classifier, predict the unit labels of the updated multimodal feature matrix to obtain the predicted unit labels of each analysis unit in each target slice.

[0130] In steps S701 to S704 of some embodiments, this application embodiment can employ an adversarial mechanism between the domain classifier and the initial feature extractor to effectively remove batch effects in transcriptomics data from different slices. Based on this, the pre-defined neural network model further includes a Gradient Reversal Layer (GRL), located between the domain classifier and the initial feature extractor, used to reverse the direction of the gradient during backpropagation, thereby achieving adversarial training. Specifically, during backpropagation, the gradient of the domain classifier passed to the initial feature extractor can be reversed first, that is, the initial gradient data is multiplied by a negative number to achieve gradient reversal. Then, the parameters of the initial feature extractor are adjusted based on the obtained reversed gradient data, so that feature extraction can be performed through the updated initial feature extractor in the next round of training. Thus, this application can perform feature extraction on multimodal data based on the parameter-adjusted initial feature extractor to update the multimodal feature matrix, and perform unit label prediction on the updated multimodal feature matrix based on the domain classifier to obtain the predicted unit labels for each analysis unit in each target slice.

[0131] Understandably, GRL can multiply the gradient passed to this layer by a negative number, making the training objectives of the networks before and after GRL opposite, thus achieving an adversarial effect. Then, based on the multimodal inverted feature matrix, unit label prediction is performed to obtain the unit predicted labels for each analysis unit in each target slice.

[0132] Understandably, the information extracted by the initial feature extractor is fed into the domain classifier, which then determines which slice the information comes from. In other words, the domain classifier needs to correctly identify the target slice to which each analysis unit belongs from the input multimodal feature matrix. Therefore, the training objective of the domain classifier at this stage is to correctly identify the target slice to which each analysis unit belongs. However, due to the presence of the gradient reversal layer, the training objective of the initial feature extractor is exactly the opposite: it aims to prevent the domain classifier from correctly identifying the target slice to which each analysis unit belongs, thus creating an adversarial relationship.

[0133] In steps S701 to S704 above, this application effectively removes batch effects in transcriptomics data from different slices by setting an adversarial mechanism between the initial feature extractor and the domain classifier, thus addressing the need for joint analysis of multiple slices. Furthermore, through adversarial training, the pre-defined neural network model can achieve better generalization ability on different batches of data.

[0134] In step S105 of some embodiments, after determining the target label data and predicted label data, a preset neural network model can be trained based on these data, and the initial feature extractor in the trained preset neural network model can be used as the target feature extractor. This target feature extractor is used to perform multi-slice feature extraction on multiple target slices, which can effectively remove batch effects in transcriptomics data from different slices and achieve multi-slice feature fusion.

[0135] Reference Figure 8 In some embodiments, step S105 may include, but is not limited to, steps S801 to S805.

[0136] Step S801: Determine the first cluster label and target unit label of each analysis unit in each target slice based on the target label data;

[0137] Step S802: Construct a first loss based on the first unit clustering label and the clustering prediction label;

[0138] Step S803: Construct a second loss based on the target unit label and the predicted unit label;

[0139] Step S804: Construct the target loss based on the first loss and the second loss;

[0140] Step S805: Train the preset neural network model based on the target loss.

[0141] In step S801 of some embodiments, after the architecture of the preset neural network model is built, this application can construct a target loss based on the target label data and the predicted label data to guide the training of the model parameters of the preset neural network model. This application can first determine the first unit cluster label and the target unit label of each analysis unit in each target slice from the target label data, and the process of obtaining the first unit cluster label and the target unit label has been described in the above embodiments and will not be repeated.

[0142] In step S802 of some embodiments, this application can construct a first loss based on a preset loss function for the first unit cluster label and the cluster prediction label. The preset loss function can be Maximum Mean Discrepancy (MMD), Cross Entropy Loss, etc., and is not specifically limited. In subsequent embodiments, this application uses Cross Entropy Loss as an example for illustration. Therefore, the construction process of the first loss can be seen in Formula 1 below.

[0143]

[0144] In Formula 1, L ClusterPredictor This represents the first loss function, CrossEntropy(·) represents the cross-entropy loss function, and ClusterLabel represents the cluster label of the first unit. ClusterLabel represents the clustering prediction label, N represents the number of target slices, C represents the number of analysis unit bins per target slice, and ClusterLabel. ji This represents the cluster label of the first unit of the i-th analysis unit in the j-th target slice. This represents the clustering prediction label of the i-th analysis unit of the j-th target slice.

[0145] In step S803 of some embodiments, if the cross-entropy loss function is used as the preset loss function, the process of constructing the second loss based on the target cell label and the cell prediction label can be seen in Formula 2 below. The preset loss functions used to construct the second loss and the first loss can be the same or different, and are not specifically limited.

[0146]

[0147] In Formula 2, L DomainClassifier This represents the first loss function, CrossEntropy(·) represents the cross-entropy loss function, and DomainClassifier represents the target cell label. The domain classifier represents the predicted label for each target slice, N represents the number of target slices, C represents the number of analysis unit bins per target slice, and DomainClassifier. ji This represents the target cell label of the i-th analysis unit in the j-th target slice. This represents the cell prediction label of the i-th analysis unit of the j-th target slice.

[0148] In step S804 of some embodiments, the present application may sum the first loss and the second loss to construct the target loss. Thus, the process of constructing the target loss can be seen in Formula 3 below.

[0149] L total =L ClusterPredictor +L DomainClassifier (Formula 3)

[0150] In other embodiments, this application may also assign corresponding weights to the first loss and the second loss, respectively, so as to calculate the target loss by weighting the first loss and the second loss according to the weights. The weights set are used to reflect the degree of influence of the cluster classifier and the domain classifier on the model training process, thereby improving the flexibility of the target loss determination.

[0151] In step S805 of some embodiments, this application can train the cluster classifier, domain classifier and initial feature extractor in the pre-constructed preset neural network model based on the target loss, and after training for a preset number of times (i.e., epoch, one epoch represents a complete traversal of the entire training set by the model, which is a basic unit in the model training process. The number of epochs in the training process is a hyperparameter that needs to be set according to the specific task and model performance), the model parameters are retained to obtain the trained preset neural network model.

[0152] It should be noted that after training the preset neural network model, the model can be set as an inference model, meaning no training or parameter updates are performed. During inference, the clustering classifier and domain classifier can be temporarily discarded, and only the target feature extractor can be used for multi-slice feature extraction. The output result is the final multi-slice batch result. Additionally, the output of the target feature extractor can be saved in the `multislide_embedding.pkl` file for subsequent downstream analysis.

[0153] In steps S801 to S805 above, this application constructs a target loss based on the prediction results of the cluster classifier and the domain classifier during the model training process. This target loss is used to guide the training of the preset neural network model, which is better suited for multi-slice data processing, effectively removes the batch effect between multi-slice data, and improves the consistency between multi-slice data.

[0154] In one specific embodiment, reference is made to Figure 9 , Figure 9 This is a schematic diagram of the training preset neural network model provided in the embodiments of this application. Multiple target slices, including target slice 1 to target slice N, are processed by a multi-model fusion module to obtain fused data for each target slice (e.g., multiplex immunofluorescence (MIF) images, staining images, and spatial transcriptome data). The fused data from multiple target slices are then fused to construct multimodal data. Further, an initial feature extractor extracts features from the multimodal data, and the extracted multimodal feature matrices are input into a cluster classifier and a domain classifier, respectively, to obtain the first unit cluster label (e.g., ...) for each analysis unit in each target slice. Figure 9 In the clustering (from cluster 1 to cluster P, where P is a positive integer greater than 1 and P represents the total number of cluster labels in the first unit) and the target unit label (e.g., Figure 9 The target slices 1 to N are predicted by the mid-domain classifier. Further, the model parameters in the preset neural network model are trained by combining the above steps for determining the target loss and the backpropagation mechanism.

[0155] Reference Figure 10In some embodiments, after step S105, the method provided in this application embodiment may also include, but is not limited to, steps S1001 to S1004.

[0156] Step S1001: Determine the multi-slice feature matrix of multiple target slices based on the target feature extractor;

[0157] Step S1002: Clustering is performed based on the multi-slice feature matrix to determine the second unit clustering label for each analysis unit;

[0158] Step S1003: Generate a multi-slice projection image based on multiple second-unit clustering labels;

[0159] Step S1004: Determine the de-batch status of multiple target slices based on the multi-slice projection image.

[0160] In step S1001 of some embodiments, after training the target feature extractor, this application can determine a multi-slice feature matrix of multiple target slices based on the target feature extractor. At this time, the multi-slice feature matrix is ​​the data after eliminating the batch effect of multiple slices.

[0161] In step S1002 of some embodiments, the first unit clustering label obtained in the above embodiments is the original clustering result of a single analysis unit before batch removal, and the second unit clustering label is used to characterize the clustering result of a single analysis unit after batch removal. The second unit clustering label is used to check the removal effect of batch effects. In addition, the clustering methods used for these two clusterings can be the same or different, such as Leiden clustering, K-means clustering, etc., without specific limitations.

[0162] In step S1003 of some embodiments, a multi-slice projection image is generated based on multiple second-unit clustering labels, i.e., a UMAP map is generated based on multiple second-unit clustering labels to visualize the features after batch removal. UMAP visualization is a visualization method based on nonlinear dimensionality reduction, whose main goal is to map high-dimensional data to two- or three-dimensional space while preserving the relative distance and structure between data, making clustering, heterogeneity, and differences between samples more apparent. (Refer to...) Figure 11 , Figure 11 This is a schematic diagram of a UMAP map generated based on multiple second-unit clustering labels provided in an embodiment of this application. Feature points of the same color belong to the same second-unit clustering label.

[0163] It should be noted that since multiple target slices may originate from different objects, data without batch effect removal, after dimensionality reduction visualization using UMAP and clustered by object, shows no overlap between data points from different objects, indicating significant batch effect. However, after removing batch effect, data points from objects overlap, and the overlap is more pronounced, indicating better batch effect removal.

[0164] In step S1004 of some embodiments, after obtaining the multi-slice projection image, this application can determine the de-batching state of multiple target slices by manual or automatic identification of the multi-slice projection image. Manual identification refers to determining whether the de-batching state of multiple target slices is de-batched or not by visual inspection. Automatic identification involves inputting the multi-slice projection image into a pre-trained de-batching state detection model for state detection. Based on the overlap between data points, a score for the de-batching state is obtained; a higher score indicates a better de-batching effect, and a lower score indicates a poorer effect. Furthermore, when the de-batching effect is poor, the preset neural network model can be further trained and optimized to improve the ability to remove batch effects between multi-slice data.

[0165] In steps S1001 to S1004 above, UMAP plots are used to visualize the data, which can intuitively show the distribution and clustering effect of the data, making it easier to understand and interpret the analysis results.

[0166] The multi-slice data processing method provided in this application can determine target label data based on multimodal labeled data obtained by clustering multimodal data, and train a preset neural network model based on the target label data and predicted label data. This allows the obtained target feature extractor to be better used for multi-slice feature extraction, reducing the situation in related technologies where it is impossible to effectively integrate relevant data from multiple target slices collected under different experimental conditions and time points, thus affecting the accuracy of the analysis results. Therefore, when performing joint analysis based on the extracted multi-slice features, this application embodiment can better improve the accuracy of the multi-slice analysis results.

[0167] Reference Figure 12 In one specific embodiment, the method provided by this application may include the following steps:

[0168] Step S1201: Obtain multimodal data of multiple target slices, and cluster the multimodal data based on the analysis unit of each target slice to determine multimodal labeled data.

[0169] In this context, the multimodal data can be data obtained from target slices in diffuse large B-cell lymphoma Stereo-seq, represented by the feature data of each original target slice clustered by object in the UMAP graph. During the data preprocessing stage, the multimodal labeled data includes the first cluster label of each analysis unit in each target slice, the slice representation character (the slice name to which the analysis unit belongs), and the target unit label (the numerical form corresponding to the slice name to which the analysis unit belongs). In other words, this application can first obtain all labels related to each analysis unit, and then annotate and merge all labels from multiple analysis units to obtain multimodal labeled data.

[0170] Step S1202: Based on the initial feature extractor, feature extraction is performed on the multimodal data to obtain a multimodal feature matrix, and based on the label prediction unit, label prediction is performed on the multimodal feature matrix to obtain predicted label data.

[0171] The adversarial mechanism between the initial feature extractor and the domain classifier in the label prediction unit effectively removes batch effects from transcriptomics data across different slices, addressing the need for joint analysis of multiple slices. Furthermore, adversarial training enhances the generalization ability of the pre-defined neural network model across different batches of data.

[0172] Step S1203: Train the preset neural network model based on the target label data and the predicted label data, and obtain the target feature extractor based on the trained preset neural network model.

[0173] Among them, the target feature extractor obtained through adversarial training can effectively remove batch effects in transcriptomics data from different slices, thus addressing the need for joint analysis of multiple slices.

[0174] Step S1204: Determine the multi-slice feature matrix of multiple target slices based on the target feature extractor.

[0175] Here, the multi-slice feature matrix is ​​the fusion matrix obtained by eliminating batch effects between different slices.

[0176] Step S1205: Clustering is performed based on the multi-slice feature matrix to determine the second unit clustering label for each analysis unit.

[0177] Step S1206: Generate a multi-slice projection image based on multiple second unit clustering labels, and determine the de-batch status of multiple target slices based on the multi-slice projection image.

[0178] Among them, such as Figure 11 As shown, by visualizing the data using UMAP plots, the effectiveness of the trained target feature extractor in removing batch effects from multiple target slices can be determined.

[0179] Reference Figure 13 This application also provides a multi-slice data processing apparatus, which includes:

[0180] Data acquisition module 1301 is used to acquire multimodal data of multiple target slices;

[0181] The clustering and labeling module 1302 is used to cluster multimodal data based on the analysis units of each target slice to determine multimodal labeled data; wherein, the multimodal labeled data is used to describe the relationship between the target slice and the target label data, and the target label data is used to describe the label information of each analysis unit in the target slice after clustering;

[0182] The feature extraction module 1303 is used to extract features from multimodal data based on the initial feature extractor of the preset neural network model to obtain a multimodal feature matrix;

[0183] The label prediction module 1304 is used to perform label prediction on the multimodal feature matrix based on the label prediction unit of the preset neural network model to obtain predicted label data. The predicted label data is used to describe the label information of each analysis unit in the predicted target slice.

[0184] The training module 1305 is used to train a preset neural network model based on target label data and predicted label data, and to obtain a target feature extractor based on the trained preset neural network model. The target feature extractor is used to extract features from multiple target slices.

[0185] In some embodiments, the label prediction unit includes a clustering classifier and a domain classifier, and the label prediction module 1304 is further configured to:

[0186] Clustering label prediction is performed on the multimodal feature matrix based on the clustering classifier to obtain the clustering prediction label of each analysis unit in each target slice;

[0187] Based on the domain classifier, the multimodal feature matrix is ​​used to predict the unit labels, and the predicted unit labels of each analysis unit in each target slice are obtained. The predicted unit labels are used to describe the label information of the target slice to which each predicted analysis unit belongs.

[0188] Predictive label data for each target slice is constructed based on cluster prediction labels and unit prediction labels.

[0189] In some embodiments, the label prediction module 1304 is further configured to:

[0190] Obtain the initial gradient data passed from the domain classifier to the initial feature extractor;

[0191] The initial gradient data is inverted to obtain inverted gradient data, and the parameters of the initial feature extractor are adjusted based on the inverted gradient data.

[0192] Feature extraction is performed on the multimodal data based on the parameter-adjusted initial feature extractor to update the multimodal feature matrix;

[0193] Based on the domain classifier, the updated multimodal feature matrix is ​​used to predict the unit labels, thereby obtaining the predicted unit labels for each analysis unit in each target slice.

[0194] In some embodiments, the training module 1305 is further configured to:

[0195] Based on the target label data, determine the first unit cluster label and the target unit label for each analysis unit in each target slice;

[0196] The first loss is constructed based on the clustering label of the first unit and the clustering prediction label;

[0197] A second loss is constructed based on the target unit label and the predicted unit label;

[0198] Construct the target loss based on the first loss and the second loss;

[0199] The pre-defined neural network model is trained based on the target loss.

[0200] In some embodiments, the data acquisition module 1301 is further configured to:

[0201] Acquire spatial transcriptomic data and staining images of the target slice;

[0202] Analysis units are constructed based on the pre-defined spatial coordinate system of the merged data and the target slice;

[0203] The spatial location data of the target slice were determined based on spatial transcriptome data and analysis units;

[0204] Spatial transcriptome data, staining images, and spatial location data of multiple target slices were fused to obtain multimodal data of multiple target slices.

[0205] In some embodiments, the clustering annotation module 1302 is further configured to:

[0206] The unit representation data of each analysis unit is determined from the multimodal data based on the coordinate information of the analysis unit;

[0207] Cluster the data expressed by multiple units and determine the first cluster label of each analysis unit;

[0208] The slice representation characters of the target slice are converted into slice representation labels by character label conversion;

[0209] Determine the target unit label for each analysis unit based on the slice representation label;

[0210] Multimodal labeled data is constructed based on the first unit cluster label, slice representation characters, and target unit labels.

[0211] In some embodiments, the apparatus further includes a clustering identification module, which is used for:

[0212] A multi-slice feature matrix for multiple target slices is determined based on the target feature extractor;

[0213] Clustering is performed based on the multi-slice feature matrix to determine the second unit clustering label for each analysis unit;

[0214] Generate multi-slice projection images based on multiple second-unit clustering labels;

[0215] Determine the de-batch status of multiple target slices based on multi-slice projection images.

[0216] It is evident that the content of the above-described multi-slice data processing method embodiments is applicable to the embodiments of this multi-slice data processing device. The specific functions implemented by this multi-slice data processing device embodiment are the same as those of the above-described multi-slice data processing method embodiments, and the beneficial effects achieved are also the same as those achieved by the above-described multi-slice data processing method embodiments.

[0217] Reference Figure 14 , Figure 14 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0218] The processor 1401 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0219] The memory 1402 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1402 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1402 and is called and executed by the processor 1401 using the multi-slice data processing method of the embodiments of this application.

[0220] The input / output interface 1403 is used to implement information input and output;

[0221] The communication interface 1404 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0222] Bus 1405 transmits information between various components of the device (e.g., processor 1401, memory 1402, input / output interface 1403, and communication interface 1404);

[0223] The processor 1401, memory 1402, input / output interface 1403 and communication interface 1404 are connected to each other within the device via bus 1405.

[0224] This application also provides a computer program product, which includes a computer program. A processor of a computer device reads and executes the computer program, causing the computer device to perform the multi-slice data processing method described above.

[0225] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in this disclosure and the foregoing drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “including,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatuses.

[0226] It should be understood that in this disclosure, "at least one item" means one or more, and "more than one" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0227] It should be understood that in the description of the embodiments of this application, "multiple" means two or more, "greater than", "less than", "exceeding" etc. are understood to exclude the number itself, and "above", "below", "within" etc. are understood to include the number itself.

[0228] In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0229] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0230] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0231] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0232] It should also be understood that the various implementation methods provided in this application can be combined arbitrarily to achieve different technical effects.

[0233] The above is a detailed description of the embodiments of this disclosure. However, this disclosure is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this disclosure. All such equivalent modifications or substitutions are included within the scope defined by the claims of this disclosure.

Claims

1. A method for processing multi-slice data, characterized in that, The method includes: Acquire multimodal data from multiple target slices; Based on the analysis unit of each target slice, the multimodal data is clustered to determine multimodal labeled data; wherein, the multimodal labeled data is used to describe the association between the target slice and the target label data, and the target label data is used to describe the label information after clustering of each analysis unit in the target slice; An initial feature extractor based on a preset neural network model extracts features from the multimodal data to obtain a multimodal feature matrix; The label prediction unit based on the preset neural network model performs label prediction on the multimodal feature matrix to obtain predicted label data, which is used to describe the label information of each analysis unit in the predicted target slice. The preset neural network model is subjected to adversarial training based on the target label data and the predicted label data, and a target feature extractor is obtained based on the trained preset neural network model. The target feature extractor is used to extract multi-slice features from multiple target slices.

2. The method according to claim 1, characterized in that, The label prediction unit includes a clustering classifier and a domain classifier. The label prediction unit based on the preset neural network model performs label prediction on the multimodal feature matrix to obtain predicted label data, including: Based on the clustering classifier, clustering label prediction is performed on the multimodal feature matrix to obtain the clustering prediction label of each analysis unit in each target slice; Based on the domain classifier, the multimodal feature matrix is ​​used to predict unit labels to obtain the predicted unit labels for each analysis unit in each target slice; the predicted unit labels are used to describe the label information of the target slice to which each analysis unit belongs; The predicted label data for each target slice is constructed based on the cluster prediction label and the unit prediction label.

3. The method according to claim 2, characterized in that, The step of predicting unit labels for the multimodal feature matrix based on the domain classifier to obtain the predicted unit labels for each analysis unit in each target slice includes: Obtain the initial gradient data passed from the domain classifier to the initial feature extractor; The initial gradient data is inverted to obtain inverted gradient data, and the parameters of the initial feature extractor are adjusted based on the inverted gradient data. Based on the parameter-adjusted initial feature extractor, features are extracted from the multimodal data to update the multimodal feature matrix; Based on the domain classifier, the updated multimodal feature matrix is ​​used to predict unit labels, thereby obtaining the predicted unit labels for each analysis unit in each target slice.

4. The method according to claim 2, characterized in that, The step of training the preset neural network model based on the target label data and the predicted label data includes: Based on the target label data, determine the first unit cluster label and target unit label of each analysis unit in each target slice; A first loss is constructed based on the first unit clustering label and the clustering prediction label; A second loss is constructed based on the target unit label and the predicted unit label; Construct a target loss based on the first loss and the second loss; The preset neural network model is trained based on the target loss.

5. The method according to claim 1, characterized in that, The acquisition of multimodal data from multiple target slices includes: Acquire spatial transcriptome data and staining images of the target slice; An analysis unit is constructed based on the preset merged data and the spatial coordinate system of the target slice; The spatial location data of the target slice are determined based on the spatial transcriptome data and the analysis unit. Data fusion is performed on the spatial transcriptome data, the staining images, and the spatial location data of multiple target slices to obtain the multimodal data of multiple target slices.

6. The method according to claim 5, characterized in that, The analysis unit based on each target slice clusters the multimodal data to determine multimodal labeled data, including: Based on the coordinate information of the analysis unit, the unit representation data of each analysis unit is determined from the multimodal data; Cluster the expressed data of multiple units to determine the first cluster label of each analysis unit; The slice representation characters of the target slice are converted into slice representation tags by character tag conversion; The target unit label for each analysis unit is determined based on the slice representation label; The multimodal annotation data is constructed based on the first unit clustering label, the slice representation character, and the target unit label.

7. The method according to claim 1, characterized in that, The method further includes: Based on the target feature extractor, a multi-slice feature matrix is ​​determined for multiple target slices; Clustering is performed based on the multi-slice feature matrix to determine the second unit clustering label for each analysis unit; Generate a multi-slice projection image based on multiple second-unit clustering labels; The de-batch status of multiple target slices is determined based on the multi-slice projection image.

8. A multi-slice data processing device, characterized in that, The device includes: The data acquisition module is used to acquire multimodal data from multiple target slices; The clustering and labeling module is used to cluster the multimodal data based on the analysis units of each target slice to determine multimodal labeled data; wherein, the multimodal labeled data is used to describe the association between the target slice and the target label data, and the target label data is used to describe the label information of each analysis unit in the target slice after clustering; The feature extraction module is used to extract features from the multimodal data based on the initial feature extractor of the preset neural network model to obtain a multimodal feature matrix; The label prediction module is used to perform label prediction on the multimodal feature matrix based on the label prediction unit of the preset neural network model to obtain predicted label data. The predicted label data is used to describe the label information of each analysis unit in the predicted target slice. The training module is used to train the preset neural network model based on the target label data and the predicted label data, and to obtain a target feature extractor based on the trained preset neural network model. The target feature extractor is used to perform multi-slice feature extraction on multiple target slices.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the multi-slice data processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the multi-slice data processing method according to any one of claims 1 to 7.

11. A computer program product comprising a computer program that is read and executed by a processor of a computer device, causing the computer device to perform the multi-slice data processing method according to any one of claims 1 to 7.

Images