Multisomic graph representation learning method and device based on multi-head self-attention mechanism

Abstract

The application provides a multi-omics graph representation learning method and device based on a multi-head self-attention mechanism, pre-processes multi-omics original data, converts the multi-omics original data into a matrix form, constructs or acquires a multi-source priori knowledge network according to existing priori knowledge, encodes the priori knowledge network by using a graph neural network to obtain a feature-level embedding representation, inputs the multi-omics feature matrix into the multi-head self-attention mechanism to capture potential correlations between different multi-omics data and between different multi-omics data, and performs dimension reduction processing on the original high-dimensional data to obtain a low-dimensional sample-level embedding representation, splices and fuses the feature-level embedding and the sample-level embedding to obtain a unified representation of comprehensive data features and priori knowledge, and finally inputs the fused representation into a task-specific module, performs supervised optimization by combining a cross-entropy loss function of a real label, and learns model parameters. In the application stage, only the multi-omics data to be analyzed needs to be input into the trained model, and a classification result can be output.

Description

Technical Field

[0001] This invention relates to the field of multi-omics cancer subtype classification technology, specifically to a multi-omics graph representation learning method and apparatus based on a multi-head self-attention mechanism. Background Technology

[0002] Against the backdrop of rapid advancements in precision medicine, cancer is no longer viewed as a single disease, but rather as a complex syndrome encompassing multiple molecular subtypes. In recent years, with the development of biotechnology, the maturation and widespread adoption of multi-omics technologies (such as genomics, transcriptomics, epigenetics, proteomics, and metabolomics) have become crucial for understanding the biological mechanisms of complex diseases (especially cancer) and developing precision treatment plans. This allows researchers to create a panoramic characterization of tumors at different biological levels, providing unprecedented data resources for subtype classification, prognostic assessment, and the discovery of therapeutic targets. Multi-omics data inherently possess the characteristics of "high dimensionality, small sample size, and strong heterogeneity." Compared to single-omics data, multi-omics data can comprehensively depict biological systems from different dimensions, providing richer information for tasks such as disease subtype classification, prognostic prediction, and drug target discovery.

[0003] To overcome the "curse of dimensionality" and inject biological interpretability, existing methods attempt to incorporate prior knowledge networks into the modeling process. Graph Neural Networks (GNNs), due to their natural adaptability to graph-structured data, have been widely applied in multi-omics data analysis. Most existing research employs GNNs to fuse specific types of biological prior knowledge (e.g., gene-gene interaction networks) to enhance the model's ability to learn relationships between features. However, existing methods often rely on prior knowledge networks from a single source, thus failing to effectively integrate information from multiple heterogeneous networks, thereby limiting the model's overall cognitive ability towards complex biological systems.

[0004] Furthermore, multi-omics data are typically high-dimensional and sparse, posing a significant challenge to feature extraction. Traditional methods or simple multilayer perceptrons (MLPs) often struggle to capture the deep nonlinear relationships within the data and between different omics when processing such data, leading to the loss of feature information. Although some methods attempt to use simple MLPs for feature learning, their ability to extract features and reduce dimensionality for high-dimensional data still has room for improvement.

[0005] To address these issues, researchers have recently explored more advanced feature extraction techniques. For example, some methods incorporate multi-head self-attention mechanisms into multi-omics data integration to learn the interdependencies between different features and effectively reduce feature dimensionality. However, these methods primarily focus on self-attention learning between samples and do not integrate multi-source prior knowledge networks into their core framework for collaborative optimization.

[0006] In view of the above, this application is hereby submitted. Summary of the Invention

[0007] This invention provides a multi-omics graph representation learning method and apparatus based on a multi-head self-attention mechanism, which can at least partially improve the above-mentioned problems.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] A multi-origin graph representation learning method based on a multi-head self-attention mechanism includes:

[0010] Multiple types of raw multi-omics data and multi-source prior knowledge networks are acquired, and the raw multi-omics data are preprocessed to obtain multi-omics data represented in matrix form.

[0011] A pre-defined graph neural network is used to encode the multi-source prior knowledge network to obtain a feature-level embedding representation. The feature-level embedding representation is a low-dimensional vector representation that reflects the known correlation between biological features and is learned from the multi-source prior knowledge network using the graph neural network.

[0012] The multi-omics data is input into a multi-head self-attention mechanism to learn and reduce the potential features within and between the multi-omics data, extract the feature information within different omics, and then perform linear transformation fusion after splicing them to obtain a low-dimensional sample-level embedding representation.

[0013] The feature-level embedding representation and the sample-level embedding representation are fused, and the fused representation is input into a preset classifier. The classification is optimized based on the cross-entropy loss function to complete the multi-omics cancer subtype classification task and obtain a trained multi-omics cancer subtype classification model.

[0014] Obtain the multi-omics data to be classified and the multi-source prior knowledge network, input the multi-omics data to be classified and the multi-source prior knowledge network into the pre-trained multi-omics cancer subtype classification model, and generate classification results.

[0015] The present invention also provides a multi-omics graph representation learning device based on a multi-head self-attention mechanism, comprising:

[0016] The preprocessing unit is used to acquire various types of raw multi-omics data and multi-source prior knowledge networks, and to preprocess the raw multi-omics data to obtain multi-omics data represented in matrix form.

[0017] The encoding unit is used to encode the multi-source prior knowledge network using a preset graph neural network to obtain a feature-level embedding representation, wherein the feature-level embedding representation is a low-dimensional vector representation that can reflect the known correlation between biological features, learned from the multi-source prior knowledge network using the graph neural network.

[0018] The dimensionality reduction unit is used to input the multi-omics data into the multi-head self-attention mechanism, learn and reduce the dimensionality of the potential features within and between the multi-omics data, extract the feature information within different omics, and then perform linear transformation fusion after splicing them to obtain a low-dimensional sample-level embedding representation.

[0019] The fusion unit is used to fuse feature-level embedding representations and sample-level embedding representations, and input the fused representation into a preset classifier. Based on the cross-entropy loss function, the classification is optimized to complete the multi-omics cancer subtype classification task and obtain a trained multi-omics cancer subtype classification model.

[0020] The classification unit is used to acquire the multi-omics data to be classified and the multi-source prior knowledge network. It inputs the multi-omics data to be classified and the multi-source prior knowledge network into the pre-trained multi-omics cancer subtype classification model to generate classification results.

[0021] In summary, this method preprocesses the original multi-omics data to obtain a feature matrix usable for the model; simultaneously, it constructs or acquires a multi-source prior knowledge network based on existing prior knowledge. This invention utilizes a graph neural network to encode the prior knowledge network, thereby obtaining a feature-level embedding representation. The multi-omics feature matrix is ​​input into a multi-head self-attention mechanism, which effectively captures potential correlations within and between different omics data and reduces the dimensionality of the original high-dimensional data to obtain a low-dimensional sample-level embedding representation. Subsequently, the feature-level embedding and the sample-level embedding are concatenated and fused to obtain a unified representation integrating data features and prior knowledge. Finally, the fused representation is input into a task-specific module, and supervised optimization is performed using a cross-entropy loss function combined with the true labels to learn the model parameters.

[0022] During the inference or application phase, the multi-omics data to be analyzed is simply input into the trained model to output classification results. This invention captures the deep nonlinear correlations and interdependencies within and between different omics data by introducing a multi-head self-attention mechanism. Simultaneously, it integrates feature-level embedding representations learned from multi-source prior knowledge networks that reflect biological prior knowledge, thereby achieving more efficient and accurate performance in multi-omics data analysis tasks, particularly suitable for disease classification, subtype prediction, and prognostic analysis. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating the multi-omics graph representation learning method based on a multi-head self-attention mechanism provided in the first embodiment of the present invention.

[0024] Figure 2 This is a schematic diagram of the flowchart of the multi-omics graph representation learning method based on the multi-head self-attention mechanism provided in the embodiments of the present invention.

[0025] Figure 3 This is a flowchart of the graph neural network used to obtain feature-level representations, as provided in an embodiment of the present invention.

[0026] Figure 4 This is a flowchart of the multi-head self-attention mechanism module provided in an embodiment of the present invention.

[0027] Figure 5 This is a schematic diagram of a module of a multi-omics graph representation learning device based on a multi-head self-attention mechanism provided in the second embodiment of the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0029] refer to Figure 1 , Figure 2 As shown, the first embodiment of the present invention discloses a multi-omics graph representation learning method based on a multi-head self-attention mechanism, which can be executed by a multi-omics graph representation learning device based on a multi-head self-attention mechanism (hereinafter referred to as the learning device), specifically, by one or more processors within the learning device, to implement the following method:

[0030] S1, acquire various types of raw multi-omics data and multi-source prior knowledge networks, preprocess the raw multi-omics data to obtain multi-omics data represented in matrix form;

[0031] Specifically, step S1 further includes: acquiring various types of raw multi-omics data and multi-source prior knowledge networks; cleaning each omics data of the raw multi-omics data; removing features or samples with too many missing values; and performing standardization or normalization preprocessing on each feature to ensure consistent data scale, so that each omics data is transformed into a feature matrix X that can be used in a multi-omics cancer subtype classification model, where rows of the multi-omics data represent biological samples and columns represent omics features.

[0032] The multi-omics data refers to the data set obtained by measuring the same batch of biological samples at different biological levels, including but not limited to at least two of the following: genomics, transcriptomics, epigenomics, proteomics, and metabolomics.

[0033] The multi-source prior knowledge network is a heterogeneous graph structure. The nodes of this network correspond to different types of biological characteristic information carriers, while the edges represent interactions or relationships between biomolecular entities, such as protein interactions, gene regulation, metabolic pathway involvement, or disease-gene associations. These relationships have all been validated based on existing biological experiments or large-scale data analysis.

[0034] In this embodiment, various types of high-dimensional raw multi-omics data (such as gene expression, DNA methylation, etc.) and multi-source prior knowledge networks (such as protein interaction networks, metabolic pathway maps, etc.) from multiple biological databases are acquired. The raw data first undergoes a unified preprocessing stage, where necessary preprocessing, such as standardization and normalization, is performed to ensure data usability.

[0035] Multi-omics datasets are collections of data composed of multiple sets of different biological characteristics or molecular descriptions, such as genomics, transcriptomics, and epigenetics. Each type of omics data can be regarded as a view. In addition, this method integrates multi-source prior knowledge networks as supplementary views. These views encode the complex relationships between known biomolecules, such as protein interactions or metabolic pathway relationships, providing the model with structured external biological knowledge.

[0036] Each view offers an observation and understanding of biological samples (such as cancer cell samples) from a different perspective. For example: Genomics views focus on the DNA sequence information of the sample, such as somatic mutations and copy number variations, providing a fundamental understanding of cancer at the gene level. Transcriptomics views focus on gene expression levels, reflecting the activity of genes in specific states (such as cancer subtypes), providing a dynamic observation of gene function execution. Multi-source prior knowledge network views include protein-protein interaction networks and metabolic pathway networks. These views do not come directly from the sample but provide known relationships between biomolecules, such as which proteins interact and which genes are on the same metabolic pathway. This prior knowledge enhances the model's understanding of biological mechanisms. These multi-source, multi-type view data are integrated by the model to achieve a more comprehensive and accurate classification of cancer subtypes.

[0037] S2, a preset graph neural network is used to encode the multi-source prior knowledge network to obtain a feature-level embedding representation, wherein the feature-level embedding representation is a low-dimensional vector representation that can reflect the known correlation between biological features, learned from the multi-source prior knowledge network using the graph neural network.

[0038] Please see Figure 3 Specifically, step S2 further includes: encoding the multi-source prior knowledge network using a preset graph neural network, aggregating gene-level information, and gradually updating the node representation of the multi-source prior knowledge network through a multi-layer message passing mechanism, wherein the preset graph neural network is a graph convolutional network.

[0039] In each layer, node features are updated by aggregating neighbor node information to improve the representation of the central node, thereby capturing information about the local structure and obtaining a feature-level embedding representation. The update formula for node features is as follows: , Let l be the node feature matrix of the l-th layer. The feature matrix of the nodes in the (l+1)th layer. For activation function, It is an adjacency matrix with self-connections. for The degree matrix, Let be the learnable weight matrix of the l-th layer.

[0040] In this embodiment, a Graph Neural Network (GNN) module is constructed to encode the multi-source prior knowledge network. This GNN module learns and obtains a feature-level embedding representation that reflects biological prior knowledge by aggregating the neighbor information of nodes (such as genes) in the network. Specifically, using a self-connected adjacency matrix and a node feature matrix as input, a three-layer message passing process is completed according to a formula. Each layer first symmetrically normalizes the adjacency, then aggregates neighbor features, and outputs a new node feature matrix after linear transformation of the weight matrix and activation function, ultimately obtaining the feature-level embedding representation. This embedding, by aggregating local structural information, transforms the heterogeneous graph into a low-dimensional vector, providing trained and optimized graph priors when concatenated with sample-level embeddings, enabling the fused representation to possess both biological relationships and task-oriented features. This mechanism allows each node to integrate the feature information of its neighboring nodes at each layer, thereby gradually constructing a feature-level embedding representation containing both local and global structural information.

[0041] S3, the multi-omics data is input into a multi-head self-attention mechanism to learn and reduce the potential features within and between the multi-omics data, extract the feature information within different omics, and then perform linear transformation fusion after splicing them to obtain a low-dimensional sample-level embedding representation.

[0042] Please see Figure 4Specifically, step S3 further includes: inputting the multi-omics data into a multi-head self-attention mechanism, performing a linear transformation on the feature matrix X of the multi-omics data to obtain a query matrix Q, a key matrix K, and a value matrix V;

[0043] in, Figure 4 Formulas in This can be represented as passing the input feature matrix X through a learnable weight matrix W. q , can be linearly transformed into a query matrix Q; Figure 4 Formulas in This can be represented as passing the input feature matrix X through a learnable weight matrix W. k It can be linearly transformed into a bond matrix K; Figure 4 Formulas in This can be represented as passing the input feature matrix X through a learnable weight matrix W. v This can be linearly transformed into a value matrix V. These three steps are the pre-projection layers of multi-head attention, mapping the same input X to three different semantic spaces QKV, so that attention scores can be calculated and information can be aggregated later.

[0044] The attention score is calculated and normalized using the softmax function to obtain the attention weight matrix. Attention weight matrix This is used to represent the association strength of different features in a priori knowledge network, where T is the transpose. Let be the dimension of the key matrix;

[0045] Multiplying the attention weight matrix and the value matrix yields the single-head attention output, the formula of which is: Multiple attention heads are executed, their outputs are concatenated, and a linear transformation is performed to generate the final multi-head attention output Z, resulting in a low-dimensional sample-level embedding representation. The calculation formula is as follows: , 'i' represents the nth attention head, which is an index variable whose value range is 1. , For the output of a single attention head, For the output of the i-th attention head, For the output of the h-th attention head, , , All of these are learnable weight matrices for the i-th attention head. The output weight matrix is ​​h, where h is the total number of attention heads.

[0046] In this embodiment, preprocessed multi-omics data is input into a multi-head self-attention mechanism. This mechanism captures deep nonlinear correlations within each omics dataset and between different omics datasets by computing multiple attention heads in parallel. More importantly, by controlling the output dimension within the attention mechanism, dimensionality reduction of the high-dimensional original data is achieved, resulting in a low-dimensional, semantically rich sample-level embedding representation.

[0047] Specifically, X undergoes three independent linear projections to obtain the query matrix Q, key matrix K, and value matrix V. The projection dimensions are all lower than the original feature dimensions, achieving initial dimensionality reduction while preserving key information. Subsequently, attention scores are calculated using a formula, and softmax normalization yields the attention weight matrix. This matrix quantifies the association strength between each feature and all other features, essentially automatically learning cross-omics coupling weights under the guidance of a prior knowledge network, avoiding biases introduced by manual selection. The weight matrix is ​​multiplied by V to obtain the single-head attention output, which is repeated in parallel multiple times. Each head independently focuses on signals in different subspaces. Multiple outputs are then concatenated along the feature dimensions, and finally, a linear fusion is performed using the output weight matrix to generate a sample-level embedding representation Z with dimensions far lower than the original input. This process completes "feature selection-weight allocation-dimensionality reduction integration" in one step, compressing high-dimensional sparse omics data into low-dimensional dense vectors, significantly reducing the number of parameters in subsequent classifiers. Simultaneously, the multi-head mechanism ensures that complementary information from different omics is captured synchronously, avoiding the information bottleneck caused by traditional sequential fusion.

[0048] This mechanism sets the model's output dimension to be lower than the original multi-omics data dimension, thus achieving effective feature extraction and dimensionality reduction. This reduces the dimensionality of high-dimensional input data while effectively extracting key features. This approach not only effectively captures the complex relationships within and between multi-omics data but also improves the robustness and interpretability of feature representations.

[0049] S4. The feature-level embedding representation and the sample-level embedding representation are fused, and the fused representation is input into the preset classifier. The classification is optimized based on the cross-entropy loss function to complete the multi-omics cancer subtype classification task and obtain the trained multi-omics cancer subtype classification model.

[0050] Specifically, step S4 further includes: fusing the feature-level embedding representation and the sample-level embedding representation, specifically: the fusion method adopts concatenation fusion, wherein the feature-level embedding representation is fused according to dimension. and sample-level embedding representation The images are then stitched together to obtain a unified representation. .

[0051] The fused representation is input into a pre-defined classifier to obtain the output prediction result. Combined with the true labels, classification optimization is performed using the cross-entropy loss function to complete the multi-omics cancer subtype classification task. The formula for the cross-entropy loss function is: N is the number of samples, and C is the total number of categories. Let be the true label of the i-th sample in class c. The unified representation after fusion is passed through a classifier to predict the probability of class c.

[0052] The pre-processed multi-omics data with category labels is input into the multi-omics cancer subtype classification model for pre-training, contrastive learning, and parameter fine-tuning to obtain a trained multi-omics cancer subtype classification model.

[0053] In this embodiment, the feature-level embedding representations and sample-level embedding representations learned by the two modules are concatenated and fused to form a unified embedding space that simultaneously includes the data's own features and biological prior knowledge. Finally, this unified embedding is input into a fully connected network as a classifier to complete the final cancer subtype classification task. End-to-end supervised learning training of the entire model is performed using the cross-entropy loss function.

[0054] Specifically, feature-level embeddings and sample-level embeddings are fed into the same fusion unit and directly concatenated along the column dimension to form a unified representation. This operation requires no additional hyperparameters and can simultaneously preserve the "graph structure prior" and the "omics data's own signals" in the vector space, allowing subsequent classifiers to see both biological relationships and sample features, avoiding information dilution that may result from weighting or summing. The concatenated unified representation is then input into a fully connected classifier. The output dimension of this classifier is consistent with the number of cancer subtypes, and the activation function is softmax, obtaining the predicted probability of each sample in each subtype. Combined with the ground truth labels, the error is calculated using the cross-entropy loss function. The loss value is updated simultaneously with backpropagation of the graph neural network, multi-head self-attention, and all weights of the classifier, achieving end-to-end optimization.

[0055] In this embodiment, a preprocessed and labeled multi-omics training set is input into the constructed multi-omics classification model for training. The model parameters are adjusted, and the adjusted model is saved to obtain the trained multi-omics classification model. For model testing, a publicly available multi-omics dataset is used, and all experiments are conducted under the same hardware environment. The training dataset uses a dataset publicly available on TCGA, containing 9664 samples and two types of omics data, each with a feature dimension of 5000. The experiments utilize three prior biological knowledge sources: gene-gene interaction (GGI) networks, protein-protein interaction (PPI) networks, and co-expression (COEXP) networks. The GGI plots are derived from the Biogrid database, while the PPI and CoEXP plots are constructed from string databases.

[0056] Furthermore, during training, a graph construction method was used to generate edges for each node by calculating the nearest neighbor of each node through Euclidean distance. Three common evaluation metrics—Accuracy (ACC), Mean Average Precision (MAP), Macro Average F1 Score (MAF), and Macro Average Recall (MAR)—were used to evaluate the model's test results. When the model's evaluation results reached a set threshold, the trained multi-omics classification model was obtained. The newly acquired multi-view dataset, after preprocessing, was then input into the trained multi-omics classification model to obtain multi-omics cancer subtype classification results.

[0057] In summary, this method addresses the shortcomings of existing methods in integrating multi-source prior knowledge and effectively fusing heterogeneous data by modeling the specific structures of each omics during the graph construction phase and using a multi-head self-attention mechanism for interpretable domain aggregation during the message passing phase.

[0058] S5: Obtain the multi-omics data and multi-source prior knowledge network to be classified, input the multi-omics data and multi-source prior knowledge network to be classified into the pre-trained multi-omics cancer subtype classification model, and generate classification results.

[0059] Specifically, step S5 further includes: the formula for the classification result is as follows: , This means identifying the class with the highest predicted score from class C.

[0060] In this embodiment, after training is completed, new sample data to be classified is acquired and input into the trained model to obtain the predicted cancer subtype classification results. Specifically, after the model completes end-to-end training and fixes the optimal parameters, it can be used for actual subtype determination: simply collect a new batch of tumor samples from the same platform and process as the training set, and complete missing value filtering, standardization, and normalization according to step S1 to obtain the multi-omics feature matrix to be classified; at the same time, extract the multi-source prior knowledge network of the corresponding species from the same version of the database, keeping the node ID completely consistent with the training stage to ensure the semantic continuity of the graph structure. Input both into the trained multi-omics cancer subtype classification model simultaneously. The graph neural network part immediately outputs feature-level embeddings, and the multi-head self-attention module generates sample-level embeddings in parallel. The two are concatenated in the same way in the inference mode, and the predicted scores of each subtype are obtained after one forward propagation; finally, the index with the highest score from C categories is selected as the subtype label of the sample according to the formula. The whole process does not require manual setting of thresholds or additional post-processing. Because the training phase has combined biological priors with data features for optimization, the model is tolerant of a small number of missing features or changes in network edges, and can complete single-case inference within seconds, meeting real-time clinical needs. At the same time, the prediction results are aligned with the true labels and can be directly mapped to molecular subtypes in existing treatment guidelines, providing doctors with actionable intervention suggestions and achieving a seamless connection from laboratory models to clinical decision-making.

[0061] Specifically, in the inference / application phase, the multi-omics data to be classified and the corresponding prior knowledge network are first acquired and input into the trained model. The model outputs the predicted probabilities of each category and determines the target category, thus obtaining the final classification result. For example, in cancer subtype classification, new patient samples can be characterized through multiple omics data (such as genomics and transcriptomics) and combined with known biological pathway networks. The trained model can be used to classify samples into categories such as "breast cancer subtype A" and "breast cancer subtype B," thereby guiding accurate diagnosis and subsequent treatment for patients. In disease prognosis analysis, the trained model can be used to classify patients into categories such as "low-risk recurrence" and "high-risk recurrence," thereby assisting clinical decision-making. Thus, the classification results not only provide numerical predictions from the model but also provide clear category labels that can directly support decision-making at the application level.

[0062] In summary, to address the shortcomings of multi-omics data analysis methods in simultaneously and effectively integrating multi-source prior knowledge and performing efficient feature extraction, this invention proposes a multi-omics graph representation learning method based on a multi-head self-attention mechanism. The core idea is to utilize the multi-head self-attention mechanism to perform dimensionality reduction and feature extraction on high-dimensional multi-omics data, and then effectively fuse it with features learned from multi-source prior knowledge networks, thereby improving the model's performance in downstream classification tasks.

[0063] This invention effectively addresses the issues of feature redundancy and information loss in high-dimensional multi-omics data by replacing traditional sample-level modules with a multi-head self-attention mechanism in the core architecture. This mechanism can deeply mine the potential correlations within the data and achieve efficient feature dimensionality reduction, enabling the model to better utilize multi-source prior knowledge and significantly improving accuracy and generalization ability on cancer subtype classification tasks.

[0064] Compared with existing technologies, this method offers the following advantages: Based on multi-head self-attention mechanisms and graph neural networks, it extracts and fuses features from multi-omics data to achieve cancer subtype classification. This method utilizes graph neural networks to encode a multi-source prior knowledge network, obtaining feature-level embedding representations that effectively integrate biological prior knowledge. Simultaneously, a multi-head self-attention mechanism is employed to capture potential correlations within and between different omics data, yielding sample-level embedding representations. Subsequently, the two embeddings are concatenated and fused to construct a unified representation space integrating data features and biological prior knowledge. Finally, supervised optimization is performed using a cross-entropy loss function based on the true labels. This unique fusion strategy enables the model to utilize multi-source prior knowledge more efficiently and accurately, achieving superior performance in multi-omics data analysis tasks, particularly suitable for disease classification and subtype prediction.

[0065] Furthermore, by introducing the cross-entropy loss function and continuously optimizing and updating the model parameters, the accuracy of the multi-omics classification model is improved. This method can achieve accurate classification of unlabeled samples, thereby significantly improving overall classification performance while ensuring training efficiency.

[0066] Please see Figure 5 A second embodiment of the present invention provides a multi-omics graph representation learning device based on a multi-head self-attention mechanism, comprising:

[0067] Preprocessing unit 101 is used to acquire various types of raw multi-omics data and multi-source prior knowledge networks, and preprocess the raw multi-omics data to obtain multi-omics data represented in matrix form;

[0068] The encoding unit 102 is used to encode the multi-source prior knowledge network using a preset graph neural network to obtain a feature-level embedding representation, wherein the feature-level embedding representation is a low-dimensional vector representation that can reflect the known correlation between biological features, learned from the multi-source prior knowledge network using the graph neural network.

[0069] The dimensionality reduction unit 103 is used to input the multi-omics data into the multi-head self-attention mechanism, learn and reduce the dimensionality of the potential features within and between the multi-omics data, extract the feature information within different omics, and then perform linear transformation fusion after splicing them to obtain a low-dimensional sample-level embedding representation.

[0070] The fusion unit 104 is used to fuse the feature-level embedding representation and the sample-level embedding representation, and input the fused representation into the preset classifier. Based on the cross-entropy loss function, the classification is optimized to complete the multi-omics cancer subtype classification task and obtain the trained multi-omics cancer subtype classification model.

[0071] The classification unit 105 is used to acquire the multi-omics data to be classified and the multi-source prior knowledge network, and input the multi-omics data to be classified and the multi-source prior knowledge network into the pre-trained multi-omics cancer subtype classification model to generate classification results.

[0072] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A multi-origin graph representation learning method based on a multi-head self-attention mechanism, characterized in that, include: Multiple types of raw multi-omics data and multi-source prior knowledge networks are acquired, and the raw multi-omics data are preprocessed to obtain multi-omics data represented in matrix form. A pre-defined graph neural network is used to encode the multi-source prior knowledge network to obtain a feature-level embedding representation. The feature-level embedding representation is a low-dimensional vector representation that can reflect the known correlation between biological features, which is learned from the multi-source prior knowledge network by the graph neural network. The multi-source prior knowledge network is a heterogeneous graph structure in which the nodes correspond to different types of biological feature information carriers, and the edges represent the interaction or correlation between biological molecular entities. The multi-omics data is input into a multi-head self-attention mechanism to learn and reduce the potential features within and between the multi-omics data, extract the feature information within different omics, and then perform linear transformation fusion after splicing them to obtain a low-dimensional sample-level embedding representation. The feature-level embedding representation and the sample-level embedding representation are fused, and the fused representation is input into a preset classifier. The classification is optimized based on the cross-entropy loss function to complete the multi-omics cancer subtype classification task and obtain a trained multi-omics cancer subtype classification model. Obtain the multi-omics data to be classified and the multi-source prior knowledge network, input the multi-omics data to be classified and the multi-source prior knowledge network into the pre-trained multi-omics cancer subtype classification model, and generate classification results; The feature-level embedding representation and the sample-level embedding representation are fused, specifically by concatenation. The feature-level embedding representation is then fused according to its dimension. and sample-level embedding representation The images are then stitched together to obtain a unified representation. ; The fused representation is input into a pre-defined classifier, and classification optimization is performed based on the cross-entropy loss function to complete the multi-omics cancer subtype classification task, resulting in a trained multi-omics cancer subtype classification model, specifically: The fused representation is input into a pre-defined classifier to obtain the output prediction result. Combined with the true labels, classification optimization is performed using the cross-entropy loss function to complete the multi-omics cancer subtype classification task. The formula for the cross-entropy loss function is: N is the number of samples, and C is the total number of categories. Let be the true label of the i-th sample in class c. The unified representation after fusion is passed through a classifier to predict the probability of class c. The pre-processed multi-omics data with category labels is input into the multi-omics cancer subtype classification model for pre-training, comparative learning and parameter fine-tuning to obtain a trained multi-omics cancer subtype classification model. The formula for the classification result is: , This means identifying the class with the highest predicted score from class C.

2. The multi-omics graph representation learning method based on multi-head self-attention mechanism according to claim 1, characterized in that, Multiple types of raw multi-omics data and multi-source prior knowledge networks are acquired. The raw multi-omics data is preprocessed to obtain a matrix representation of the multi-omics data, specifically: Multiple types of raw multi-omics data and multi-source prior knowledge networks are acquired. Data cleaning is performed on each omics data of the raw multi-omics data to remove features or samples with too many missing values. Each feature is preprocessed by standardization or normalization to ensure data scale consistency. This transforms each omics data into a feature matrix X that can be used in a multi-omics cancer subtype classification model. The rows of the multi-omics data represent biological samples, and the columns represent omics features. The multi-omics data refers to the data set obtained by measuring the same batch of biological samples at different biological levels, including but not limited to at least two of the following: genomics, transcriptomics, epigenetics, proteomics, and metabolomics data.

3. The multi-omics graph representation learning method based on multi-head self-attention mechanism according to claim 1, characterized in that, The multi-omics data is input into a multi-head self-attention mechanism to learn and reduce the dimensionality of the latent features within and between the multi-omics data, extract the feature information within different omics, and then concatenate and fuse them using linear transformation to obtain a low-dimensional sample-level embedding representation, specifically: The multi-omics data is input into a multi-head self-attention mechanism, and a linear transformation is performed on the feature matrix X of the multi-omics data to obtain the query matrix Q, the key matrix K, and the value matrix V. The attention score is calculated, and then normalized using the softmax function to obtain the attention weight matrix. Attention weight matrix Used to represent the correlation strength between different features, where T is the transpose. Let be the dimension of the key matrix; Multiplying the attention weight matrix and the value matrix yields the single-head attention output, the formula of which is: Multiple attention heads are executed, their outputs are concatenated, and a linear transformation is performed to generate the final multi-head attention output Z, resulting in a low-dimensional sample-level embedding representation. The calculation formula is as follows: , 'i' represents the nth attention head, which is an index variable whose value range is 1. , For the output of a single attention head, For the output of the i-th attention head, For the output of the h-th attention head, , , All of these are learnable weight matrices for the i-th attention head. The output weight matrix is ​​h, where h is the total number of attention heads.

4. A multi-origin graph representation learning device based on a multi-head self-attention mechanism, characterized in that, include: The preprocessing unit is used to acquire various types of raw multi-omics data and multi-source prior knowledge networks, and to preprocess the raw multi-omics data to obtain multi-omics data represented in matrix form. The encoding unit is used to encode the multi-source prior knowledge network using a preset graph neural network to obtain a feature-level embedding representation. The feature-level embedding representation is a low-dimensional vector representation that can reflect the known correlation between biological features, learned from the multi-source prior knowledge network using the graph neural network. The multi-source prior knowledge network is a heterogeneous graph structure in which the nodes correspond to different types of biological feature information carriers, and the edges represent the interactions or correlations between biological molecular entities. The dimensionality reduction unit is used to input the multi-omics data into the multi-head self-attention mechanism, learn and reduce the dimensionality of the potential features within and between the multi-omics data, extract the feature information within different omics, and then perform linear transformation fusion after splicing them to obtain a low-dimensional sample-level embedding representation. The fusion unit is used to fuse feature-level embedding representations and sample-level embedding representations, and input the fused representation into a preset classifier. Based on the cross-entropy loss function, the classification is optimized to complete the multi-omics cancer subtype classification task and obtain a trained multi-omics cancer subtype classification model. The classification unit is used to acquire the multi-omics data to be classified and the multi-source prior knowledge network. It inputs the multi-omics data to be classified and the multi-source prior knowledge network into the pre-trained multi-omics cancer subtype classification model to generate classification results. The feature-level embedding representation and the sample-level embedding representation are fused, specifically by concatenation. The feature-level embedding representation is then fused according to its dimension. and sample-level embedding representation The images are then stitched together to obtain a unified representation. ; The fused representation is input into a pre-defined classifier, and classification optimization is performed based on the cross-entropy loss function to complete the multi-omics cancer subtype classification task, resulting in a trained multi-omics cancer subtype classification model, specifically: The fused representation is input into a pre-defined classifier to obtain the output prediction result. Combined with the true labels, classification optimization is performed using the cross-entropy loss function to complete the multi-omics cancer subtype classification task. The formula for the cross-entropy loss function is: N is the number of samples, and C is the total number of categories. Let be the true label of the i-th sample in class c. The unified representation after fusion is passed through a classifier to predict the probability of class c. The pre-processed multi-omics data with category labels is input into the multi-omics cancer subtype classification model for pre-training, comparative learning and parameter fine-tuning to obtain a trained multi-omics cancer subtype classification model. The formula for the classification result is: , This means identifying the class with the highest predicted score from class C.

Images