Multi-omics data integration and classification method, system and equipment based on hierarchical attention

Abstract

The invention discloses a multi-omics data integration and classification method, system and device based on hierarchical attention, and is applied to the field of precise medical big data analysis. The method comprises the following steps: firstly, generating feature embedding and feature importance scores through a plurality of parallel feature-level attention modules; then, embedding and inputting all the characteristics of the omics into a unified omics-level attention module, and generating omics embedding and omics importance scores; and finally, a classification prediction task is executed based on omics embedding, and a classification result is output for disease classification. The invention completely abandons a traditional dependency graph convolutional network and an integration normal form of variants of the dependency graph convolutional network, and provides a universal hierarchical attention integration architecture. The framework supports classification tasks of any complex diseases, is not limited by omics data types and combination modes, not only is remarkably superior to a traditional integration normal form in classification performance, but also shows a unique negative generalization distance, and proves that the framework has excellent generalization ability. Meanwhile, features and omics importance scores automatically output by the model provide a powerful analysis tool for biomarker discovery and precise diagnosis and treatment of complex diseases.

Description

Technical Field

[0001] This invention relates to the field of precision medicine big data analysis, and in particular to a novel, general, hierarchical attention-based multi-omics data integration and classification framework for the accurate classification of any complex disease. It can automatically assess the importance of features and omics, and improve the interpretability of the model. Background Technology

[0002] Existing Techniques Based on Graph Convolutional Networks and Their Variants: Integrating and analyzing multi-omics data (such as genomics, transcriptomics, epigenetics, proteomics, metabolomics, etc.) of complex diseases is an important method for studying their pathogenesis. However, current mainstream methods are mainly based on graph models of graph convolutional networks and their variants (such as MoGCN, MOGONET, SUPREME, MOGAT, HyperTMO, etc.), which heavily rely on predefined or adaptively learned patient similarity graph structures. This approach has inherent drawbacks, including complex graph construction processes, sensitivity to similarity metric selection, and inability to optimally capture features and complex interactions between omics.

[0003] Improvements and Limitations of Graph Models Incorporating Attention Mechanisms: While some studies have attempted to introduce attention mechanisms, most have treated them as independent components paired with graph models (such as MOSGAT, MOGLAM, MoAGL-SA, DMOIT, and DeepKEGG), failing to fundamentally escape the constraints of graph structures. Furthermore, some non-graph-based attempts exist, such as the CLCLSA model, which describes a "pure attention scheme." However, it's important to clarify that CLCLSA's "attention" is achieved through a multilayer perceptron combined with a sigmoid activation function; its essence is a conventional feature projection and gating mechanism, rather than the universally accepted attention mechanism based on query-key-value interaction. More importantly, in the omics integration phase, CLCLSA completes integration through simple omics embedding concatenation. This integration method cannot achieve dynamic interaction and learning between different omics embeddings during the integration process. Therefore, CLCLSA fails to address the problem of insufficient cross-omics information fusion in existing technologies, and its model interpretability is also relatively limited.

[0004] The technical problem to be solved by this invention is that there is an urgent need in the field for a new integration framework that does not rely on graph structure modeling, is highly versatile, and has high interpretability. Summary of the Invention

[0005] Purpose of the Invention: The purpose of this invention is to overcome the shortcomings of existing technologies and provide a multi-omics data integration and classification method, system, and device based on hierarchical attention and independent of graph structure. This framework is versatile, not only achieving excellent classification performance but also automatically providing importance scores at the feature and omics levels, and possessing strong interpretability.

[0006] Technical Solution: To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a multi-omics data integration and classification method based on hierarchical attention, comprising the following steps: (1) obtaining feature tensors from multiple omics data sources; (2) inputting the feature tensors of each omics data source into the corresponding feature-level attention module, and generating feature embeddings and feature importance scores; (3) inputting the feature embeddings from all omics data sources into a unified omics-level attention module, and generating omics embeddings and omics importance scores; based on the omics embeddings, performing a classification prediction task, and outputting classification results for disease classification. According to some embodiments of the present invention, the processing of the feature-level attention module includes: performing a first element-level fusion operation on the input feature tensor and the learnable feature semantic embedding; the processing of the omics-level attention module includes: performing a second element-level fusion operation on the feature embeddings of each omics data source and the learnable omics semantic embedding; wherein the first element-level fusion operation and the second element-level fusion operation are independently selected from element addition or element multiplication. According to some embodiments of the present invention, at least one of the feature-level attention module and the omics-level attention module is implemented through an attention mechanism. According to some embodiments of the present invention, the attention mechanism is a multi-head self-attention mechanism. According to some embodiments of the present invention, the feature-level attention module and the omics-level attention module further include: at least one feedforward network layer. According to some embodiments of the present invention, the feature-level attention module and the omics-level attention module further include: at least one residual connection layer. According to some embodiments of the present invention, the feature importance score and the omics importance score are generated by normalizing the normalized feature or omics embedding output by the attention mechanism after projection. According to some embodiments of the present invention, the projection is implemented through a linear projection layer; the normalization is implemented through a softmax function with a temperature parameter. In a second aspect, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method described above. In a third aspect, the present invention provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method described above.

[0007] The core innovation and universality of this invention: It should be noted that the core innovation of the MOMLA (Multi-Omics Integration Framework Based on Multi-Level Attention Mechanisms) framework proposed in this invention lies in its universal architecture design. It does not depend on any specific combination of omics data types and can support the integration of any two or more omics data such as genomics, transcriptomics, epigenetics, proteomics, and metabolomics. At the same time, this framework is applicable to any complex disease classification task, including but not limited to breast cancer, lung cancer, and Alzheimer's disease. Its strong universality ensures its effectiveness in different complex disease classification application scenarios. Compared with the prior art, this invention has the following significant advantages: (1) Architectural innovation and excellent performance: The "hierarchical attention integration architecture" is proposed for the first time, completely abandoning the constraints of traditional graph convolutional networks. This architecture achieves excellent performance on the breast cancer PAM50 classification task and exhibits a unique "negative generalization distance", indicating that it has learned more essential biological laws; (2) High interpretability and strong biomarker mining ability: The model can automatically output feature importance scores and omics importance scores. This goes beyond the "black box" model that only provides prediction results, providing biologists with clear insights that can be directly used to identify the most critical biomarkers for disease classification and to understand the contributions of different omics types; (3) Dynamic collaborative integration: Unlike the direct splicing integration methods such as CLCLSA, the omics-level modules of this invention achieve true dynamic and collaborative fusion through a unified attention mechanism; (4) Universality and robustness: This framework is highly universal, not designed for specific omics types or diseases, and can support the integration of multi-omics data of arbitrary combinations and the classification tasks of arbitrarily complex diseases. The residual connections, feedforward networks, and other components included in the structure ensure the stability of the training process and the generalization ability of the model.

[0008] Attached image description: Figure 1 (A) is the overall architecture diagram of the MOMLA model. Figure 1 (B) is a detailed structural diagram of the feature-level attention module. Figure 1 (C) is a detailed structural diagram of the omics-level attention module. Figure 2 This is a comparison chart of the confusion matrix of the method of this invention and existing technologies (HyperTMO, MOGLAM) on the test set. Detailed Implementation

[0009] To make the objectives, technical solutions, and advantages of this invention clearer, the embodiments of this invention will be described in further detail below with reference to the accompanying drawings, using the PAM50 classification of breast cancer as an example. It should be emphasized that this embodiment is only used to illustrate the specific application of this invention and is not intended to limit the scope of protection of this invention. The technical solutions of this invention are also applicable to classification tasks of any other disease, as well as any other combination of multi-omics data.

[0010] Algorithm flow overview: The core algorithm flow of this invention can be summarized as follows: (1) Input: Multi-omics feature tensor dictionary and feature name list dictionary; (2) Steps: Initialize an empty dictionary to store feature embeddings and importance score lists; For each omics type, call its corresponding feature-level attention module to process the input feature tensor, obtain feature embeddings and feature importance score lists, and store them in the dictionary; Input all feature embeddings into the omics-level attention module, perform cross-omics integration, and obtain the final omics embeddings and omics importance score lists; Normalize and randomly deactivate the omics embeddings; Input the processed omics embeddings into the multilayer perceptron classifier to obtain the prediction results; (3) Output: Feature importance list dictionary, omics importance score list, classification prediction results.

[0011] Data Preparation and Preprocessing: This embodiment uses the TCGA-BRCA dataset, which includes three omics datasets: mRNA expression, miRNA expression, and DNA methylation. After standardization and other preprocessing, the raw data is organized into feature tensors. The dataset is then randomly divided into training, validation, and test sets at a ratio of 60%, 20%, and 20%, respectively, while maintaining the same proportions of each category in each set as in the original dataset.

[0012] Model architecture and process: Please refer to Figure 1 (A) illustrates the overall architecture of the invention (MOMLA). Its core consists of multiple parallel feature-level attention modules and a unified omics-level attention module. The first step is to input the feature tensor of each type of omics data (e.g., mRNA expression) into the corresponding feature-level attention module. Figure 1(B) Shows the details of this module): (1) Residual projection: Linear projection is performed on the input feature tensor to reduce the dimensionality and retain information for subsequent residual connections; (2) Feature semantic embedding and fusion: Each feature name is mapped to a learnable embedding vector. The feature semantic embedding is multiplied and fused with the original feature tensor to form a feature semantically aware feature embedding; (3) Attention mechanism and layer normalization: Attention mechanism (multi-head self-attention is used in this embodiment) is performed on the fused feature embedding to capture the complex nonlinear dependencies between features. After calculation, the model is stabilized by residual connections and layer normalization; (4) Feedforward network and layer normalization: The feature embedding is further refined by a feedforward network (a feedforward network layer containing the GELU activation function is used in this embodiment), and residual connections and layer normalization are also used to enhance the model's fitting ability; (5) Importance score generation and aggregation: The feature importance score of each feature is generated by projecting and normalizing the feature embedding output by the attention mechanism (a softmax function with temperature parameters is used in this embodiment). Subsequently, the feature embeddings are weighted and summed according to their importance scores to obtain a weighted feature embedding. This weighted feature embedding is then added to the initial residual projection to output the feature embedding for that omics. This process is repeated for miRNA expression and DNA methylation data. The second step involves inputting the feature embeddings from all omics (mRNA expression, miRNA expression, DNA methylation) into a unified omics-level attention module. Figure 1 (C) Shows the details of the module): (1) Omics semantic embedding and fusion: Assign a learnable embedding vector to each omics type and fuse it with the corresponding feature embedding to inject omics identity information; (2) Concatenation and attention mechanism: Concatenate all the fused feature embeddings together and input them into a unified attention mechanism (multi-head self-attention is used in this embodiment) layer, which dynamically models the collaboration and dependency between different omics types; (3) Feedforward network and layer normalization: Similarly, the output of the attention mechanism is nonlinearly transformed by the feedforward network and residual connection and layer normalization are used; (4) Importance score generation and aggregation: Omics importance scores are generated by projecting the omics embeddings output by the module and normalizing them (softmax function with temperature parameter is used in this embodiment). Finally, the omics embeddings are weighted and aggregated according to these scores to obtain a unified omics embedding. In the third step, after the omics embeddings are processed by layer normalization and random deactivation, they are input into a multilayer perceptron classifier and finally output the predicted category of PAM50 subtype. It also outputs feature importance scores and omics importance scores for model interpretation and biomarker discovery.

[0013] Performance Comparison Experiments and Result Analysis: To fairly and rigorously verify the effectiveness and advancement of the proposed framework (MOMLA), we designed independent repeated experiments. For each repeated experiment, the proposed model (MOMLA) and all baseline models used the exact same random seed and followed the same data preprocessing procedure and model training strategy. A total of 10 independent repeated experiments were conducted, each using a different random seed. This approach aims to ensure, to the greatest extent possible, that performance differences stem from differences in the model architecture itself, rather than randomness or differences in model training strategies. In the model performance comparison, the existing baseline model HyperTMO has a balance accuracy of 0.726±0.006, a macro F1 score of 0.727±0.006, a macro AUC score of 0.901±0.005, and a relative training-test set distance of (4.613±0.761)%; another baseline model, MOGLAM, has a balance accuracy of 0.779±0.015, a macro F1 score of 0.742±0.019, a macro AUC score of 0.922±0.015, and a relative training-test set distance of (14.466±1.914)%. In comparison, the MOMLA model proposed in this invention achieves better performance, with a balance accuracy of 0.811±0.013, a macro F1 score of 0.792±0.015, a macro AUC score of 0.944±0.003, and a significant reduction in the relative training-test set distance to (-76.418±1.628)%.

[0014] Based on the above strictly controlled comparative experiments, the following conclusions can be drawn: (1) Significant and stable performance improvement: In 10 repeated experiments, the MOMLA model proposed in this invention showed significantly higher mean values ​​than existing baseline models in the three core classification performance indicators of balanced accuracy, macro F1 score, and macro AUC score, and the standard deviation indicated that the results had good stability. This fully demonstrates that the hierarchical attention integration architecture proposed in this invention has a solid technical advantage in solving multi-omics data classification problems; (2) Excellent and reliable generalization ability: Existing baseline models all showed positive "relative training-test set distance", indicating that the model has the risk of overfitting. However, the model of this invention showed a large negative distance, that is, a unique "negative generalization distance". Ablation experiments further proved that when at least one of the feature-level and omics-level attention modules is implemented through the attention mechanism in the hierarchical architecture, the model can maintain a significant "negative generalization distance". This shows that the hierarchical attention architecture itself is the key source of the model's excellent generalization ability.

[0015] In summary, under strictly controlled experimental conditions, the data strongly demonstrate that the integration method provided by this invention not only consistently outperforms existing baseline models in classification performance, but more importantly, it fundamentally improves the model's generalization ability and robustness, providing a more reliable and powerful general tool for solving multi-omics data integration analysis of complex diseases.

[0016] It is important to note that the "multi-omics data integration and classification method based on hierarchical attention" claimed in this invention is fundamentally different in its technical means, architectural nature, and application goals from schemes that use multilayer perceptrons combined with sigmoid function activation to simulate attention, and generative models whose main goal is to solve the problem of missing data.

Claims

1. A multi-omics data integration and classification method based on hierarchical attention, applied to the field of precision medicine big data analysis, characterized in that... Includes the following steps: (1) Obtain feature tensors from multiple omics data sources; (2) Input the feature tensors of each omics data source into the corresponding feature-level attention module and generate feature embeddings and feature importance scores; (3) The features from all omics data sources are embedded into a unified omics-level attention module, and omics embeddings and omics importance scores are generated; (4) A classification prediction task is performed based on the omics embeddings, and the classification results are output for disease classification.

2. The method according to claim 1, characterized in that, The processing of the feature-level attention module includes: performing a first element-level fusion operation on the input feature tensor and the learnable feature semantic embedding; the processing of the omics-level attention module includes: performing a second element-level fusion operation on the feature embedding of each omics data source and the learnable omics semantic embedding; wherein the first element-level fusion operation and the second element-level fusion operation are independently selected from element-wise addition or element-wise multiplication.

3. The method according to claim 1, characterized in that, At least one of the feature-level attention module and the omics-level attention module is implemented through an attention mechanism.

4. The method according to claim 3, characterized in that, The attention mechanism is a multi-head self-attention mechanism.

5. The method according to claim 3, characterized in that, The feature-level attention module and the omics-level attention module further include: at least one feedforward network layer.

6. The method according to claim 3, characterized in that, The feature-level attention module and the omics-level attention module further include: at least one residual connection layer.

7. The method according to claim 3, characterized in that, The feature importance score and omics importance score are generated by normalizing the normalized result of embedding the features or omics output by the attention mechanism into the projection.

8. The method according to claim 7, characterized in that, The projection is achieved through a linear projection layer; the normalization process is achieved through a softmax function with a temperature parameter.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 8.

Images