An artificial intelligence-based syndrome marker prediction algorithm and application thereof

Abstract

The application provides an artificial intelligence-based syndrome marker prediction algorithm and applies the algorithm in Yinqiao Qingre Tablets and Jinhong Tablets. The application first constructs a low-dimensional vector embedding method based on a pre-trained model to intelligently extract disease / phenotype / syndrome and molecular features. The application constructs a single classification autoencoder neural network model for screening negative samples, and divides the data set into a pre-training data set and a fine-tuning data set. Furthermore, the extracted disease / phenotype / syndrome and molecular features are used to design a syndrome marker prediction algorithm based on a mutual attention mechanism. Finally, the method is used to analyze two embodiments, i.e., Yinqiao Qingre Tablets for treating wind-heat and dryness syndrome cold and Jinhong Tablets for treating liver-stomach disharmony syndrome chronic superficial gastritis. The experimental results show that the method is superior to the baseline method in disease / phenotype and molecular correlation prediction performance, and is superior to the baseline method and two widely used tools in syndrome and molecular correlation prediction performance. The method promotes the development of personalized treatment, and through the prediction of syndrome markers, the possible marker molecules can be predicted according to the individual disease and syndrome state, which promotes the development of traditional Chinese medicine diagnosis and treatment in a more accurate direction, and provides objective technical support for clinical practice.

Description

Technical Field

[0001] This invention relates to an artificial intelligence-based disease biomarker prediction algorithm and its application. Background Technology

[0002] With the continuous advancement of medical science, the traditional "one-size-fits-all" treatment model has gradually revealed its limitations, especially in dealing with complex diseases (such as cancer, cardiovascular disease, and metabolic diseases), often failing to achieve ideal treatment results. Precision medicine has emerged to address this need, aiming to tailor treatment plans by integrating a patient's molecular information, environmental factors, and lifestyle habits. Leveraging cutting-edge technologies such as genomics, big data analysis, and artificial intelligence, precision medicine can select the most suitable treatment plan for each patient. Its core objective is to improve treatment effectiveness and safety, while simultaneously driving the medical model towards prevention and prediction.

[0003] With the rapid development of artificial intelligence and the emergence of large language models, more and more researchers are focusing on using AI methods to advance precision medicine. The key to achieving precision medicine lies in revealing the diverse macro- and micro-states of different patients. Previous phenotypic-molecular association studies mainly targeted specific phenotypes or diseases, limiting their ability to predict molecular associations of diseases and symptoms. However, the phenotypic characteristics of different patients are often unique, and simply using a certain predefined combination of phenotypes is insufficient to fully capture their individual differences. Therefore, predicting individualized biomarkers for diseases and symptoms is of great significance.

[0004] Artificial intelligence is widely used in precision medicine, but research in the molecular prediction of diseases and syndromes is limited. Traditional Chinese medicine (TCM) syndromes (or TCM "patterns") are one of the core concepts in TCM used to describe a patient's overall health status and have significant clinical importance. They are a holistic judgment of disease or health status formed by identifying and integrating factors such as the patient's medical history, symptoms, signs, and living environment. Furthermore, TCM syndromes recorded in different databases often lack standardized names and descriptions of primary and secondary symptoms. Therefore, how to predict disease and syndrome markers based on the specific descriptions of TCM syndromes for different patients is a problem that urgently needs to be solved. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention proposes an artificial intelligence-based disease biomarker prediction algorithm for predicting disease biomarkers in different patients (see...). Figure 1 This provides valuable means for accurate diagnosis and customized treatment, and specifically applies it to the treatment of colds with wind-heat and dryness syndrome by Yinqiao Qingre tablets and the treatment of chronic superficial gastritis with liver-stomach disharmony by Jinhong tablets, respectively, to analyze the mechanism.

[0006] To achieve the above objectives, this invention provides an artificial intelligence-based disease biomarker prediction algorithm, characterized by the following steps:

[0007] Step 1: Using existing pre-trained large models, digital and vectorized representations of diseases / phenotypes, syndromes, and molecules were achieved, with molecular features including protein-protein association network information and protein sequence information.

[0008] Step 2: Fusing disease / phenotype / syndrome and molecular feature information through a mutual attention mechanism allows disease / phenotype / syndrome features and molecular features to learn the representation information of the other feature, which is helpful for subsequent association prediction.

[0009] Step 3: By constructing a single-class autoencoder with a similar structure to the neural network prediction model, an equal number of associated negative samples are calculated and filtered from the unlabeled associated samples using existing associated positive samples.

[0010] Step 4: Use the molecular association data of the disease / phenotype as the pre-training dataset for the pre-training of the neural network prediction model, and use the molecular association data of the syndrome as the fine-tuning dataset for the fine-tuning of the neural network prediction model. Divide these datasets into training, validation, and test sets, following a randomization principle.

[0011] Step 5: Construct and train a neural network prediction model. This model consists of two main modules: a mutual attention module based on a multi-head mutual attention mechanism and a prediction module constructed from multiple fully connected layers. The neural network prediction model is trained sequentially on a pre-training dataset and a fine-tuning dataset. Pre-training on disease / phenotype-molecular association data helps it learn the association between diseases / phenotypes and molecules, while fine-tuning on syndrome-molecular association data adapts it to predict syndrome-molecular associations.

[0012] Step Six: Evaluation of the Neural Network Prediction Model. Both the pre-trained and fine-tuned neural network prediction models were compared with other baseline methods. The fine-tuned neural network prediction model was also compared with the current state-of-the-art syndrome-molecular prediction method. Evaluation metrics included Precision, Recall, F1-score, Accuracy, AUC (Area Under the Curve), and AUPR (Area Under the Precision-Recall Curve).

[0013] Step 7: Using disease marker prediction algorithms combined with network target analysis, identify the key mechanisms by which Yinqiao Qingre tablets intervene in colds with wind-heat and dryness syndrome and Jinhong tablets intervene in chronic superficial gastritis with liver-stomach disharmony syndrome. Attached Figure Description

[0014] Figure 1Schematic diagram of the artificial intelligence-based prediction algorithm for disease-syndrome markers;

[0015] Figure 2 Key mechanism diagram of Yinqiao Qingre Tablets in intervening in wind-heat and dryness syndrome colds obtained by using the artificial intelligence-based prediction algorithm for disease-syndrome markers of the present invention;

[0016] Figure 3 Key mechanism diagram of Jinhong Tablets in intervening in chronic superficial gastritis with disharmony between the liver and stomach obtained by using the artificial intelligence-based prediction algorithm for disease-syndrome markers of the present invention. Detailed implementation manners

[0017] Taking the key mechanism identification of Yinqiao Qingre Tablets in intervening in wind-heat and dryness syndrome colds and Jinhong Tablets in intervening in chronic superficial gastritis with disharmony between the liver and stomach as examples, the present invention's embodiments designed and implemented an artificial intelligence-based prediction algorithm for disease-syndrome markers. First, we collected and screened data related to molecules, diseases / phenotypes, and syndromes from public databases as raw data. Protein-protein interaction network data came from the STRING database, and protein sequence information came from the UniRef50 database. Disease / phenotype text description data was sourced from the Unified Medical Language System (UMLS) database, and syndrome text description data was sourced from the TCM GPT large language model fine-tuned based on ChatGPT-4. In addition, disease / phenotype-molecule association data was collected from the DisGeNET database, and syndrome-molecule association data came from the GEO database. The implementation steps mainly included: feature extraction of diseases / phenotypes / syndromes and molecules, constructing a single-class autoencoder and predicting negative samples, dividing the training set, validation set, and test set of pre-training data and fine-tuning data, constructing a neural network prediction model and performing pre-training and fine-tuning successively, evaluating the neural network prediction model, and analyzing the mechanisms of Yinqiao Qingre Tablets in intervening in wind-heat and dryness syndrome colds and Jinhong Tablets in intervening in chronic superficial gastritis with disharmony between the liver and stomach. Specific embodiments illustrate the present invention in detail.

[0018] Embodiment:

[0019] An artificial intelligence-based prediction algorithm for disease-syndrome markers and its application according to the present invention comprises the following steps:

[0020] I. Data collection

[0021] In this embodiment, the inventors collected and screened molecular-related, disease / phenotype-related, and syndrome-related data from public databases. Protein-protein interaction networks and protein sequences were obtained from the STRING and UniRef50 databases, respectively. Disease / phenotype text descriptions were obtained from the Unified Medical Language System (UMLS) database, and syndrome text descriptions were obtained from the TCM GPT large language model fine-tuned based on ChatGPT-4. Disease / phenotype-molecular association data reported in existing literature were collected from the DisGeNET database, and syndrome-molecular association data were collected from the GEO database, involving two steps:

[0022] First, three transcriptome datasets covering 12 different disease-syndrome groups and their healthy control groups were collected from the database. Then, the R package limma was used to perform differential expression analysis on these transcriptome datasets, and the syndrome-molecule associations were classified as positive or negative samples based on whether there was significant differential expression of molecules, i.e., a p-value less than 0.05.

[0023] In summary, the inventors constructed a comprehensive dataset that integrates network and sequence information of 19,505 molecules, descriptive information of 17,082 diseases / phenotypes and 12 syndromes, as well as disease / phenotype / syndrome-molecule association data. This dataset contains 909,673 positive and negative samples of disease / phenotype-molecule associations, 99,755 positive samples of syndrome-molecule associations, and 97,811 negative samples of syndrome-molecule associations (see: https: / / github.com / WBY20 / GPS-MAP / tree / main).

[0024] II. Feature extraction of diseases / phenotypes, syndromes, and molecules

[0025] The methods used for feature extraction of diseases / phenotypes, syndromes, and molecules are all existing low-dimensional vector embedding representations. For the text description of each disease / phenotype or syndrome, we encode it into 768-dimensional features using a pre-trained language model (BioBERT-Base v1.2, https: / / github.com / dmis-lab / biobert) specifically optimized for large-scale literature in the biomedical field based on BERT. For each molecule, the model simultaneously considers the linguistic information contained in the protein sequence and the network information of protein-protein interactions across the entire genome. ProFlash (https: / / github.com / ISYSLAB-HUST / ProtFlash / ) is a pre-trained lightweight protein language model based on an attention mechanism. This model is used to obtain the sequence information features of molecules. From the perspective of network connectivity, we use the Node2Vec algorithm (https: / / github.com / aditya-grover / node2vec) to represent the feature information of molecular networks. The two features are concatenated to form the 1024-dimensional features of the molecule.

[0026] III. Constructing a single-class autoencoder and predicting negative samples

[0027] To construct the dataset, this invention builds a neural network model for filtering negative samples, referred to as a single-class autoencoder. The single-class autoencoder consists of an encoder and a decoder. The encoder architecture is the same as the Mutual-AttentionModule in the neural network prediction model constructed in step 5, and its output is CombinedFeature:

[0028] Encoder=Mutual-AttentionModule

[0029] The encoder's parameters are the same as those in the mutual attention module. The decoder consists of three fully connected layers, with batch normalization and ReLU activation functions between each pair of layers. The computational details of the decoder can be represented as follows:

[0030] Hidden1=FC1(ReLU(BatchNorm1(CombinedFeature)))

[0031] Hidden2=FC2(ReLU(BatchNorm2(Hidden1)))

[0032] Output=FC3(ReLU(BatchNorm3(Hidden2)))

[0033] Here, BatchNorm represents batch normalization, FC represents a fully connected layer, Hidden represents the output of the hidden layer, and Output represents the reconstructed molecular and disease / phenotypic features. During training, the encoder learns how to compress data into a low-dimensional latent space representation, and the decoder learns how to reconstruct the input molecular and disease / phenotypic features. After training, the autoencoder is used to calculate the loss for unlabeled samples and filter out outliers (negative samples). We use the mean squared error loss function (MSELoss) to train the model. The final autoencoder hyperparameters are: learning rate 8e-4; optimizer Adam with fixed weight decay; weight decay of 1e-7; batch size of 2048; dropout of 0.1.

[0034] IV. Divide the pre-training data and fine-tuning data into training, validation, and test sets.

[0035] Using the constructed comprehensive dataset, it was first divided into a pre-training dataset and a fine-tuning dataset. The pre-training dataset included disease / phenotype and molecular correlation data, while the fine-tuning dataset included syndrome and molecular correlation data. The pre-training dataset was divided into a training set, a validation set, and a test set in a 7:2:1 ratio. From the 12 syndromes in the fine-tuning dataset, 3 syndromes and their molecular correlation data were randomly selected as the test set, and the molecular correlation data of the remaining 9 syndromes were used as the training set.

[0036] V. Construct a neural network prediction model and perform pre-training and fine-tuning sequentially.

[0037] The neural network prediction model includes a mutual attention module and a prediction module.

[0038] 5A, Mutual Attention Module

[0039] The mutual attention module comprises a projection layer, a multi-head attention layer, and a fully connected layer. The neural network prediction model is first pre-trained using pre-training data and then fine-tuned using fine-tuning data to obtain the final neural network prediction model.

[0040] First, the neural network prediction model uses 768 (=N1)-dimensional disease / phenotype / syndrome feature vectors and 1024 (=N2)-dimensional molecular feature vectors as input, simultaneously projecting both vectors onto the same feature space. To learn the latent molecular representations related to the disease / phenotype / syndrome and the latent disease / phenotype / syndrome representations related to the molecular features, the neural network prediction model utilizes a multi-head mutual attention mechanism to fuse disease / phenotype / syndrome features and molecular features.

[0041] The steps for calculating the mutual attention module for molecular latent representations related to disease / phenotype / syndrome are as follows:

[0042] First, query (Q), key (K), and value (V) matrices are generated from molecular features and disease / phenotype / syndrome features respectively through a linear projection layer, defined as follows: and Where i is a superscript matrix representing the matrix corresponding to the i-th head in multi-head attention, disease is a subscript matrix representing the matrix corresponding to disease / phenotype / syndrome features, and molecule is a subscript matrix representing the matrix corresponding to molecular features. Then, single-head attention from disease to molecule is calculated as follows:

[0043]

[0044] Used to obtain The weights are assigned to the components. After projection, a multi-head mutual attention mechanism is used to fuse disease / phenotype / syndrome features and molecular features. Finally, the single-head attention results from disease / phenotype / syndrome to molecular features are concatenated into a single output fused feature through a fully connected layer.

[0045] MultiHead d2m (Q,K,V)=Concat(head1,…,head h W O

[0046]

[0047] Where h represents the number of heads, W O MultiHead represents a randomly initialized parameter matrix that is automatically optimized and updated during training. Concat concatenates the matrices corresponding to different heads row by row. d2m This represents a potential representation of a disease response molecule that combines mutual attention information from different representation subspaces.

[0048] By applying the mutual attention module, we obtain the molecular latent representation F of the disease / phenotype / symptom response. d2m And molecular response to disease / phenotype / symptom potential representation F m2d .

[0049] 5B. Prediction Module

[0050] The prediction module is used to predict disease / phenotype / syndrome-molecular associations. This module is a feedforward neural network consisting of three fully connected layers. Each pair of layers contains batch normalization and ReLU activation functions to output the association probability between the disease / phenotype / syndrome and the molecular marker. The calculation steps are as follows:

[0051] CombinedFeature = Concat(F d2m ,F m2d )

[0052] Hidden=FC1(ReLU(BatchNorm1(CombinedFeature)))

[0053] Output=FC2(ReLU(BatchNorm2(Hidden)))

[0054] Here, Concat means concatenating the matrix row by row, BatchNorm means batch normalization, FC means fully connected layer, Hidden means the output of the hidden layer, and Output means the final two-dimensional output of the neural network prediction model.

[0055] The neural network prediction model was first pre-trained on a constructed disease / phenotype-molecular dataset to learn about the correlations between disease / phenotype features and molecular features. Then, it was fine-tuned on a syndrome-molecular dataset after initialization using the weights of the pre-trained neural network prediction model. The goal of fine-tuning was to leverage the biomedical knowledge learned during pre-training to enhance the neural network prediction model's ability to predict molecular associations in the field of TCM syndromes. The model was trained using the binary cross-entropy loss function (BCEWithLogitsLoss). The same hyperparameters were used in both the pre-training and fine-tuning processes, and these hyperparameters were optimized to the following values: learning rate 1e-4; optimizer, using Adam with weight decay correction; weight decay 1e-5; batch size 512; dropout 0.1.

[0056] VI. Evaluation of Neural Network Prediction Models

[0057] The evaluation of the neural network prediction model includes evaluating both the pre-trained and fine-tuned models. Both models are compared to a baseline model, and the fine-tuned model is further compared to widely used syndrome molecular prediction tools in related fields. The main evaluation metrics include Precision, Recall, F1-score, Accuracy, AUC (area under the curve), and AUPR (area under the precision-recall curve).

[0058] The neural network prediction model trained on disease / phenotype-molecular association data was compared with baseline models such as random forest, multilayer perceptron, k-nearest neighbors, and decision tree. Random forest, k-nearest neighbors, and decision tree were implemented using sklearn with default parameters. The architecture of the multilayer perceptron was similar to that of the neural network prediction model of this invention, and the models were compared after undergoing the same training process.

[0059] The neural network prediction model, fine-tuned with syndrome-molecule association data, was compared not only with baseline models such as random forest and multilayer perceptron (the baseline models had different parameters added in addition to the default parameters for comparison), but also with existing syndrome-molecule association prediction tools. Specifically, considering that the transcriptome in the test set contains disease-syndrome samples, we also retrieved disease-related molecule sets corresponding to the test set from the DisGeNET database. For each test set, the union of the syndrome-related molecule sets and the disease-related molecule sets from SoFDA and SymMap was used to form the positive samples predicted by SoFDA and SymMap, respectively. The remaining molecules in the test set were used as negative samples for prediction.

[0060] VII. Application Cases

[0061] 7A. Analysis of the Mechanism of Yinqiao Qingre Tablets in Intervening in Common Cold with Wind-Heat and Dryness Syndrome

[0062] Yinqiao Qingre tablets are tablets made from extracts of nine Chinese medicinal herbs, including honeysuckle, kudzu root, and forsythia. They have the effects of relieving exterior syndromes with pungent and cool properties and clearing heat and detoxifying [1]. The predicted target spectrum of Yinqiao Qingre tablets was calculated by the drugCIPHER [2] algorithm, and the overall target of the whole formula of Yinqiao Qingre tablets was modeled and constructed based on the statistical model established in previous studies [3]. The overall target was enriched to obtain the pathways of potential effects of Yinqiao Qingre tablets. At the disease level, the text description of wind-heat and dryness syndrome cold was input into the disease marker prediction algorithm, and the molecular markers of wind-heat and dryness syndrome cold were obtained through whole genome-wide association prediction, including CD3D, LCK, PIK3CD, MAPK1, TOP2B and PIK3R3. Furthermore, the markers of wind-heat and dryness syndrome cold were enriched to obtain the relevant pathways and biological processes of the disease.

[0063] Based on the potential intervention pathways and biological processes of Yinqiao Qingre Tablets and the prediction and analysis of the syndrome and disease markers model, the mechanism of wind-heat and dryness syndrome cold was obtained. Through integrated analysis, the mechanism of action of Yinqiao Qingre Tablets in intervening in wind-heat and dryness syndrome cold was obtained. The results mainly focused on three modules: signal transduction, immune regulation, and antiviral, including the processes of T cell maturation and differentiation and immune-related signal pathways. It was found that some genes played potential regulatory roles in multiple modules including immunity. Through integrated analysis, a modular molecular mechanism network of Yinqiao Qingre Tablets intervening in wind-heat and dryness syndrome through overall targets was constructed (see Figure 2 ). Based on these key markers and pathway mechanisms, it was suggested that white blood cells (WBC) and lymphocytes (Lym) were potential clinical markers, providing molecular marker analysis, mechanism of action research, and individualized treatment strategies for the subsequent precise clinical application of Yinqiao Qingre Tablets.

[0064] Analysis of the Mechanism of Jinhong Tablets in Intervening in Chronic Superficial Gastritis with Disharmony between the Liver and Stomach

[0065] Jinhong Tablets are composed of four medicinal materials: Toosendan Fruit, Corydalis Rhizoma Yanhusuo Vinegar-processed, Aucklandiae Radix, and Illicium Verum Hook.f. Leaves. It has the effects of soothing the liver and relieving depression, regulating qi and activating blood circulation, and harmonizing the stomach and relieving pain. The predicted target spectrum of Jinhong Tablets was calculated by the drugCIPHER algorithm, and based on the statistical model established in previous studies, the overall target of the whole formula of Jinhong Tablets was modeled and constructed. Enrichment analysis was performed on the overall target to obtain the potential pathways of Jinhong Tablets. At the syndrome and disease level, the text description of chronic superficial gastritis with disharmony between the liver and stomach was input into the syndrome and disease marker prediction algorithm, and molecular markers of chronic superficial gastritis with disharmony between the liver and stomach were obtained through genome-wide association prediction, including CXCL8, GER, IL1B, POMC, and TNF, etc. Further, enrichment analysis was performed on the markers of chronic superficial gastritis with disharmony between the liver and stomach to obtain the related pathways and biological processes of this syndrome and disease.

[0066] Based on the potential intervention pathways and biological processes of Jinhong Tablets and the prediction and analysis of the syndrome and disease markers model of chronic superficial gastritis with disharmony between the liver and stomach, a multi-level biomolecular network of Jinhong Tablets intervening in chronic superficial gastritis with disharmony between the liver and stomach was integratively constructed (see Figure 3 ). The results showed that Jinhong Tablets mainly exerted the traditional effects of soothing the liver and relieving depression, regulating qi and activating blood circulation, and harmonizing the stomach and relieving pain by intervening in key modules such as immune regulation, inflammatory response, cell adhesion, and nervous system. Based on these key markers and pathway mechanisms, it was suggested that IL1B and TNF were potential clinical molecular markers, providing support for the subsequent precise clinical application and individualized treatment of Jinhong Tablets.

[0067] Advantages and Positive Effects

[0068] In the comparison with other methods, the evaluation results of the pre-trained neural network prediction model are as follows:

[0069] The area under the curve (AUC) was 0.996, the area under the precision-recall curve (AUPR) was 0.996, the precision was 0.972, the recall was 0.972, the F1 score was 0.972, and the accuracy was 0.972, all of which exceeded other baseline models (Table 1).

[0070] Furthermore, we trained and tested the performance of the fine-tuned neural network prediction model on 12 syndrome-molecular datasets. Baseline models such as Multilayer Perceptron, Random Forest, k-Nearest Neighbors, and Decision Tree were all trained on the same disease / phenotype-molecular and syndrome-molecular data. The results show that the neural network prediction model of this invention outperforms all baseline models, with AUC improvements of 28.6%–39.2% and AUPR improvements of 34.4%–43.7% (Table 2).

[0071] Finally, the fine-tuned neural network prediction model was compared with the prediction results of two widely used tools, SoFDA and SymMap, and significant performance improvements were observed, with accuracy improving by 38.8% to 39.2% and F1 score improving by 73.2% to 73.5% (Table 3).

[0072] Table 1. Performance comparison between the pre-trained neural network prediction model and the baseline model

[0073] This invention Multilayer perceptron Random Forest k-nearest neighbor Decision Tree AUC 0.9964 0.9734 0.9889 0.9778 0.9051 AUPR 0.996 0.9697 0.9833 0.9656 0.8896 Precision 0.972 0.8924 0.9541 0.9508 0.8523 Recall 0.9719 0.8743 0.9541 0.9508 0.8511 F1 score 0.9719 0.8728 0.9541 0.9508 0.851 Accuracy 0.9719 0.8743 0.9541 0.9508 0.8511

[0074] Table 2. Performance comparison between the fine-tuned neural network prediction model and the baseline model

[0075]

[0076]

[0077] Table 3. Performance comparison between the fine-tuned neural network prediction model and widely used prediction tools

[0078] This invention SoFDA SymMap Precision 0.823 0.500 0.500 Recall 0.819 0.087 0.084 F1 score 0.820 0.149 0.144 Accuracy 0.819 0.431 0.427

[0079] References:

[0080] [1] Li Shao et al., Research and development practice of traditional Chinese medicine based on network target theory and technology, China Journal of Traditional Chinese Medicine, Vol. 48, No. 22, pp. 5965-5976, 2023, doi:10.19540 / j.cnki.cjcmm.20230923.701.

[0081] [2] S. Zhao and S. Li, Network-based relating pharmacological and genomicspaces for drug target identification, PloS One, Volume 5, Issue 7, Page e11764, July 2010, doi:10.1371 / journal.pone.0011764.

[0082] [3] X. Liang, H. Li and S. Li, “A novel network pharmacology approach to analyze traditional herbal formulae: the Liu-Wei-Di-Huang pill as a case study”, Mol. Biosyst., Vol. 10, No. 5, pp. 1014-1022, May 2014, doi:10.1039 / c3mb70507b.

Claims

1. A modeling method for neural network prediction models for predicting pathological biomarkers based on artificial intelligence, characterized in that... include: A) Construct a mutual attention module, which includes a projection layer, a multi-head attention layer, and a fully connected layer. The mutual attention module includes parts for performing the following operations: A1) By using N1-dimensional disease / phenotype / syndrome feature vectors and N2-dimensional molecular feature vectors as input, the disease / phenotype / syndrome feature vectors and molecular feature vectors are simultaneously projected onto the same feature space to construct a projection layer, where the molecular features include protein-protein association network information and protein sequence information. A2) Utilizing a multi-head mutual attention mechanism to fuse disease / phenotype / syndrome features and molecular features, learn potential molecular representations related to disease / phenotype / syndrome and potential molecular representations of disease / phenotype / syndrome, including: A21) Generate query matrices from molecular features and disease / phenotype / syndrome features respectively through projection layers. Key matrix Sum matrix and superscript The first in the bullish attention The matrix corresponding to each head, subscript A matrix representing disease / phenotype / syndrome characteristics, with subscripts... The matrix representing the molecular characteristics A22) Then, calculate the single-head attention from disease / phenotype / syndrome to molecule, as follows: in, express transpose, express Dimensions It is a function used to get Weight on, Following A3), a multi-head mutual attention mechanism is used to fuse disease / phenotype / syndrome features and molecular features. This includes connecting the single-head attention results from disease / phenotype / syndrome to molecular features through a fully connected layer into a single output fused feature. in: Subscript Indicates the number of heads. Represents a parameter matrix, This indicates the splicing operation of different heads. This represents the potential representation of disease response molecules that incorporates mutual attention information from different representation subspaces. This leads to the potential molecular representation of disease / phenotype / symptom response. and molecular responses to potential manifestations of disease / phenotype / syndrome ; B) Construct a prediction module for predicting disease / phenotype / syndrome-molecular associations. This module is a feedforward neural network consisting of three fully connected layers, with batch normalization and batch normalization occurring between each pair of fully connected layers. The activation function, used to output the probability of association between disease / phenotype / syndrome and molecules, performs the following operations: in, This indicates that the matrix will be concatenated row by row. This indicates a batch normalization operation. Indicates a fully connected layer. This represents the output of the hidden layer. This represents the final two-dimensional output of the neural network prediction model.

2. The modeling method for the neural network prediction model according to claim 1, characterized in that... Further includes: C1) First, the neural network prediction model is pre-trained on the constructed disease / phenotype-molecular dataset to learn the relevant information between disease / phenotype features and molecular features. C2) Then, fine-tuning was performed on the syndrome-molecular dataset by initializing the model using the weights of the pre-trained neural network prediction model. C3) Model training is performed, including using the binary cross-entropy loss function BCEWithLogitsLoss.

3. The modeling method for the neural network prediction model according to claim 2, characterized in that: The pre-training and fine-tuning processes used the same hyperparameters, which were optimized to the following final values: learning rate 1e-4; optimizer, using Adam with weight decay correction; weight decay 1e-5; batch size 512; dropout 0.

1. N1 = 768, N2 = 1024 The goal of fine-tuning is to leverage the biomedical knowledge learned during pre-training to enhance the ability of neural network prediction models to predict molecular associations in the field of traditional Chinese medicine syndromes.

Images