Drug-drug interaction prediction method based on multi-modal semantic alignment

Abstract

The embodiment of the invention discloses a drug-drug interaction prediction method based on multi-modal semantic alignment, and relates to the technical field of biomedical prediction. The drug-drug interaction prediction method based on multi-modal semantic alignment aims at solving the technical problem that the DDI classification effect of a model is poor. Comprising the following steps: acquiring a text feature set, a molecular structure feature set and a subgraph feature set of a SemEval-2013Task 9 data set; performing feature fusion on the sub-graph feature set and the molecular structure feature set through a sub-graph-molecule cross attention module, and then inputting the feature fused sub-graph feature set and the molecular structure feature set together with the text feature set into a text guide fusion module to obtain a text fusion feature set; and splicing the text feature set, the molecular structure feature set and the sub-graph feature set, performing residual connection processing and splicing processing on the text feature set, the molecular structure feature set and the sub-graph feature set, and performing layer normalization processing to obtain target multi-modal fusion data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of biomedical prediction, and in particular to a drug-drug interaction prediction method based on multi-modal semantic alignment. BACKGROUND

[0002] In the field of biomedical text mining, accurately identifying drug-drug interactions (DDI) is of great significance to improving drug safety and supporting clinical decision-making. However, existing methods have the problem of insufficient semantic alignment when fusing molecular structure, biomedical entity knowledge graph, and biomedical entity text semantics. Biomedical entities include information such as diseases, pathways, and target points. The main problems are as follows: first, it is difficult to fully mine the deep mechanism of drug molecular structure affecting efficacy through biomedical entities; second, the modeling ability of long-distance semantic dependency in text is limited, resulting in inaccurate identification of complex DDI relationships. These problems limit the accuracy and depth of DDI extraction, affecting the effectiveness of drug safety and clinical decision support. SUMMARY

[0003] The main purpose of the present application is to provide a drug-drug interaction prediction method based on multi-modal semantic alignment, aiming to solve the technical problem of poor effect of model DDI classification.

[0004] To achieve the above purpose, the technical solutions adopted by the embodiments of the present application are as follows:

[0005] In a first aspect, the embodiments of the present application provide a drug-drug interaction prediction method based on multi-modal semantic alignment, comprising the following steps:

[0006] Obtain a text feature set, a molecular structure feature set, and a subgraph feature set of the SemEval-2013 Task 9 dataset; perform feature fusion on the subgraph feature set and the molecular structure feature set through a subgraph-molecule cross-attention module, and input the text feature set, the subgraph feature set, and the molecular structure feature set into a text-guided fusion module to obtain a text fusion feature set;

[0007] After performing splicing processing on the text feature set, the molecular structure feature set, and the subgraph feature set, performing residual connection processing and splicing processing on the text fusion feature set, and performing layer normalization processing, a target multi-modal fusion data is obtained.

[0008] As some optional embodiments of the present application, the text feature set is a plurality of text features of words extracted based on a text feature extractor;

[0009] The text features satisfy the following relationship:

[0010] V = [v1, v2,..., vn] = IntensityMatrix 0 (E N ] = IntensityMatrix 0 (E token +E pos )

[0011] wherein IntensityMatrix is a diagonal matrix, diagonal elements are intensity i , v i is the adaptive position interaction vector of the i-th word, E token is the word embedding matrix, and E pos is the position embedding matrix.

[0012] As some optional embodiments of the present application, the adaptive position interaction vector v i of the i-th word satisfies the following relationship:

[0013]

[0014] wherein intensity i is an attention weight, is a word embedding, is a position embedding.

[0015] As some optional embodiments of the present application, the attention weight intensity i satisfies the following relationship:

[0016] intensity i = Uniform (0.9-t·dis, 1.1+t·dis)

[0017] wherein dis represents the ratio of the total length of the entity to the number of tokens N of the sentence, reflecting the spatial proportion of the entity in the sentence, and the parameter t is a strength adjustment coefficient to control the fluctuation range of the strength value.

[0018] As some optional embodiments of the present application, the molecular structure feature set is obtained based on the following steps:

[0019] a plurality of molecular fingerprint features are input into the PS-AEGNN as initial vectors; the molecular fingerprint features include atom types, bond types and functional group identifiers;

[0020] linear transformation and activation processing are performed on all node vectors of the first layer, and attention weight values thereof and all bonds are obtained;

[0021] based on the attention weight values, neighbor value vectors are weighted and summed to obtain updated nodes combined with residual connections;

[0022] all atomic vectors are aggregated to obtain a molecular structure feature set.

[0023] As some optional embodiments of the present application, the linear transformation and activation processing of all node vectors of the lth layer satisfy the following relationship:

[0024]

[0025] In the formula, is the intermediate vector of the ith atom after linear transformation and activation processing of the lth layer; is the vector representation of the ith atom in the lth layer; i is the ith; is the linear transformation weight matrix of the lth layer; ReLU is the activation function.

[0026] As some optional embodiments of the present application, the attention weight value satisfies the following relationship:

[0027]

[0028] In the formula, α ij is the attention weight value; d k is the dimension of the key vector; are vectors obtained by linear transformation from the query and key value pairs respectively; S i is a small part selected from all possible keys according to the sparse query selection algorithm; exp is the exponential function; is the score of the real degree of each key in the screened key set S i , and the score is exponentially operated, and finally the exponential results are added up; j and i are the atoms (key objects) associated by attention and the atom (query subject) initiating attention respectively; d k k in d is the symbolic identifier (scaling parameter) of the key vector dimension.

[0029] As some optional embodiments of the present application, the combined residual connection update node satisfies the following relationship:

[0030]

[0031] In the formula, is the combined residual connection update node; α ij is the attention weight value; is the feature vector of the jth neighbor node in the lth layer; is the updated feature of node i (the ith atom) in the l+1th layer; is the intermediate vector of the ith atom after linear transformation and activation processing of the lth layer; i is the ith; j is the jth neighbor node representing the ith atom.

[0032] As some optional embodiments of the present application, the set of molecular structure features satisfies the following relationship:

[0033]

[0034] wherein h mol is a set of molecular structure features, which is obtained after processing by the PS-AEGNN and can represent the entire molecular structure features of the drug; f() is a non-linear transformation function representing an aggregation of all atomic vectors; M is the number of atoms, L is the total number of network layers, W output and b output are the linear transformation matrix and bias of the output layer.

[0035] As some optional embodiments of the present application, the target multi-modal fusion data satisfies the following relationship:

[0036]

[0037] Finally, the training process is stabilized by layer normalization (Layer Normalization), and the final fusion features are output:

[0038]

[0039] wherein f residual is a residual fusion feature, which is derived from the middle layer fusion feature f1 and the original splicing feature f0;

[0040] :,:T fused .shape[-1] is a slicing operation on a tensor, and the first ":" means selecting all rows; :T fused .shape[-1] means selecting the first T fused .shape[-1] columns from the column dimension, the purpose being to match the dimensions of f0 and f1 so as to perform subsequent residual connection calculation; T fused is a feature tensor generated in the middle layer fusion process; shape[-1] means taking the size of the last dimension in the shape of the tensor;

[0041] is a matrix to which the relevant vector or tensor belongs in the real number field R, and its shape is B x d t ;

[0042] h final is a subgraph-molecule fusion feature; LayerNorm(f residual ) is a layer normalization operation, which normalizes the residual fusion feature f residual .

[0043] Compared with the prior art, the present application proposes a multi-modal framework named MultiMod-DDI for drug-drug interaction (DDI) extraction. The framework significantly improves the DDI classification performance by integrating three data modalities of text, molecular structure and knowledge graph, and combining a series of innovative methods. Specifically, in the text modality processing, the BioBERT model is used to encode the drug-related text, and an adaptive position interaction vector is innovatively introduced to accurately capture the long-distance dependency between drug entities and semantic units. In the aspect of molecular structure feature extraction, a molecular graph neural network (PS-AEGNN) integrating ProbSparse self-attention mechanism (sparse self-attention mechanism) is constructed, which can effectively handle the long-distance dependency problem in complex molecular structure and accurately capture key chemical features. In the utilization of knowledge graph modality, a knowledge graph is constructed based on authoritative databases, and the SubAGCN method is used to filter the subgraph information related to specific drug pairs, providing additional biomedical background support for DDI prediction. In addition, to realize the effective fusion of multi-modal data, the framework designs a multi-stage adaptive fusion module, which includes three key steps of bottom feature splicing, middle cross-modal attention fusion, and high-level residual fusion and normalization, which can ensure the collaborative action of different modal data in a unified semantic space. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 The workflow of the MultiMod-DDI framework involved in the embodiments of the present application;

[0045] Figure 2 The visual representation of the adaptive interaction vector involved in the embodiments of the present application; the horizontal axis represents each token in the sentence, and the vertical axis represents the different weights assigned to them according to the interaction attention vector;

[0046] Figure 3 The molecular graph neural network (PS-AEGNN) integrating sparse self-attention mechanism involved in the embodiments of the present application. DETAILED DESCRIPTION

[0047] It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0048] As mentioned above, in the field of biomedical text mining, early methods mainly rely on rule-based models and statistical learning algorithms. Among them, classic models such as Support Vector Machine (SVM) and Conditional Random Fields (CRF) are difficult to adapt to the complexity of biomedical text due to their reliance on manually designed features. Although Bhasuran and other scholars have improved the accuracy of gene-disease association extraction by integrating multi-source features and ensemble learning strategies, such models are highly sensitive to the distribution of training data, and their generalization ability is significantly limited when dealing with new disease-gene associations.

[0049] With the rise of deep learning technology, biomedical text mining has ushered in a technological revolution. Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN) support relationship classification tasks with local feature extraction capabilities, while the Transformer architecture achieves global semantic association modeling through self-attention mechanisms, effectively breaking through the technical bottlenecks of traditional methods. Christopoulou and others proposed an integrated framework that combines Bidirectional Long Short-Term Memory Networks (BiLSTM), Walk-based models, and Transformers, achieving end-to-end extraction of Adverse Drug Events (ADEs) and drug relationships in the n2c2 shared task, fully demonstrating the potential of deep learning in complex clinical text processing. However, there is still room for improvement in the aspect of complex semantic analysis.

[0050] In recent years, pre-trained language models such as BERT have been widely used in Drug-Drug Interaction (DDI) extraction tasks and have made breakthrough progress. Asada et al. first introduced drug molecular structure information into the BERT framework, encoding molecular features through a graph convolutional network, demonstrating that multi-modal fusion can significantly improve DDI recognition accuracy. Duan et al. proposed the IMSE model, which innovatively designed interactive attention vectors and entity attention vectors to handle overlapping relationships, and combined molecular structure features to strengthen the importance of functional group representation, further optimizing DDI extraction performance. Shi et al. introduced the SubAGCN module to extract k-hop subgraphs of drug pairs from knowledge graphs, and cooperated with text features generated by PubMedBERT to model more biomedical entity information, deepening the understanding of drug interaction mechanisms.

[0051] Despite the significant progress made in research, existing technologies still have two major limitations:

[0052] Firstly, the long-distance dependence modeling capability is insufficient. At the molecular structure level, traditional graph neural networks (GNN) adopt local feature capture and neighbor node uniform aggregation strategy, which makes it difficult to model the long-distance interactions (such as hydrogen bond, hydrophobic interaction) between remote atoms or functional groups in complex molecules, and the modeling capability decreases exponentially with the increase of distance, which makes it difficult to analyze the biological significance. At the text semantic level, the DDI related key information may be distributed in different positions of the text, and the semantic association often spans a long text distance. The traditional model based on local context is difficult to effectively capture such long-range semantic association.

[0053] Secondly, the cross-modal semantic association mining is insufficient. The existing methods have semantic alignment obstacles when fusing molecular structure, knowledge graph and text data, and fail to fully mine the internal association between "molecular structure-biomedical entity-text semantic". This semantic fragmentation leads to the neglect of the deep association between biomedical entities, drug molecular structures and text descriptions, which limits the model to learn the DDI mechanism from multiple dimensions, and finally affects the classification performance.

[0054] Based on this, the present application proposes a MultiMod-DDI framework composed of a text feature extractor, a molecular structure feature extractor, a subgraph feature extractor, and a multi-stage adaptive fusion module. The working process diagram is shown as Figure 1 APIV is an adaptive position interaction vector generation mechanism. That is, the following steps are included:

[0055] 1) Data input and single-modal feature extraction

[0056] Text modality: input the text content containing drug-drug interaction related information, use

[0057] BioBERT model to encode it to get the text feature vector, and then further process it through average (A) operation to generate text features.

[0058] Molecular structure modality: for the input drug molecular structure (presented in the form of drug pair, including specific molecular structure diagram), first perform molecular graph structure representation, then use the molecular graph neural network (PS-AEGNN) with fusion ProbSparse self-attention mechanism (sparse self-attention mechanism) to process and extract molecular structure features to form molecular features.

[0059] Knowledge graph modality: build a knowledge graph based on authoritative database, use SubAGCN method to process the knowledge graph, filter out the subgraph information related to the specific drug pair, and get the subgraph features after encoding by the encoder.

[0060] 2) Multi-modal feature interaction and fusion

[0061] Bottom-layer feature splicing: The text features, molecular features, and subgraph features are spliced to prepare for subsequent fusion.

[0062] Middle-layer cross-modal attention fusion: The middle layer of the multi-stage adaptive fusion module is entered, and the text features are used as a guide to realize the interaction of different modal features through a cross-modal attention mechanism. First, the subgraph-molecule cross-modal attention operation is performed, the molecule features are used as queries (Q), the subgraph features are used as keys (K) and values (V), and after being projected to the same dimension through a linear layer, the attention weights are calculated through a scaled dot-product and normalized, and then the value vector is weighted and summed to obtain the subgraph-molecule fusion features, realizing the semantic interaction of the molecule structure and biomedical knowledge. Subsequently, the subgraph-molecule fusion features are projected to the text feature dimension through a text-guided gating mechanism, and a gating vector is generated by combining the text features, dynamically adjusting the fusion ratio of the text features and the subgraph-molecule fusion features, and generating middle-layer fusion features to lay a semantic-consistent feature foundation for subsequent high-layer fusion.

[0063] High-layer residual fusion and normalization: The high layer of the multi-stage adaptive fusion module is entered. To avoid gradient disappearance and preserve the details of the bottom-layer features, a residual path is designed to superimpose and integrate the middle-layer fusion features and the original spliced features. First, the original spliced features are extracted through a tensor slicing operation to match the dimension of the middle-layer fusion features, and then the two are added to obtain residual fusion features. Subsequently, the residual fusion features are subjected to layer normalization to stabilize the training process by standardizing the feature distribution, and finally, the multi-modal fusion comprehensive features containing text semantic global guidance, molecular structure local characteristics, and knowledge subgraph association information are output.

[0064] 3) Final prediction: After the multi-modal fused features are processed by a multi-layer perceptron, the text features, molecular features, and subgraph features are spliced (C) and input into the relevant classification prediction module, and finally the drug-drug interaction (DDI) prediction is realized, the DDI classification result is output, and the entire process from multi-modal input to DDI prediction is completed. Through the above process, the MultiMod-DDI framework integrates text, molecular structure, and knowledge graph data modalities, and uses the multi-stage adaptive fusion module to make different modal data work together in a unified semantic space to realize DDI extraction.

[0065] It should be noted that the data used by the MultiMod-DDI framework is mainly from

[0066] SemEval-2013 Task 9 dataset. This dataset plays an important role in biomedical literature mining, with the core goal of advancing drug-drug interaction (DDI) extraction technology. It integrates literature resources from two authoritative public databases, DrugBank and MedLine, with DrugBank providing 730 professional articles and MedLine contributing 175 relevant articles. These articles cover a wealth of DDI information, and all annotation work is completed by domain experts, ensuring the accuracy and reliability of the data. The construction of this dataset aims to provide researchers with a standardized testing platform to promote the continuous innovation of DDI extraction algorithms. After oversampling and undersampling processing, the DDI 2013 dataset still has a long-tail distribution, and in this dataset, the proportion of negative samples is extremely high, and the proportion of positive samples is relatively small. Therefore, the application adopts the method of combining undersampling and oversampling to optimize the training set. Compared with the original data, undersampling can effectively filter out a large number of duplicate negative samples, while oversampling can significantly increase the number of sparse samples. At the same time, in order to avoid the impact of sample imbalance on model performance, a multi-focus loss function is introduced.

[0067] In specific embodiments, the method for predicting drug-drug interactions based on multi-modal semantic alignment described in the application comprises the following steps:

[0068] Step S10: First, obtain the text information in the SemEval-2013 Task 9 dataset and extract the corresponding text feature set; second, obtain the SMILES sequence of the drug from the DrugBank database and convert it into a graph representation of the molecular structure using the RDKit toolkit, further extract the graph structure and subgraph feature set. In addition, a drug pair knowledge subgraph is constructed, which is derived from a large drug knowledge graph integrating multiple authoritative public biomedical databases, including DrugBank, TWOSIDES, OFFSIDES, SIDER, SMPDB (Small Molecule Pathway Database), DrugCentral, Entrez Gene, and CTD (Comparative Toxicogenomics Database);

[0069] Step S20: After feature fusion of the subgraph feature set and the molecular structure feature set through the subgraph-molecule cross-attention module, input the text feature set together into the text-guided fusion module to obtain the text fusion feature set.

[0070] Step S30: After concatenation processing of the text feature set, the molecular structure feature set and the subgraph feature set, residual connection processing and concatenation processing of the text fusion feature set are performed, and layer normalization processing is performed to obtain the target multi-modal fusion data.

[0071] To solve the long-distance dependency problem in DDI text description, the application designs an adaptive position interaction vector generation mechanism with dynamic intensity adjustment function based on the text context vector and position vector output by BioBERT. This mechanism can effectively strengthen the modeling ability of long-distance dependency by quantifying the position relationship and semantic association strength between words and drug entities.

[0072] The specific implementation process is as follows:

[0073] First, the sentence is converted into Token sequence by using the word segmenter, and the start and end indexes of the drug entities DRUG1 and DRUG2 are located. Considering that the entity is the core carrier of the interaction relationship, in order to highlight the semantic focus, higher weight fluctuation range is given to the words within the scope of the drug entity. For such words, the calculation formula of the weight value is:

[0074] intensity i =Uniform(0.9-t·dis,1.1+t·dis)

[0075] Where dis represents the ratio of the total length of the entity to the Token number N of the sentence, reflecting the spatial proportion of the entity in the sentence, and the parameter t is the intensity adjustment coefficient, controlling the fluctuation range of the intensity value.

[0076] Then, the dynamic intensity and position interaction are fused to take intensity i as the attention weight, and the word embedding and the position embedding are fused (the output of this step is shown in Figure 2 ), to generate the adaptive position interaction vector of the ith word:

[0077]

[0078] Where intensity i is the attention weight, is the word embedding, is the position embedding.

[0079] Further, the v i of all N words in the sentence is collected to get:

[0080] V=[v1,v2,...,v N ]=IntensityMatrix⊙(E token +E pos )

[0081] wherein IntensityMatrix is a diagonal matrix, diagonal elements of which are intensity i , v i is the adaptive position interaction vector of the i-th word, E token is the word embedding matrix, E pos is the position embedding matrix.

[0082] As shown in Figure 2 , through this multi-level text feature representation method, the pre-training semantic understanding ability of BioBERT for biomedical terms is retained, and the modeling accuracy of long-distance dependency between entities is enhanced with the help of position interaction vectors, thereby providing more rich text feature inputs for subsequent multi-modal fusion. Among them Figure 2 , the English sentence in the figure contains a DDI relationship, which is an example of the original data sent into the model; the column chart shown is a visualization of the adaptive position interaction vector; the horizontal axis represents each word in the sentence, and the vertical axis represents different weights assigned to them according to the interaction attention vector.

[0083] In another embodiment of the present application, the molecular structure feature set is obtained based on the following steps:

[0084] A plurality of molecular fingerprint features are input into the PS-AEGNN as initial vectors; the molecular fingerprint features include atom types, bond types, and functional group identifiers;

[0085] Linear transformation and activation processing are performed on all node vectors of the l-th layer, and attention weight values of all bonds are obtained;

[0086] Based on the attention weight values, neighbor value vectors are weighted and summed to obtain a combined residual connection updated node;

[0087] All atomic vectors are aggregated to obtain a molecular structure feature set.

[0088] It should be noted that the "combined residual connection updated node" is not a full operation, but a key step in the node vector updating phase of the PS-AEGNN (molecular graph neural network integrating ProbSparse self-attention mechanism), and only occurs in the "neighbor information aggregation and node updating" link of the molecular structure feature extraction. In the subsequent process of "aggregating all atomic vectors to obtain a drug molecule vector", the core role of the residual connection is to ensure the effectiveness of the node vector and provide high-quality input for aggregation.

[0089] The atomic vector refers to a vector that is an initial input of molecular fingerprint features, and is a continuous and refined feature after linear transformation, activation processing, attention weighted summation, and residual connection updating in the PS-AEGNN layer.

[0090] In this step, the ability to model long-range dependencies between remote atoms or functional groups decreases exponentially with distance, making it difficult to interpret its biological significance. To solve this problem, the ProbSparse self-attention mechanism is introduced, and PS-AEGNN is proposed (as shown in Figure Three ). This model combines local chemical structure and global semantics of the molecular graph through the "sparse query-key neighbor" interaction mechanism, which can accurately capture features such as drug functional group activity and metabolic pathways. Although it increases the number of model parameters and computational steps, it effectively improves the ability to model long-range dependencies, laying the foundation for cross-modal alignment of molecular structure and text semantics, and achieving accurate mining of key information of complex molecular structures.

[0091] As shown in Figure 3 , the specific implementation process is as follows:

[0092] First, feature extraction is performed to generate molecular fingerprint features containing atom types, bond types, and functional group identifiers. Each atom is represented by a fingerprint vector, which is input as an initial vector to the

[0093] PS-AEGNN.

[0094] In the update function, the ProbSparse self-attention mechanism is introduced to select the most important neighbor information. Linear transformation and activation operations are performed on all node vectors of the l-th layer:

[0095]

[0096] In the formula, is the intermediate vector obtained after linear transformation and activation of the i-th atom at the l-th layer; is the vector representation of the i-th atom at the l-th layer; i is the i-th; is the linear transformation weight matrix of the l-th layer; ReLU is the activation function.

[0097] For each query vector, calculate its divergence from the uniform distribution, and select the u key queries with the largest divergence to calculate their attention weights with all keys:

[0098]

[0099] Where, α ij is the attention weight value; d k is the dimension of the key vector; are vectors obtained by linear transformation from the query and key value pairs, respectively; S i is a small part selected from all possible keys according to the sparse query selection algorithm; exp is the exponential function; This refers to the selected set of key keys S. i For each key in the array, first calculate its relation to... The reality score is calculated, then an exponential operation is performed on that score, and finally these exponential results are summed; j and i refer to the atoms (key objects) associated with attention and the atoms (query subjects) that initiated the attention, respectively. d k In this context, k refers to the symbol for the dimension of the key vector (scaling parameter).

[0100] The attention weights are used to perform a weighted summation of the neighbor value vectors, and the node representation is updated using residual connections:

[0101]

[0102] in, This refers to updating nodes by combining residual connections; α ij This refers to the attention weight value; among which, This refers to updating nodes by combining residual connections; α ij This refers to the attention weight value; It refers to the feature vector of the j-th neighbor node in the l-th layer; This refers to the updated features of node i (the i-th atom) at layer l+1; It refers to the intermediate vector obtained after the i-th atom undergoes linear transformation and activation processing in the l-th layer; i refers to the i-th atom; j refers to the j-th neighbor node representing the i-th atom.

[0103] Ultimately, the molecular structure feature set is generated by aggregating all atom vectors:

[0104]

[0105] Among them, h mol This refers to the molecular structure feature set, which is a vector representing the structural features of the entire drug molecule obtained after processing by PS-AEGNN; f() represents a nonlinear transformation function used to aggregate all atom vectors; M is the number of atoms, L is the total number of network layers, and W... output and b output The linear transformation matrix and bias of the output layer.

[0106] By using summation aggregation and nonlinear transformation, the molecular graph is transformed into a fixed-dimensional semantic vector representation.

[0107] To solve the problem that the model is difficult to realize semantic alignment when fusing molecular structure, knowledge graph and DDI text, and cannot fully mine the internal relationship between "molecular structure-biomedical entity-text semantic", the application introduces a multi-stage adaptive fusion module. The module is responsible for integrating text features, molecular structure features, and biomedical entity subgraph features, etc. multi-modal features to realize accurate classification of DDI. Through hierarchical interaction and dynamic weight adjustment, the module can solve the differences between different modal data in semantic and feature space, and construct a "structure-semantic-knowledge" complementary joint representation.

[0108] The specific implementation process is as follows:

[0109] First, the text features extracted by BioBERT, the molecular features generated by PS-AEGNN, and the subgraph features extracted by SubAGCN are spliced to obtain the preliminary fusion features f0, and the bottom layer splicing is completed:

[0110]

[0111] In the formula, f0 refers to the original spliced features; h mol refers to the molecular structure features, which are vectors representing the entire drug molecular structure features obtained after processing by PS-AEGNN; refers to the knowledge subgraph features; h text refers to the text features.

[0112] To solve the limitations of bottom layer splicing, a cross-modal attention mechanism is introduced. With text features as the guide, the weights of molecular graph and knowledge subgraph features are dynamically calculated to realize middle layer cross-modal attention fusion. The specific operation is as follows:

[0113] 1) Subgraph-molecule cross-modal attention:

[0114] The molecular graph features are taken as the query Q, and the subgraph features are taken as the key K and the value V. With the help of attention weight, the key subgraph information is dynamically focused. The specific steps are as follows:

[0115] First, in the feature projection part, the molecular graph features and the subgraph features are projected to dimension d through linear layer respectively:

[0116]

[0117] Where B refers to the batch size, and Linear refers to the linear transformation from dimension a to b.

[0118] Then the attention score is calculated by scaling dot product and normalized:

[0119]

[0120] wherein a denotes the attention weight vector, which is obtained by Softmax normalization of the calculation results of the molecular feature (query Q) and the subgraph feature (key K). In the subgraph-molecule cross-modal attention mechanism, it is used to measure the importance of each part of the subgraph feature to the molecular feature, so as to dynamically focus on the key subgraph information; wherein K T T in the formula (1) denotes the transpose of the subgraph feature vector K; d k denotes the dimension of the key vector, and the normalization factor ensures the stability of the gradient.

[0121] Then the value vector V is weighted aggregated to obtain the fusion feature, realizing the interaction between the molecular structure and the biomedical knowledge:

[0122] h fused = attnV∈R B×d

[0123] wherein "attn" represents the attention mechanism, indicating that the calculation process is to weight sum the value vector V based on the attention weight a, so as to obtain the feature representation of the fusion of the molecular structure and the biomedical knowledge. In actual calculation, a and V are multiplied by corresponding elements and then accumulated, so as to highlight the important subgraph information and realize the interactive fusion of the molecular structure and the biomedical knowledge.

[0124] 2) Text-guided gating mechanism:

[0125] The text-guided gating fusion module aims to realize the guided fusion of the text semantics to the graph modal feature, and adjusts the fusion ratio of the text feature and the graph feature through the dynamic gating mechanism. The specific steps are as follows:

[0126] First, the subgraph-molecule fusion feature is projected to the text feature dimension:

[0127]

[0128] Subsequently, the gating signal generator (2-layer ReLU activation fully connected layer based on the text feature) processes the spliced text feature and the projected subgraph-molecule feature to generate a gating vector with a dimension of [0, 1], and finally the text feature and the graph feature are dynamically weighted fused according to the gating value:

[0129]

[0130] Finally, the original spliced feature and the middle layer fusion feature are superimposed, and layer normalization is performed, realizing high layer residual fusion and normalization to obtain the final multi-modal fusion data:

[0131]

[0132] wherein: fresidual is referred to as the residual fusion feature, which is generated by the middle layer fusion feature f1 and the original stitching feature f0;

[0133] :,:T fused .shape[-1] is referred to as a slicing operation on a tensor, and the first ":" represents selecting all rows; :T fused .shape[-1] represents selecting the first T columns from the column dimension; fused .shape[-1] columns, the purpose of which is to match the dimensions of f0 and f1 so as to perform subsequent residual connection calculations; fused is referred to as a feature tensor generated in the middle layer fusion process; shape[-1] is referred to as representing the size of the last dimension in the shape of the tensor;

[0134] is referred to as a matrix to which the relevant vector or tensor belongs in the real number field R, and the shape is Bxd t ;

[0135] h final is referred to as a subgraph-molecule fusion feature; LayerNorm(f residual ) is referred to as a layer normalization operation, which normalizes the residual fusion feature f residual .

[0136] As can be seen, the MultiMod-DDI framework proposed in the present application, by integrating multi-modal data such as text semantics, molecular structure features and external biomedical knowledge graphs, and using an adaptive fusion mechanism, not only significantly improves the accuracy and interpretability of drug-drug interaction (DDI) extraction, but also lays a solid foundation for future multi-modal data integration. The framework exhibits many significant advantages in terms of technical effects, as follows:

[0137] First, in terms of accuracy improvement, the experimental results of the MultiMod-DDI model on the SemEval-2013 Task 9 DDI Extraction dataset show that the macro F1 value and the micro F1 value reach 85.57% and 85.20% respectively, which is significantly better than the existing state-of-the-art model. This achievement is mainly due to the in-depth mining of the internal relationship between "molecular structure-biomedical entity-text semantics". By constructing a "molecular structure-pathway-text" triple evidence chain, the framework can comprehensively understand the interaction between drugs from multiple dimensions, thereby providing more accurate prediction results. For example, for some complex DDI cases, it is difficult to accurately determine them by relying on a single information source, but

[0138] MultiMod-DDI can capture deeper relationship clues through cross-modal information fusion, greatly improving classification accuracy. In addition, the PS-AEGNN introduced in this application, with the help of the "sparse query-key neighbor" interaction mechanism, fuses local chemical structure and global semantics of the molecule graph, accurately captures features such as drug functional group activity and metabolic pathways, effectively solves the problem that the long-distance dependency modeling capability of traditional GNNs decreases exponentially with distance when dealing with complex molecular structures, and significantly improves the modeling accuracy of molecular structure features. At the same time, the adaptive position interaction vector generation mechanism strengthens the modeling capability of long-distance dependency relationships between drug entities in the text by dynamically adjusting the position weight, further improving the efficiency of capturing complex semantic relationships.

[0139] Secondly, in terms of explanatory enhancement, the MultiMod-DDI framework integrates the chemical properties of drugs (such as molecular structure), biological associations (such as pathway nodes), and text descriptions (such as interaction sentences) through the inclusion of biomedical entities such as diseases and pathways, breaking through the limitations of single-modal analysis in traditional methods. The molecular structure features extracted by PS-AEGNN reveal the chemical nature of drugs, and the adaptive position interaction vector generation mechanism enhances the long-distance dependency modeling of text semantics. Together with the biomedical entity subgraph features, they form a multi-level evidence chain, which not only helps to reveal the deep mechanisms behind drug interactions, but also provides more persuasive support for clinical decision-making.

[0140] Furthermore, to address the problem of data imbalance, MultiMod-DDI designs a multi-focus loss function to optimize the training process, ensuring that different types of DDI samples are given appropriate attention. In addition, by adjusting the sample distribution in the training set through undersampling and oversampling strategies, the model's ability to identify rare types is further enhanced. These improvements effectively alleviate the model bias caused by class imbalance, making MultiMod-DDI more robust and adaptable when faced with complex and variable data environments in the real world.

[0141] Finally, the MultiMod-DDI framework provides the possibility for integrating more modal data in the future. As new data types and technological means continue to emerge in the biomedical field, how to effectively integrate this information becomes an important issue. This application builds a flexible and extensible multi-modal fusion model, demonstrating strong capabilities in handling heterogeneous data. The successful application of PS-AEGNN and the adaptive position interaction vector generation mechanism proves the effectiveness and flexibility of the framework in integrating different modal data, laying a solid foundation for further integrating more modal data such as protein structure and gene expression in the future. This not only promotes a comprehensive understanding of drug interaction mechanisms, but also opens up new paths for the development of more intelligent and personalized medical auxiliary systems.

[0142] In summary, the MultiMod-DDI framework brings revolutionary changes to drug safety evaluation and clinical decision support with its outstanding performance in accuracy, interpretability, robustness, and scalability, marking the beginning of a new era for drug-drug interaction extraction technology.

[0143] The above description is merely preferred embodiments of the application, and not intended to limit the application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application shall be included in the protection scope of the application.

Claims

1. A drug-drug interaction prediction method based on multi-modal semantic alignment, characterized in that, The method comprises the following steps: obtaining a text feature set, a molecular structure feature set and a subgraph feature set of a SemEval-2013 Task 9 data set; performing feature fusion on the subgraph feature set and the molecular structure feature set through a subgraph-molecule cross attention module, and inputting the text feature set, the subgraph feature set and the molecular structure feature set into a text-guided fusion module to obtain a text fusion feature set; performing splicing processing on the text feature set, the molecular structure feature set and the subgraph feature set, performing residual connection processing and splicing processing on the text fusion feature set, and performing layer normalization processing to obtain target multi-modal fusion data. 2.The method of claim 1, wherein, The text feature set is a plurality of text features of words obtained based on a text feature extractor; The text feature satisfies the following relationship: V = [v1, v2,..., v N ] = IntensityMatrix © (E token + E pos ) wherein IntensityMatrix is a diagonal matrix with diagonal elements of intensity i i is the adaptive position interaction vector for the ith word, E token is the word embedding matrix, and Epos is the position embedding matrix.​ 3.The method of claim 2, wherein, the adaptive positional interaction vector v of the ith word i satisfies the following relation: where intensity i is an attention weight, is a word embedding, is a position embedding. 4.The method of claim 3, wherein, The attention weight intensity i satisfies the following relation: intensity i = Uniform(0.9 - t · dis, 1.1 + t · dis) wherein dis represents the ratio of the total length of the entity to the number N of tokens in the sentence, reflects the spatial proportion of the entity in the sentence, and uses the parameter t as the intensity adjustment coefficient to control the fluctuation range of the intensity value. 5.The method of claim 1, wherein, The molecular structure feature set is obtained based on the following steps: A feature extraction method is used to generate molecular fingerprint features containing atom types, bond types and functional group identifiers. First, we use a fingerprint vector to represent each atom and input it as an initial vector into PS-AEGNN. Linear transformation and activation processing are performed on all node vectors of the lth layer, and attention weight values of all bonds are obtained. Based on the attention weight values, the neighbor value vectors are weighted and summed to obtain a combined residual connection update node. All atom vectors are aggregated to obtain a molecular structure feature set. 6.The method of claim 5, wherein, The linear transformation and activation processing of all node vectors of the lth layer satisfy the following relationship: In the formula, is referred to as the intermediate vector obtained after the linear transformation and the activation processing of the i-th atom in the l-th layer; is referred to as the vector representation of the i-th atom in the l-th layer; i is referred to as the i-th; is referred to as the linear transformation weight matrix of the l-th layer; and ReLU is an activation function. 7.The method of claim 5, wherein, The attention weight values satisfy the following relationship: where α ij is the attention weight value; d k is the dimension of the key vector; are vectors obtained from the query and key-value pair through linear transformation, respectively; S i is a small part of all possible keys selected from the sparse query selection algorithm; exp is the exponential function; means that for each key in the selected key set S i , first calculate its degree of reality score with , then do the exponential operation on the score, and finally add up the exponential results; j and i respectively refer to the atom to which attention is associated and the atom that initiates attention; k in d k refers to the symbolic identifier of the key vector dimension. 8.The method of claim 5, wherein, The combined residual connection update node satisfies the following relationship: wherein, denotes the combined residual connection update node; a ij denotes the attention weight value; denotes the feature vector of the jth neighbor node at the lth layer; denotes the updated feature of the node i (the ith atom) at the (l+1)th layer; denotes the intermediate vector of the ith atom after linear transformation and activation processing at the lth layer; i denotes the ith; j denotes the jth neighbor node of the ith atom. 9.The method of claim 5, wherein, The molecular structure feature set satisfies the following relationship: wherein h mol denotes the molecular structure feature set, which is obtained after processing by the PS-AEGNN and can represent the entire molecular structure feature of the drug; f() denotes a nonlinear transformation function for aggregating all atomic vectors; M is the number of atoms, L is the total number of network layers, W output and b output are the linear transformation matrix and bias of the output layer. 10.The method of claim 1, wherein, To avoid gradient vanishing and preserve low-level details, a residual path is designed to superimpose the original spliced features and the middle layer fusion features: Finally, layer normalization (Layer Normalization) is used to stabilize the training process, and the final fusion features are output: wherein: f residual denotes the residual fusion feature, from the mid-level fusion feature f1 and the original stitching feature f0; :,:T fused .shape[-1] is used to slice the tensor, the first : represents selecting all rows; :T fused .shape[-1] represents selecting the first :T fused .shape[-1] columns, the purpose is to match the dimensions of f0 and f1 so that the subsequent residual connection calculation can be performed;T fused is the feature tensor generated in the middle layer fusion process; shape[-1] means taking the size of the last dimension in the tensor shape; denotes the matrix of shape B x d to which the related vector or tensor belongs in the real field R t ; h final refers to sub-graph-molecule fusion feature; LayerNorm(f residual ) refers to a layer normalization operation, which normalizes the residual fusion feature f residual .

Images