A disease recognition prediction method based on social media knowledge graph embedding

Abstract

The application discloses a disease recognition prediction method based on social media knowledge graph embedding, mainly aiming at specific disease patient recognition in a social media scene. In order to optimize the representation learning process of the social media user self-report knowledge graph triple, a knowledge embedding optimization algorithm based on hyperbolic space is designed, the ComplEx model is embedded into the hyperbolic space, the representation ability of the model for complex data with hierarchical structure is enhanced, and the Riemannian gradient descent method is used for gradient updating of the Poincaré ball model, so that the embedded points are always in the effective area. Based on the embedding optimization, the social media user self-report text information and the knowledge graph information are knowledge fused by combining the channel attention mechanism and the spatial attention mechanism on the basis of the BERT+LSTM model, the spatial feature expression ability of the model is enhanced by using the external information in the knowledge graph, so that the recognition accuracy of the specific disease patient user in the social media is improved.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, specifically to a disease identification and prediction method based on social media knowledge graph embedding. Background Technology

[0002] With the widespread use of social media platforms, many patients record their symptoms on social media to seek help, providing researchers and public health departments with an effective channel for data acquisition. This social media data contains a wealth of medical knowledge, disease information, patient feedback, and related descriptions. However, because social media data is often unstructured and contains a lot of noise and uncertainty, effectively extracting valuable health information from this massive amount of data for disease identification and prediction has become a pressing challenge.

[0003] In recent years, knowledge graphs, as a graphical method for representing complex information, have been widely applied in intelligent systems across various fields. Knowledge graphs represent things and their relationships through the relationships between nodes and edges. In the medical field, knowledge graphs can help integrate and link information related to diseases, symptoms, and treatment plans. Knowledge graph embedding technology can transform entities and relationships in the graph into dense representations in a vector space, enabling efficient comparison and computation of different types of information within that vector space. Therefore, by embedding relevant information from social media and combining it with deep learning techniques, good performance can be achieved in disease identification and prediction tasks.

[0004] Currently, while some research has embedded knowledge graphs into disease prediction models, these methods typically rely on knowledge graphs from traditional medicine or other structured clinical data. When facing infectious diseases such as SARS, Ebola, and COVID-19, these methods are often limited by the sensitivity of patient data and the difficulty in timely access and sharing. Furthermore, although existing knowledge graph embedding models can learn knowledge representations of entities and relationships within knowledge graphs, their representation capabilities are weak when dealing with complex, hierarchical data. The highly personalized and social characteristics of social media data necessitate more flexible processing methods for disease prediction models. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention proposes a disease identification and prediction method based on social media knowledge graph embedding to solve the problems that existing disease identification models are unable to effectively identify patients with specific diseases in social media and that traditional knowledge graph embedding methods are inadequate in processing hierarchical data.

[0006] The technical solution adopted by this invention to solve its technical problem is:

[0007] A disease identification and prediction method based on social media knowledge graph embedding includes the following steps:

[0008] Step 1: Obtain self-reported data from social media users and construct a social media disease identification dataset. The social media disease identification dataset includes basic information of social media users and their self-reported disease posts. The social media disease identification dataset is divided into a training set, a test set, and a validation set.

[0009] Step 2: Construct a social media user self-report knowledge graph. Extract relevant information such as user age, disease, symptoms, time, and medication from the social media user self-reports as entities. The corresponding relations are constructed by discretizing each entity. The social media user self-report knowledge graph will include a set of triples {(h1,r1,t1),(h2,r2,t2),...}. Here, h represents the head entity, r represents the relation, and t represents the tail entity.

[0010] Step 3: Construct the knowledge graph embedding optimization model KG2ER. The knowledge graph embedding optimization model extends the ComplEx model embedding into hyperbolic space, performs representation learning on triples in the social media user self-report knowledge graph, and uses the Riemannian gradient descent method to optimize the Poincaré sphere model to ensure that the embedding point is always in the effective region, thereby realizing the embedding of the knowledge graph.

[0011] Step 4: Construct a disease identification and prediction model KGBertCS based on social media knowledge graph embedding. The disease identification and prediction model based on social media knowledge graph embedding includes: a social media user self-report text encoding module, a knowledge embedding module, a span representation module, a joint attention knowledge fusion module, and a identification and prediction module. The disease identification and prediction model based on social media knowledge graph embedding is trained using the data in the training set to obtain a trained disease identification and prediction model based on social media knowledge graph embedding.

[0012] Step 5: Use the disease identification and prediction model based on social media knowledge graph embedding trained in Step 4 to identify diseases and evaluate the prediction indicators.

[0013] Furthermore, the process of step 3 is as follows:

[0014] Step 3.1: Extend the ComplEx model embedding to hyperbolic space, i.e., hyperbolic space embedding, and obtain the score of triple (h,r,t) in hyperbolic space by defining a scoring function.

[0015] Furthermore, the process of step 3.1 is as follows:

[0016] The Poincaré sphere model is used to represent entities and relations in hyperbolic space. The embedding vectors h, r, and t of entities and relations are all located in the unit sphere of hyperbolic space. Within the hyperbolic space, a new scoring function is defined to replace the complex inner product-based scoring method in the original ComplEx model. The score of the triple (h, r, t) is calculated using the hyperbolic cosine distance to measure the triple's fit in the embedding space. The specific formula is as follows: in, -cosh represents the inner product of embedding vectors h, r, t in hyperbolic space. -1 () represents the inverse hyperbolic cosine function. In the Poincaré sphere model, the inner product in hyperbolic space is defined as: <h,r·t> This represents the similarity between the head entity h and relation r and the tail entity t. Based on the scoring function described above, the logistic sigmoid function is used to calculate the probability that each triplet is true. Specifically, for each triplet (h, r, t), its probability of being true is... For each positive-negative sample pair, the cross-entropy loss function of the Poincaré sphere model is calculated to quantify the difference between the model's prediction and the actual label: Among them, y i For the labels of positive samples, y i =1 indicates a positive sample, y i =0 indicates a negative sample.

[0017] Step 3.2: Optimize using Riemannian gradient descent to ensure the updated embedding remains in hyperbolic space. In each optimization step, first multiply the Euclidean gradient by the inverse of the Poincaré metric tensor to obtain the Riemannian gradient of the loss function with respect to the embedding: Where g(θ) -1 The inverse of the Poincaré metric tensor. This is the Euclidean gradient. For the D-dimensional Poincaré sphere model, its metric tensor... Where ∥θ∥ is the norm of the embedding vector θ in Euclidean space. It is a d×d identity matrix. Then, it is updated along the gradient direction, and the gradient update embedding vector is calculated: An exponential mapping from the tangent space to the manifold, where η is the learning rate.

[0018] Furthermore, the process of step 4 is as follows:

[0019] Step 4.1: The social media user self-report text encoding module described in Step 4. First, the social media user self-report is serialized into a text sequence, resulting in a text sequence X = [x1, x2, ..., x...]. n The pre-trained model BERT-Base-Chinese is used as the text encoder to encode the text sequence X: Obtain the text embedding vector

[0020] Step 4.2: The knowledge embedding module described in Step 4. Using the knowledge graph embedding model described in Step 3, each entity e in the triple (h,r,t) of the social media user self-reported knowledge graph is processed. i and relation r j Encoding yields the embedding vector θ = [e h ,r j ,e t A linear transformation is used to convert the knowledge graph embedding vector θ into a vector similar to the text embedding vector H. x Same dimension: θ' = W1θ + b1, where W1 is a weight matrix of a linear transformation and b1 is a bias vector.

[0021] Step 4.3: The span representation module described in Step 4. The text embedding vector H obtained in Steps 4.1 and 4.2 is used... x The knowledge graph embedding vector θ′ is input into the LSTM model. The LSTM model dynamically adjusts the representation of each word based on contextual information from the user's self-reported text and dynamically captures the temporal features of relationships between entities in the graph, thus understanding the semantics and structure of the text and the graph. Specifically, the process includes calculating the unit state at each time step through the LSTM model's input gate, forget gate, and output gate. and hidden state h t =o t ⊙tanh(c t ), where i t f t o t These represent the input gate, forget gate, and output gate, respectively. This indicates the candidate state of the current cell.

[0022] Step 4.4: The joint attention knowledge fusion module described in Step 4. This module utilizes channel attention and spatial attention mechanisms for knowledge fusion. The input feature is the hidden layer h of the text embedding vector obtained in Step 4.3. text The hidden layer h of the knowledge graph embedding vector kg , B is the batch size, C1 is the number of channels for text embedding, C2 is the number of channels for knowledge graph embedding, and H and W are the height and width of the feature map, respectively.

[0023] The channel attention mechanism uses two convolutional layers and a batch normalization layer to generate channel attention weights. First, the number of channels of the input feature is reduced through a convolutional operation: h′=Conv1(h), resulting in vector h′. text and h′ kg The number of channels is reduced from C1 to C′1 and from C2 to C′2, respectively. Then, the ReLU activation function is used to introduce nonlinearity and adjust it: h″ = ReLU(h′), resulting in the vector h″. text and h″ kg The activated feature map is then passed through a convolutional layer to restore the original number of channels, h. final =Conv2(h″) to obtain vectors respectively and The average activation value for each channel is calculated using global average pooling, and then the attention weight for each channel is obtained using the sigmoid activation function: α. c =σ(GAP(h) final )),in, It is the Sigmoid activation function. This is performed using global average pooling. The resulting channel attention weights are then... and Multiply by the corresponding elements of the original feature maps of the text and knowledge graph: h att1 =α c ·h, to obtain the weighted channel attention feature map and

[0024] The spatial attention mechanism further processes the feature map after applying channel attention, generating attention weights to emphasize or suppress spatial locations, thereby enhancing the model's spatial feature representation capability. First, the input feature map... and Perform global max pooling and global average pooling along the channel dimension to obtain two H×W×1 feature maps, respectively. and The obtained feature maps are concatenated by channels to obtain a feature map of size H×W×2. A 7×7 convolution operation is then performed on the concatenated result to obtain a feature map of size H×W×1. Finally, the spatial attention weights α are obtained by applying the Sigmoid activation function. s =σ(f 7×7 [h catt——ax h catt_max The spatial attention weight matrix is ​​then multiplied by the original feature map h.satt =α s ·h catt This will yield the weighted spatial attention feature map. and Finally, additive fusion is used to fuse the user self-reported text feature map and the knowledge graph feature map processed by the channel-space attention mechanism:

[0025] Step 4.3: The recognition and prediction module described in Step 4. It integrates the features h after channel-spatial attention knowledge fusion. fused The input feature vector is fed into the classifier. The classifier consists of a fully connected layer that maps the input feature vector to a single numerical value, i.e., performs a linear transformation: z = W²·h fused +b2, where W2 is the weight matrix and b2 is the bias term. Since this invention addresses a binary classification task, specifically predicting whether a user has a particular disease, the sigmoid activation function is used to predict the probability that the user has the disease. If P(y|z) >= 0.5, it means the model predicts the user has the disease; if P(y|z) < 0.5, it means the model predicts the user does not have the disease.

[0026] The beneficial effects of adopting the above technical solution are as follows:

[0027] This invention first proposes a knowledge embedding optimization algorithm based on hyperbolic space, extending complex embeddings in ComplEx to hyperbolic space to enhance the model's ability to represent complex data with hierarchical structures. Riemannian gradient descent is used for optimization to ensure that embedding points are always within the effective region, thereby achieving embedding into social media knowledge graphs. Based on the BERT+LSTM model, channel attention and spatial attention mechanisms are combined to fuse social media user self-reported text with the knowledge graph. Information from the knowledge graph enhances the model's spatial feature representation capabilities, enabling the identification and prediction of users with specific diseases on social media. Attached Figure Description

[0028] Figure 1 This is a flowchart of the present invention;

[0029] Figure 2 This is a network architecture diagram of the present invention. Detailed Implementation

[0030] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes, but the embodiments of the present invention are not limited thereto.

[0031] Example 1:

[0032] Combined with appendix Figure 1 As shown, a disease identification and prediction method based on social media knowledge graph embedding includes:

[0033] Step 1: Obtain self-reported data from social media users and construct a social media disease identification dataset. The social media disease identification dataset includes basic information of social media users and their self-reported disease posts. The social media disease identification dataset is divided into a training set and a test set.

[0034] Step 2: Construct a social media user self-report knowledge graph. Extract relevant information such as user age, disease, symptoms, time, and medication from the social media user self-reports as entities. The corresponding relations are constructed by discretizing each entity. The social media user self-report knowledge graph will include a set of triples {(h1, r1, t1), (h2, r2, t2), ...}. Here, h represents the head entity, r represents the relation, and t represents the tail entity.

[0035] Step 3: Construct the knowledge graph embedding optimization model KG2ER. The knowledge graph embedding optimization model extends the ComplEx model embedding into hyperbolic space, performs representation learning on triples in the social media user self-report knowledge graph, and uses the Riemannian gradient descent method to optimize the Poincaré sphere model to ensure that the embedding point is always in the effective region, thereby realizing the embedding of the knowledge graph.

[0036] Step 4: Construct a disease identification and prediction model KGBertCS based on social media knowledge graph embedding. The disease identification and prediction model based on social media knowledge graph embedding includes: a social media user self-report text encoding module, a knowledge embedding module, a span representation module, a joint attention knowledge fusion module, and a identification and prediction module. The disease identification and prediction model based on social media knowledge graph embedding is trained using the data in the training set to obtain a trained disease identification and prediction model based on social media knowledge graph embedding.

[0037] Step 5: Use the disease identification and prediction model based on social media knowledge graph embedding trained in Step 4 to identify diseases and evaluate the prediction indicators.

[0038] Example 2:

[0039] Based on Example 1, combined with Appendix Figure 1 As shown, step 3 specifically includes:

[0040] Step 3.1: Embed the ComplEx model into the hyperbolic space, i.e., hyperbolic space embedding, and obtain the score of the triple (h, r, t) in the hyperbolic space by defining a scoring function. The Poincaré ball model is used to represent entities and relationships in the hyperbolic space. The embedding vectors h, r, t of entities and relationships are all located within the unit ball of the hyperbolic space inside, and the is that the norm of each embedding vector is less than 1. Among them, represents the d-dimensional Euclidean space, that is, the set of all real vectors with dimension d; ||x|| is the norm of the vector x, that is, the length or magnitude of the vector.

[0041] In the hyperbolic space, redefine the scoring function to replace the scoring method based on complex inner product in the original ComplEx model. Use the hyperbolic cosine distance to calculate the score of the triple (h, r, t) and measure the matching degree of the triple in the embedding space. The specific formula is: Among them, represents the inner product of the embedding vectors h, r, t in the hyperbolic space, and -cosh -1 () is the inverse hyperbolic cosine function. In the Poincaré ball model, the inner product in the hyperbolic space is defined as: <h, r·t> represents the similarity between the head entity h, the relationship r and the tail entity t. Since the hyperbolic space has geometric properties different from the Euclidean space, additional normalization factors (1 - ||h|| 2 ) and (1 - ||r·t|| 2 ) are used to ensure the consistency and stability of the score calculation. According to the above scoring function, use the logistic sigmoid function to calculate the probability that each triple is true. Specifically, for each triple (h, r, t), the probability that it is true For each positive and negative sample pair, calculate the cross-entropy loss function of the Poincaré ball model to quantify the difference between the prediction of the model and the actual label: Among them, yi is the label of the positive sample, yi = 1 means it is a positive sample, and yi = 0 means it is a negative sample.

[0042] Step 3.2: Since in the hyperbolic space, the standard Euclidean optimization method is not applicable, the Riemannian gradient descent method is used for optimization to ensure that the updated embedding is still within the hyperbolic space. In each optimization step, first multiply the Euclidean gradient by the inverse of the Poincaré metric tensor to obtain the Riemannian gradient of the loss function with respect to the embedding: where g(θ) -1 is the inverse of the Poincaré metric tensor, This is the Euclidean gradient. For the D-dimensional Poincaré sphere model, its metric tensor... Where ||θ|| is the norm of the embedding vector θ in Euclidean space. It is a d×d identity matrix. Then, it is updated along the gradient direction, and the gradient update embedding vector is calculated: An exponential mapping from the tangent space to the manifold, where η is the learning rate.

[0043] Example 3:

[0044] Based on Example 2, combined with Appendix Figure 1 As shown, step 4 specifically includes:

[0045] Step 4.1: The social media user self-report text encoding module described in Step 4. First, the social media user self-report is represented as a text serialization, resulting in a text sequence X = [x1, x2, ..., x...]. n The pre-trained model BERT-Base-Chinese is used as the text encoder to encode the text sequence X: Obtain the text embedding vector

[0046] Step 4.2: The knowledge embedding module described in Step 4. Using the knowledge graph embedding model described in Step 3, each entity e in the triple (h, r, t) of the social media user self-reported knowledge graph is processed. i and relation r j Encoding yields the embedding vector θ = [e h r j e t The knowledge graph embedding vector θ is transformed using a linear transformation into a vector similar to the text embedding vector H. x Same dimension: θ′=W1θ+b1, where W1 is a weight matrix of a linear transformation and b1 is a bias vector.

[0047] Step 4.3: The span representation module described in Step 4. The text embedding vector H obtained in Steps 4.1 and 4.2 is used... x The knowledge graph embedding vector θ′ is input into the LSTM model. The LSTM model dynamically adjusts the representation of each word based on contextual information from the user's self-reported text and dynamically captures the temporal features of relationships between entities in the graph, thus understanding the semantics and structure of the text and the graph. Specifically, the process includes calculating the unit state at each time step through the LSTM model's input gate, forget gate, and output gate. and hidden state h t=o t ⊙tanh(c t ), where i t f t o t These represent the input gate, forget gate, and output gate, respectively. This indicates the candidate state of the current cell.

[0048] Step 4.4: The joint attention knowledge fusion module described in Step 4. This module utilizes channel attention and spatial attention mechanisms for knowledge fusion. The input feature is the hidden layer h of the text embedding vector obtained in Step 4.3. text The hidden layer h of the knowledge graph embedding vector kg , B is the batch size, C1 is the number of channels for text embedding, C2 is the number of channels for knowledge graph embedding, and H and W are the height and width of the feature map, respectively.

[0049] The channel attention mechanism uses two convolutional layers and a batch normalization layer to generate channel attention weights. First, the number of channels of the input feature is reduced through a convolutional operation: h′=Conv1(h), resulting in vector h′. text and h′ kg The number of channels is reduced from C1 to C′1 and from C2 to C′2, respectively. Then, the ReLU activation function is used to introduce nonlinearity and adjust it: h″ = ReLU(h′), resulting in the vector h″. text and h″ kg The activated feature map is then passed through a convolutional layer to restore the original number of channels, h. final =Conv2(h″) to obtain vectors respectively and The average activation value for each channel is calculated using global average pooling, and then the attention weight for each channel is obtained using the sigmoid activation function: α. c =σ(GAP(h) final )),in, It is the Sigmoid activation function. This is performed using global average pooling. The resulting channel attention weights are then... and Multiply by the corresponding elements of the original feature maps of the text and knowledge graph: h att1 =α c ·h, to obtain the weighted channel attention feature map and

[0050] The spatial attention mechanism further processes the feature map after applying channel attention, generating attention weights to emphasize or suppress spatial locations, thereby enhancing the model's spatial feature representation capability. First, the input feature map... and Perform global max pooling and global average pooling along the channel dimension to obtain two H×W×1 feature maps, respectively. and The obtained feature maps are concatenated by channels to obtain a feature map of size H×W×2. A 7×7 convolution operation is then performed on the concatenated result to obtain a feature map of size H×W×1. Finally, the spatial attention weights α are obtained by applying the Sigmoid activation function. s =σ(f 7×7 [h catt_max h catt_max The spatial attention weight matrix is ​​then multiplied by the original feature map h. satt =α s ·h catt This will yield the weighted spatial attention feature map. and Finally, additive fusion is used to fuse the user self-reported text feature map and the knowledge graph feature map processed by the channel-space attention mechanism:

[0051] Step 4.3: The recognition and prediction module described in Step 4. It integrates the features h after channel-spatial attention knowledge fusion. fused The input feature vector is fed into the classifier. The classifier consists of a fully connected layer that maps the input feature vector to a single numerical value, i.e., performs a linear transformation: z = W²·h fused +b2, where W2 is the weight matrix and b2 is the bias term. Since this invention addresses a binary classification task, specifically predicting whether a user has a particular disease, the sigmoid activation function is used to predict the probability that the user has the disease. If P(y|z) >= 0.5, it means the model predicts the user has the disease; if P(y|z) < 0.5, it means the model predicts the user does not have the disease. During training, the classifier uses a binary cross-entropy loss function to evaluate the difference between the model output and the actual label. The loss function is Loss = -[ylog(p) + (1-y)log(1-p)], where y is the true label. If the user has the disease, then y = 1; if the user does not have the disease, then y = 0.

[0052] Example 4:

[0053] Based on Example 3, to verify the effectiveness of the present invention, further explanation is provided below using experimental data:

[0054] This invention treats COVID-19 sequelae as a specific disease for experimental purposes. Using previous research findings and a joint extraction model, a knowledge graph embedding experiment was conducted on the WLE dataset of triples extracted from self-reported text by Weibo users regarding COVID-19 sequelae. This dataset contains 27,096 triples, including four entities: symptoms, disease, medication, and time; and five relationships: accompaniment, symptom stage, illness stage, medication period, and treatment. The WLE dataset was constructed through data cleaning and manual annotation of the self-reported text from Weibo users regarding COVID-19 sequelae. This WLE dataset, containing 260 million Chinese text entries, was then used to train and test a disease identification and prediction model, combining the triple vector data obtained from knowledge graph embedding.

[0055] The dataset size used in the experiment is shown in Table 1.

[0056] Table 1 Dataset Size

[0057] Number of training set items Number of test set items Number of validation sets WLE 21660 2926 2510 WLP 20800 2600 2600

[0058] For the knowledge graph embedding experiments, minimum loss (Minloss), mean rank (MR), and Hits@10 were used as evaluation metrics. Minloss represents the minimum loss value obtained by the model during training; MR represents the average rank of correct triples among all candidate triples; and Hits@10 represents the proportion of triples correctly predicted by the model that rank in the top 10. The calculation formulas are as follows:

[0059]

[0060] in This is the loss function calculated in each epoch, where θ is the model parameters; N is the total number of triples, and Rank is... i Let i be the rank of the i-th triplet among all candidate triplets; It is the indicator function (1 if the condition is true, 0 otherwise).

[0061] The experiment aimed at identifying and predicting the sequelae of COVID-19 used precision, recall, and F1 score as evaluation metrics. The calculation formulas are as follows:

[0062]

[0063] Where TP represents the number of actual and predicted positive examples; FP represents the number of predicted positive examples but actual negative examples; and FN represents the number of predicted negative examples but actual positive examples.

[0064] The proposed hyperbolic space-based knowledge graph embedding optimization algorithm was compared with four other methods on the WLE dataset. The experimental results are shown in Table 2. The results demonstrate that the proposed KG2E algorithm significantly outperforms traditional models in the knowledge graph embedding task of self-reported COVID-19 sequelae users. Its lower minimum loss, best average rank, and high HITS@10 index of 84.15% indicate that KG2E possesses stronger expressive and predictive capabilities when processing knowledge graph data with hierarchical structures and complex relationships.

[0065] Table 2 Comparative Experiments on Knowledge Graph Embedding

[0066] Model Minloss MR HITS@10 TransA 0.4234 78.64 0.5536 TransD 0.4102 70.85 0.6486 TransE 0.3850 66.23 0.7224 TransH 0.3645 66.01 0.7356 KG2ER 0.3260 54.68 0.8415

[0067] To demonstrate the performance of the proposed disease identification and prediction model KGBertCS based on social media knowledge graph embedding, comparative experiments were conducted on the WLP dataset with other baseline models. The experimental results are shown in Table 3. The results show that KGBertCS significantly outperforms traditional BERT models (such as BERT and BERT LSTM) and other variant models (such as BERT DPCNN and BERT HAN), indicating that such models have limitations in feature extraction from colloquial social media text. While convolutional neural network variants (such as BERT CNN and BERT DPCNN) improve performance to some extent, they still cannot match the overall performance of KGBertCS. The attention mechanism variant BERT ATT performs well, but still lags behind KGBertCS. This is mainly because the present invention incorporates semantic embedding of knowledge graphs, providing more external knowledge about the sequelae of COVID-19 and its related symptoms. This allows the model to better understand and process complex information related to user self-reports in social media. By introducing knowledge graphs, the model can not only mine deep-level entity information but also transform unstructured data in text into structured knowledge, thereby improving prediction accuracy and recall.

[0068] Table 3 Comparative Experiment on Identification and Prediction of Post-COVID-19 Sequelae

[0069] Model Precision Recall F1 Bert 84.56 87.02 85.77 BertCNN 88.25 90.12 89.18 BertCNNPlus 88.68 90.56 89.61 BertDPCNN 89.63 91.68 90.64 BertHAN 90.23 93.05 91.62 BertLSTM 87.35 89.02 88.18 BertRCNN 88.80 90.36 89.57 BertATT 90.89 91.26 91.07 KGBertCS 94.85 96.23 95.54

[0070] The embodiments of the present invention have been described in detail above. These embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Those skilled in the art can design many other modifications and implementations based on the principles of the present invention, and these modifications and implementations will fall within the scope and spirit of the principles disclosed in this application.

Claims

1. A disease identification and prediction method based on social media knowledge graph embedding, characterized in that, Includes the following steps: Includes the following steps: Step 1: Obtain self-reported data from social media users and construct a social media disease identification dataset. The social media disease identification dataset includes basic information of social media users and their self-reported disease posts. The social media disease identification dataset is divided into a training set, a test set, and a validation set. Step 2: Construct a knowledge graph of social media user self-reports, extract user-related information from the social media user self-reports as entities, and the corresponding relationships are formed by the discretization of each entity; Step 3: Construct a knowledge graph embedding optimization model. The knowledge graph embedding optimization model extends the ComplEx model embedding into hyperbolic space, performs representation learning on triples in social media user self-report knowledge graphs, and uses the Riemannian gradient descent method to optimize the Poincaré sphere model to ensure that the embedding point is always in the effective region, thereby realizing the embedding of the knowledge graph. Step 4: Construct a disease identification and prediction model based on social media knowledge graph embedding. The disease identification and prediction model based on social media knowledge graph embedding includes: a social media user self-report text encoding module, a knowledge embedding module, a span representation module, a joint attention knowledge fusion module, and an identification and prediction module. The disease identification and prediction model based on social media knowledge graph embedding is trained using the data in the training set to obtain a trained disease identification and prediction model based on social media knowledge graph embedding. Step 5: Use the disease identification and prediction model based on social media knowledge graph embedding trained in Step 4 to identify diseases and evaluate the prediction indicators; The social media user self-report text encoding module in step 4 is specifically implemented as follows: First, the self-reports of social media users are represented as text sequences, resulting in a text sequence X = [x1, x2, ..., x...]. n The pre-trained model BERT-Base-Chinese is used as the text encoder to encode the text sequence X: Obtain the text embedding vector The span representation module in step 4 is specifically implemented as follows: embed the text vector H x and knowledge graph embedding vector θ ' The data are input into the LSTM model, and the specific process includes: calculating the cell state at each time step through the input gate, forget gate, and output gate of the LSTM model. and hidden state h t =o t ⊙tanh(c t ), where i t f t o t These represent the input gate, forget gate, and output gate, respectively. This indicates the candidate state of the current cell.

2. The disease identification and prediction method based on social media knowledge graph embedding according to claim 1, characterized in that, Step 3 specifically includes: Step 3.1: Extend the ComplEx model embedding to hyperbolic space, and obtain the score of the triple (h,r,t) in hyperbolic space by defining a scoring function; use the Poincaré sphere model to represent entities and relations in hyperbolic space; the embedding vectors h,r,t of entities and relations are all located in the unit sphere of hyperbolic space. Within the hyperbolic space, a new scoring function is defined to replace the complex inner product-based scoring method in the original ComplEx model; the score of the triple (h,r,t) is calculated using hyperbolic cosine distance to measure the matching degree of the triple in the embedding space, with the specific formula as follows: in, -cosh represents the inner product of embedding vectors h, r, t in hyperbolic space. -1 () represents the inverse hyperbolic cosine function; in the Poincaré sphere model, the inner product in hyperbolic space is defined as: <h,r·t> Let represent the similarity between the head entity h and relation r and the tail entity t. Based on the scoring function described above, the logistic sigmoid function is used to calculate the probability that each triplet is true. Specifically, for each triplet (h, r, t), its probability of being true is... For each positive-negative sample pair, the cross-entropy loss function of the Poincaré sphere model is calculated to quantify the difference between the model's prediction and the actual label: Among them, y i For the labels of positive samples, y i =1 indicates a positive sample, y i =0 indicates a negative sample; Step 3.2: Optimize using Riemannian gradient descent to ensure the updated embedding remains in hyperbolic space. In each optimization step, first multiply the Euclidean gradient with the inverse of the Poincaré metric tensor to obtain the Riemannian gradient of the loss function with respect to the embedding: Where g(θ) -1 The inverse of the Poincaré metric tensor. The Euclidean gradient; for the D-dimensional Poincaré sphere model, its metric tensor Where ||θ|| is the norm of the embedding vector θ in Euclidean space. It is a d×d identity matrix; then it is updated along the gradient direction, and the gradient update embedding vector is calculated: An exponential mapping from the tangent space to the manifold, where η is the learning rate.

3. The disease identification and prediction method based on social media knowledge graph embedding according to claim 1, characterized in that, The knowledge embedding module in step 4 is specifically implemented as follows: Using the knowledge graph embedding model described in step 3, each entity e in the triple (h,r,t) of the social media user self-reported knowledge graph is analyzed. i and relation r j Encoding yields the embedding vector θ = [e h ,r j ,e t A linear transformation is used to convert the knowledge graph embedding vector θ into a vector similar to the text embedding vector H. x Same dimension: θ' = W1θ + b1, where W1 is a weight matrix of a linear transformation and b1 is a bias vector.

4. The disease identification and prediction method based on social media knowledge graph embedding according to claim 1, characterized in that, The joint attention knowledge fusion module in step 4 is specifically implemented as follows: Knowledge fusion is achieved using channel attention and spatial attention mechanisms, with the hidden layer h containing text embedding vectors as input features. text Hidden layers of knowledge graph embedding vectors B is the batch size, C1 is the number of channels for text embedding, C2 is the number of channels for knowledge graph embedding, and H and W are the height and width of the feature map, respectively. The channel attention mechanism uses two convolutional layers and a batch normalization layer to generate channel attention weights. First, the number of channels of the input feature is reduced through a convolutional operation: h' = Conv1(h), resulting in vectors h'. text and h' kg The number of channels is reduced from C1 to C'1 and C2 to C'2, respectively; then, the ReLU activation function is used to introduce nonlinearity and adjust it: h” = ReLU(h'), resulting in vector h”. text and h” lg The activated feature map is then passed through a convolutional layer to restore the original number of channels, h. final =Conv2(h”), respectively, to obtain vectors and The average activation value for each channel is calculated using global average pooling, and then the attention weight for each channel is obtained using the sigmoid activation function: α. c =σ(GAP(h) final )),in, It is the Sigmoid activation function. For global average pooling; the resulting channel attention weights and Multiply by the corresponding elements of the original feature maps of the text and knowledge graph: h att1 =α c ·h, to obtain the weighted channel attention feature map and The spatial attention mechanism further processes the feature map after applying channel attention, generating attention weights to emphasize or suppress spatial locations, thereby enhancing the model's spatial feature representation capability. First, the input feature map... and Perform global max pooling and global average pooling along the channel dimension to obtain two H×W×1 feature maps, respectively. and The obtained feature maps are concatenated by channels to obtain a feature map of size H×W×2. A 7×7 convolution operation is then performed on the concatenated result to obtain a feature map of size H×W×1. Finally, the spatial attention weights α are obtained by applying the Sigmoid activation function. s =σ(f 7×7 [h catt_max h catt_max The spatial attention weight matrix is ​​then multiplied by the original feature map h. satt =α s ·h catt This will yield the weighted spatial attention feature map. and Finally, additive fusion is used to fuse the user self-reported text feature map and the knowledge graph feature map processed by the channel-spatial attention mechanism:

5. The disease identification and prediction method based on social media knowledge graph embedding according to claim 1, characterized in that, The identification and prediction module in step 4 is specifically implemented as follows: The feature h after channel-space attention knowledge fusion fused The input feature vector is fed into a classifier; the classifier consists of a fully connected layer that maps the input feature vector to a single numerical value, i.e., performs a linear transformation: z = W²·h fused +b2, where W2 is the weight matrix and b2 is the bias term; for binary classification tasks, the sigmoid activation function is used for prediction.

Images