Medical knowledge explicit attribution system based on medical literature information

Abstract

The invention discloses a medical knowledge explicit attribution system based on medical literature information, belongs to the field of natural language processing in artificial intelligence in the field of computers, and relates to the medical knowledge explicit attribution system based on the medical literature information. The objective of the invention is to solve the problem of lack of reliability verification of medical knowledge in practical application caused by information uncertainty in medical attribution and diagnosis due to diversity and complexity of existing medical information and knowledge. The system comprises a query text module, a query vector module, a document vector module, a semantic similarity module, a document collection screening module, a basic principle acquisition module, a to-be-verified medical knowledge judgment module and a result output module. The basic principle acquisition module is used for acquiring a basic principle; the to-be-verified medical knowledge judgment module is used for judging whether to-be-verified medical knowledge is true or false; and the result output module is used for outputting a result of the to-be-verified medical knowledge judgment module and a result of the basic principle acquisition module.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing in artificial intelligence in the computer field, and relates to a medical knowledge explicit attribution system based on medical literature information. Background Art

[0002] In modern medical research and clinical diagnosis, artificial intelligence, especially large-scale pre-trained models based on deep learning, have gradually become an important tool for decision-making assistance (Touvron et al. Llama: Open and efficient foundation language models [J]. arXiv: 2302.13971, 2023.). These models have shown excellent results in various tasks (Wang et al. Knowledge-tuning Large Language Models with Structured Medical Knowledge Bases for Reliable Response Generation in Chinese [J]. arXiv: 2309.04175, 2023.), but their decision-making process is often opaque. When dealing with complex medical problems, the model must have sufficient credibility to obtain the approval of medical experts, and the existing "black box" characteristics limit the widespread application of these models in the medical field.

[0003] Although current large-scale pre-trained models perform well, their decisions are usually based on learning complex patterns of input data. Studies have shown that these models often rely on surface features (Geirhos et al. Shortcut learning in deep neural networks [J]. Nature Machine Intelligence, 2020, 2 (11): 665-673.), which means that their decisions may not be entirely based on medically valid logic. In the medical field, the judgment of any model needs to be trusted and verified to ensure that its decisions are consistent with medical knowledge and experience. Therefore, it is crucial to improve the interpretability and transparency of the model, especially to clarify the logical basis behind its decision-making process.

[0004] Existing explanation techniques can be mainly divided into pipeline-based explanations and parallel explanation frameworks. In the pipeline-based framework, the more common approach is "post-hoc explanation", that is, after the model makes a decision, humans or other algorithms analyze the reasons for its decision. Commonly used methods include attention mechanisms (Tenney I. BERT rediscovers the classical NLP pipeline [J]. arXiv: 1905.05950, 2019.), LIME (Ribeiro et al. "Why should I trust you?" Explaining the predictions of any classifier [C] / / Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and datamining. 2016: 1135-1144.) and gradient methods (Sundararajan et al. Axiomatic attribution for deep networks [C] / / International conference on machine learning. PMLR, 2017: 3319-3328.), etc. These methods attempt to trace the factors that the model pays attention to when making decisions. However, the credibility of these post-hoc interpretation methods has been questioned (Grissom et al. Pathologies of Neural Models Make Interpretation Difficult [J]. 2019.). Studies have found that attention scores or gradient calculations sometimes produce counterintuitive results, suggesting that these methods may not be able to truly reflect the basis for model decisions, especially in high-risk applications such as medical literature analysis.

[0005] In order to improve the transparency of the model, some "ex ante explanation" methods have been proposed. This type of method improves the transparency of the model by encouraging it to generate corresponding reasons before making a decision. For example, the BERT2BERT framework (Chen et al. BERT2BERT: Towards Reusable Pretrained Language Models [C] / / Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022: 2134-2148.) uses two independent models, one for extracting reasons and the other for making decisions, and jointly optimizes the two models during training through reparameterization methods. In addition, the IB framework (Paranjape et al. An Information Bottleneck Approach for Controlling Conciseness in Rationale Extraction [C] / / Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020: 1938-1952.) further improved the BERT2BERT framework and introduced an information bottleneck method to jointly optimize the model instead of reparameterization. The QUASER framework (Ghoshal et al. Quaser: Question answering with scalable extractive rationalization [C] / / Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2022: 1208-1218.) combines sentence selectors and multi-task training objectives to achieve sequence-to-sequence model effects. These methods have improved the transparency and interpretability of the model to a certain extent, but there are still some challenges.

[0006] Although existing frameworks excel in generating high-quality evidence and achieving high task performance, there is still the problem of unreliable connection between evidence and decision making. The model may make the right decision when provided with wrong evidence, or make the wrong judgment when provided with correct evidence (Wiegreffe et al. Measuring Association Between Labels and Free-Text Rationales [C] / / Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2021: 10266-10284.). This uncertainty is particularly dangerous in the medical field, because inaccurate interpretations may mislead medical professionals and affect the patient's diagnosis and treatment process.

[0007] In the medical field, the application of models is not only to provide accurate diagnostic results, but more importantly, to provide understandable and verifiable evidence for their judgments, especially when complex clinical information is involved. For example, when a doctor is faced with the diagnosis of a certain disease, he may need to refer to multiple relevant documents to determine whether a decision is reasonable. Therefore, the intelligent auxiliary system for medical decision-making needs to have the following characteristics: First, a reliable decision-making basis is required, based on the information of large-scale medical literature, to output accurate and medically meaningful explanations to support the credibility of the decision. Secondly, the model should have good human-computer collaboration capabilities, and doctors can understand and question the judgments of the model. This two-way interaction is a safety guarantee in the medical field. Finally, the model should have the ability of explicit attribution, that is, it can clearly attribute the decision-making process to known medical knowledge and literature, which not only improves the reliability of the model, but also provides a scientific basis for doctors to make final decisions.

[0008] In summary, existing large-scale pre-trained models and their interpretation methods still have limitations in generating credible evidence for medical decision-making. To address the above problems, this paper proposes a new method that aims to ensure the reliability and rationality of medical decisions by explicitly attributing knowledge in medical literature, thereby improving the overall credibility and practicality of intelligent medicine. Summary of the invention

[0009] The purpose of the present invention is to solve the problem that the diversity and complexity of existing medical information and knowledge often lead to information uncertainty in medical attribution and diagnosis, resulting in a lack of reliability verification of medical knowledge in practical applications, and to propose a medical knowledge explicit attribution system based on medical literature information.

[0010] A medical knowledge explicit attribution system based on medical literature information includes:

[0011] Query text module, query vector module, document vector module, semantic similarity module, literature collection screening module, basic principle acquisition module, medical knowledge judgment module to be verified and result output module;

[0012] The query text module is used to describe the medical knowledge to be verified as the query text;

[0013] The query vector module is used to call the pre-trained model to encode the query text and generate a vector of fixed length, called the query vector;

[0014] The document vector module is used to call the pre-trained language model to encode each medical document in the document library one by one, and generate a vector of fixed length, called a document vector;

[0015] The semantic similarity module is used to calculate the cosine similarity of the query vector obtained by the query vector module and each document vector obtained by the document vector module as the semantic similarity;

[0016] The document collection screening module is used to sort all medical documents in the document library from large to small according to the semantic similarity in the semantic similarity module, select the top K documents with the highest similarity, and form the document collection most relevant to the query text;

[0017] The basic principle acquisition module is used to apply the pre-trained model to encode each sentence in each document in the document collection screened by the document collection screening module, and encode each sentence into a sentence vector; calculate the semantic similarity between each sentence vector and the query vector obtained by the query vector module, and use the T sentences with the highest similarity as the T basic principles related to the input query in the corresponding document;

[0018] The to-be-verified medical knowledge judgment module is used to judge the truth or falsity of the to-be-verified medical knowledge;

[0019] The result output module is used to output the result of the medical knowledge judgment module to be verified and the result of the basic principle acquisition module.

[0020] The beneficial effects of the present invention are:

[0021] The medical knowledge explicit attribution system based on medical literature information of the present invention is adopted, and the description of the medical knowledge to be verified and the related retrieved documents are input, prompting the pre-trained language model to locate the basic principles related to the knowledge to be verified in the document content, such as medical theorems or clinical experience. These basic principles are extracted and sorted as the basis for supporting the knowledge to be verified. The basic principles and the description of the knowledge to be verified are input into the model together. The model judges the authenticity of the knowledge based on the consistency between the two, and outputs a "true" or "false" verification result.

[0022] The effect of the explicit attribution framework was evaluated using the accuracy and F1 indicators on the open source dataset PubmedQA. In terms of the decision-making performance of the model, as shown in Table 1, the explicit attribution framework is significantly better than the medical BioBERT and WT5 frameworks in terms of accuracy and F1 indicators.

[0023] Table 1 Model decision performance

[0024] Model Accuracy (%) F1(％) BioBERT 61.0 42.7 WT5 56.0 35.7 SADM of the present invention 64.7 45.2

[0025] In addition to evaluating the performance of model decisions, the quality of the rationale provided by the model is also evaluated from three aspects. Aspect 1: whether the rationale provided by the evaluation model is factually consistent with the problem; Aspect 2: whether the rationale provided by the evaluation model is semantically consistent with the annotated rationale; Aspect 3: whether there is a reliable connection between the model's decision and the rationale it provides. The experimental results are shown in Table 2. In terms of factual consistency between the rationale provided by the model and the problem, the explicit attribution framework and the WT5 framework are basically the same. In terms of semantic consistency between the rationale provided by the model and the annotated rationale, and reliable connection between the model decision and the rationale it provides, the explicit attribution framework is significantly better than the WT5 framework.

[0026] Model Aspect 1 Aspect 2 Aspect 3 WT5 96 58 60 SADM of the present invention 95 69 94 BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a flow chart of the present invention;

[0028] Figure 2 This is a schematic diagram of the calculation process of the medical literature retrieval model;

[0029] Figure 3 This is a framework diagram of an explicit attribution system based on medical literature;

[0030] Figure 4 An embodiment of an explicit attribution system based on medical literature. DETAILED DESCRIPTION

[0031] Specific implementation method 1: This implementation method is a medical knowledge explicit attribution system based on medical literature information, including: a query text module, a query vector module, a document vector module, a semantic similarity module, a document collection screening module, a basic principle acquisition module, a to-be-verified medical knowledge judgment module and a result output module;

[0032] The query text module is used to describe the medical knowledge to be verified as the query text;

[0033] "Aspirin is effective in preventing myocardial infarction" is used as the medical knowledge description to be verified and input into the system as the query content;

[0034] The query vector module is used to call the pre-trained model to encode the query text and generate a high-dimensional floating-point vector of fixed length, called the query vector;

[0035] The document vector module is used to call the pre-trained language model to encode each medical document in the document library one by one, and generate a high-dimensional floating-point vector of fixed length, called a document vector;

[0036] The semantic similarity module is used to calculate the cosine similarity of the query vector obtained by the query vector module and each document vector obtained by the document vector module as the semantic similarity;

[0037] The document collection screening module is used to sort all medical documents in the document library from large to small according to the semantic similarity in the semantic similarity module, select the first K documents with the highest similarity (K is a positive integer), and form a document collection most relevant to the query text;

[0038] The basic principle acquisition module is used to apply the pre-trained model to encode each sentence in each document in the document collection screened by the document collection screening module, and encode each sentence into a sentence vector; calculate the semantic similarity between each sentence vector and the query vector obtained by the query vector module, and use the T sentences with the highest similarity as the T basic principles related to the input query in the corresponding document;

[0039] The to-be-verified medical knowledge judgment module is used to judge the truth or falsity of the to-be-verified medical knowledge;

[0040] The result output module is used to output the result of the medical knowledge judgment module to be verified and the result of the basic principle acquisition module.

[0041] Specific implementation method 2: The difference between this implementation method and specific implementation method 1 is that the pre-trained model in the query vector module is BERT or GPT.

[0042] The other steps and parameters are the same as those in the first embodiment.

[0043] Specific implementation method three: This implementation method is different from specific implementation methods one or two in that: the query vector module is used to call the pre-trained model to encode the query text and generate a high-dimensional floating-point vector of a fixed length, called a query vector; the expression is:

[0044] Q=Encode(query)=[q1,q2,…,q n ]

[0045] Among them, n is the dimension of the vector, and Encode represents the encoding process of the language model;

[0046] Q is the query vector, query is the query text, q1 is the first position in the query vector Q, q2 is the second position in the query vector Q, and q n is the nth bit in the query vector Q;

[0047] The pre-trained language model is called to encode the query text "Aspirin is effective in preventing myocardial infarction" to generate a query vector.

[0048] The other steps and parameters are the same as those in the first or second embodiment.

[0049] Specific implementation method 4: This implementation method is different from any one of the specific implementation methods 1 to 3 in that: the document vector module is used to call the pre-trained language model to encode each medical document in the document library one by one, and generate a high-dimensional floating-point vector of fixed length, called a document vector; the expression is:

[0050] V i =Encode(D i )=[v i1 ,v i2 ,…,v in ]

[0051] Among them, V i is the document vector of the i-th medical article, v i1 V i No. 1, v i2 V i No. 2, v in V i The nth position in the

[0052] D i is the i-th medical article in the literature library;

[0053] i is the i-th one.

[0054] The other steps and parameters are the same as those in Specific Embodiments 1 to 3.

[0055] Specific implementation method five: This implementation method is different from any one of specific implementation methods one to four in that the pre-trained language model in the document vector module is the same as the pre-trained model in the query vector module.

[0056] The other steps and parameters are the same as those in Specific Implementation 1 to 4-1.

[0057] Specific implementation method 6: This implementation method is different from any one of the specific implementation methods 1 to 5 in that the calculation formula of the cosine similarity in the semantic similarity module is:

[0058]

[0059] Among them, Q is the query vector, Vi is the document vector of the i-th medical article;

[0060] q j is the jth position in the query vector Q, v ij V i The jth position in the middle;

[0061] j is the jth dimension of the vector;

[0062] are vectors Q and V i The dot product of

[0063] is the modulus of vector Q;

[0064] is the vector V i The mold length.

[0065] The other steps and parameters are the same as those in Specific Implementation Methods 1 to 5-1.

[0066] Specific implementation method 7: This implementation method is different from any one of specific implementation methods 1 to 6 in that: the document collection screening module is used to sort all medical documents in the document library from large to small according to the semantic similarity in the semantic similarity module, and select the first K documents with the highest similarity (K is a positive integer) to form a document collection most relevant to the query text;

[0067] The specific process is:

[0068] S={D1,D2,…,D K}

[0069] Among them, S is the document set most relevant to the query text in step 1;

[0070] D1 is the first document in the document set that is most relevant to the query text, D2 is the second document in the document set that is most relevant to the query text, and D K is the Kth document in the document set that is most relevant to the query text;

[0071] The value of K is 20;

[0072] Sort all medical documents in the document library from large to small according to the semantic relevance measure obtained in step 4, select the top 20 documents with the highest similarity, and form the document set most relevant to the query text "Aspirin is effective in preventing myocardial infarction";

[0073] S={D1,D2,…,D 20}

[0074] Where S is the document set most relevant to the query text in step 1, D1 is the first document in the document set most relevant to the query text in step 1, D2 is the second document in the document set most relevant to the query text in step 1, and D 20 It is the 20th document in the document set that is most relevant to the query text in step 1.

[0075] The other steps and parameters are the same as those in Specific Embodiments 1 to 6.

[0076] Specific implementation eight: This implementation differs from any one of specific implementations one to seven in that: the basic principle acquisition module is used to apply the pre-trained model to encode each sentence in each document in the document collection screened by the document collection screening module, and encode each sentence into a sentence vector; calculate the semantic similarity between each sentence vector and the query vector obtained by the query vector module, and use the T sentences with the highest similarity as the T basic principles related to the input query in the corresponding document;

[0077] The specific implementation is:

[0078] Apply the pre-trained model to each document D i Each sentence in s j Encode each sentence s j Encoded into sentence vector

[0079] in, for No. 1 in for 2nd place in the middle; for The nth position in the

[0080] Calculate sentence vector Cosine similarity with the query vector Q

[0081]

[0082] in, for The kth position in the k is the kth position in the query vector Q;

[0083] For each document, the sentence with the highest similarity is selected as the rationale in the corresponding document that is relevant to the input query;

[0084] The pre-trained model is the same as the pre-trained model in the query vector module.

[0085] The value of T is 5.

[0086] The other steps and parameters are the same as those in Specific Implementation Methods 1 to 67.

[0087] Specific implementation method 9: This implementation method is different from any one of specific implementation methods 1 to 8 in that: the to-be-verified medical knowledge judgment module is used to judge the truth or falsity of the to-be-verified medical knowledge;

[0088] The specific process is:

[0089] Calculate the semantic similarity between the T basic principles corresponding to each document and the query vector to obtain T semantic similarity scores;

[0090] Set thresholds;

[0091] If there is a semantic similarity score greater than the set threshold among the T semantic similarity scores, the medical knowledge to be verified is determined to be "true";

[0092] If there is no semantic similarity score greater than the set threshold among the T semantic similarity scores, the medical knowledge to be verified is determined to be "false".

[0093] The threshold is set to 0.5. If there is a semantic similarity score greater than the set threshold among the T semantic similarity scores, the medical knowledge to be verified "Aspirin is effective in preventing myocardial infarction" is judged to be "true"; if there is no semantic similarity score greater than the set threshold among the T semantic similarity scores, the medical knowledge to be verified "Aspirin is effective in preventing myocardial infarction" is judged to be "false".

[0094] The other steps and parameters are the same as those in Specific Implementations 1 to 68.

[0095] Specific implementation ten: This implementation differs from any one of specific implementations one to nine in that: the result output module is used to output the result of the to-be-verified medical knowledge judgment module and the result of the basic principle acquisition module;

[0096] Specifically:

[0097] Output the authenticity label ("true" or "false") of the medical knowledge to be verified and the basic principle content fragment supporting the judgment. The output result clearly shows the verification conclusion and its theoretical basis.

[0098] The other steps and parameters are the same as those in Specific Embodiments 1 to 9.

[0099] The present invention may also have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A medical knowledge explicit attribution system based on medical literature information, characterized by: The system includes: a query text module, a query vector module, a document vector module, a semantic similarity module, a document collection screening module, a basic principle acquisition module, a to-be-verified medical knowledge judgment module and a result output module; The query text module is used to use the medical knowledge description to be verified as the query text; The query vector module is used to call the pre-trained model to encode the query text and generate a vector of fixed length, called the query vector; The document vector module is used to call the pre-trained language model to encode each medical document in the document library one by one, and generate a vector of fixed length, called a document vector; The semantic similarity module is used to calculate the cosine similarity of the query vector obtained by the query vector module and each document vector obtained by the document vector module as the semantic similarity; The document collection screening module is used to sort all medical documents in the document library from large to small according to the semantic similarity in the semantic similarity module, select the top K documents with the highest similarity, and form the document collection most relevant to the query text; The basic principle acquisition module is used to apply the pre-trained model to encode each sentence in each document in the document collection screened by the document collection screening module, and encode each sentence into a sentence vector; calculate the semantic similarity between each sentence vector and the query vector obtained by the query vector module, and use the T sentences with the highest similarity as the T basic principles related to the input query in the corresponding document; The to-be-verified medical knowledge judgment module is used to judge the truth or falsity of the to-be-verified medical knowledge; The result output module is used to output the result of the medical knowledge judgment module to be verified and the result of the basic principle acquisition module.

2. According to claim 1, a medical knowledge explicit attribution system based on medical literature information is characterized by: The pre-trained model in the query vector module is BERT or GPT.

3. According to claim 2, a medical knowledge explicit attribution system based on medical literature information is characterized by: The query vector module is used to call the pre-trained model to encode the query text and generate a vector of fixed length, called the query vector; the expression is: Q=Encode(query)=[q1,q2,...,q n ] Where n is the dimension of the vector; Encode represents the encoding process of the language model; Q is the query vector; query is the query text; q1 is the first position in the query vector Q, q2 is the second position in the query vector Q, and q n is the nth bit in the query vector Q.

4. The medical knowledge explicit attribution system based on medical literature information according to claim 3 is characterized by: The document vector module is used to call the pre-trained language model to encode each medical document in the document library one by one, and generate a vector of fixed length, called a document vector; the expression is: V i =Encode(D i )=[v i1 ,v i2 ,…,v in ] Among them, V i is the document vector of the i-th medical article; v i1 V i No. 1, v i2 V i No. 2, v in V i The nth position in the D i is the i-th medical article in the literature library; i is the i-th one.

5. The medical knowledge explicit attribution system based on medical literature information according to claim 4 is characterized by: The pre-trained language model in the document vector module is the same as the pre-trained model in the query vector module.

6. The medical knowledge explicit attribution system based on medical literature information according to claim 5 is characterized by: The calculation formula of cosine similarity in the semantic similarity module is: Where Q is the query vector; V i is the document vector of the i-th medical article; q j is the jth position in the query vector Q; v ij V i The jth position in the middle; j is the jth dimension of the vector; is the vector Q and V i The dot product of is the modulus of vector Q; is the vector V i The mold length.

7. The medical knowledge explicit attribution system based on medical literature information according to claim 6 is characterized by: The document collection screening module is used to sort all medical documents in the document library from large to small according to the semantic similarity in the semantic similarity module, select the top K documents with the highest similarity, and form the document collection most relevant to the query text; The specific process is: S={D1,D2,…,D K } Among them, S is the document set most relevant to the query text in step 1; D1 is the first document in the document set that is most relevant to the query text, D2 is the second document in the document set that is most relevant to the query text, and D K is the Kth document in the document set that is most relevant to the query text.

8. The medical knowledge explicit attribution system based on medical literature information according to claim 7 is characterized by: The basic principle acquisition module is used to apply the pre-trained model to encode each sentence in each document in the document collection screened by the document collection screening module, and encode each sentence into a sentence vector; calculate the semantic similarity between each sentence vector and the query vector obtained by the query vector module, and use the T sentences with the highest similarity as the T basic principles related to the input query in the corresponding document; The specific process is: Apply the pre-trained model to each document D i Each sentence in s j Encode each sentence s j Encoded into sentence vector in, for No. 1 in for 2nd place in the middle; for The nth position in the Calculate sentence vector Cosine similarity with the query vector Q in, for The kth position in the middle; . is the dot product; q k is the kth position in the query vector Q; For each document, the sentence with the highest similarity is selected as the rationale in the corresponding document that is relevant to the input query; The pre-trained model is the same as the pre-trained model in the query vector module.

9. The medical knowledge explicit attribution system based on medical literature information according to claim 8, characterized in that: The to-be-verified medical knowledge judgment module is used to judge the truth or falsity of the to-be-verified medical knowledge; the specific process is: Calculate the semantic similarity between the T basic principles corresponding to each document and the query vector to obtain T semantic similarity scores; Set thresholds; If there is a semantic similarity score greater than the set threshold among the T semantic similarity scores, the medical knowledge to be verified is determined to be true; If there is no semantic similarity score greater than the set threshold among the T semantic similarity scores, the medical knowledge to be verified is determined to be false.

10. The medical knowledge explicit attribution system based on medical literature information according to claim 9, characterized in that: The result output module is used to output the result of the medical knowledge judgment module to be verified and the result of the basic principle acquisition module; The specific process is: Outputs true or false labels of the medical knowledge to be verified and a snippet of the rationale content supporting the judgment.

Images