A medical knowledge explicit attribution system based on medical literature information
By using an explicit attribution system based on medical literature information, the problem of opaque decision-making in the medical field by large-scale pre-trained models is solved, the credibility and reliability of medical knowledge are realized, reliable medical evidence is provided, and the transparency and interpretability of the model are improved.
Patent Information
- Application Number
- CN202411653554.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-19
- Publication Date
- 2025-11-18
- Estimated Expiration
- 2044-11-19
AI Technical Summary
The decision-making process of existing large-scale pre-trained models in the medical field is opaque, and existing interpretation methods cannot reliably provide medical evidence, resulting in a lack of credibility and reliability of medical knowledge in practical applications.
Design an explicit attribution system based on medical literature information. Through a query text module, query vector module, document vector module, semantic similarity module, literature set filtering module, basic principle acquisition module, and medical knowledge to be verified module, clarify the decision-making basis of medical knowledge and generate credible medical evidence.
It significantly improves the transparency and credibility of the model in medical decision-making, provides reliable medical evidence support, and enhances the interpretability and reliability of the model.
Smart Images
Figure CN119577113B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of natural language processing in the field of artificial intelligence in computer science, and relates to an explicit attribution system for medical knowledge based on medical literature information. Background Technology
[0002] In modern medical research and clinical diagnosis, artificial intelligence, especially large-scale pre-trained models based on deep learning, has gradually become an important tool for assisting decision-making (Touvron et al. Llama: Open and efficient foundation language models[J].arXiv:2302.13971,2023.). These models have shown excellent performance in various tasks (Wang et al. Knowledge-tuning Large Language Models with Structured Medical Knowledge Bases for Reliable Response Generation in Chinese[J].arXiv:2309.04175,2023.), but their decision-making process is often opaque. When dealing with complex medical problems, models must have sufficient credibility to gain the approval of medical experts, but the existing "black box" characteristics limit the widespread application of these models in the medical field.
[0003] While current large-scale pre-trained models perform admirably, their decisions are often based on learning complex patterns from the input data. Research shows that these models tend to rely on surface features (Geirhos et al. Shortcut learning in deep neural networks[J]. Nature Machine Intelligence,2020,2(11):665-673.), which makes their decisions potentially not entirely based on valid medical logic. In the medical field, the judgments of any model need to be trusted and validated to ensure that its decisions align with medical knowledge and experience. Therefore, improving the interpretability and transparency of models, especially clarifying the logical basis behind their decision-making process, becomes crucial.
[0004] Existing explanation techniques can be mainly divided into pipeline-based explanations and parallel explanation frameworks. In pipeline-based frameworks, a common approach is "post-hoc explanation," where the model makes a decision, and then humans or other algorithms analyze the reasons for that decision. Commonly used methods include attention mechanisms (Tenney I. BERT rediscovers the classical NLP pipeline[J].arXiv:1905.05950,2019.), LIME (Ribeiro et al. "Why should I trust you?" Explaining the predictions of any classifier[C] / / Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and datamining.2016:1135-1144.), and gradient methods (Sundararajan et al. Axiomatic attribution for deep networks[C] / / International conference on machine learning.PMLR,2017:3319-3328.), etc. These methods attempt to trace the factors that the model focuses on when making decisions. However, the credibility of these ex post facto interpretation methods has been questioned (Grissom et al. Pathologies of Neural Models Make Interpretation Difficult[J]. 2019.). Studies have found that attention scores or gradient calculations sometimes produce counterintuitive results, suggesting that these methods may not accurately reflect the basis of model decisions, especially in high-risk applications such as medical literature analysis.
[0005] To improve model transparency, some "pre-explanation" methods have been proposed. These methods improve transparency by encouraging the model to generate reasons before making decisions. For example, the BERT2BERT framework (Chen et al. BERT2BERT: Towards Reusable Pretrained Language Models[C] / / Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics(Volume 1:Long Papers).2022:2134-2148.) uses two independent models, one for extracting reasons and the other for making decisions, and jointly optimizes these two models during training through a reparameterization method. Furthermore, the IB framework (Paranjape et al. An Information Bottleneck Approach for Controlling Conciseness in Rationale Extraction [C] / / Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020:1938-1952.) further improves the BERT2BERT framework by introducing an information bottleneck method to replace the reparameterization approach for jointly optimizing the model. The QUASER framework (Ghoshal et al. Quaser: Question answering with scalable extractiverationalization [C] / / Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2022:1208-1218.) combines a sentence selector and a multi-task training objective to achieve sequence-to-sequence model performance. These methods improve the transparency and interpretability of the model to some extent, but some challenges still exist.
[0006] While existing frameworks excel in generating high-quality evidence and achieving high task performance, the unreliable link between evidence and decision-making remains a concern. Models may make correct decisions when presented with incorrect evidence, or incorrect judgments when presented with correct evidence (Wiegreffe et al. Measuring Association Between Labels and Free-Text Rationales[C] / / Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing.2021:10266-10284.). This uncertainty is particularly dangerous in the medical field, as inaccurate interpretations can mislead healthcare professionals and consequently affect patient care.
[0007] In the medical field, the application of models goes beyond simply providing accurate diagnostic results; more importantly, it provides understandable and verifiable evidence for these judgments, especially when dealing with complex clinical information. For example, when faced with the diagnosis of a disease, a doctor may need to consult multiple relevant articles to determine the rationality of a decision. Therefore, intelligent decision-making support systems in medicine need to possess the following characteristics: First, they require reliable decision-making basis, outputting accurate and medically meaningful interpretations based on information from a large-scale medical literature to support the credibility of the decision. Second, the model should have good human-machine collaboration capabilities, enabling doctors to understand and question the model's judgments; this two-way interaction is a guarantee of safety in the medical field. Finally, the model should have the ability to explicitly attribute causes, that is, to clearly attribute the decision-making process to known medical knowledge and literature. This not only improves the reliability of the model but also provides a scientific basis for doctors to make their final decisions.
[0008] In summary, existing large-scale pre-trained models and their interpretation methods still have limitations in generating credible medical decision-making evidence. This invention proposes a novel method to address these issues, aiming to ensure the reliability and rationality of medical decisions by explicitly attributing knowledge to medical literature, thereby improving the overall credibility and practicality of intelligent healthcare. Summary of the Invention
[0009] The purpose of this invention is to address the problem that the diversity and complexity of existing medical information and knowledge often lead to information uncertainty in medical attribution and diagnosis, resulting in a lack of reliability verification of medical knowledge in practical applications. Therefore, this invention proposes an explicit attribution system for medical knowledge based on medical literature information.
[0010] A medical knowledge explicit attribution system based on medical literature information includes:
[0011] The system includes a query text module, a query vector module, a document vector module, a semantic similarity module, a document collection filtering module, a basic principle acquisition module, a medical knowledge verification module, and a result output module.
[0012] The query text module is used to use the description of the medical knowledge to be verified as the query text;
[0013] The query vector module is used to call a pre-trained model to encode the query text and generate a fixed-length vector, called the query vector.
[0014] The document vector module is used to call a pre-trained language model to encode each medical document in the literature database one by one, generating a fixed-length vector called a document vector;
[0015] The semantic similarity module is used to calculate the cosine similarity between the query vector obtained by the query vector module and each document vector obtained by the document vector module, as the semantic similarity.
[0016] The document set filtering module is used to sort all medical documents in the document database from largest to smallest according to the semantic similarity in the semantic similarity module, and select the top K documents with the highest similarity to form a document set most relevant to the query text.
[0017] The basic principle acquisition module is used to encode each sentence in each document in the document set selected by the document set filtering module by applying a pre-trained model to each document, and encode each sentence into a sentence vector; calculate the semantic similarity between each sentence vector and the query vector obtained by the query vector module, and take the T sentences with the highest similarity as the T basic principles in the corresponding document that are related to the input query;
[0018] The medical knowledge to be verified module is used to determine the authenticity of the medical knowledge to be verified.
[0019] The result output module is used to output the results of the medical knowledge judgment module to be verified and the results of the basic principle acquisition module.
[0020] The beneficial effects of this invention are as follows:
[0021] This invention employs a medical knowledge explicit attribution system based on medical literature information. The input includes a description of the medical knowledge to be verified and relevant retrieved literature. A pre-trained language model is prompted to locate fundamental principles, such as medical theorems or clinical experience, within the literature content related to the knowledge to be verified. These fundamental principles are extracted and organized as supporting evidence for the knowledge to be verified. Both the fundamental principles and the description of the knowledge to be verified are input into the model. The model judges the authenticity of the knowledge based on their consistency and outputs a verification result of "true" or "false".
[0022] The performance of the explicit attribution framework was evaluated using accuracy and F1 score on the open-source dataset PubmedQA. As shown in Table 1, the explicit attribution framework significantly outperformed the BioBERT and WT5 frameworks in both accuracy and F1 score in terms of model decision performance.
[0023] Table 1 Model Decision Performance
[0024] Model Accuracy (%) F1(％) BioBERT 61.0 42.7 WT5 56.0 35.7 This invention SADM 64.7 45.2
[0025] In addition to evaluating the performance of model decision-making, the quality of the basic principles provided by the model was assessed from three aspects: Aspect 1: whether the basic principles provided by the model are factually consistent with the problem; Aspect 2: whether the basic principles provided by the model are semantically consistent with the labeled basic principles; Aspect 3: whether there is a reliable connection between the model's decisions and the basic principles it provides. The experimental results are shown in Table 2. Regarding the factual consistency between the basic principles provided by the model and the problem, the explicit attribution framework and the WT5 framework are roughly equal. However, in terms of the semantic consistency between the basic principles provided by the model and the labeled basic principles, and the reliable connection between the model's decisions and the basic principles it provides, the explicit attribution framework significantly outperforms the WT5 framework.
[0026] Model Aspect 1 Aspect 2 Aspect 3 WT5 96 58 60 This invention SADM 95 69 94 Attached Figure Description
[0027] Figure 1 This is a flowchart of the present invention;
[0028] Figure 2 This is a schematic diagram of the calculation process of the medical literature retrieval model;
[0029] Figure 3 A framework diagram of an explicit attribution system based on medical literature;
[0030] Figure 4 This is an example of an explicit attribution system based on medical literature. Detailed Implementation
[0031] Specific Implementation Method 1: This implementation method is a medical knowledge explicit attribution system based on medical literature information, which includes: a query text module, a query vector module, a document vector module, a semantic similarity module, a literature set filtering module, a basic principle acquisition module, a medical knowledge to be verified judgment module, and a result output module.
[0032] The query text module is used to use the description of the medical knowledge to be verified as the query text;
[0033] The statement "Aspirin is effective in preventing myocardial infarction" was entered into the system as a text to be verified as the query content.
[0034] The query vector module is used to call a pre-trained model to encode the query text and generate a fixed-length high-dimensional floating-point vector, called the query vector.
[0035] The document vector module is used to call a pre-trained language model to encode each medical document in the literature database one by one, generating a fixed-length high-dimensional floating-point vector, which is called a document vector.
[0036] The semantic similarity module is used to calculate the cosine similarity between the query vector obtained by the query vector module and each document vector obtained by the document vector module, as the semantic similarity.
[0037] The document set filtering module is used to sort all medical documents in the document database from largest to smallest according to the semantic similarity in the semantic similarity module, and select the top K documents with the highest similarity (K is a positive integer) to form the document set most relevant to the query text.
[0038] The basic principle acquisition module is used to encode each sentence in each document in the document set selected by the document set filtering module by applying a pre-trained model to each document, and encode each sentence into a sentence vector; calculate the semantic similarity between each sentence vector and the query vector obtained by the query vector module, and take the T sentences with the highest similarity as the T basic principles in the corresponding document that are related to the input query;
[0039] The medical knowledge to be verified module is used to determine the authenticity of the medical knowledge to be verified.
[0040] The result output module is used to output the results of the medical knowledge judgment module to be verified and the results of the basic principle acquisition module.
[0041] Specific Implementation Method Two: This implementation method differs from Specific Implementation Method One in that the pre-trained model in the query vector module is BERT or GPT.
[0042] The other steps and parameters are the same as in Specific Implementation Method 1.
[0043] Specific Implementation Method Three: This implementation method differs from Specific Implementation Method One or Two in that: the query vector module is used to call the pre-trained model to encode the query text, generating a fixed-length high-dimensional floating-point vector, called the query vector; the expression is:
[0044] Q=Encode(query)=[q1,q2,…,q n ]
[0045] Where n is the dimension of the vector, and Encode represents the encoding process of the language model;
[0046] Q is the query vector, query is the query text, q1 is the first element in query vector Q, q2 is the second element in query vector Q, and q... n To query the nth element in vector Q;
[0047] The pre-trained language model is invoked to encode the query text "Aspirin is effective in preventing myocardial infarction" to generate a query vector.
[0048] Other steps and parameters are the same as in specific implementation method one or two.
[0049] Specific Implementation Method Four: This implementation method differs from Specific Implementation Methods One to Three in that: the document vector module is used to call a pre-trained language model to encode each medical document in the literature database, generating a fixed-length high-dimensional floating-point vector, called a document vector; the expression is:
[0050] V i =Encode(D i ) = [v i1 ,v i2 ,…,v in ]
[0051] Among them, V i Let v be the document vector of the i-th medical document. i1 For V i Ranked #1 in China, v i2 For V i Second in the middle, v in For V i The nth position;
[0052] D i This refers to the i-th medical document in the database.
[0053] i represents the i-th element.
[0054] The other steps and parameters are the same as those in one of the specific implementation methods one to three.
[0055] Specific Implementation Method Five: This implementation method differs from Specific Implementation Methods One to Four in that the pre-trained language model in the document vector module is the same as the pre-trained model in the query vector module.
[0056] The other steps and parameters are the same as those in specific implementation methods one through four.
[0057] Specific Implementation Method Six: This implementation method differs from Specific Implementation Methods One to Five in that the formula for calculating cosine similarity in the semantic similarity module is:
[0058]
[0059] Where Q is the query vector, Vi Let i be the document vector of the i-th medical document;
[0060] q j To query the j-th element in vector Q, v ij For V i The j-th position;
[0061] j is the j-th dimension of the vector;
[0062] They are vectors Q and V i The dot product;
[0063] Let Q be the magnitude of vector Q;
[0064] For vector V i The length of the module.
[0065] The other steps and parameters are the same as those in specific implementation methods one through five.
[0066] Specific Implementation Method Seven: This implementation method differs from Specific Implementation Methods One to Six in that: the document set filtering module is used to sort all medical documents in the document database according to the semantic similarity in the semantic similarity module from largest to smallest, and select the top K documents with the highest similarity (K is a positive integer) to form a document set most relevant to the query text;
[0067] The specific process is as follows:
[0068] S = {D1,D2,…,D} K}
[0069] Where S is the set of documents most relevant to the query text in step one;
[0070] D1 is the first document in the set of documents most relevant to the query text, and D2 is the second document in the set of documents most relevant to the query text. K The Kth document in the set of documents most relevant to the query text;
[0071] The value of K is 20;
[0072] Sort all medical literature in the literature database from largest to smallest according to the semantic relevance measure obtained in step four, and select the top 20 literatures with the highest similarity to form the literature set most relevant to the query text "Aspirin is effective in preventing myocardial infarction".
[0073] S = {D1,D2,…,D} 20}
[0074] Where S is the set of documents most relevant to the query text in step one, D1 is the first document in the set of documents most relevant to the query text in step one, D2 is the second document in the set of documents most relevant to the query text in step one, and D... 20 The 20th document in the set of documents most relevant to the query text in step one.
[0075] The other steps and parameters are the same as those in one of the specific implementation methods one to six.
[0076] Specific Implementation Method Eight: This implementation method differs from Specific Implementation Methods One to Seven in that: the basic principle acquisition module is used to encode each sentence in each document in the document set selected by the document set filtering module by applying a pre-trained model to each document, and encode each sentence into a sentence vector; calculate the semantic similarity between each sentence vector and the query vector obtained by the query vector module, and take the T sentences with the highest similarity as the T basic principles in the corresponding document that are related to the input query;
[0077] The specific implementation method is as follows:
[0078] Applying a pre-trained model to each document D i Each sentence in the middle j Encode each sentence s j Encoding into sentence vectors
[0079] in, for Ranked #1 in China; for Ranked 2nd in the middle; for The nth position;
[0080] Calculate sentence vectors cosine similarity to query vector Q
[0081]
[0082] in, for The k-th position in the middle; · represents the dot product, q k To query the k-th element in vector Q;
[0083] For each document, the sentence with the highest similarity is selected as the basic principle related to the input query in the corresponding document;
[0084] The pre-trained model is the same as the pre-trained model in the query vector module.
[0085] The value of T is 5.
[0086] The other steps and parameters are the same as those in one of the specific implementation methods one to six or seven.
[0087] Specific Implementation Method Nine: This implementation method differs from Specific Implementation Methods One to Eight in that: the medical knowledge to be verified judgment module is used to judge the authenticity of the medical knowledge to be verified;
[0088] The specific process is as follows:
[0089] Calculate the semantic similarity between the T basic principles corresponding to each document and the query vector to obtain T semantic similarity scores;
[0090] Set a threshold;
[0091] If any of the T semantic similarity scores is greater than a set threshold, then the medical knowledge to be verified is determined to be "true".
[0092] If none of the T semantic similarity scores is greater than the set threshold, the medical knowledge to be verified is determined to be "false".
[0093] A threshold of 0.5 is set. If any of the T semantic similarity scores is greater than the threshold, the medical knowledge to be verified, "Aspirin is effective in preventing myocardial infarction," is determined to be "true." If none of the T semantic similarity scores is greater than the threshold, the medical knowledge to be verified, "Aspirin is effective in preventing myocardial infarction," is determined to be "false."
[0094] The other steps and parameters are the same as those in any of the specific implementation methods one to six.
[0095] Specific Implementation Method Ten: This implementation method differs from Specific Implementation Methods One to Nine in that: the result output module is used to output the results of the medical knowledge judgment module to be verified and the results of the basic principle acquisition module;
[0096] Specifically:
[0097] Output the authenticity label ("true" or "false") of the medical knowledge to be verified, as well as fragments of basic principles supporting the judgment. The output results clearly show the verification conclusion and its theoretical basis.
[0098] The other steps and parameters are the same as those in any of the specific implementation methods one to nine.
[0099] This invention may have other embodiments. Without departing from the spirit and essence of this invention, those skilled in the art can make various corresponding changes and modifications according to this invention, but these corresponding changes and modifications should all fall within the protection scope of the appended claims.
Claims
1. A medical knowledge explicit attribution system based on medical literature information, characterized in that: The system includes: a query text module, a query vector module, a document vector module, a semantic similarity module, a document set filtering module, a basic principle acquisition module, a medical knowledge verification module, and a result output module. The query text module is used to use the description of the medical knowledge to be verified as the query text; The query vector module is used to call a pre-trained model to encode the query text and generate a fixed-length vector, called the query vector. The document vector module is used to call a pre-trained language model to encode each medical document in the literature database one by one, generating a fixed-length vector called a document vector; The semantic similarity module is used to calculate the cosine similarity between the query vector obtained by the query vector module and each document vector obtained by the document vector module, as the semantic similarity. The document set filtering module is used to sort all medical documents in the document database from largest to smallest according to the semantic similarity in the semantic similarity module, and select the top K documents with the highest similarity to form a document set most relevant to the query text. The basic principle acquisition module is used to encode each sentence in each document in the document set selected by the document set filtering module by applying a pre-trained model to each document, and encode each sentence into a sentence vector; calculate the semantic similarity between each sentence vector and the query vector obtained by the query vector module, and take the T sentences with the highest similarity as the T basic principles in the corresponding document that are related to the input query; The medical knowledge to be verified module is used to determine the authenticity of the medical knowledge to be verified. The result output module is used to output the results of the medical knowledge judgment module to be verified and the results of the basic principle acquisition module.
2. The explicit attribution system for medical knowledge based on medical literature information according to claim 1, characterized in that: The pre-trained model in the query vector module is either BERT or GPT.
3. The explicit attribution system for medical knowledge based on medical literature information according to claim 2, characterized in that: The query vector module is used to encode the query text using a pre-trained model, generating a fixed-length vector called the query vector; the expression is: Q=Encode(query)=[q1,q2,...,q n ] Where n is the dimension of the vector; Encode represents the encoding process of a language model; Q is the query vector; The query is the query text; q1 is the first element in query vector Q, q2 is the second element in query vector Q, and q n To query the nth element in vector Q.
4. The explicit attribution system for medical knowledge based on medical literature information according to claim 3, characterized in that: The document vector module is used to call a pre-trained language model to encode each medical document in the literature database, generating a fixed-length vector called a document vector; the expression is: V i =Encode(D i )=[v i1 ,v i2 ,…,v in ] Among them, V i Let i be the document vector of the i-th medical document; v i1 For V i Ranked #1 in China, v i2 For V i Second in the middle, v in For V i The nth position; D i This refers to the i-th medical document in the database. i represents the i-th element.
5. A medical knowledge explicit attribution system based on medical literature information according to claim 4, characterized in that: The pre-trained language model in the document vector module is the same as the pre-trained model in the query vector module.
6. A medical knowledge explicit attribution system based on medical literature information according to claim 5, characterized in that: The formula for calculating cosine similarity in the semantic similarity module is as follows: Where Q is the query vector; V i Let i be the document vector of the i-th medical document; q j To query the j-th element in vector Q; v ij For V i The j-th position; j is the j-th dimension of the vector; They are vectors Q and V i The dot product; Let Q be the magnitude of vector Q; For vector V i The length of the module.
7. A medical knowledge explicit attribution system based on medical literature information according to claim 6, characterized in that: The document set filtering module is used to sort all medical documents in the document database from largest to smallest according to the semantic similarity in the semantic similarity module, and select the top K documents with the highest similarity to form a document set most relevant to the query text. The specific process is as follows: S={D1,D2,…,D K } Where S is the set of documents most relevant to the query text in step one; D1 is the first document in the set of documents most relevant to the query text, and D2 is the second document in the set of documents most relevant to the query text. K The Kth document in the set of documents most relevant to the query text.
8. A medical knowledge explicit attribution system based on medical literature information according to claim 7, characterized in that: The basic principle acquisition module is used to encode each sentence in each document in the document set selected by the document set filtering module by applying a pre-trained model to each document, and encode each sentence into a sentence vector; calculate the semantic similarity between each sentence vector and the query vector obtained by the query vector module, and take the T sentences with the highest similarity as the T basic principles in the corresponding document that are related to the input query; The specific process is as follows: Applying a pre-trained model to each document D i Each sentence in the middle j Encode each sentence s j Encoding into sentence vectors in, for Ranked #1 in China; for Ranked 2nd in the middle; for The nth position; Calculate sentence vectors cosine similarity to query vector Q in, for The kth position in the middle; . represents the dot product; q k To query the k-th element in vector Q; For each document, the sentence with the highest similarity is selected as the basic principle related to the input query in the corresponding document; The pre-trained model is the same as the pre-trained model in the query vector module.
9. A medical knowledge explicit attribution system based on medical literature information according to claim 8, characterized in that: The medical knowledge verification module is used to determine the authenticity of the medical knowledge to be verified; the specific process is as follows: Calculate the semantic similarity between the T basic principles corresponding to each document and the query vector to obtain T semantic similarity scores; Set a threshold; If any of the T semantic similarity scores is greater than a set threshold, then the medical knowledge to be verified is determined to be true. If none of the T semantic similarity scores is greater than the set threshold, then the medical knowledge to be verified is determined to be false.
10. A medical knowledge explicit attribution system based on medical literature information according to claim 9, characterized in that: The result output module is used to output the results of the medical knowledge judgment module to be verified and the results of the basic principle acquisition module. The specific process is as follows: Output true or false labels for the medical knowledge to be verified, along with fragments of the basic principles supporting the judgment.
Citation Information
Patent Citations
Medical literature retrieval method and device, electronic equipment and storage medium
CN112287217A
Vectorization knowledge base construction method and device
CN117422135A