Medical question generation method and apparatus

Abstract

The specification provides a medical question generation method and device, the method comprising: obtaining medical information for generating a medical question, and extracting a medical entity from the medical information; finding a medical knowledge item related to the medical entity in a medical knowledge graph constructed based on a plurality of medical knowledge items, and extracting a subgraph related to the medical entity from the medical knowledge graph based on the found medical knowledge item; converting the subgraph into structured medical knowledge according to a preset medical knowledge format, and identifying a medical question pattern matched with the subgraph; determining an evidence-based question type corresponding to the medical question based on the structured medical knowledge and the medical question pattern, and generating a medical question based on a medical question template corresponding to the evidence-based question type.

Description

TECHNICAL FIELD

[0001] One or more embodiments of the present specification relate to the field of artificial intelligence, and in particular, to a medical question generation method and device. BACKGROUND

[0002] In modern clinical medical practice, proposing accurate and operable medical questions, especially clinical questions, is the core prerequisite for realizing evidence-based medical (EBM) decision-making. Clinical questions not only guide the selection of diagnosis and treatment paths and the development of individualized treatment plans, but also play a key role in multidisciplinary team collaboration, guideline updating, research topic selection, and medical quality improvement. In this case, how to obtain reasonable medical questions has become a problem to be solved. SUMMARY

[0003] One or more embodiments of the present specification provide technical solutions as follows:

[0004] According to a first aspect of one or more embodiments of the present specification, a medical question generation method is provided, the method comprising:

[0005] obtaining medical information for generating a medical question, and extracting a medical entity from the medical information;

[0006] finding a medical knowledge item related to the medical entity in a medical knowledge graph constructed based on a plurality of medical knowledge items, and extracting a subgraph related to the medical entity from the medical knowledge graph based on the found medical knowledge item;

[0007] transforming the subgraph into structured medical knowledge according to a preset medical knowledge format, and identifying a medical question pattern matched with the subgraph;

[0008] determining an evidence-based question type corresponding to the medical question based on the structured medical knowledge and the medical question pattern, and generating a medical question based on a medical question template corresponding to the evidence-based question type.

[0009] According to a second aspect of one or more embodiments of the present specification, a medical question generation device is provided, the device comprising:

[0010] an entity extraction module that obtains medical information for generating a medical question, and extracts a medical entity from the medical information;

[0011] a subgraph extraction module that finds a medical knowledge item related to the medical entity in a medical knowledge graph constructed based on a plurality of medical knowledge items, and extracts a subgraph related to the medical entity from the medical knowledge graph based on the found medical knowledge item;

[0012] a graph processing module, configured to convert the sub-graph into structured medical knowledge according to a preset medical knowledge format, and identify a medical question pattern matched with the sub-graph;

[0013] a question generation module, configured to determine an evidence-based question type corresponding to the medical question based on the structured medical knowledge and the medical question pattern, and generate a medical question based on a medical question template corresponding to the evidence-based question type.

[0014] According to a third aspect of one or more embodiments of the present specification, an electronic device is provided, comprising:

[0015] a processor;

[0016] a memory for storing processor-executable instructions;

[0017] The processor implements the steps of the method according to any one of the above embodiments by running the executable instructions.

[0018] According to a fourth aspect of one or more embodiments of the present specification, a computer-readable storage medium is provided, which stores computer instructions, and the instructions are executed by a processor to implement the steps of the method according to any one of the above embodiments.

[0019] According to a fifth aspect of one or more embodiments of the present specification, a computer program product is provided, comprising computer program / instructions, and the computer program / instructions are executed by a processor to implement the steps of the method according to any one of the above embodiments.

[0020] As can be seen from the above embodiments, first, medical information used for generating a medical question can be obtained, and medical entities can be extracted from the medical information. Then, medical knowledge items related to the medical entities can be searched in a medical knowledge graph constructed based on a plurality of medical knowledge items, and sub-graphs related to the medical entities can be extracted from the medical knowledge graph based on the searched medical knowledge items. Then, the sub-graphs can be converted into structured medical knowledge according to a preset medical knowledge format, and a medical question pattern matched with the sub-graphs can be identified. Finally, an evidence-based question type corresponding to a medical question to be generated can be determined based on the structured medical knowledge and the medical question pattern, and a medical question can be generated based on a medical question template corresponding to the evidence-based question type.

[0021] In the above manner, automated medical question generation can be achieved. By determining an evidence-based question type based on structured medical knowledge converted from a specific subgraph extracted from a medical knowledge graph and a medical question pattern matched with the subgraph, and generating a medical question based on a medical question template corresponding to the evidence-based question type, the comprehensiveness, accuracy, and standardization of the generated medical question can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 FIG. 1 is a schematic diagram of a medical question generation system according to an example embodiment.

[0023] Figure 2 FIG. 1 is a flowchart of a medical question generation method according to an example embodiment.

[0024] Figure 3 FIG. 1 is a flowchart of an evidence evaluation method according to an example embodiment.

[0025] Figure 4 FIG. 1 is a flowchart of a medical question output method according to an example embodiment.

[0026] Figure 5 FIG. 1 is a structural schematic diagram of an apparatus according to an example embodiment.

[0027] Figure 6 FIG. 1 is a block diagram of a medical question generation device according to an example embodiment. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the technical solutions in the specification, the technical solutions in the specification will be clearly and completely described below in conjunction with the drawings in the specification. Obviously, the described embodiments are only some of the embodiments of the specification, not all. Based on the embodiments in the specification, all other embodiments obtained by those skilled in the art without creative labor should fall within the scope of protection of the specification.

[0029] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the specification are information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0030] Evidence-based medicine (EBM) is a medical practice method, whose core idea is to combine the best available research evidence, the physician's professional skills and clinical experience, and the patient's values and preferences in the process of clinical decision-making to develop the most scientific and individualized diagnosis and treatment plan. In short, it is "using the best evidence to make the most appropriate decision".

[0031] The three core elements of EBM include the best research evidence, clinical professional skills, and patient values and preferences. Among them, the best research evidence comes from rigorously designed clinical research (e.g., randomized controlled trials, systematic reviews, Meta-analysis, etc.); evidence has levels, and systematic reviews / Meta-analysis is usually the highest level. Clinical professional skills refer to the physician's ability to diagnose diseases, judge treatment options, and grasp individual differences in patients, which can evaluate the applicability of evidence and apply it to specific patients. Patient values and preferences refer to the patient's personal wishes for treatment goals, risk tolerance, quality of life, economic burden, cultural beliefs, etc.; respect for patient choice and achieve "patient-centered" care.

[0032] Different research designs provide different strengths of evidence. The levels of evidence from high to low are: systematic review / Meta-analysis, randomized controlled trial (RCT), cohort study, case-control study, case series / case report, expert opinion / basic research. The higher the level of evidence, the lower the risk of bias, and the more reliable the results.

[0033] The five-step method of practicing EBM includes: asking clinical questions, i.e., transforming clinical uncertainties into answerable and structured questions (commonly using PICO or PECO format); searching for evidence, i.e., efficiently finding the best current evidence; evaluating evidence, i.e., evaluating the authenticity (Validity), importance (Importance), and applicability (Applicability) of the research (e.g., is the research design reasonable? Is the sample size sufficient? Is there bias? Are the results clinically meaningful? Is it applicable to my patients?); applying evidence, i.e., combining evidence with clinical experience and patient wishes to develop individualized programs; evaluating the effect, i.e., evaluating the clinical effect and efficiency after applying evidence, summarizing experience, and continuously improving.

[0034] As we can see, asking precise and operational medical questions, especially clinical questions, is the core prerequisite for implementing EBM decision-making.

[0035] However, the formulation of medical questions and their classification has long relied on the subjective experience and manual induction of field experts, with inherent limitations such as limited coverage, strong subjectivity, and difficulty in standardization.

[0036] One or more embodiments of the present specification provide a technical solution for realizing medical question generation, in which medical information for generating a medical question can be acquired first, and medical entities can be extracted from the medical information, then medical knowledge items related to the medical entities can be searched for in a medical knowledge graph constructed based on a plurality of medical knowledge items, and subgraphs related to the medical entities can be extracted from the medical knowledge graph based on the searched medical knowledge items, then the subgraphs can be converted into structured medical knowledge according to a preset medical knowledge format, and a medical question pattern matched with the subgraphs can be identified, and finally a type of evidence-based question corresponding to the medical question to be generated can be determined based on the structured medical knowledge and the medical question pattern, and a medical question can be generated based on a medical question template corresponding to the type of evidence-based question.

[0037] In the above manner, automated medical question generation can be realized, and by determining a type of evidence-based question based on structured medical knowledge converted from a specific subgraph extracted from a medical knowledge graph and a medical question pattern matched with the subgraph, and generating a medical question based on a medical question template corresponding to the type of evidence-based question, the comprehensiveness, accuracy, and standardization of the generated medical question can be improved.

[0038] Reference Figure 1 , Figure 1 is a schematic diagram of a medical question generation system provided by an exemplary embodiment.

[0039] As shown in Figure 1 , the above intelligent dialogue system can include a server and at least one client accessing the server through any type of wired or wireless network.

[0040] The above server can correspond to a server including a single independent physical host, or a server cluster composed of a plurality of independent physical hosts; or can correspond to a virtual server, a cloud server, etc. carried by a host cluster.

[0041] The above client can correspond to a terminal device such as a smart phone, a tablet computer, a notebook computer, a desktop computer, a PC (Personal Computer), a PDA (Personal Digital Assistant), a wearable device (e.g., smart glasses, a smart watch, etc.), a smart vehicle device, or a game console, etc.

[0042] In actual applications, the above client can be used to upload medical information for generating a medical question to the above server, and the medical information can be derived from a clinical scenario, and the server can perform specific steps of medical question generation.

[0043] In some embodiments, the server can be equipped with a large language model (LLM). The LLM can work with various functional components or functional subsystems on the server to generate medical questions.

[0044] A large language model is a natural language processing model based on deep learning technology, with strong language understanding and generation capabilities. Large language models are typically deep learning models trained on large amounts of text data and can be used to understand the meaning of natural language text or generate natural language text. Large language models can handle a variety of natural language tasks, such as text classification, named entity recognition (NER), question answering, dialogue, etc., and are an important way to artificial intelligence.

[0045] In the field of natural language processing (NLP), large-scale text datasets are often referred to as corpora. Corpora can contain various types of text data, such as literary works, academic papers, legal documents, news reports, daily conversations, emails, online forum posts, etc. By learning from the text data in the corpus, large language models can acquire and understand the rules and patterns of natural language, and thus effectively process and generate human language.

[0046] Large language models typically use the Transformer architecture, i.e. large language models are typically deep learning models based on the Transformer architecture. Deep learning models based on the Transformer architecture are a class of neural network models that use the Transformer architecture, which performs well in natural language processing and other fields.

[0047] Transformer is a neural network model for sequence-to-sequence modeling. Transformer does not rely on recursive structure and can parallelize training and inference, speeding up model processing. In deep learning models based on the Transformer architecture, a multi-layer Transformer encoder is usually used to extract features from the input sequence, and a Transformer decoder is used to convert the extracted features into an output sequence. At the same time, such models usually use self-attention mechanisms to capture long-distance dependencies in input sequences, and use residual connections and normalization methods to speed up training and improve model performance.

[0048] A pre-trained model is a large language model pre-trained on a large-scale of unlabeled text data. A pre-trained model is a general-purpose model that is not designed and optimized for a specific task. In order to adapt the pre-trained model to a specific application scenario and task requirement, fine-tuning is needed to improve the performance of the model on the specific task. The large language model finally put into use is usually a model that is further fine-tuned on the basis of the pre-trained model and is supervised learning based on labeled text data. Pre-training and fine-tuning are complementary processes. Pre-training enables the model to have a broad language understanding ability, and fine-tuning makes the model more professional and accurate on specific tasks.

[0049] That is, the training process of a large language model can be divided into two stages: pre-training and fine-tuning. In the pre-training stage, unsupervised learning (for example, self-supervised learning) can be used to pre-train on a large-scale, unlabeled text dataset (for example, web encyclopedias, web articles, books, etc.). Specifically, the missing part or the next word can be predicted according to the context, the statistical rules and language structure such as semantics and syntax are learned, and the model parameters are iteratively updated by minimizing the prediction loss through backpropagation and optimization algorithms (for example, gradient descent method) to gradually improve the model's understanding of language. In the fine-tuning stage, according to the specific application scenario and task requirement, the corresponding supervised learning task (for example, text classification, named entity recognition, question and answer system, dialogue system, etc.) can be selected, and the task-specific text dataset can be prepared. Then the pre-trained model can be used as the starting point for fine-tuning, and supervised learning can be used to fine-tune on the task-specific text dataset. Specifically, the task can be performed based on the text dataset, and the model parameters are iteratively updated by minimizing the loss that measures the performance of the model in handling the specific task through backpropagation and optimization algorithms (for example, gradient descent method) to gradually improve the performance of the model on the specific task. In practical applications, fine-tuning can flexibly choose supervised learning, unsupervised learning or semi-supervised learning according to the specific application scenario and the type of available data.

[0050] The language understanding ability learned by the large language model in the pre-training stage and the fine-tuning stage enables the large language model to understand, analyze and synthesize text information, make logical deductions, knowledge reasoning or problem-solving abilities when facing complex problems or tasks. This ability is usually referred to as the reasoning ability of the large language model.

[0051] In practical applications, the pre-trained large language model is usually referred to as the base model of the large language model, and the fine-tuned large language model is referred to as the service model of the large language.

[0052] A large language model usually performs a specific task under the guidance or inspiration of a prompt, which can be referred to as a Prompt. The prompt can be an initial text or a piece of text provided to the large language model, such as a sentence, a question, a dialogue, etc., which is intended to guide or inspire the model to generate corresponding output. The prompt is a key tool for guiding the model output, which can be very simple or designed to be quite complex, such as including instructions, examples, descriptions of expected output format, etc. Through the prompt, the large language model can be explicitly told what task it is expected to perform, such as answering a question, simulating a dialogue, writing an article, translating text, etc. At the same time, the prompt can provide the large language model with necessary background information and context, enabling the large language model to understand the logic, style, theme, or position that should be followed when generating content. In addition, the prompt can also inspire the large language model to exhibit its inherent knowledge reserve or specific language ability, such as explaining complex concepts, citing regulations, imitating the writing style of a specific author, etc.

[0053] Since the large language model is mainly used for understanding and generating human language based on text processing, the prompt usually appears in the form of text. However, in actual applications, the large language model can also accept other forms of input as prompts, such as images, audio, and even video, provided that the large language model is designed or trained to handle multi-modal data (e.g., text, images, audio, video, etc.).

[0054] In actual applications, the large language model carried on the server can be one and only one, which can be used to perform the evidence evaluation task, and can be used to perform the entity extraction task, the graph pattern recognition task, etc. Alternatively, the carried large language models can be multiple, which can include, in addition to the large language model for performing the evidence evaluation task, a large language model for performing the entity extraction task, a large language model for performing the graph pattern recognition task, etc.

[0055] It should be noted that various functional components or functional subsystems can also be carried on the server. These components or subsystems can work together with the large language model carried on the server to jointly implement medical question generation.

[0056] Reference Figure 2 , Figure 2 is a flowchart of a medical question generation method provided by an exemplary embodiment.

[0057] In the present embodiment, the above medical question generation method can be applied to a server as shown in Figure 1 The medical question generation method can specifically include the following steps:

[0058] Step 202: Obtain medical information for generating a medical question, and extract medical entities from the medical information.

[0059] In this embodiment, medical information for generating a medical question can be acquired first.

[0060] In some embodiments, the above medical information can be a real case and / or a diagnosis and treatment scheme (e.g., a doctor's suggestion, a doctor's recommended scheme, etc.) that has been strictly privacy-protected, all direct identifiers (e.g., name, ID number, address, etc.) and indirect identifiers (e.g., exact date of birth, rare disease combination, etc.) are removed, and key clinical information (e.g., age interval, gender, diagnosis, laboratory index, medication history, etc.) is retained.

[0061] After the above medical information is acquired, entity extraction can be performed on the medical information to extract medical entities from the medical information. It should be noted that entity extraction refers to identifying entities with specific meanings from text and classifying these entities into predefined categories. It should be noted that medical entities can include not only entities such as diseases / stages / layers, biomarkers, drugs / combinations, symptoms, test / inspection items, lines / contraindications / necessities / time windows, but also entities such as thresholds / units of biomarkers, dosages / cycles of drugs / combinations, etc.

[0062] In some embodiments, after the medical entities are extracted from the above medical information, the extracted medical entities can be normalized to keep medical entities with the same meaning uniform. The normalization process can refer to the process of mapping the extracted medical entities from the medical information to unique identifiers or standard names in a standard medical terminology system.

[0063] For example, the extracted entities such as "heart attack", "acute myocardial infarction", etc. can be normalized to "acute myocardial infarction" to achieve disease name normalization; the extracted entities such as "aspirin", "acetylsalicylic acid", "Aspirin", "Aspirin (trade name)", etc. can be normalized to "Aspirin" to achieve drug name normalization; the extracted entities such as "fever", "fever", "elevated body temperature", etc. can be normalized to "Fever" to achieve symptom normalization; the extracted entities such as "blood routine", "complete blood count", "CBC (Complete Blood Count)", etc. can be normalized to "Complete Blood Count" to achieve test / inspection item normalization.

[0064] Step 204: Find medical knowledge items related to the medical entities in the medical knowledge graph constructed based on a plurality of medical knowledge items, and extract a subgraph related to the medical entities from the medical knowledge graph based on the found medical knowledge items.

[0065] In this embodiment, in the case that the medical entity is obtained, a subgraph related to the obtained medical entity can be extracted from the medical knowledge graph.

[0066] It should be noted that the medical knowledge graph described above can be a knowledge graph constructed based on a plurality of medical knowledge items, and a medical knowledge item can be a specific structured recommendation / contraindication / annotation information in a medical guideline, and has metadata (for example: category, version number, applicable condition, footnote, etc.). The medical guideline refers to a systematic recommendation document prepared by an authoritative medical organization, a professional society, or a government health agency based on the current best scientific evidence and expert consensus, for guiding clinicians to make decisions in the prevention, diagnosis, treatment, and management of specific diseases or health problems, such as NCCN guidelines. A medical knowledge item can include a plurality of medical entities and their attributes; in addition, the category, version number, applicable condition, footnote, and other metadata can also be used as attributes of the medical knowledge item.

[0067] Specifically, the medical knowledge item related to the medical entity can be first searched in the medical knowledge graph, and then based on the searched medical knowledge item, a subgraph related to the medical entity can be extracted from the medical knowledge graph. For example, a graph structure formed by the medical knowledge item containing the medical entity can be first searched in the medical knowledge graph, and then the nodes and edges connected to the graph structure in the medical knowledge graph can be extracted to form a subgraph related to the medical entity.

[0068] It should be noted that the medical knowledge item related to the medical entity searched in the medical knowledge graph and the subgraph related to the medical entity extracted from the medical knowledge graph can essentially be regarded as evidence corresponding to the medical question generated based on the medical information.

[0069] In some embodiments, in order to improve the accuracy of medical knowledge item searching and subgraph extraction, that is, to improve the accuracy of evidence retrieval, the medical entity extracted from the medical information can be aligned with the medical entity in the medical knowledge item used to construct the medical knowledge graph, so that the medical entities with the same meaning in the extracted medical entity and the medical entity in the medical knowledge item used to construct the medical knowledge graph are kept uniform; subsequently, the medical knowledge item related to the aligned medical entity can be first searched in the medical knowledge graph, and then based on the searched medical knowledge item, a subgraph related to the aligned medical entity can be extracted from the medical knowledge graph.

[0070] Step 206: converting the subgraph into structured medical knowledge according to a preset medical knowledge format, and identifying a medical question pattern matched with the subgraph.

[0071] In this embodiment, after obtaining the subgraph, on the one hand, the subgraph can be converted into structured medical knowledge according to a preset medical knowledge format, so as to generate a medical question by using the standardized and normalized structured medical knowledge subsequently.

[0072] In some embodiments, the medical knowledge format can include the following fields: Population (population); Interventions (intervention measures); Evidence (evidence); and Constraints (constraints, such as contraindications, indications, etc.).

[0073] On the other hand, a medical question pattern matching the subgraph can be identified. The medical question pattern can include multiple-scheme branching, category inconsistency, threshold gating, sequencing, contraindication, scarcity, diagnosis path, and other logic patterns. Identifying the medical question pattern matching the subgraph is to determine whether the logic of the medical question pattern exists in the subgraph.

[0074] For example, assuming that the logic represented by the subgraph contains the logic of “if PD-L1≥50%, recommend A; otherwise, recommend B”, multiple-scheme branching can be determined as the medical question pattern matching the subgraph; assuming that the logic represented by the subgraph contains the logic of “a certain drug can be used only when eGFR>30mL / min”, threshold gating can be determined as the medical question pattern matching the subgraph; and assuming that the logic represented by the subgraph contains the logic of “EGFR mutation positive is contraindicated for immune monotherapy”, contraindication can be determined as the medical question pattern matching the subgraph.

[0075] By performing graph pattern recognition on the above subgraph, a "comparison type" question can be identified, i.e., multiple optional solutions in a medical guideline are identified, the graph pattern of which is that multiple intervention paths are branched out under the same population node, at this time, the "Comparison" part in the PICO question can be automatically generated; conflicts can also be detected, i.e., conflicts between doctor's recommendations or doctor's recommended solutions and contraindications in medical guidelines are detected, the graph pattern of which is that the combination of medical entities in the doctor's recommendations or the doctor's recommended solutions triggers the "contraindication edge" or "mutually exclusive constraint" in the graph, at this time, unmet_conditions (unmet conditions) can be marked, triggering warnings or reviews; knowledge gaps can also be found, i.e., patient characteristics (e.g., rare mutations) are detected in the medical guideline without corresponding paths, the graph pattern of which is that medical entities cannot hit any valid subgraph, at this time, research-oriented questions (e.g., "What is the best first-line treatment for MET exon14 skip mutation patients?") can be generated; boundary situations can also be captured, i.e., the dose, time window, and biomarker threshold are at the critical value, the graph pattern of which includes the "threshold gating" mode, i.e., the node has a numerical constraint (e.g., PD-L1≥50%), and the "sequential" mode, i.e., there is a time-dependent edge between nodes (e.g., postoperative→start adjuvant therapy within 4-8 weeks), at this time, it can be checked whether the doctor's recommendations or the doctor's recommended solutions are out of bounds, and accurate questions (e.g., "Can immunotherapy still be considered for PD-L1=49%?") can be generated; multiple solution categories can also be distinguished, i.e., multiple treatment paths for the same disease are distinguished (e.g., early stage, late stage, maintenance therapy), the graph pattern of which is that the subgraph presents a multi-branch structure, each branch corresponds to a different stage / hierarchy, at this time, it can be ensured that the question generation strictly matches the patient's stage.

[0076] In some embodiments, identifying a medical question pattern matching the above subgraph can specifically include performing a graph query on the subgraph based on a medical question pattern library to determine a medical question pattern matching the subgraph. The graph query can be used to "describe the logical structure to be searched" in a declarative language, and can be efficiently executed by a graph engine, i.e., to determine whether the logic represented by the subgraph contains the logic of a certain medical question pattern, a graph query can be used to efficiently implement it.

[0077] In some embodiments, the graph pattern recognition can also be performed in a machine learning manner. Specifically, a graph neural network (GNN) can be used to identify a medical question pattern matching the above subgraph.

[0078] Step 208: determining an evidence-based question type corresponding to the medical question based on the structured medical knowledge and the medical question pattern, and generating a medical question based on a medical question template corresponding to the evidence-based question type.

[0079] In this embodiment, in the case that the structured medical knowledge and the medical question pattern are obtained, a type of evidence-based question corresponding to the medical question to be generated can be determined based on the structured medical knowledge and the medical question pattern, so that a medical question can be generated based on a medical question template corresponding to the type of evidence-based question, based on the structured medical knowledge or even the medical question pattern.

[0080] The type of evidence-based question generally includes four types of G1-G4. Among them, G1 refers to abstract fact and standard alignment (verification type): verification around abstract facts such as definition / standard / process / hierarchy / threshold, without explaining the reason, without involving decision-making. G2 refers to mechanism and evidence interpretation (explanation type): answering "why", comparing evidence differences, evaluating extrapolation and bias, and not giving clinical decision-making. G3 refers to direct evidence support for clinical decision-making (decision-making type): containing specific patient information, giving clear and executable action plan (do / not do / when do / how much do), including threshold / time window / contraindication and applicable boundary. G4 refers to clinical decision-making without direct evidence support (speculative type): in the case of insufficient / evidence conflict / population mismatch, giving semi-quantitative / intervalized conclusion, extrapolation hypothesis and uncertainty, alternative path and trigger condition.

[0081] It should be noted that since the structured medical knowledge can be regarded as evidence corresponding to the medical question generated based on the medical information, the medical question generated based on the structured medical knowledge can be regarded as an evidence-based question.

[0082] In some embodiments, a type of evidence-based question corresponding to the medical question to be generated can be determined based on the structured medical knowledge and the medical question pattern according to a rule base. The rules stored in the rule base can be rules representing the association relationship between each field in the structured medical knowledge, the medical question pattern, and the type of evidence-based question.

[0083] In some embodiments, in order to improve the correctness of the determined type of evidence-based question corresponding to the medical question to be generated, a large language model can be used to perform evidence evaluation based on the structured medical knowledge and the medical question pattern to obtain an evidence evaluation result including the type of evidence-based question corresponding to the medical question to be generated.

[0084] Reference Figure 3 , Figure 3 is a flowchart of an evidence evaluation method provided by an exemplary embodiment.

[0085] The evidence evaluation method can specifically include the following steps:

[0086] Step 302: inputting the structured medical knowledge and the medical question pattern into a large language model, and generating, by the large language model, an evidence evaluation result corresponding to the medical question based on the structured medical knowledge and the medical question pattern; wherein the evidence evaluation result comprises an evidence-based question type.

[0087] Specifically, determining the evidence-based question type corresponding to the medical question to be generated based on the structured medical knowledge and the medical question pattern can include: inputting the structured medical knowledge and the medical question pattern into a large language model, and generating, by the large language model, an evidence evaluation result corresponding to the medical question based on the structured medical knowledge and the medical question pattern.

[0088] It should be noted that the content of the evidence evaluation result generated by the above-mentioned large language model can be constrained by SOP (Standard Operating Procedure). SOP is a detailed and written operation guide used to standardize the execution of a specific process or task to ensure consistency, safety, efficiency and compliance.

[0089] The above-mentioned evidence evaluation result can include g_class (G1-G4), i.e. the evidence-based question type. In addition, the evidence evaluation result can also include: supports_doctor_plan, i.e. whether there is a doctor's suggestion or a doctor's recommended scheme; evidence_flags / levels, i.e. evidence markers / levels; rationale_links, i.e. reasoning basis (medical knowledge items); unmet_conditions, i.e. unmet conditions.

[0090] Step 304: verifying the evidence evaluation result. If the verification fails, step 306 is performed; if the verification passes, step 316 is performed.

[0091] In some embodiments, in order to improve the accuracy of the evidence evaluation result and thus improve the accuracy of the medical question generated based on the evidence evaluation result, the evidence evaluation result can be verified after obtaining the evidence evaluation result.

[0092] In some embodiments, verifying the above-mentioned evidence evaluation result can include determining whether the confidence corresponding to the evidence evaluation result is greater than a preset threshold. For example, the evidence evaluation result can also include confidence, i.e. confidence. In this case, it can be directly determined whether the confidence in the evidence evaluation result is greater than the preset threshold. Alternatively, the confidence corresponding to the evidence evaluation result can also be determined according to the preset rule, and then it can be determined whether the confidence corresponding to the evidence evaluation result is greater than the preset threshold.

[0093] If the confidence corresponding to the evidence evaluation result is greater than a preset threshold, it can be considered that the verification of the evidence evaluation result passes; otherwise, it can be considered that the verification of the evidence evaluation result fails.

[0094] Alternatively, verifying the evidence evaluation result can include determining whether the evidence evaluation result has consistency conflicts. The consistency can include factual consistency, internal consistency, self-consistency, etc. The factual consistency refers to the consistency between the content generated by the model and the known facts of the external world; if the content generated by the model contradicts the true facts, it is called a factual consistency conflict. The internal consistency refers to whether the output of the model is self-consistent in logic, semantics, or position in a single generation or multiple interactions; if not, it is called an internal consistency conflict. The self-consistency refers to the consistency of the content generated by the model through multiple reasoning; if the content generated by the model through multiple reasoning has a high degree of difference, it is called a self-consistency conflict.

[0095] It should be noted that the specific content of the consistency conflict can be set according to actual conditions and actual needs, and the present specification does not have special restrictions thereon.

[0096] If the evidence evaluation result does not have a consistency conflict, it can be considered that the verification of the evidence evaluation result passes; otherwise, it can be considered that the verification of the evidence evaluation result fails.

[0097] Step 306: If the verification of the evidence evaluation result fails, the large language model generates multiple evidence evaluation results corresponding to the medical question based on the structured medical knowledge and the medical question pattern, and determines a final evidence evaluation result from the generated multiple evidence evaluation results.

[0098] In some embodiments, if the verification of the evidence evaluation result fails, the large language model can generate multiple evidence evaluation results corresponding to the medical question based on the structured medical knowledge and the medical question pattern. In this case, a final evidence evaluation result can be determined from the generated multiple evidence evaluation results. For example, a voting mechanism can be used to vote for an evidence evaluation result from the generated multiple evidence evaluation results as the final evidence evaluation result.

[0099] Step 308: Verify the final evidence evaluation result. If the verification fails, perform step 310; if the verification passes, perform step 316.

[0100] In some embodiments, after obtaining the final evidence evaluation result, the evidence evaluation result can be reverified.

[0101] Similarly, the verification on the final evidence evaluation result can include determining whether the confidence corresponding to the evidence evaluation result is greater than a preset threshold, and / or determining whether the evidence evaluation result has consistency conflicts.

[0102] Step 310: If the verification on the final evidence evaluation result fails, an evidence evaluation result corresponding to the medical question is generated based on an evidence evaluation rule.

[0103] In some embodiments, if the verification on the final evidence evaluation result fails, an evidence evaluation result corresponding to the medical question can be generated based on an evidence evaluation rule. The evidence evaluation rule can be an evidence evaluation rule preset according to the characteristics of the evidence evaluation result.

[0104] Step 312: The evidence evaluation result generated based on the evidence evaluation rule is verified. If the verification fails, step 314 is performed; if the verification passes, step 316 is performed.

[0105] In some embodiments, in the case where the evidence evaluation result is generated based on the evidence evaluation rule, the evidence evaluation result can be re-verified.

[0106] Similarly, the verification on the evidence evaluation result generated based on the evidence evaluation rule can include determining whether the confidence corresponding to the evidence evaluation result is greater than a preset threshold, and / or determining whether the evidence evaluation result has consistency conflicts.

[0107] Step 314: If the verification on the evidence evaluation result generated based on the evidence evaluation rule fails, an evidence evaluation result annotated by an annotator based on the structured medical knowledge and the medical question pattern for the medical question is obtained.

[0108] In some embodiments, if the verification on the evidence evaluation result generated based on the evidence evaluation rule fails, manual review and annotation can be performed. Therefore, an evidence evaluation result annotated by an annotator based on the structured medical knowledge and the medical question pattern for the medical question can be obtained.

[0109] Step 316: If the verification passes or the annotated evidence evaluation result is obtained, the evidence evaluation result is converted into a structured evidence-based question according to an evidence-based question format.

[0110] In some embodiments, if the verification on the evidence evaluation result in step 304 passes, the evidence evaluation result can be converted into a structured evidence-based question according to an evidence-based question format.

[0111] If the check on the final evidence evaluation result is passed in step 308, the final evidence evaluation result can be converted into a structured evidence-based question according to the evidence-based question format.

[0112] If the check on the evidence evaluation result generated based on the evidence evaluation rule is passed in step 312, the evidence evaluation result generated based on the evidence evaluation rule can be converted into a structured evidence-based question according to the evidence-based question format.

[0113] If the evidence evaluation result annotated by the annotator is obtained, the annotated evidence evaluation result can be converted into a structured evidence-based question according to the evidence-based question format.

[0114] In actual applications, the PICO or PECO format can be used to convert the evidence evaluation result into a structured evidence-based question. In evidence-based medicine, PICO and PECO are two commonly used structured frameworks for constructing clinical questions, which are used to help clinicians clarify research questions, develop retrieval strategies, and guide systematic reviews or clinical decision-making.

[0115] In PICO, P represents Patient / Population / Problem, i.e., patients, populations, or problems, such as middle-aged men with type 2 diabetes; I represents Intervention, i.e., intervention measures, such as treatment with metformin; C represents Comparison, i.e., control measures (which can be placebo, standard treatment, or other interventions), such as treatment with sulfonylurea drugs; and O represents Outcome, i.e., outcome indicators, such as a decrease in hemoglobin A1c levels, a low incidence of hypoglycemia.

[0116] In PECO, P represents Population, i.e., target populations, such as long-term adult smokers; E represents Exposure, i.e., exposure factors, such as daily smoking ≥ 20 cigarettes; C represents Comparison, i.e., control / non-exposed group, such as non-smokers or daily smokers < 10 cigarettes; and O represents Outcome, i.e., outcome (e.g., disease occurrence, death, etc.), such as lung cancer incidence.

[0117] Correspondingly, generating a medical question based on the medical question template corresponding to the evidence-based question type can include: generating a medical question based on the medical question template corresponding to the evidence-based question type and the structured evidence-based question.

[0118] In some embodiments, to ensure the quality of the generated medical question, the generated medical question can be checked, and the generated medical question can be output according to the check result.

[0119] Reference Figure 4 ,Figure 4 is a flowchart of a medical question output method provided by an example embodiment.

[0120] The medical question output method can specifically include the following steps:

[0121] Step 402: Verify the generated medical question. If the verification passes, perform step 404; if the verification fails, perform step 406.

[0122] Specifically, the terms, dosages, dosage units, thresholds, threshold units, etc. in the generated medical question can be verified to determine whether these contents in the generated medical question are correct.

[0123] Step 404: If the verification of the generated medical question fails, adjust the evidence-based question type corresponding to the generated medical question.

[0124] In some embodiments, if the verification of the generated medical question fails, the evidence-based question type corresponding to the generated medical question can be adjusted. For example, the evidence-based question type corresponding to the generated medical question can be downgraded to G2, i.e., explanatory. Or, the generated medical question can be prompted for manual review.

[0125] After adjusting the evidence-based question type corresponding to the generated medical question or the generated medical question itself, the adjusted medical question can also be sorted according to the application scenario corresponding to the generated medical question.

[0126] Step 406: If the verification of the generated medical question passes, sort the generated medical question according to the application scenario corresponding to the generated medical question.

[0127] In some embodiments, if the verification of the generated medical question passes, the generated medical question can be sorted according to the application scenario corresponding to the generated medical question to improve the usability and information density of the generated medical question. Specifically, when the application scenario is clinical, the generated medical question can be sorted in the order of evidence-based question type G3>G2>G1>G4, and for medical questions of the same type, the medical questions can be sorted in the order of the lexicographic order of evidence markers or the high-low order of evidence levels; and when the application scenario is research, the generated medical question can be sorted in the order of evidence-based question type G4>G2>G3>G1, and for medical questions of the same type, the medical questions can be sorted in the order of medical questions with evidence markers of controversy / blank first.

[0128] It should be noted that the generated medical questions can also be de-duplicated when the generated medical questions are sorted.

[0129] Step 408: output the generated medical question, and the evidence-based question type and medical knowledge item corresponding to the generated medical question.

[0130] In some embodiments, the generated medical question, and the evidence-based question type and medical knowledge item corresponding to the generated medical question can be output. The medical knowledge item corresponding to the generated medical question is the evidence traceability information of the generated medical question. The evidence traceability information can be the reasoning basis in the above evidence evaluation result, or the medical knowledge item related to the above medical entity found in the above medical knowledge graph. In addition, the evidence traceability information can also include the version and footnote metadata of the medical knowledge item corresponding to the generated medical question, so as to trigger offline / online reconstruction and reordering when the version or footnote changes, and provide evidence path level explanation.

[0131] In the technical scheme provided by one or more embodiments of the present application, first, medical information used to generate a medical question can be obtained, and a medical entity can be extracted from the medical information. Then, a medical knowledge item related to the medical entity can be found in a medical knowledge graph constructed based on a plurality of medical knowledge items, and a subgraph related to the medical entity can be extracted from the medical knowledge graph based on the found medical knowledge item. Then, the subgraph can be converted into structured medical knowledge according to a preset medical knowledge format, and a medical question pattern matching the subgraph can be identified. Finally, an evidence-based question type corresponding to the medical question to be generated can be determined based on the structured medical knowledge and the medical question pattern, and a medical question can be generated based on a medical question template corresponding to the evidence-based question type.

[0132] In the above manner, automatic medical question generation can be achieved. Furthermore, by determining an evidence-based question type based on structured medical knowledge converted from a specific subgraph extracted from a medical knowledge graph, and a medical question pattern matching the subgraph, and generating a medical question based on a medical question template corresponding to the evidence-based question type, the comprehensiveness, accuracy, and standardization of the generated medical question can be improved.

[0133] Corresponding to the above-mentioned embodiments of the medical question generation method, the present specification also provides embodiments of a medical question generation device.

[0134] Reference Figure 5 , Figure 5 is a structural schematic diagram of a device provided by an exemplary embodiment. As shown in Figure 5As shown, the device 500 is comprised primarily of a communication interface 502, a user interface 504, a processor 506, and a data store 508, which are interconnected and communicate with each other through a system bus, network, or other connection mechanism 510. The communication interface 502 enables the device 500 to communicate with other devices, access networks, and transport networks through analog or digital modulation. For example, the communication interface 502 can include a chipset and antenna for wireless communication with a radio access network or access point. In addition, the communication interface 502 can be a wired interface such as an Ethernet, Token Ring, or USB port, or a wireless interface such as Wifi, Bluetooth, Global Positioning System (GPS), or a wide-area wireless interface (e.g., WiMAX or LTE). Of course, the communication interface 502 can support other forms of physical layer interface and standard or proprietary communication protocol. The communication interface 502 can also include multiple physical communication interfaces, such as a Wifi interface, a Bluetooth interface, and a wide-area wireless interface.

[0135] The user interface 504 includes receiving user input and providing output to a user. Thus, the user interface 504 can include input components such as a keypad, keyboard, touch- sensitive or presence-sensitive panel, computer mouse, trackball, joystick, microphone, still camera, and video camera, and output components such as a display screen (which can be combined with a touch-sensitive panel), CRT, LCD, LED, display using DLP technology, printer, other known or future developed equivalent devices. The user interface 504 can also generate audible output through a speaker, speaker jack, audio output port, audio output device, earphones, and other known or future developed equivalent devices. In some embodiments, the user interface 504 can include software, circuitry, or other form of logic that is capable of transmitting data to and receiving data from external user input / output devices. Additionally or alternatively, the device 500 can support remote access from other devices through the communication interface 502 or another physical interface (not shown). The user interface 504 can be configured to receive user input, the location and movement of which can be indicated by an indicator or cursor described herein. The user interface 504 can also be configured as a display device for rendering or displaying a text segment.

[0136] The processor 506 can include one or more general processors and / or specialized processors.

[0137] The data store 508 can include one or more volatile and / or non-volatile storage components, and can be integrated in whole or in part with the processor 506. The data store 508 can include removable and / or non-removable components.

[0138] The processor 506 can execute program instructions stored in the data storage 508 (e.g., compiled or interpreted program logic and / or machine code) to implement various functionality described herein. The data storage 508 can include a non-transitory computer-readable medium having stored thereon program instructions that, when executed by the device 500, enable the device 500 to perform any of the methods, processes, or functions disclosed in the specification and / or drawings. Execution of the program instructions 518 by the processor 506 can cause the processor 506 to utilize data 512.

[0139] For example, the program instructions 518 can include an operating system 522 (e.g., an operating system kernel, device drivers, and / or other modules) installed on the device 500 as well as one or more application programs 520 (e.g., a browser, a social application, or a game application). Similarly, the data 512 can include operating system data 516 and application data 514. The operating system data 516 is primarily accessible to the operating system 522, while the application data 514 is primarily accessible to the one or more application programs 520. The application data 514 can be in a file system that is visible to or hidden from the user of the device 500.

[0140] The application programs 520 can communicate with the operating system 522 through one or more application programming interfaces (APIs). These APIs facilitate the application programs 520 in reading and / or writing application data 514, communicating or receiving information via the communication interface 502, receiving or displaying information on the user interface 504, and the like.

[0141] In some terminology, the application programs 520 can be referred to simply as “apps.” Furthermore, the application programs 520 can be downloaded to the device 500 through one or more online application stores or application markets. However, the application programs can also be installed on the device 500 through other means, such as through a web browser or a physical interface (e.g., a USB port) on the device 500.

[0142] Reference Figure 6 , Figure 6 is a block diagram of an exemplary embodiment of a medical question generation apparatus. The medical question generation apparatus can be applied in a device as shown in Figure 5 to implement the technical solutions of the present specification.

[0143] The medical question generation apparatus described above can include:

[0144] An entity extraction module 602 obtains medical information used to generate a medical question, and extracts medical entities from the medical information;

[0145] The subgraph extraction module 604 searches for medical knowledge items related to the medical entity in a medical knowledge graph constructed based on a plurality of medical knowledge items, and extracts a subgraph related to the medical entity from the medical knowledge graph based on the searched medical knowledge items;

[0146] The graph processing module 606 converts the subgraph into structured medical knowledge according to a preset medical knowledge format, and identifies a medical question pattern matched with the subgraph;

[0147] The question generation module 608 determines an evidence-based question type corresponding to the medical question based on the structured medical knowledge and the medical question pattern, and generates a medical question based on a medical question template corresponding to the evidence-based question type.

[0148] In some embodiments, the searching for medical knowledge items related to the medical entity in a medical knowledge graph constructed based on a plurality of medical knowledge items, and extracting a subgraph related to the medical entity from the medical knowledge graph based on the searched medical knowledge items, comprises:

[0149] aligning the medical entity with medical entities in the medical knowledge items;

[0150] searching for medical knowledge items related to the aligned medical entity in the medical knowledge graph constructed based on a plurality of medical knowledge items, and extracting a subgraph related to the aligned medical entity from the medical knowledge graph based on the searched medical knowledge items.

[0151] In some embodiments, the identifying a medical question pattern matched with the subgraph comprises:

[0152] performing graph query on the subgraph based on a medical question pattern library to determine a medical question pattern matched with the subgraph.

[0153] In some embodiments, the determining an evidence-based question type corresponding to the medical question based on the structured medical knowledge and the medical question pattern comprises:

[0154] inputting the structured medical knowledge and the medical question pattern into a large language model, and generating an evidence evaluation result corresponding to the medical question based on the structured medical knowledge and the medical question pattern by the large language model; wherein the evidence evaluation result comprises an evidence-based question type.

[0155] In some embodiments, the apparatus further comprises a first verification module configured to:

[0156] verify the evidence evaluation result;

[0157] If the verification on the evidence evaluation result fails, the large language model generates multiple evidence evaluation results corresponding to the medical question based on the structured medical knowledge and the medical question pattern, and determines a final evidence evaluation result from the generated multiple evidence evaluation results.

[0158] In some embodiments, the first verification module is further configured to:

[0159] verify the final evidence evaluation result;

[0160] If the verification on the final evidence evaluation result fails, generate an evidence evaluation result corresponding to the medical question based on an evidence evaluation rule.

[0161] In some embodiments, the first verification module is further configured to:

[0162] verify the evidence evaluation result generated based on the evidence evaluation rule;

[0163] If the verification on the evidence evaluation result generated based on the evidence evaluation rule fails, obtain an evidence evaluation result labeled by a labeler for the medical question based on the structured medical knowledge and the medical question pattern.

[0164] In some embodiments, the first verification module is further configured to:

[0165] If the verification passes or the labeled evidence evaluation result is obtained, convert the evidence evaluation result into a structured evidence-based question according to an evidence-based question format.

[0166] The generating of the medical question based on the medical question template corresponding to the evidence-based question type comprises:

[0167] Generating a medical question based on the medical question template corresponding to the evidence-based question type and the structured evidence-based question.

[0168] In some embodiments, verifying the evidence evaluation result comprises:

[0169] determining whether the confidence corresponding to the evidence evaluation result is greater than a preset threshold; and / or,

[0170] determining whether the evidence evaluation result has a consistency conflict.

[0171] In some embodiments, the apparatus further comprises a second verification module configured to:

[0172] verify the generated medical question;

[0173] If the check on the generated medical question passes, the generated medical question is sorted according to an application scenario corresponding to the generated medical question.

[0174] In some embodiments, the second checking module is further configured to:

[0175] If the check on the generated medical question fails, an evidence-based question type corresponding to the generated medical question is adjusted.

[0176] In some embodiments, the apparatus further comprises a question output module configured to:

[0177] output the generated medical question, and the evidence-based question type and the medical knowledge item corresponding to the generated medical question.

[0178] For ease of description, the above apparatus is described in various modules or units in terms of functions. Of course, the functions of each module or unit can be implemented in one or more software and / or hardware when implementing one or more of the present specification, and the modules implementing the same function can be implemented by a combination of multiple sub-modules or sub-units. The apparatus embodiments described above are only illustrative, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0179] Based on the same idea as the above method, the present specification also provides an electronic device, comprising: a processor; a memory for storing processor executable instructions; wherein the processor implements the steps of the method according to any one of the above embodiments by running the executable instructions.

[0180] Based on the same idea as the above method, the present specification also provides a computer readable storage medium, which stores computer instructions, and the instructions are executed by a processor to implement the steps of the method according to any one of the above embodiments.

[0181] Based on the same idea as the above method, the present specification also provides a computer program product, comprising computer program / instructions, which are executed by a processor to implement the steps of the method according to any one of the above embodiments.

[0182] Those skilled in the art can understand that:

[0183] In this specification, the term "comprising" or "including" or any other variant thereof is intended to cover non-exclusive inclusions, such that processes, methods, products, or devices that comprise a list of elements are not limited to those elements, but can include other elements not expressly listed or inherent to such processes, methods, products, or devices. Without further limitation, additional same or equivalent elements in processes, methods, products, or devices that include the recited elements are not excluded.

[0184] In this specification, "one", "a", and "the" do not refer to the singular, but can include the plural as well, unless otherwise indicated.

[0185] In this specification, the first, second, and the like ordinal numbers do not necessarily indicate a sequence or order, and in many cases, are used to distinguish a corresponding component from another component. For example, a first server and a second server are used to distinguish the two servers from each other. Of course, the two servers can be the same server in some cases.

[0186] In this specification, unless explicitly described otherwise, "receiving and transmitting of data" is not necessarily direct reception and transmission, and can be indirect reception and transmission. For example, A receiving data transmitted by B can be understood as A directly receiving data transmitted by B, or A indirectly receiving data transmitted by B through C or another subject. Similarly, B transmitting data to A can be understood as B directly transmitting data to A, or B indirectly transmitting data to A through C or another subject. Here, C can be one subject, or two or more subjects.

[0187] In this specification, unless explicitly described otherwise, the association relationship between structures can be a direct association relationship or an indirect association relationship. For example, when "A is connected to B" is described, unless it is explicitly described that A is directly connected to B, it should be understood that A can be directly connected to B, or A can be indirectly connected to B. For another example, when "A is on B", unless it is explicitly described that A is directly on the top of B (AB is adjacent and A is on the top of B), it should be understood that A can be directly on the top of B, or A can be indirectly on the top of B (AB is separated by other elements, and A is on the top of B). Similarly.

[0188] This specification uses specific terms to describe embodiments thereof. Terms such as "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Furthermore, those skilled in the art can combine and integrate the different embodiments or examples described herein, as well as the features of those different embodiments or examples, without contradiction.

[0189] Although one or more embodiments of this specification provide method steps as described in the embodiments or flowcharts, it is understood that the order of steps listed in the embodiments or flowcharts is only one of many possible execution orders and does not represent the only execution order. Therefore, when the claims involve method steps, any changes or adjustments to the order of such steps, or the parallelism between steps, are also within the scope of protection of the claims.

Claims

1. A method for generating medical questions, the method comprising: Obtain medical information used to generate medical questions, and extract medical entities from the medical information; Search for medical knowledge items related to the medical entity in a medical knowledge graph constructed based on several medical knowledge items, and extract subgraphs related to the medical entity from the medical knowledge graph based on the found medical knowledge items. Based on a preset medical knowledge format, the subgraph is transformed into structured medical knowledge, and medical problem patterns matching the subgraph are identified. Based on the structured medical knowledge and the medical problem pattern, an evidence evaluation result corresponding to the medical problem is generated; wherein, the evidence evaluation result includes the evidence-based problem type; According to the evidence-based question format, the evidence evaluation results are transformed into structured evidence-based questions, and medical questions are generated based on the medical question template corresponding to the evidence-based question type and the structured evidence-based questions.

2. The method according to claim 1, wherein searching for medical knowledge items related to the medical entity in a medical knowledge graph constructed based on several medical knowledge items, and extracting a subgraph related to the medical entity from the medical knowledge graph based on the found medical knowledge items, comprises: Align the medical entity with the medical entities in the medical knowledge item; In a medical knowledge graph constructed based on several medical knowledge items, medical knowledge items related to the aligned medical entities are searched, and based on the searched medical knowledge items, subgraphs related to the aligned medical entities are extracted from the medical knowledge graph.

3. The method according to claim 1, wherein identifying the medical problem pattern matching the subgraph comprises: Based on a medical problem pattern library, a graph query is performed on the subgraph to determine the medical problem pattern that matches the subgraph.

4. The method according to claim 1, wherein generating an evidence evaluation result corresponding to the medical problem based on the structured medical knowledge and the medical problem pattern includes: The structured medical knowledge and the medical problem pattern are input into a large language model, which then generates an evidence evaluation result corresponding to the medical problem based on the structured medical knowledge and the medical problem pattern.

5. The method according to claim 4, further comprising: The results of the evidence evaluation were verified. If the verification of the evidence evaluation result fails, the large language model generates multiple evidence evaluation results corresponding to the medical problem based on the structured medical knowledge and the medical problem pattern, and determines the final evidence evaluation result from the multiple generated evidence evaluation results.

6. The method according to claim 5, further comprising: The final evidence assessment results are verified. If the verification of the final evidence assessment result fails, an evidence assessment result corresponding to the medical problem is generated based on the evidence assessment rules.

7. The method according to claim 6, further comprising: The evidence evaluation results generated based on the aforementioned evidence evaluation rules are verified. If the verification of the evidence evaluation result generated based on the evidence evaluation rules fails, then the evidence evaluation result annotated by the annotator based on the structured medical knowledge and the medical problem pattern for the medical problem is obtained.

8. The method according to any one of claims 5-7, wherein converting the evidence evaluation result into a structured evidence-based question according to the evidence-based question format includes: If the verification passes, or if the marked evidence evaluation results are obtained, the evidence evaluation results are transformed into structured evidence-based questions according to the evidence-based question format.

9. The method according to any one of claims 5-7, wherein verifying the evidence evaluation result includes: Determine whether the confidence level corresponding to the evidence assessment result is greater than a preset threshold; And / or, Determine whether there are any inconsistencies in the results of the evidence assessment.

10. The method according to claim 1, further comprising: The generated medical questions are validated. If the generated medical questions pass the verification, they are then sorted according to the application scenarios they correspond to.

11. The method according to claim 10, further comprising: If the validation of the generated medical question fails, the evidence-based question type corresponding to the generated medical question will be adjusted.

12. The method according to claim 1, further comprising: Output the generated medical question, along with the corresponding evidence-based question type and medical knowledge item.

13. A medical problem generation device, the device comprising: The entity extraction module acquires medical information used to generate medical questions and extracts medical entities from the medical information. The subgraph extraction module searches for medical knowledge items related to the medical entity in a medical knowledge graph constructed based on several medical knowledge items, and extracts subgraphs related to the medical entity from the medical knowledge graph based on the found medical knowledge items. The graph processing module converts the subgraph into structured medical knowledge according to a preset medical knowledge format and identifies medical problem patterns that match the subgraph. The question generation module generates evidence evaluation results corresponding to the medical question based on the structured medical knowledge and the medical question pattern; wherein, the evidence evaluation results include the evidence-based question type; Furthermore, based on the evidence-based question format, the evidence evaluation results are transformed into structured evidence-based questions, and medical questions are generated based on the medical question template corresponding to the evidence-based question type and the structured evidence-based questions.

14. An electronic device comprising: processor; Memory used to store processor-executable instructions; The processor implements the steps of the method as described in any one of claims 1 to 12 by executing the executable instructions.

15. A computer-readable storage medium having stored thereon computer instructions that, when executed by a processor, implement the steps of the method as claimed in any one of claims 1 to 12.

16. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method as claimed in any one of claims 1 to 12.

Images