Medical subject knowledge service method and device, electronic equipment and medium

Abstract

The invention provides a medical subject knowledge service method and device, electronic equipment and a medium, and the method comprises the steps: obtaining medical literatures, and constructing a medical subject knowledge base corresponding to each subject field based on the medical literatures; obtaining a query problem input by a user, and pre-judging the query problem to determine a subject field and a query mode to which the query problem belongs; performing medical subject knowledge base query and / or medical subject big language model interaction corresponding to the subject field to which the query problem belongs based on the query mode to obtain a result of the query problem; if a second feedback result is obtained, correlation evaluation scoring is carried out on the second feedback result, and if the scoring result meets a second preset condition, a third feedback result is obtained based on the medical subject large language model, and secondary correlation evaluation scoring is carried out on the third feedback result; and finally, feeding back the second feedback result or the third feedback result meeting the requirement to the user. According to the invention, the efficiency and accuracy of acquiring the related knowledge of the medical subject are improved.

Description

Technical Field

[0001] The present invention relates to the field of medical technology, and in particular to a medical subject knowledge service method, device, electronic equipment and medium. Background Art

[0002] At present, with the gradual deepening of aging, chronic diseases such as Alzheimer's disease (AD) and tumors have brought heavy burdens to patients, families and society. In order to better deal with Alzheimer's disease, tumors, etc., researchers in the field of medical disciplines need to efficiently obtain relevant literature and obtain relevant field progress knowledge.

[0003] At present, the most important means for researchers in the field to obtain literature and knowledge of field progress is mainly through manual acquisition and reading of literature, which is time-consuming, inefficient, and low-quality. Some existing auxiliary tools also have great deficiencies in terms of convenience, accuracy, and efficiency. Therefore, in the context of knowledge explosion, the low efficiency and low quality of obtaining literature, evidence, knowledge, ideas, etc. related to medical disciplines have become a problem that needs to be solved urgently. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a medical subject knowledge service method, device, electronic device and medium to improve the efficiency and accuracy of acquiring medical subject related knowledge, reduce workload and complexity, and improve the effect of knowledge service.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] In a first aspect, the present invention provides a medical discipline knowledge service method, comprising: obtaining medical literature, and constructing a medical discipline knowledge base corresponding to each subject field based on the medical literature; wherein the medical discipline knowledge base includes: a medical discipline literature base, a medical discipline literature entity base, a medical discipline evidence base, a medical discipline atlas base, and a medical discipline index base; obtaining a query question input by a user, and pre-judging the query question to determine the subject field and query method to which the query question belongs; based on the query method, querying the medical discipline knowledge base corresponding to the subject field to which the query question belongs and / or interacting with a medical discipline large language model to obtain a result of the query question; wherein the result of the query question includes one of the following: a first query result, a first feedback result, a second query result, and a second feedback result based on the second query result; if the first query result or If a first feedback result is obtained, the first query result or the first feedback result is fed back to the user; if a second feedback result is obtained, a relevance evaluation score is performed on the second feedback result, and when the score result of the relevance evaluation score of the second feedback result meets the first preset condition, a consistency evaluation score is performed on the second feedback result; when the score result of the relevance evaluation score of the second feedback result meets the second preset condition, a third feedback result is obtained based on the medical discipline large language model, and a secondary relevance evaluation score is performed on the third feedback result; when the score result of the secondary relevance evaluation score of the third feedback result meets the first preset condition, a consistency evaluation score is performed on the third feedback result; when the score result of the consistency evaluation score of the second feedback result or the third feedback result meets the third preset condition, the second feedback result or the third feedback result is fed back to the user.

[0007] In a second aspect, the present invention provides a medical discipline knowledge service device, including: a medical discipline knowledge base construction module, which is used to obtain medical literature and construct a medical discipline knowledge base corresponding to each subject field based on the medical literature; wherein the medical discipline knowledge base includes: a medical discipline literature library, a medical discipline literature entity library, a medical discipline evidence library, a medical discipline atlas library, and a medical discipline index library; a pre-judgment module, which is used to obtain a query question input by a user, and pre-judgment the query question to determine the subject field and query method to which the query question belongs; a query and interaction module, which is used to query the medical discipline knowledge base corresponding to the subject field to which the query question belongs and / or interact with a medical discipline large language model based on the query method to obtain the result of the query question; wherein the result of the query question includes one of the following: a first query result, a first feedback result, a second query result, and a second feedback result based on the second query result; A feedback and evaluation module is used for feeding back the first query result or the first feedback result to the user if a first query result or a first feedback result is obtained; if a second feedback result is obtained, performing a relevance evaluation and scoring on the second feedback result, and when the scoring result of the relevance evaluation and scoring of the second feedback result meets the first preset condition, performing a consistency evaluation and scoring on the second feedback result; when the scoring result of the relevance evaluation and scoring of the second feedback result meets the second preset condition, obtaining a third feedback result based on a large language model of medical disciplines, and performing a secondary relevance evaluation and scoring on the third feedback result; when the scoring result of the secondary relevance evaluation and scoring of the third feedback result meets the first preset condition, performing a consistency evaluation and scoring on the third feedback result; when the scoring result of the consistency evaluation and scoring of the second feedback result or the third feedback result meets the third preset condition, feeding back the second feedback result or the third feedback result to the user.

[0008] In a third aspect, the present invention provides an electronic device comprising a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the steps of any one of the methods provided in the first aspect above.

[0009] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any one of the methods provided in the first aspect are executed.

[0010] The present invention brings the following beneficial effects:

[0011] The medical subject knowledge service method, device, electronic device and medium provided by the present invention first obtain medical literature and build a medical subject knowledge base corresponding to each subject field based on the medical literature; secondly, obtain the query question input by the user, and pre-judge the query question to determine the subject field and query method to which the query question belongs; then, based on the query method, query the medical subject knowledge base corresponding to the subject field to which the query question belongs and / or interact with the medical subject large language model to obtain the result of the query question; finally, if the first query result or the first feedback result is obtained, the first query result or the first feedback result is fed back to the user; if the second feedback result is obtained, the second feedback result is processed accordingly. A relevance evaluation score is given, and when the scoring result of the relevance evaluation score of the second feedback result meets the first preset condition, a consistency evaluation score is given to the second feedback result; when the scoring result of the relevance evaluation score of the second feedback result meets the second preset condition, a third feedback result is obtained based on the medical subject big language model, and a secondary relevance evaluation score is given to the third feedback result; when the scoring result of the secondary relevance evaluation score of the third feedback result meets the first preset condition, a consistency evaluation score is given to the third feedback result; when the scoring result of the consistency evaluation score of the second feedback result or the third feedback result meets the third preset condition, the second feedback result or the third feedback result is fed back to the user.

[0012] The above method can ensure the comprehensiveness and timeliness of knowledge in different subject areas by constructing a real-time updated medical subject knowledge base in different subject areas, including a medical subject literature base, a medical subject literature entity base, a medical subject evidence base, a medical subject atlas base, and a medical subject index base; by pre-judging the query questions input by the user, determining the subject area and query method to which it belongs, and then querying and / or interacting based on the medical subject knowledge base and / or the medical subject large language model corresponding to the subject area, it can improve the accuracy and quality of the feedback results on the basis of improving service efficiency; finally, by performing relevance evaluation and scoring, secondary relevance evaluation and scoring, and consistency evaluation and scoring on the query results and feedback results, the results that meet the requirements are fed back to the user, further improving the accuracy and quality of obtaining medical subject-related knowledge.

[0013] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0014] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the preferred embodiments are specifically mentioned below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 A flowchart of a medical subject knowledge service method provided by an embodiment of the present invention;

[0017] Figure 2 A schematic diagram of the structure of a medical subject knowledge service device provided by an embodiment of the present invention;

[0018] Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] At present, in the context of knowledge explosion, the low efficiency and low quality of obtaining medical-related literature, evidence, knowledge, ideas, etc. has become a problem that needs to be solved urgently.

[0021] Based on this, the medical subject knowledge service method, device, electronic device and medium provided by the embodiments of the present invention can improve the efficiency and accuracy of obtaining medical subject related knowledge, reduce workload and complexity, and improve the effect of knowledge service.

[0022] To facilitate understanding of this embodiment, a medical subject knowledge service method disclosed in an embodiment of the present invention is first introduced in detail. The method can be executed by an electronic device, such as a smart phone, a computer, a tablet computer, etc. Figure 1 The flowchart of a medical subject knowledge service method shown in FIG. 1 illustrates that the method mainly includes the following steps S101 to S107:

[0023] Step S101: Obtain medical literature, and construct a medical subject knowledge base corresponding to each subject area based on the medical literature.

[0024] In one embodiment, in order to support intelligent knowledge services for different disciplines, it is necessary to build a knowledge base of relevant research results in different disciplines, that is, a medical discipline knowledge base in different disciplines (such as: Alzheimer's disease field, tumor field, etc.), including: a medical discipline literature library, a medical discipline literature entity library, a medical discipline evidence library, a medical discipline atlas library, and a medical discipline index library.

[0025] In order to ensure the integrity of the content of the medical subject knowledge base, in this embodiment, multiple databases, such as pubmed, embase, cochrane, Wanfang Medical, etc., can be searched and documents in various medical fields can be obtained at the same time. For the obtained documents, a machine learning method is used to accurately distinguish the documents in different subject fields, and then the documents in each subject field are respectively extracted metadata, processed based on rules (pre-set), constructed classification models for processing, and constructed information extraction models for processing, etc., to construct a medical subject document library in each subject field; and based on the pre-constructed entity information extraction model, the documents are entity extracted to construct a medical subject document entity library in each subject field; for each document in the medical subject document library in each subject field, combined with the above-constructed medical subject document entity library, a medical subject atlas library in each subject field is constructed; for documents with RCT labels in their document types, evidence information is extracted based on the pre-constructed evidence extraction model to construct a medical subject evidence library in each subject field; finally, a medical subject index library in each subject field is constructed.

[0026] Step S102: obtaining a query question input by a user, and pre-judging the query question to determine the subject field and query method to which the query question belongs.

[0027] In one implementation, the user service interface includes traditional keyword input, and the user can input query questions by inputting keywords, or can express his query questions in natural language.

[0028] In order to ensure the quality of knowledge services and improve service efficiency, in this embodiment, a pre-judgment of the query question input by the user can be performed to determine the subject field to which the query question belongs and the query method (that is, the method of answering the query question). Then, according to the subject field to which the query question belongs, the corresponding medical subject knowledge base and / or medical subject large language model can be used to perform query and / or interaction.

[0029] Step S103: Based on the query method, a medical subject knowledge base query corresponding to the subject field to which the query question belongs and / or a medical subject large language model interaction are performed to obtain a result of the query question.

[0030] In one embodiment, based on the above query question, a corresponding medical subject knowledge base query and / or medical subject large language model interaction is performed to obtain the result of the query question; wherein the result of the query question includes one of the following: a first query result (corresponding to the result of the medical subject knowledge base query), a first feedback result (corresponding to the result of the medical subject large language model interaction), a second query result, and a second feedback result based on the second query result (corresponding to the result of the combination of the medical subject knowledge base query and the medical subject large language model interaction). The medical subject knowledge base query process includes the following three links: querying the medical subject literature library, querying the medical subject literature entity library, and querying the medical subject evidence library.

[0031] Step S104: if the first query result or the first feedback result is obtained, the first query result or the first feedback result is fed back to the user.

[0032] Step S105: If a second feedback result is obtained, a relevance evaluation score is performed on the second feedback result, and when the score result of the relevance evaluation score of the second feedback result meets the first preset condition, a consistency evaluation score is performed on the second feedback result, and when the score result of the relevance evaluation score of the second feedback result meets the second preset condition, a third feedback result is obtained based on the medical discipline large language model, and a secondary relevance evaluation score is performed on the third feedback result.

[0033] Step S106: When the scoring result of the secondary relevance evaluation of the third feedback result meets the first preset condition, a consistency evaluation score is performed on the third feedback result.

[0034] Step S107: When the consistency evaluation score of the second feedback result or the third feedback result meets the third preset condition, the second feedback result or the third feedback result is fed back to the user.

[0035] In one embodiment, if a first query result is obtained by querying the medical subject knowledge base, or a first feedback result is obtained by interacting with a medical subject large language model, the first query result or the first feedback result is directly fed back to the user; if a second feedback result is obtained by querying the medical subject knowledge base and interacting with a medical subject large language model, the second feedback result of the obtained medical subject large language model is subjected to content inspection and filtering, as well as relevance evaluation and scoring.

[0036] If the relevance evaluation score of the second feedback result meets the first preset condition (the score is greater than or equal to the second threshold but less than the first threshold), the second feedback result is scored for consistency; if the relevance evaluation score of the second feedback result meets the second preset condition (the score is less than the second threshold), the third feedback result is obtained based on the entity list generated in the process of processing the user's query question and the constructed medical subject atlas library, as well as the medical subject large language model, and the third feedback result is scored for secondary relevance evaluation; otherwise (that is, the score is greater than or equal to the first threshold), the second feedback result is fed back to the user.

[0037] If the result of the secondary correlation evaluation score of the third feedback result meets the first preset condition, the third feedback result is evaluated and scored for consistency; if the result of the secondary correlation evaluation score of the third feedback result meets the second preset condition, the medical subject large language model interaction is re-performed to obtain a new second feedback result, and the above steps are repeated; otherwise, the third feedback result is fed back to the user; if the result of the consistency score of the second feedback result or the third feedback result meets the third preset condition (the score is greater than or equal to the third threshold), the second feedback result or the third feedback result is fed back to the user.

[0038] The above-mentioned medical subject knowledge service method provided by the embodiment of the present invention can ensure the comprehensiveness and timeliness of knowledge in different subject areas by constructing a medical subject knowledge base in different subject areas that is updated in real time and includes a medical subject literature base, a medical subject literature entity base, a medical subject evidence base, a medical subject atlas base, and a medical subject index base; by pre-judging the query question input by the user, determining the subject area and query method to which it belongs, and then querying and / or interacting based on the medical subject knowledge base and / or the medical subject large language model corresponding to the subject area to which it belongs, it can improve the accuracy and quality of the feedback results on the basis of improving service efficiency; finally, by performing relevance evaluation and scoring, secondary relevance evaluation and scoring, and consistency evaluation and scoring on the query results and feedback results, the results that meet the requirements are fed back to the user, further improving the accuracy and quality of obtaining medical subject-related knowledge.

[0039] The aforementioned step S101, i.e. building a medical subject knowledge base corresponding to each subject field based on medical literature, includes the following steps S1011 to S1016:

[0040] Step S1011: Classify the medical literature according to subject areas to obtain an initial medical subject literature library in different subject areas.

[0041] In one embodiment, after obtaining relevant literature in the medical field, the medical literature is first classified to obtain literature in different subject areas. Specifically, a machine learning method can be used to distinguish the subject areas of the literature based on the title, abstract, keyword, journal name, etc. of the literature, such as building a model based on the BERT+ tuning method, so as to obtain an initial medical subject literature library in different subject areas.

[0042] Step S1012: For each document in the initial medical subject document library, the document is processed based on the pre-built metadata extraction model, knowledge rules, classification model and information extraction model, and a medical subject document library is constructed according to the content and metadata corresponding to each tag of each document.

[0043] In one embodiment, the medical subject literature library framework can be constructed using a multi-level and multi-link method. Overall, the first-level categories of the medical subject literature library framework (the first-level categories of the knowledge system's labels) include literature title information, content location information, literature type information, disease information, focus information, link information, measurement method information, intervention classification information, intervention information, evaluation standard information, evaluation indicator information, quality evaluation information, methodology evaluation information, comprehensive evaluation information, etc. Further, the first-level categories can be subdivided, for example, literature title information can be divided into title, abstract, keywords, author, affiliated institution, literature publication medium name (journal), impact factor, publication time, etc.; content location information can be further subdivided into title, abstract, keywords, introduction, background, method, key, experiment, discussion, prospect, etc.; literature type information can be further subdivided into clinical trials, randomized controlled trials (Randomized Controlled Trials), etc. Clinical Trial, RCT), meta-analysis, systematic review, review, guideline, others, etc.; disease information can be further subdivided into AD, MCI, tumor, etc.; focus information can be further subdivided into calculation, reasoning, problem solving, decision-making, perception, memory, attention, visual space, execution, learning, language, etc. (taking AD as an example); measurement method information can be further subdivided into imaging, EEG, magnetoencephalogram, scale, etc. (taking AD as an example); link information can be further subdivided into screening, diagnosis, treatment, prognosis, etc.; intervention classification information can be further subdivided into Chinese medicine, Western medicine, surgery, non-drug methods, etc.; intervention information can be further subdivided into Yizhi Jiannao Granules, Yizhi Drink, Rehmannia Yizhi Recipe, etc. (taking Chinese medicine intervention as an example); evaluation standard information can be further subdivided into diagnostic criteria, efficacy evaluation table, etc.; evaluation indicator information can be further subdivided into Mini-mental State Examination (Mini-mental State Examination Examination, MMSE), Montreal Cognitive Assessment (MoCA), etc. (taking AD as an example); report quality evaluation can be further subdivided into dimensions such as experimental design, data analysis, and data interpretation methods; methodological evaluation can be further divided into random sequence generation methods, whether the allocation scheme is hidden, implementation bias, measurement bias, follow-up bias, etc.; comprehensive evaluation information can be further subdivided into quality evaluation, recommendation strength, etc. (taking grade as an example). Moreover, the above categories can be further subdivided according to the situation.

[0044] For the documents in each subject area, according to the above-mentioned knowledge system, each document in the document library is processed by extracting metadata, processing based on rules (pre-set), processing based on a pre-built classification model, processing based on a pre-built information extraction model, etc., and the corresponding label content of each component in the system is obtained and saved in the corresponding fields in the medical subject document library (including the paragraph and structural position, sentence and specific character position, etc., if any), to form a complete medical subject document library.

[0045] Step S1013: For each document in the medical subject literature library, entities in each document are extracted based on a pre-built entity information extraction model to build a medical subject literature entity library.

[0046] In one embodiment, entity categories in the medical subject literature entity library include: diseases, symptoms, examinations, tests, operations, scales, brain regions / parts, drugs, prescriptions, Chinese patent medicines, Chinese herbal medicines, surgeries, adverse reactions, data objects, processing algorithms, etc. For each document in the medical subject literature library, based on a pre-built entity information extraction model (such as an entity extraction model based on seq2seq), entities in the document are extracted, and saved according to the document ID, entity ID, entity category, specific entity, location (including paragraph and structural position, sentence and specific character position, etc.), related document data (including the document library knowledge system label and content corresponding to the document) and other fields to form a medical subject literature entity library.

[0047] Step S1014: For each document in the medical subject literature library, a relationship map between the document and the entity is constructed based on the document and the medical subject literature entity library, and a triple map is constructed based on the relationship between the entities in the document in the medical subject literature entity library, and a medical subject map library is constructed based on the relationship map and the triple map.

[0048] In one embodiment, the medical subject atlas library may be a two-layer structure, including: a first-layer relationship graph consisting of entities and documents, and a second-layer triple graph consisting of entities and relationships between entities within documents.

[0049] In the specific implementation, for each document in the medical subject document library, combined with the above-mentioned medical subject document entity library, a medical subject atlas library is constructed. First, based on the document and the entity inside it, a relationship map between the entity and the document is constructed (the interaction between the document ID and the entity ID, that is, if there is an entity in the document, then this entity and the corresponding document establish a connection relationship, otherwise there is no connection relationship); then based on the relationship between the entities in the document, a triple map is constructed. Specifically, the relationship between the entities in the document can be extracted based on a pre-constructed relationship information extraction model (such as a relationship extraction model based on a pre-trained model such as Bert), and a triple map (interaction between entity IDs) can be constructed. In order to ensure the distinction of the context of the triple, in this embodiment, attribute values ​​can be added to the relationship of the triple (the value domain includes the document ID-can take multiple values, and the relevant document data includes the document library knowledge system label and content corresponding to the document); finally, the extracted map is saved in the medical subject atlas library (such as neo4j).

[0050] Step S1015: For each document in the medical subject literature library, if the document type tag content of the document includes RCT, the evidence information in each document is extracted based on the pre-built evidence extraction model to build a medical subject evidence library; wherein the evidence information is information related to RCT.

[0051] In one embodiment, the medical discipline evidence library includes at least: patient status, diagnostic criteria (such as DSM-IV diagnostic criteria, etc.), grouping situation, grouping basis (such as random number table method, computer random number method, odd-even numbering method, etc.), sample size, demographic data (such as age distribution, gender distribution, etc.), intervention measures (such as placebo treatment, Western medicine treatment, Chinese medicine treatment, etc.), implementation methods (such as type of medicine, dosage, course of treatment), evaluation indicators, group intervention results, adverse events, experimental results, experimental significance, bias risk assessment, etc.

[0052] For each document in the medical subject library, if the document type label of the document includes RCT, evidence information is extracted based on a pre-built evidence extraction model (such as an evidence extraction model based on seq2seq), and saved according to the document ID, evidence ID, evidence field, evidence content, location (including paragraph and structural position, sentence and specific character position, etc.), related document data (including the document library knowledge system label and content corresponding to the document) and other fields to form a medical subject evidence library.

[0053] Step S1016: construct sub-indexes of the medical subject document library, the medical subject document entity library and the medical subject evidence library respectively, and integrate and associate the medical subject document library, the medical subject document entity library and the medical subject evidence library based on the association IDs between the data of the medical subject document library, the medical subject document entity library and the medical subject evidence library to construct an overall index, thereby obtaining a medical subject index library; wherein the sub-index and the overall index include: an inverted index and a vector index.

[0054] In one embodiment, a medical subject index library is constructed. First, sub-indexes are constructed for the medical subject document library, the medical subject document entity library, and the medical subject evidence library. Then, based on the association ID (such as document ID) between the data of the medical subject document library, the medical subject document entity library, and the medical subject evidence library, the data of each library is integrated and associated to construct an overall index for the entire medical subject; secondly, for each sub-index and the overall index, an inverted index and a vector index are constructed respectively. In the process of constructing the vector index, the long text is first segmented (for example, segmented according to a length of 500 characters), and then the segmented text is vectorized (based on a pre-trained embedding model), and then stored in the library respectively. In order to further improve the efficiency and effect of retrieval, in this embodiment, in the process of constructing the vector index, for each segmented text block, the entire document (to which the text block belongs) and the segmented text block can be input into the large model based on the fine-tuned large model, and the context summary for the segmented text block is obtained. Then, the obtained context summary and the segmented text block are connected to obtain the input text for constructing the index. In addition, for the medical subject evidence base, a digital index is also constructed for the digital fields therein.

[0055] Therefore, a medical discipline knowledge base was constructed, which is specifically composed of a medical discipline literature library, a medical discipline literature entity library, a medical discipline atlas library, a medical discipline evidence library, and a medical discipline index library.

[0056] It is worth mentioning that the medical discipline knowledge base includes various parts such as the document library, document entity library, atlas library, evidence library, index library, etc., which will be updated in a timely manner according to the acquisition of source document data, thus ensuring the latest content of the entire medical discipline knowledge base system.

[0057] The aforementioned step S102, i.e., making a preliminary judgment on the query question to determine the subject field and query method to which the query question belongs, comprises: firstly, making a judgment on the subject field to which the query question belongs to determine the subject field to which the query question belongs; then making an objectivity judgment and an answerability judgment on the query question. If the question is an objective question and an answerable question, the query method of the query question is determined based on a pre-constructed query method judgment model.

[0058] In one implementation, for the query question input by the user, the user question is first pre-judged, specifically including: query question objectivity judgment, query question subject field judgment, and query question answerability judgment. The question objectivity judgment is to judge whether the query question is objective and whether it requires a true and definite response (the opposite of this includes subjective questions, questions that have not yet reached a consensus, and imaginary questions, etc.), the question subject field judgment is to judge the subject field to which the query question input by the user belongs, and the question answerability judgment is to judge whether the query question can be answered correctly.

[0059] The above-mentioned objectivity judgment of the query question, the subject field judgment of the query question, and the answerability judgment of the query question can be completed by a pre-built judgment model (the judgment model is fine-tuned based on a pre-trained model such as T5). For questions that are judged as non-objective or non-answerable by the judgment model, the user is directly prompted and then returns; for questions that are judged as objective and answerable by the judgment model, the corresponding medical subject knowledge base and medical subject large language model (select a base model such as llama, and obtain it through steps such as fine-tuning in the entire medical field and fine-tuning in the medical subject) can be called according to the judgment result of the subject field to which the judgment model belongs to perform the following processing.

[0060] Furthermore, the query method of the query question input by the user is judged: the method of answering the question is judged, that is, the method of querying the medical subject knowledge base corresponding to the subject field to which the query question belongs, the method of interacting with the medical subject large language model corresponding to the subject field to which the query question belongs, and the method of combining the query of the medical subject knowledge base corresponding to the subject field to which the query question belongs and the interaction of the medical subject large language model. The query method judgment can be completed by a query method judgment model constructed in advance (the model is fine-tuned based on a pre-trained model such as T5).

[0061] In a possible implementation, based on the query mode, a medical subject knowledge base query corresponding to the subject field to which the query question belongs and / or a medical subject large language model interaction are performed, and the query question results obtained include the following three situations:

[0062] Case 1: If the query method is an interactive combination of querying the medical subject knowledge base corresponding to the subject field to which the query question belongs and the medical subject big language model, the query question is preprocessed, and the medical subject knowledge base corresponding to the subject field to which the query question belongs is queried based on the preprocessed query question to obtain a second query result, and based on the second query result, a first prompt content of the medical subject big language model corresponding to the subject field to which the query question belongs is constructed, and the first prompt content is input into the medical subject big language model to obtain a second feedback result based on the second query result.

[0063] Specifically, for questions that need to be answered correctly based on the medical subject knowledge base corresponding to the subject field to which the query question belongs and the medical subject large language model, the query question is first preprocessed, and then the medical subject knowledge base corresponding to the subject field to which the query question belongs is queried based on the preprocessed query question to obtain the second query result. Specifically, the following steps (1) to (8) are included:

[0064] Step (1): Input the query question into the medical subject large language model corresponding to the subject field to which the query question belongs to perform problem decomposition to obtain a decomposed question, and concatenate the decomposed question with the query question to obtain a concatenated question.

[0065] In the specific implementation, the user's query question Query is input into the medical subject language model corresponding to the subject field to which the query question belongs. By requiring the medical subject language model to solve the problem step by step, the problem is prompted to decompose the problem and obtain the decomposed answer. 1 , and concatenate the decomposition problem and the query problem to obtain the concatenated problem Query con =(Query, Answer 1 ).

[0066] Step (2): Perform word segmentation and keyword extraction on the concatenation problem to obtain a keyword list, and input the keyword list into a pre-built synonym large language model to obtain a synonym list of the keywords.

[0067] In the specific implementation, a word segmenter (such as Jieba) is used to query the splicing problem con Perform word segmentation and extract keywords (such as Textrank algorithm) to obtain a keyword list Query syn , and input the keyword into a fine-tuned synonym large language model (a base model such as llama can be used) to obtain a synonym list of the keyword Query synon (Including keyword list Query syn ).

[0068] Step (3): Extract the entities of the query question based on the pre-built information extraction model to obtain an entity list.

[0069] In the specific implementation, for the query question Query input by the user, the pre-established information extraction model is used to extract entities to obtain the entity list Query entity .

[0070] Step (4): Based on the pre-built judgment model, the query condition judgment is performed on the query question to obtain the query condition result; wherein the query condition at least includes: the document type to which the query content belongs, and the location of the query content in the document.

[0071] In the specific implementation, the query question Query input by the user is judged by using a pre-established judgment model (which can be based on a pre-trained model, such as T5 model) to obtain the query condition result Query condition Among them, the query conditions at least include: the possible document type to which the query content may belong, the possible location of the query content in the document, such as possible location: abstract, keywords, introduction, background, methods, key, experiment, discussion, outlook, or possible document type: clinical trial, RCT, meta-analysis, systematic review, review, guideline, others, etc.

[0072] Step (5): Based on the synonym list of the keywords, the query condition results and the splicing problem, the first query logic and the second query logic are respectively constructed for the inverted index and the vector index of the medical subject literature library, and the inverted index query and the vector index query are respectively performed based on the first query logic and the second query logic to obtain the literature query results.

[0073] In specific implementation, a query is first performed on the medical subject literature library, including: a query on the inverted index of the medical subject literature library and a query on the vector index of the medical subject literature library.

[0074] Query the inverted index of the medical subject literature library, and input the synonym list of the keyword Query synon And the query condition results Query condition , the logical relationship between keywords is logical OR, the logical relationship between query conditions is logical OR, and the logical relationship between keywords and query conditions is logical AND. The final first query logic is: And(Or(Query synon ), Or(Query condition )) Based on the first query logic, perform an inverted index query of the medical subject literature library to obtain the sorted topK (pre-set, such as 10) query results Search doc-inv .

[0075] Query the vector index of the medical subject literature library, and input the concatenation problem Query con and query condition results Query condition , the logical relationship between the query conditions is logical OR, and the logical relationship between the concatenation question and the query condition is logical AND, so the final second query logic is: AND(Query con ,Or(Query condition)) Based on the second query logic, a vector index query is performed on the medical subject literature library to obtain the sorted topK (pre-set, for example, 10) query results Search doc-vec In this embodiment, the query result Search doc-inv and the query results doc-vec Together they constitute the literature search results.

[0076] Step (6): construct a third query logic for the medical subject literature entity library based on the entity list and the query condition results, and perform a query on the medical subject literature entity library based on the third query logic to obtain an entity query result.

[0077] In the specific implementation, the query is performed on the medical subject literature entity library, and the input is the entity list Query entity , Query condition results condition , the logical relationship between entity lists is logical OR, the logical relationship between query conditions is logical OR, and the logical relationship between entity lists and query conditions is logical AND. The final third query logic is: And(Or(Query entity ), Or(Query condition )) Based on the third query logic, the medical subject literature entity database is searched to obtain the sorted topK (pre-set, such as 10) entity query results Search entity (Including: document content obtained through document ID association, etc.).

[0078] Step (7): If the document type in the query condition result includes RCT, a fourth query logic for the medical subject evidence library is constructed based on the entity list and the synonym list of the keywords, and the medical subject evidence library is queried based on the fourth query logic to obtain the evidence query result.

[0079] In the specific implementation, in the query condition result Query condition In the query, if the result of judging RCT in the document type information is true (indicated by 1), that is, the document type in the query condition result includes RCT, then the medical subject evidence database is queried, and the input is the entity list Query entity 、Keyword synonym list Query synon , the logical relationship between entity list queries is logical OR, the logical relationship between keyword list queries is logical OR, and the logical relationship between entity list queries and keyword list queries is logical OR, then the final fourth query logic is: Or(Or(Query entity ), Or(Query synon)) Based on the fourth query logic, the medical discipline evidence database is queried to obtain the sorted topK (pre-set, such as 10) evidence query results Search evidence (Including: the names and content of each relevant field in the medical discipline evidence library, etc.).

[0080] Step (8): Input the query question, document query results, entity query results and evidence query results into a pre-built evaluation and scoring model to obtain a scoring result for each query result, and determine the second query result based on the scoring result.

[0081] In the specific implementation, for the query results obtained by the above query, each search result is evaluated and scored by the pre-built evaluation and scoring model, where the evaluation and scoring model consists of two sub-models Model q 、Model a The two models can have the same structure (such as siamese, etc.), but the training corpus is different. Each training corpus of the former is composed of (question 1: question 2) (the semantics of the two questions in the positive sample are similar, while those in the negative sample are different), and each training corpus of the latter is composed of (question: answer) (the content of the question and answer in the positive sample is matched, while that in the negative sample is not matched). The training goal of the former is to maximize the similarity index of similar (positive sample) questions, and the training goal of the latter is to maximize the correlation index between the positive question and its answer.

[0082] The input of the evaluation and scoring model is the query question Query entered by the user, each of the above query results (including literature query results, entity query results and evidence query results), and the output scoring results are Score q (i.e. sub-model Model q Scoring results), Score a (i.e. sub-model Model a The scoring result of the query result is obtained by q *Score q +W a *Score a (Weight W q and W a can be determined in advance).

[0083] For each query result, you can sort them from large to small according to the above scoring results, and then intercept the scoring results greater than or equal to the pre-set threshold L min The top L (pre-set) results of the query are used as the second query result Search last If the second query result is lastIf the result is empty, step S103 is repeated. If the result is empty again, the user is directly fed back a "cannot answer" result.

[0084] Case 2: If the query method is a method of querying the medical subject knowledge base corresponding to the subject field to which the query question belongs, then the medical subject knowledge base corresponding to the subject field to which the query question belongs is queried based on the query question to obtain a first query result.

[0085] Specifically, if the query mode judgment model determines that a correct answer can be obtained by simply querying the medical subject knowledge base corresponding to the subject field to which the query question belongs, the first query result is obtained by directly querying the medical subject knowledge base.

[0086] In one embodiment, the following methods may be used, including but not limited to:

[0087] First, the query question Query entered by the user is segmented using a word segmenter (such as Jieba), and keywords are extracted (for example, using the Textrank algorithm) to obtain a keyword list Query syn1 , and input the keyword into the fine-tuned synonym language model to obtain the keyword synonym list Query synon1 (Including keyword list Query syn1 ). Secondly, for the query question Query input by the user, the pre-established information extraction model is used to extract entities to obtain the entity list Query entity1 Next, based on the synonym list of the keyword, Query synon1 and Entity List Query entity1 Construct query logic for medical subject literature database (Query based on synonym list of keywords) synon1 ), query logic for medical literature entity library (based on entity list query entity1 ), and query logic for medical subject evidence base (keyword-based synonym list query synon1 and Entity List Query entity1 ). Then, based on the query logic for the medical subject document library, a medical subject document library query is performed to obtain a document query result, based on the query logic for the medical subject document entity library, a medical subject document entity library query is performed to obtain an entity query result, and based on the query logic for the medical subject evidence library, a medical subject evidence library query is performed to obtain an evidence query result. Finally, the query question, document query result, entity query result, and evidence query result are input into a pre-constructed evaluation and scoring model (which may be the same as the evaluation and scoring model in the aforementioned example) to obtain a scoring result for each query result, and the query result is determined based on the scoring result.

[0088] In this embodiment, query logic is constructed, and queries are performed on the medical subject literature library, the medical subject literature entity library, and the medical subject evidence library. After the query results are merged and sorted, the top M (pre-set) results are returned to the user. The specific implementation method is similar to the previous embodiment and will not be repeated here.

[0089] Case three: If the query method is based on the interaction with the medical discipline big language model corresponding to the discipline field to which the query question belongs, the query question is input into the medical discipline big language model corresponding to the discipline field to which the query question belongs to obtain the first feedback result.

[0090] If the query judgment model determines that the correct answer can be obtained based on the medical subject large language model corresponding to the subject field to which the query question belongs, the query question is directly sent to the medical subject large language model to obtain the first feedback result Re.

[0091] In one embodiment, after obtaining the second query result based on the above step S103, a first prompt content of the medical discipline language model corresponding to the subject field to which the query question belongs can be constructed based on the second query result, and the first prompt content is input into the medical discipline language model to obtain second feedback content based on the second query result. Specifically, in this embodiment, the first prompt content prompt constructed for interacting with the medical discipline language model includes: roles, tasks, requirements, reference materials, related prompts, etc.

[0092] In one embodiment, when constructing the first prompt content of the medical subject language model corresponding to the subject field to which the query question belongs based on the second query result, the following methods may be used, including but not limited to:

[0093] (1) Clarify the role of the medical subject language model corresponding to the subject area of ​​the query question. For example, you are an expert in the field of tumors.

[0094] (2) Clarify the task of this dialogue, including: listing the query questions input by the user (including question decomposition) and clarifying the tasks. For example: requiring the user to understand and respond to the knowledge in the subject area to which the query question belongs based on the user's query question and related reference materials.

[0095] (3) Clarify relevant requirements. For example, the word limit for responses, requiring factual answers - if you don't know, just reply "I don't know", etc.

[0096] (4) List relevant information, including: references related to the query question entered by the user.

[0097] In the specific implementation, the reference material sorting process is as follows: First, for the second query result Search lastIf the query result corresponds to a document, the document content needs to be summarized according to the abstract extraction algorithm (such as cluster-based abstract extraction) to compress the content length so that it does not exceed the limit (the allowed length of each reference (i.e., the preset length) is obtained based on the total length of the allowed references / the number of references). doc ); If the query result corresponds to the pre-split content, if the content length does not exceed the above allowed length Length doc , no processing is done, otherwise compression is also done; if the search result corresponds to the content of the evidence library, the evidence content is combined into a natural language sentence in the form of (evidence field: evidence content, ...). Then, according to the score of each query result, it is arranged in descending order to form a reference list serving the prompt.

[0098] (5) List relevant prompts (e.g., keywords and entities related to the user's query question Query). Here, the keyword list Query obtained in the previous processing process is mainly clarified. syn , and entity list Query entity wait.

[0099] Thus, a complete first prompt is formed to interact with the large medical subject language model corresponding to the subject area to which the query question belongs.

[0100] It should be noted that in the case where it is judged that "only a large language model of the medical discipline is needed to give a correct answer", the prompt content includes the first three items mentioned above.

[0101] The aforementioned step S105, i.e., performing relevance evaluation and scoring on the second feedback result, includes the following steps a1 to a3:

[0102] Step a1: Filter the second feedback result based on a preset rule.

[0103] In one embodiment, the second feedback result Re 2th Content checking and filtering are mainly completed according to pre-set rules. The rules constrain relatively clear knowledge: for example, in the field of AD, recommendations for AD drugs must be consistent with the guidelines, and discriminatory language and results must not appear. Content that does not comply with the rules will be dealt with accordingly.

[0104] Step a2: input the filtered second feedback result into the pre-built first relevance evaluation model to obtain the relevance evaluation score of the second feedback result.

[0105] In one embodiment, the filtered second feedback result is evaluated and scored. In this embodiment, the relevance evaluation score consists of two parts: on the one hand, it is given by the medical discipline language model that generates the second feedback result (by constructing a prompt consisting of roles, tasks-providing a relevance score between 0 and 100, user query questions, second feedback results and reference materials, etc., and sending it to the medical discipline language model to obtain the relevance evaluation score), specifically including: the relevance evaluation score Relevance between the second feedback result and the query question entered by the user mo-re-qu1 , and the relevance evaluation score between the second feedback result and the query result of the reference material listed in the prompt mo-re-se1 (wherein the score is the weighted average of the relevance evaluation scores of the second feedback result and each query result), and then according to the pre-set weight (W mo-rq1 and W mo-rs1 ) The above two correlation evaluation scores are weighted and calculated to obtain the correlation evaluation score given by the medical discipline large language model:

[0106] Relevance model1 =W mo-rq1 *Relevance mo-re-qu1 +W mo-rs1 *Relevance mo-re-se1 .

[0107] On the other hand, the relevance evaluation score Relevance between the second feedback result and the query question input by the user is obtained by the pre-constructed first relevance evaluation model sub-model (which can be based on a pre-trained model, such as T5 tuning). s -re-qu1 , and the relevance evaluation score of the second feedback result and the query result of the reference material listed in the prompt s-re-se1 (wherein the score is a weighted average of the relevance evaluation scores of the second feedback result and each query result), and according to the pre-set weight (W s-rq1 and W s-rs1 ) The above two correlation evaluation scores are weighted and calculated to obtain the correlation evaluation score given by the first correlation evaluation model sub-model:

[0108] Relevance s1 =W s-rq1 *Relevance s-re-qu1 +W s-rs1 *Relevance s-re-se1 .

[0109] Finally, the relevance evaluation score of the second feedback result for the medical subject large language model feedback is obtained:

[0110] Relevance1=(Relevance s1 +Relevance model1 ) / 2(During the evaluation process, obtain and record the second feedback result Re 2th for each sentence in the text and the corresponding reference number and content that exceeds a pre-set threshold).

[0111] Step a3: If the relevance evaluation score of the second feedback result is greater than or equal to the second threshold and less than the first threshold, the second feedback result is evaluated and scored for consistency; wherein the first threshold is greater than the second threshold; if the relevance evaluation score of the second feedback result is less than the second threshold, a third feedback result is obtained based on the large language model of the medical discipline, and a secondary relevance evaluation score is performed on the third feedback result; if the relevance evaluation score of the second feedback result is greater than or equal to the first threshold, the second feedback result is fed back to the user.

[0112] In one embodiment, the relevance evaluation score of the second feedback result given by the first relevance evaluation model and two pre-set thresholds (the first threshold Re max >Second threshold Re min ) is processed according to the situation. If the relevance evaluation score of the second feedback result is greater than or equal to the first threshold Re max , the second feedback result is directly fed back to the user; if the relevance evaluation score of the second feedback result is greater than or equal to the second threshold Re min But less than the first threshold Re max , then the second feedback result is scored for consistency evaluation; if the relevance evaluation score of the second feedback result is less than the second threshold Re min , then the third feedback result is obtained based on the medical discipline language model, and a secondary correlation evaluation and scoring is performed on the third feedback result.

[0113] In a possible implementation, obtaining a third feedback result based on the medical discipline large language model and performing a secondary relevance evaluation and scoring on the third feedback result include the following steps b1 to b8:

[0114] Step b1, performing entity matching based on each entity in the entity list and the medical subject atlas library to obtain a matching entity list and a document set corresponding to the matching entity.

[0115] In the specific implementation, the matching method is adopted to query the entity list entityEach entity in the medical subject atlas is matched with the medical subject atlas to obtain a list of successfully matched entities and a collection of documents corresponding to the matching entities. entity If the matching entity list is empty, step S103 is repeated. If the matching entity list is empty again, a "cannot answer" result is directly fed back to the user.

[0116] Step b2: based on the matching entity list, entities adjacent to the matching entity are extracted from the document collection to obtain entity triples, and the entity triples are filtered.

[0117] In the specific implementation, for the matching document set Docum entity , starting from the matching entity, extract the entities adjacent to the matching entity in each document to obtain entity triples, and filter the extracted entity triples to ensure that the entities in the entity triples must contain the corresponding entities.

[0118] Step b3, repeatedly extracting and filtering entity triples until a preset number of times is reached to obtain a first entity triple set.

[0119] In the specific implementation, the triple extraction is continued until a preset number (N) is reached, thereby obtaining the first entity triple set Triple entity .

[0120] Step b4, input the query question into the pre-built answer question pattern prediction model to perform answer question pattern prediction, and obtain a pattern library composed of multiple answer question patterns; wherein the answer question pattern is the entity relationship of the answer question.

[0121] In the specific implementation, the answer question pattern prediction is performed. Specifically, the query question Query input by the user is input into the pre-built answer question pattern prediction model (which can be fine-tuned based on the pre-trained model, such as the T5 model), and a pattern library Pattern consisting of multiple answer question patterns is obtained. query Among them, the question answering pattern is the entity relationship of answering questions combined with key entities, such as (drug, treatment, disease).

[0122] Step b5, based on the pre-built pre-trained model, vectorize each entity relationship in the pattern library and each entity triple relationship in the first entity triple set, and perform similarity calculation on the entity relationships and entity triple relationships in the vectorized pattern library, and intercept a preset number of entity triples based on the similarity calculation results to obtain a second entity triple set.

[0123] In the specific implementation, for the pattern library Pattern queryEach entity relationship in the pattern (such as "treatment" in (drug, treatment, disease)) and the first entity triple set Triple entity The relationship in each entity triple in (such as "therapeutic drug" in (memantine, therapeutic drug, Alzheimer's disease)) is vectorized based on the pre-built pre-trained model, and the similarity is calculated between the entity relationship of the vectorized pattern library and the relationship of the entity triple (for example, the cosine function is used to calculate the similarity), and they are sorted according to the similarity score, and the pre-selected number (P) of entity triplets is intercepted to obtain the second entity triple set Triple filterd In this embodiment, according to the need of answering the question, in addition to calculating the similarity, antonym calculation, implication calculation, etc. can also be performed.

[0124] Step b6: construct the second prompt content of the medical subject large language model based on the second entity triple set, and input the second prompt content into the medical subject large language model to obtain a third feedback result.

[0125] In the specific implementation, the second prompt content prompt of the medical subject large language model is constructed. The construction of the prompt is similar to the above steps, wherein the reference material here is the extracted second entity triple set Triple filterd , the triples are combined into natural language sentences in the form of (subject, relation, object). Specifically, the original text content associated with the triples can be included in the reference material as needed.

[0126] Then, the constructed second prompt content prompt is sent to the medical subject language model, and the third feedback result Re returned by the medical subject language model is obtained. 3th .

[0127] Step b7: filtering the third feedback result based on a preset rule, and inputting the filtered third feedback result into a pre-constructed second relevance evaluation model to obtain a relevance evaluation score of the third feedback result.

[0128] In the specific implementation, the third feedback result is first checked and filtered based on the pre-set rules, and then the filtered third feedback result is evaluated and scored based on the pre-built second relevance evaluation model. The evaluation and scoring results are composed of two parts: on the one hand, it is given by the medical discipline big language model that generates the third feedback result (by providing a prompt consisting of roles, tasks-providing a relevance score between 0 and 100, user query questions, third feedback results and reference materials, etc., and sending it to the medical discipline big language model to obtain the relevance evaluation score), specifically including: the relevance evaluation score Relevance for the third feedback result and the query question entered by the user mo-re-qu2 , and the relevance evaluation score Relevance between the third feedback result and the query result of the reference material listed in the prompt (determined according to the second entity triple set) mo-re-se2 (wherein the score is the weighted average of the relevance evaluation scores of the third feedback result and each query result), and then according to the pre-set weight (W mo-rq2 and W mo-rs2 ) The above two correlation evaluation scores are weighted and calculated to obtain the correlation evaluation score given by the medical discipline large language model:

[0129] Relevance model2 =W mo-rq2 *Relevance mo-re-qu2 +W mo-rs2 *Relevance mo-re-se2 .

[0130] On the other hand, the relevance evaluation score Relevance between the third feedback result and the query question input by the user is obtained by the pre-constructed second relevance evaluation model sub-model s-re-qu2 , and the relevance evaluation score Relevance between the third feedback result and the query result of the reference material listed in the prompt (determined according to the second entity triple set) s-re-se2 (Wherein, the score is a weighted average of the relevance evaluation scores of the third feedback result and each query result), and according to the pre-set weight (W s-rq2 and W s-rs2 ) The above two correlation evaluation scores are weighted and calculated to obtain the correlation evaluation score given by the correlation evaluation model:

[0131] Relevance s2 =W s-rq2 *Relevance s-re-qu2 +W s-rs2 *Relevance s-re-se2 .

[0132] Finally, the relevance evaluation score of the third feedback result for the medical discipline large language model feedback is obtained:

[0133] Relevance2=(Relevance s2 +Relevance model2 ) / 2(In the process, record the third feedback result Re 3th for each sentence in the text and the corresponding reference number and content that exceeds a pre-set threshold).

[0134] Step b8, if the relevance evaluation score of the third feedback result is greater than or equal to the first threshold, the third feedback result is fed back to the user; if the relevance evaluation score of the third feedback result is greater than or equal to the second threshold and less than the first threshold, the third feedback result is scored for consistency evaluation; if the relevance evaluation score of the third feedback result is less than the second threshold, the first prompt content of the medical subject large language model is reconstructed to obtain a new second feedback result, and the new second feedback result is scored for relevance evaluation; if the relevance evaluation score of the new second feedback result is less than the second threshold, the second prompt content of the medical subject large language model is reconstructed to obtain a new third feedback result; if the relevance evaluation score of the new third feedback result is less than the second threshold, the query question is preprocessed again, and based on the preprocessed query question, the medical subject knowledge base corresponding to the subject field to which the query question belongs is queried and the medical subject large language model is interacted to obtain a new second feedback result.

[0135] In the specific implementation, according to the scoring result of the third feedback result given by the second correlation evaluation model and the two pre-set thresholds (the first threshold Re max >Second threshold Re min ) is processed according to the situation. If the relevance evaluation score of the third feedback result is greater than or equal to the first threshold Re max , the third feedback result is directly fed back to the user; if the relevance evaluation score of the third feedback result is greater than or equal to the second threshold Re min But less than the first threshold Re max , then the third feedback result is scored for consistency evaluation; if the relevance evaluation score of the third feedback result is less than the second threshold Re min, then the first prompt content of the medical discipline large language model is returned to be reconstructed (at this time, the relevant prompt also includes the evaluation and scoring of the previous third feedback result content), a new second feedback result is obtained, and the new second feedback result is evaluated and scored for relevance. If the relevance evaluation score of the new second feedback result is less than the second threshold, the second prompt content of the medical discipline large language model is reconstructed to obtain a new third feedback result, and the new third feedback result is evaluated and scored for relevance twice; if the relevance evaluation scores of the two (configurable) third feedback results are both less than the second threshold Re min , then return to step (1) to re-preprocess the query question (at this time, the relevant prompt also includes the evaluation and scoring of the third feedback result content before), if the relevance evaluation scores of the third feedback results returned three times (configurable) are all less than the second threshold Re min , it will directly return to the user "cannot answer".

[0136] In a possible implementation, performing consistency evaluation and scoring on the second feedback result or the third feedback result includes the following steps c1 to c4:

[0137] Step c1, constructing a key summary prompt based on the second feedback result or the third feedback result, and inputting the key summary prompt into the medical subject large language model to obtain the key summary result of the second feedback result or the third feedback result.

[0138] In a specific implementation, the second feedback result Re for the query question Query input by the user obtained according to the above steps 2th Or the third feedback result Re 3th , construct a key summary prompt (including: roles, tasks-requirements combined with user query questions, reference materials, second feedback results or third feedback results, etc. to generate key summary), and send the key summary prompt to the medical discipline language model to generate a key summary for the second feedback result Re 2th Or the third feedback result Re 3th Key points of the results are summarized in sum .

[0139] Step c2: scoring the key summary result of the second feedback result or the third feedback result based on the pre-built consistency evaluation model to obtain a consistency evaluation score of the key summary result of the second feedback result or the third feedback result.

[0140] In the specific implementation, the key points of the second feedback result or the third feedback result of the model feedback are summarized based on the pre-built consistency evaluation model. sumThe consistency evaluation score is composed of two parts. On the one hand, it is given by the medical discipline language model that generates the key summary results. By constructing a prompt consisting of roles, tasks, providing a relevance score between 0 and 100, user query questions, second feedback results or third feedback results, and key summary results, it is sent to the medical discipline language model to obtain the consistency evaluation score of the key summary results, that is, the consistency evaluation score of the key summary results and the second feedback results Re 2th Or the third feedback result Re 3th Relevance model -sum On the other hand, the second feedback result Re is obtained by the pre-built consistency evaluation model sub-model (which can be obtained based on the pre-trained model tuning, such as the T5 model). 2th Or the third feedback result Re 3th Relevance of consistency evaluation score with key summary results s-sum ; Finally, we get the consistency evaluation score of the summary results of the key points of the large language model for medical disciplines: Relevance sum =(Relevance s-sum +Relevance model-sum ) / 2.

[0141] Step c3, if the consistency evaluation score of the key summary result of the second feedback result is greater than or equal to the third threshold, the second feedback result is fed back to the user; if the consistency evaluation score of the key summary result of the second feedback result is less than the third threshold, the first prompt content of the medical subject large language model is reconstructed to obtain a new second feedback result, if the relevance evaluation score of the new second feedback result is greater than or equal to the second threshold and less than the first threshold, the new second feedback result is scored for consistency evaluation, if the consistency evaluation score of the key summary result of the new second feedback result is less than the third threshold, the query question is preprocessed again, and based on the preprocessed query question, the medical subject knowledge base corresponding to the subject field to which the query question belongs is queried and the medical subject large language model is interacted to obtain a new second feedback result.

[0142] In the specific implementation, the consistency evaluation score of the result is summarized according to the key points of the second feedback result and the third threshold value (Re sum-min ) is processed according to the situation: if the consistency evaluation score of the key summary result of the second feedback result is greater than or equal to the third threshold value Re sum-min, the second feedback result is directly fed back to the user; otherwise, the first prompt content of reconstructing the medical discipline large language model is returned (at this time, the relevant prompt also includes the consistency evaluation score for the previous feedback result), and a new second feedback result is obtained. If the relevance evaluation score of the new second feedback result is greater than or equal to the second threshold and less than the first threshold, the new second feedback result is evaluated and scored for consistency. If the consistency evaluation scores of the key summary results of the second feedback results returned twice (configurable) are both less than the third threshold Re sum-min , then return to step (1) to re-preprocess the query question (at this time, the relevant prompts also include the consistency evaluation scores for the previous feedback results). If the consistency evaluation scores of the key summary results of the new second feedback results returned three times (configurable) are all less than the third threshold Re sum-min , it will directly return to the user "cannot answer".

[0143] Step c4, if the consistency evaluation score of the key summary result of the third feedback result is greater than or equal to the third threshold, the third feedback result is fed back to the user; if the consistency evaluation score of the key summary result of the third feedback result is less than the third threshold, the first prompt content of the medical subject large language model is reconstructed to obtain a new second feedback result, if the relevance evaluation score of the new second feedback result is less than the second threshold, the second prompt content of the medical subject large language model is reconstructed to obtain a new third feedback result, if the relevance evaluation score of the new third feedback result is greater than or equal to the second threshold and less than the first threshold, the new third feedback result is scored for consistency evaluation, if the consistency evaluation score of the key summary result of the new third feedback result is less than the third threshold, the query question is re-preprocessed, and based on the preprocessed query question, the medical subject knowledge base corresponding to the subject field to which the query question belongs is queried and the medical subject large language model is interacted to obtain a new second feedback result.

[0144] In the specific implementation, the consistency evaluation score of the result is summarized according to the key points of the third feedback result finally obtained and the third threshold value (Re sum-min ) is processed according to the situation: if the consistency evaluation score of the key summary result of the third feedback result is greater than or equal to the third threshold value Re sum-min, the third feedback result is directly fed back to the user; otherwise, the first prompt content of the reconstructed medical discipline large language model is returned (at this time, the relevant prompt also includes the consistency evaluation score for the previous feedback result), and a new second feedback result is obtained, and the new second feedback result is evaluated and scored for relevance; if the relevance evaluation score of the new second feedback result is less than the second threshold, the second prompt content of the medical discipline large language model is reconstructed to obtain a new third feedback result, and a secondary relevance evaluation score is performed on the new third feedback result. If the relevance evaluation score of the new third feedback result is greater than or equal to the second threshold and less than the first threshold, the new third feedback result is evaluated and scored for consistency. If the consistency evaluation scores of the key summary results of the third feedback results returned twice (configurable) are both less than the third threshold Re sum-min , then return to step (1) to re-preprocess the query question (at this time, the relevant prompts also include the consistency evaluation scores for the previous feedback results). If the consistency evaluation scores of the third feedback results returned three times (configurable) are all less than the third threshold Re sum-min , it will directly return to the user "cannot answer".

[0145] Further, for the second feedback result or the third feedback result that has passed the evaluation and scoring, a reference mark is generated in the second feedback result or the third feedback result according to each sentence and the corresponding reference material number in the recorded second feedback result or the third feedback result, and the reference material content and the reference material number (i.e., the document ID) are listed after the second feedback result or the third feedback result, thereby generating the final feedback result Re last , to provide feedback to users.

[0146] Compared with the prior art, the above method provided by the embodiment of the present invention has the following advantages and positive effects: (1) By constructing a real-time updated, multi-dimensional medical knowledge base including documents, entities, graphs, evidence, indexes, etc., it ensures that the medical knowledge in different disciplines is multi-dimensional, cross-granular, comprehensive and up-to-date; (2) Based on the knowledge structure of document tags, entities, graphs, evidence, etc. of the multi-dimensional knowledge base, and a fine-tuned large language model, a combination of knowledge and data-driven methods is adopted to ensure the automation and human-like effect of the medical knowledge service process in different disciplines; (3) Pre-judgment of the user's query query is performed (including query method judgment), matching the complexity of the question with the answer method, ensuring the accuracy and effect on the basis of ensuring the efficiency of knowledge service; (4) by optimizing the user's query, optimizing the knowledge query, and optimizing the prompt construction and feedback results, the accuracy of the reply feedback results is further guaranteed; (5) For more complex questions, the present invention can be combined with the constructed medical subject atlas library, and a method combined with the prediction of the answer question pattern is used to extract more accurate and specific reference materials for feedback result generation, further ensuring the accuracy of the query results; (6) by using the user's query query and query optimization The method of evaluating and scoring the correlation between the query results, the query results and the feedback results, the query results and the feedback results, and the feedback results and the key summary results ensures the self-consistency from the user's query to the feedback results and the key summary results. The method of giving the feedback result of "unable to answer" in the case of low score is adopted to ensure the accuracy of the reply result; (7) through step-by-step (loop-based) planning (based on the decomposition and processing of the problem, etc.), optimization (based on the evaluation of previous results), evaluation of interaction results and evaluation-based routing selection methods, the detailed decomposition and self-optimization of the knowledge service process are promoted, while ensuring While ensuring the efficiency of the process, the quality and effect of the knowledge service process are also guaranteed; (8) By adding reference marks and reference materials to the content of the feedback results, the interpretability of the feedback results is enhanced and the basis of the feedback results is explained; (9) Combining the knowledge base and query process with the graph structure can greatly improve the accuracy of answers to details and more advanced focus questions; (10) Based on the above process, the present invention can be widely used in different medical disciplines to obtain literature, obtain evidence, read literature, obtain research ideas, research design, data processing, and generate results, etc., reducing the workload and complexity of researchers in the medical discipline field and improving efficiency, quality and effect.

[0147] Regarding the medical subject knowledge service method provided in the above embodiment, the present invention also provides a medical subject knowledge service device, see Figure 2The schematic diagram of the structure of a medical subject knowledge service device shown in the figure shows that the device mainly includes the following parts:

[0148] The medical subject knowledge base construction module 201 is used to obtain medical literature and construct medical subject knowledge bases corresponding to various subject areas based on the medical literature; wherein the medical subject knowledge base includes: a medical subject literature base, a medical subject literature entity base, a medical subject evidence base, a medical subject atlas base, and a medical subject index base;

[0149] The pre-judgment module 202 is used to obtain the query question input by the user, and pre-judgment the query question to determine the subject field and query method to which the query question belongs;

[0150] The query and interaction module 203 is used to query the medical subject knowledge base corresponding to the subject field to which the query question belongs and / or interact with the medical subject large language model based on the query method to obtain the result of the query question; wherein the result of the query question includes one of the following: a first query result, a first feedback result, a second query result, and a second feedback result based on the second query result;

[0151] The feedback and evaluation module 204 is used for feeding back the first query result or the first feedback result to the user if a first query result or a first feedback result is obtained; if a second feedback result is obtained, performing a relevance evaluation and scoring on the second feedback result, and when the scoring result of the relevance evaluation and scoring of the second feedback result meets the first preset condition, performing a consistency evaluation and scoring on the second feedback result; when the scoring result of the relevance evaluation and scoring of the second feedback result meets the second preset condition, obtaining a third feedback result based on the medical discipline large language model, and performing a secondary relevance evaluation and scoring on the third feedback result; when the scoring result of the secondary relevance evaluation and scoring of the third feedback result meets the first preset condition, performing a consistency evaluation and scoring on the third feedback result; when the scoring result of the consistency evaluation and scoring of the second feedback result or the third feedback result meets the third preset condition, feeding back the second feedback result or the third feedback result to the user.

[0152] The above-mentioned medical subject knowledge service device provided by the embodiment of the present invention can ensure the comprehensiveness and timeliness of knowledge in different subject areas by constructing a medical subject knowledge base in different subject areas that is updated in real time and includes a medical subject literature base, a medical subject literature entity base, a medical subject evidence base, a medical subject atlas base, and a medical subject index base; by pre-judging the query question input by the user, determining the subject area and query method to which it belongs, and then querying and / or interacting based on the medical subject knowledge base and / or the medical subject large language model corresponding to the subject area to which it belongs, it can improve the accuracy and quality of the feedback results on the basis of improving service efficiency; finally, by performing relevance evaluation and scoring, secondary relevance evaluation and scoring, and consistency evaluation and scoring on the query results and feedback results, the results that meet the requirements are fed back to the user, further improving the accuracy and quality of obtaining medical subject-related knowledge.

[0153] It should be noted that the implementation principle and technical effects of the device provided in the embodiment of the present invention are the same as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0154] An embodiment of the present invention further provides an electronic device. Specifically, the electronic device includes a processor and a storage device. The storage device stores a computer program, and when the computer program is executed by the processor, it executes the method described in any one of the above implementation methods.

[0155] Figure 3 A structural diagram of an electronic device provided in an embodiment of the present invention, the electronic device 100 includes: a processor 30, a memory 31, a bus 32 and a communication interface 33, wherein the processor 30, the communication interface 33 and the memory 31 are connected via the bus 32; the processor 30 is used to execute an executable module stored in the memory 31, such as a computer program.

[0156] The memory 31 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 33 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0157] The bus 32 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0158] Among them, the memory 31 is used to store programs, and the processor 30 executes the program after receiving the execution instruction. The method executed by the device for flow process definition disclosed in any embodiment of the above-mentioned embodiments of the present invention can be applied to the processor 30 or implemented by the processor 30.

[0159] The processor 30 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 30. The above processor 30 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiments of the present invention can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 31, and the processor 30 reads the information in the memory 31 and completes the steps of the above method in combination with its hardware.

[0160] The computer program product of the readable storage medium provided in the embodiment of the present invention includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be referred to the previous method embodiments, which will not be repeated here.

[0161] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0162] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A medical subject knowledge service method, characterized in that: include: Acquire medical literature, and construct medical subject knowledge bases corresponding to various subject areas based on the medical literature; wherein the medical subject knowledge base includes: a medical subject literature base, a medical subject literature entity base, a medical subject evidence base, a medical subject atlas base, and a medical subject index base; Obtaining a query question input by a user, and pre-judging the query question to determine the subject field and query method to which the query question belongs; Based on the query method, a medical subject knowledge base query corresponding to the subject field to which the query question belongs and / or a medical subject large language model interaction are performed to obtain a result of the query question; wherein the result of the query question includes one of the following: a first query result, a first feedback result, a second query result, and a second feedback result based on the second query result; If the first query result or the first feedback result is obtained, feeding back the first query result or the first feedback result to the user; If the second feedback result is obtained, a relevance evaluation score is performed on the second feedback result, and when the score result of the relevance evaluation score of the second feedback result meets the first preset condition, a consistency evaluation score is performed on the second feedback result, and when the score result of the relevance evaluation score of the second feedback result meets the second preset condition, a third feedback result is obtained based on the medical subject large language model, and a secondary relevance evaluation score is performed on the third feedback result; When the scoring result of the secondary relevance evaluation of the third feedback result meets the first preset condition, performing consistency evaluation and scoring on the third feedback result; When the consistency evaluation score of the second feedback result or the third feedback result meets the third preset condition, the second feedback result or the third feedback result is fed back to the user.

2. The method according to claim 1, characterized in that Based on the medical literature, a medical knowledge base corresponding to each subject area is constructed, including: Classifying the medical literature according to subject areas to obtain an initial medical subject literature library in different subject areas; For each document in the initial medical subject document library, the document is processed based on a pre-built metadata extraction model, knowledge rules, classification model and information extraction model, and a medical subject document library is constructed according to the content and metadata corresponding to each tag of each document; For each document in the medical subject document library, entities in each document are extracted based on a pre-constructed entity information extraction model to construct a medical subject document entity library; For each document in the medical subject document library, a relationship map between the document and the entity is constructed based on the document and the medical subject document entity library, and a triple map is constructed based on the relationship between the entities in the document in the medical subject document entity library, and a medical subject map library is constructed based on the relationship map and the triple map; For each document in the medical subject literature library, if the document type tag content of the document includes RCT, then extract the evidence information in each document based on the pre-built evidence extraction model to build a medical subject evidence library; wherein the evidence information is information related to RCT; Sub-indexes of the medical subject document library, the medical subject document entity library and the medical subject evidence library are respectively constructed, and based on the association IDs between the data of the medical subject document library, the medical subject document entity library and the medical subject evidence library, the medical subject document library, the medical subject document entity library and the medical subject evidence library are integrated and associated to construct an overall index to obtain a medical subject index library; wherein the sub-indexes and the overall index include: an inverted index and a vector index.

3. The method according to claim 1, characterized in that The query method includes: a method based on querying the medical subject knowledge base corresponding to the subject field to which the query question belongs, a method based on interacting with the medical subject large language model corresponding to the subject field to which the query question belongs, and a method based on querying the medical subject knowledge base corresponding to the subject field to which the query question belongs and interacting with the medical subject large language model; Based on the query method, a medical subject knowledge base query corresponding to the subject field to which the query question belongs and / or a medical subject large language model interaction are performed to obtain a result of the query question, including: If the query method is a method of querying the medical subject knowledge base corresponding to the subject field to which the query question belongs, then querying the medical subject knowledge base corresponding to the subject field to which the query question belongs based on the query question to obtain a first query result; If the query method is a method of interacting with a large language model of a medical discipline corresponding to the discipline field to which the query question belongs, the query question is input into the large language model of a medical discipline corresponding to the discipline field to which the query question belongs, and a first feedback result is obtained; If the query method is a method of interactively combining a query based on a medical subject knowledge base corresponding to the subject field to which the query question belongs and a medical subject large language model, the query question is preprocessed, and a query based on the preprocessed query question is performed on the medical subject knowledge base corresponding to the subject field to which the query question belongs to obtain a second query result, and based on the second query result, a first prompt content of the medical subject large language model corresponding to the subject field to which the query question belongs is constructed, and the first prompt content is input into the medical subject large language model to obtain a second feedback result based on the second query result.

4. The method according to claim 3, characterized in that Preprocessing the query question, and querying the medical subject knowledge base corresponding to the subject field to which the query question belongs based on the preprocessed query question, to obtain a second query result, including: Inputting the query question into the medical subject large language model corresponding to the subject field to which the query question belongs to perform problem decomposition to obtain a decomposed question, and concatenating the decomposed question with the query question to obtain a concatenated question; Performing word segmentation and keyword extraction on the concatenation problem to obtain a keyword list, and inputting the keyword list into a pre-built synonym large language model to obtain a synonym list of the keywords; Extracting entities of the query question based on a pre-built information extraction model to obtain an entity list; Based on the pre-built judgment model, the query question is judged based on the query conditions to obtain the query condition results; wherein the query conditions at least include: the document type to which the query content belongs, and the location of the query content in the document; Based on the synonym list of the keyword, the query condition result and the splicing problem, a first query logic and a second query logic are respectively constructed for the inverted index and the vector index of the medical subject literature library, and an inverted index query and a vector index query are respectively performed based on the first query logic and the second query logic to obtain a literature query result; Constructing a third query logic for the medical subject literature entity library based on the entity list and the query condition result, and performing a query on the medical subject literature entity library based on the third query logic to obtain an entity query result; If the document type in the query condition result includes RCT, constructing a fourth query logic for the medical subject evidence library based on the entity list and the synonym list of the keyword, and querying the medical subject evidence library based on the fourth query logic to obtain an evidence query result; The query question, the document query result, the entity query result and the evidence query result are input into a pre-built evaluation and scoring model to obtain a scoring result for each query result, and a second query result is determined based on the scoring result.

5. The method according to claim 4, characterized in that The second feedback result is evaluated and scored for relevance, including: Filtering the second feedback result based on a preset rule; Inputting the filtered second feedback result into the pre-built first relevance evaluation model to obtain a relevance evaluation score of the second feedback result; If the relevance evaluation score of the second feedback result is greater than or equal to a second threshold and less than a first threshold, then the second feedback result is scored for consistency evaluation; wherein the first threshold is greater than the second threshold; If the relevance evaluation score of the second feedback result is less than the second threshold, obtaining a third feedback result based on the medical subject large language model, and performing a secondary relevance evaluation and scoring on the third feedback result; If the relevance evaluation score of the second feedback result is greater than or equal to the first threshold, the second feedback result is fed back to the user.

6. The method according to claim 5, characterized in that Obtaining a third feedback result based on the medical discipline large language model, and performing a secondary relevance evaluation and scoring on the third feedback result, including: Perform entity matching based on each entity in the entity list and the medical subject atlas library to obtain a matching entity list and a document set corresponding to the matching entity; Based on the matching entity list, entities adjacent to the matching entity are extracted from the document collection to obtain entity triples, and the entity triples are filtered; Repeat the entity triple extraction and filtering until a preset number of times is reached to obtain a first entity triple set; Input the query question into a pre-built answer question pattern prediction model to predict the answer question pattern, and obtain a pattern library consisting of multiple answer question patterns; wherein the answer question pattern is the entity relationship of the answer question; Based on the pre-built pre-trained model, each entity relationship in the pattern library and each entity triple relationship in the first entity triple set are vectorized, and similarity calculation is performed on the vectorized entity relationships in the pattern library and the entity triple relationships, and a preset number of entity triples are intercepted based on the similarity calculation results to obtain a second entity triple set; Constructing second prompt content of the medical subject large language model based on the second entity triple set, and inputting the second prompt content into the medical subject large language model to obtain a third feedback result; Filtering the third feedback result based on a preset rule, and inputting the filtered third feedback result into a pre-constructed second relevance evaluation model to obtain a relevance evaluation score of the third feedback result; If the relevance evaluation score of the third feedback result is greater than or equal to the first threshold, feeding back the third feedback result to the user; If the relevance evaluation score of the third feedback result is greater than or equal to the second threshold and less than the first threshold, performing consistency evaluation and scoring on the third feedback result; If the relevance evaluation score of the third feedback result is less than the second threshold, the first prompt content of the medical subject large language model is reconstructed to obtain a new second feedback result, and the new second feedback result is evaluated and scored for relevance. If the relevance evaluation score of the new second feedback result is less than the second threshold, the second prompt content of the medical subject large language model is reconstructed to obtain a new third feedback result. If the relevance evaluation score of the new third feedback result is less than the second threshold, the query question is preprocessed again, and based on the preprocessed query question, the medical subject knowledge base corresponding to the subject field to which the query question belongs is queried and the medical subject large language model is interacted to obtain a new second feedback result.

7. The method according to claim 5 or 6, characterized in that: Conducting consistency evaluation and scoring on the second feedback result or the third feedback result, including: Constructing a key summary prompt based on the second feedback result or the third feedback result, and inputting the key summary prompt into the medical subject large language model to obtain a key summary result of the second feedback result or the third feedback result; Scoring the key summary result of the second feedback result or the third feedback result based on a pre-built consistency evaluation model to obtain a consistency evaluation score of the key summary result of the second feedback result or the third feedback result; If the consistency evaluation score of the key summary result of the second feedback result is greater than or equal to the third threshold, the second feedback result is fed back to the user; if the consistency evaluation score of the key summary result of the second feedback result is less than the third threshold, the first prompt content of the medical subject large language model is reconstructed to obtain a new second feedback result, if the relevance evaluation score of the new second feedback result is greater than or equal to the second threshold and less than the first threshold, the new second feedback result is scored for consistency evaluation, if the consistency evaluation score of the key summary result of the new second feedback result is less than the third threshold, the query question is preprocessed again, and based on the preprocessed query question, the medical subject knowledge base corresponding to the subject field to which the query question belongs is queried and the medical subject large language model is interacted to obtain a new second feedback result; If the consistency evaluation score of the key summary result of the third feedback result is greater than or equal to the third threshold, the third feedback result is fed back to the user; if the consistency evaluation score of the key summary result of the third feedback result is less than the third threshold, the first prompt content of the medical subject large language model is reconstructed to obtain a new second feedback result; if the relevance evaluation score of the new second feedback result is less than the second threshold, the second prompt content of the medical subject large language model is reconstructed to obtain a new third feedback result; if the relevance evaluation score of the new third feedback result is greater than or equal to the second threshold and less than the first threshold, the new third feedback result is scored for consistency evaluation; if the consistency evaluation score of the key summary result of the new third feedback result is less than the third threshold, the query question is preprocessed again, and based on the preprocessed query question, the medical subject knowledge base corresponding to the subject field to which the query question belongs is queried and the medical subject large language model is interacted to obtain a new second feedback result.

8. A medical subject knowledge service device, characterized in that: include: A medical subject knowledge base construction module is used to obtain medical literature and construct a medical subject knowledge base corresponding to each subject area based on the medical literature; wherein the medical subject knowledge base includes: a medical subject literature base, a medical subject literature entity base, a medical subject evidence base, a medical subject atlas base, and a medical subject index base; A pre-judgment module is used to obtain a query question input by a user and pre-judgment the query question to determine the subject field and query method to which the query question belongs; A query and interaction module, used to query the medical subject knowledge base corresponding to the subject field to which the query question belongs and / or interact with the medical subject large language model based on the query method to obtain the result of the query question; wherein the result of the query question includes one of the following: a first query result, a first feedback result, a second query result, and a second feedback result based on the second query result; A feedback and evaluation module is used for, if a first query result or a first feedback result is obtained, then the first query result or the first feedback result is fed back to the user; if the second feedback result is obtained, then the second feedback result is evaluated and scored for relevance, and when the score result of the relevance evaluation score of the second feedback result meets the first preset condition, the second feedback result is evaluated and scored for consistency, when the score result of the relevance evaluation score of the second feedback result meets the second preset condition, a third feedback result is obtained based on the medical subject large language model, and the third feedback result is evaluated and scored for secondary relevance; when the score result of the secondary relevance evaluation score of the third feedback result meets the first preset condition, the third feedback result is evaluated and scored for consistency; when the score result of the consistency evaluation score of the second feedback result or the third feedback result meets the third preset condition, the second feedback result or the third feedback result is fed back to the user.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are performed.

Images