Intelligent medical question-answering system and method based on hybrid retrieval and lightweight reordering

Abstract

The invention provides an intelligent medical question-answering system and method based on hybrid retrieval and lightweight reordering, and is applied to the technical field of medical data processing. According to the method, five types of core entities are extracted through the preset Chinese medical NER model, and the structured knowledge base is formed through relation extraction modeling association. Analyzing user Chinese query, extracting medical entities, identifying four types of appeals and converting the four types of appeals into semantic vectors; a PubMedBERT is adopted to encode a medical document to generate a vector for storage, BM25 and vector retrieval are executed in parallel when query is received, and candidate documents are generated through fusion of an RRF algorithm. And generating a score data set based on the prompt template, and training the lightweight model to sort and output an evidence set. In combination with query semantics, a simplified context is generated through retrieval, sorting and compression, and FlashAttention optimization calculation is integrated. An optimized U-Net segmentation image is utilized to generate a structured report, and multi-modal information is integrated to generate an accurate answer giving consideration to the image and medical knowledge through LLM reasoning.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical data processing, in particular to an intelligent medical question and answer system and method based on hybrid retrieval and lightweight reordering. BACKGROUND

[0002] In recent years, artificial intelligence technology, especially large language models (LLM), has been increasingly applied in the medical and health field. The technology framework represented by retrieval-augmented generation (RAG) provides a new solution for clinical decision support and medical knowledge question and answer scenarios by combining external knowledge bases with the generation capabilities of LLM. However, the existing technology still faces many challenges when applied to the highly specialized and rigorous Chinese medical field. First, natural language processing (NLP) of Chinese medical text itself has inherent difficulties, such as word segmentation ambiguity, frequent appearance of new terms, and huge differences between colloquial expressions and professional terms. Second, the rapid iteration and massive growth of medical knowledge pose high requirements for the construction, maintenance, and real-time calling of knowledge bases.

[0003] More importantly, the effectiveness of the standard RAG framework is highly dependent on the quality of the retrieval stage. If the signal-to-noise ratio of the retrieved document fragments is low, i.e., containing irrelevant, outdated, or even incorrect information, these "toxic" contexts will seriously mislead the LLM, leading it to produce "hallucination" content that appears reasonable but is actually wrong. In the high-risk field of medicine, such errors can have serious consequences. In addition, processing ultra-long texts containing complex disease course descriptions and multiple examination reports is a common technical bottleneck faced by LLMs. Clinical diagnosis and treatment not only rely on text information, but also highly depend on the interpretation of medical images such as CT and MRI. The existing question and answer systems generally lack the ability to process and understand visual modal information, and have the problems of high computational cost and information forgetting ("lost in the middle"), which limit their deep application in real clinical scenarios. Therefore, how to optimize the retrieval and information processing process in the RAG framework for the Chinese medical scenario, improve the relevance and accuracy of the retrieved content, suppress model hallucination, and efficiently process long text information is a technical problem that needs to be solved in this field.

[0004] It should be noted that the information disclosed in the above background section is only used to strengthen the understanding of the background of the present disclosure, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0005] The purpose of the present application is to provide an intelligent medical question and answer system and method based on hybrid retrieval and lightweight reordering, which at least partially overcomes the problems existing in the prior art. The present application does not use a single retrieval method, but designs a precise pipeline. It first performs complementary recall through BM25 and vector retrieval, then balances keyword accuracy and semantic generality using the Reverse Rank Fusion (RRF) algorithm, and finally scores and sorts the candidate evidence through a dedicated lightweight reordering model. This architecture maximizes the context signal-to-noise ratio of the input LLM and is the key guarantee for suppressing model hallucinations. Another major innovation of the present application is the training method of the reordering model. It cleverly uses the strong understanding ability of large language models (such as GPT-4) to generate a large amount of high-quality (question, document, relevance score) labeled data through prompt engineering, and then trains a special reordering model with small parameters and fast reasoning speed using these data. This method takes into account the labeling quality and training efficiency, achieving accurate reordering with low cost and high performance.

[0006] To solve the problem of long text processing, the present application proposes to use query-aware intelligent compression technology to accurately extract the most relevant core information segments from the massive original text, significantly reducing the context size. Then, on the compressed and simplified context, the efficient attention algorithm such as FlashAttention is applied. This fundamentally avoids the huge overhead of LLM processing redundant information, and realizes efficient analysis of complex cases. Through the offline construction of a medical knowledge graph and online "entity + intent" dual analysis, the present application can accurately understand the real query intent of Chinese users, and convert the colloquial and fuzzy questions into structured retrieval signals that can be accurately executed by machines, laying a solid foundation for all subsequent processing procedures.

[0007] Other characteristics and advantages of the present application will become apparent from the following detailed description, or will be learned by practice of the application.

[0008] According to an aspect of the present application, an intelligent medical question and answer method based on hybrid retrieval and lightweight reordering is provided, comprising: acquiring multi-source authoritative Chinese medical data, Chinese medical user input query text, Chinese medical knowledge base documents, and user uploaded Chinese clinical scene medical images; extracting five types of core Chinese medical entities based on a preset Chinese medical NER model, modeling entity associations through relationship extraction, constructing a Chinese medical knowledge graph containing standard names and Chinese alias attributes, and forming a structured knowledge base hub; extracting medical entities from user Chinese queries, identifying four types of core demands for disease consultation, converting Chinese queries into semantic vectors using a fine-tuned Chinese medical Sentence-BERT model, and realizing cross-expression semantic equivalent mapping in the Chinese medical field; encoding Chinese medical documents using PubMedBERT to generate vectors and store them, performing BM25 keyword retrieval and vector semantic retrieval in parallel when receiving Chinese queries, generating a preliminary candidate document list through RRF algorithm fusion, and considering the accuracy of Chinese keywords and semantic relevance; generating a Chinese medical question and answer relevance scoring dataset based on a structured prompt template, training a lightweight model to score and sort candidate documents, outputting a Chinese medical evidence set, and improving evidence accuracy; combining user Chinese query semantics, generating a simplified context through a retrieval-sorting-compression pipeline, integrating FlashAttention technology to optimize attention calculation, and reducing Chinese long text processing memory usage; using an optimized U-Net model to segment medical image lesions and generate structured Chinese visual reports, integrating reports, user Chinese questions, and retrieval knowledge to form a multi-modal context, and generating answer information considering image features and Chinese medical knowledge through LLM reasoning.

[0009] In another aspect of the present application, an intelligent medical question and answer device based on hybrid retrieval and lightweight reordering is characterized in that it comprises: an acquisition module for acquiring multi-source authoritative Chinese medical data, Chinese medical user input query text, Chinese medical knowledge base documents, and user uploaded Chinese clinical scene medical images; a processing model for extracting five types of core Chinese medical entities based on a preset Chinese medical NER model, modeling entity associations through relationship extraction, constructing a Chinese medical knowledge graph containing standard names and Chinese alias attributes, and forming a structured knowledge base hub; extracting medical entities from user Chinese queries, identifying four types of core demands for disease consultation, converting Chinese queries into semantic vectors using a fine-tuned Chinese medical Sentence-BERT model, and realizing cross-expression semantic equivalent mapping in the Chinese medical field; generating vectors by encoding Chinese medical documents using PubMedBERT and storing them, performing BM25 keyword retrieval and vector semantic retrieval in parallel when receiving Chinese queries, generating a preliminary candidate document list through RRF algorithm fusion, and taking into account the accuracy of Chinese keywords and semantic relevance; generating a Chinese medical question and answer relevance scoring dataset based on a structured prompt template, training a lightweight model to score and sort candidate documents, outputting a Chinese medical field evidence set, and improving evidence accuracy; combining user Chinese query semantics to generate a simplified context through a retrieval-sorting-compression pipeline, integrating FlashAttention technology to optimize attention calculation, and reducing Chinese long text processing memory usage; using an optimized U-Net model to segment medical image lesions and generate structured Chinese visual reports, integrating reports, user Chinese questions, and retrieval knowledge to form a multi-modal context, and generating answer information that takes into account image features and Chinese medical knowledge through LLM reasoning.

[0010] According to still another aspect of the present application, an electronic device is characterized in that it comprises: a first processor; and a memory for storing executable instructions of the first processor; wherein the first processor is configured to execute the executable instructions to implement the above-mentioned intelligent medical question and answer method based on hybrid retrieval and lightweight reordering.

[0011] According to still another aspect of the present application, a computer-readable storage medium having a computer program stored thereon is provided, wherein the computer program is executed by a second processor to implement the above-mentioned intelligent medical question and answer method based on hybrid retrieval and lightweight reordering.

[0012] The intelligent medical question answering system and method based on hybrid retrieval and lightweight reordering provided by the application, the core "retrieval-fusion-rearrangement" architecture ensures that the evidence received by the LLM is highly relevant and reliable. In the internal CBLUE medical text understanding benchmark test, the macro-averaged F1 score of the key task of the system reaches 92.4%, which is significantly improved compared with the baseline RAG system. More importantly, the hallucination rate of the generated content is strictly controlled below 1.5%, which greatly guarantees the safety of medical question answering.

[0013] The use of the lightweight reordering model avoids calling a large and expensive LLM for sorting during online inference, and the response speed is faster. The two-stage long context processing framework reduces the average time spent by doctors in integrating and analyzing complex medical records and multiple reports by about 60%. Under the premise of ensuring information integrity, this framework reduces the dependence on high-end computing hardware, has higher cost performance and deployability.

[0014] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 A flowchart of an intelligent medical question answering method based on hybrid retrieval and lightweight reordering provided by an embodiment of the application is shown.

[0016] Figure 2 The structure of an intelligent medical question answering device based on hybrid retrieval and lightweight reordering provided by an embodiment of the application is shown. DETAILED DESCRIPTION

[0017] The preferred embodiments of the application are described below in conjunction with the accompanying drawings, and it should be understood that the preferred embodiments described herein are only for illustration and explanation of the application, and cannot limit the application.

[0018] The intelligent medical question answering method based on hybrid retrieval and lightweight reordering according to the exemplary embodiments of the application will be described below in conjunction with Figure 1 It should be noted that the following application scenarios are only shown for the purpose of facilitating understanding of the spirit and principles of the application, and the embodiments of the application are not limited in this respect. On the contrary, the embodiments of the application are applicable to any applicable scenarios.

[0019] In an embodiment, the application also provides an intelligent medical question answering system and method based on hybrid retrieval and lightweight reordering. Figure 1 A flowchart of an intelligent medical question answering method based on hybrid retrieval and lightweight reordering according to an embodiment of the application is schematically shown.

[0020] S101, obtain multi-source authoritative Chinese medical data, Chinese medical user input query text, Chinese medical knowledge base documents, and user uploaded Chinese clinical scene medical images.

[0021] In an embodiment, integrate Chinese medical data from multiple channels such as medical textbooks (e.g., the chapter on the diagnostic criteria for type 2 diabetes in Internal Medicine (9th Edition)), diagnosis and treatment guidelines (e.g., the content on blood glucose control targets in the China Type 2 Diabetes Prevention and Treatment Guidelines (2023 Edition)), and drug instructions (e.g., the indications and usage of metformin hydrochloride tablets). These data need to be screened to ensure their authority and accuracy, providing a foundation for subsequent knowledge graph construction and retrieval. In addition to the aforementioned medical textbooks, diagnosis and treatment guidelines, and drug instructions, it also covers authoritative medical journal articles (e.g., research papers on the clinical application of new hypoglycemic drugs in Chinese Internal Medicine), clinical pathway documents (e.g., the "Type 2 Diabetes Clinical Pathway" released by the National Health Commission), and medical continuing education materials (e.g., diabetes diagnosis and treatment progress training courseware organized by medical societies). After integration, these data need to undergo strict preprocessing: first, remove duplicate content (e.g., duplicate basic diagnostic criteria in different guidelines), correct data errors (e.g., dosage labeling errors caused by typographical errors in drug instructions), and unify formats (e.g., standardize different disease staging expressions from different sources to the international standard). Through screening and preprocessing, ensure that the data meet the professional standards of the Chinese medical field, and provide high-quality raw materials for subsequent extraction of five core entities such as diseases and symptoms using the pre-set Chinese medical NER model, and construction of a medical knowledge graph containing standard names and Chinese aliases.

[0022] User input Chinese query text includes not only direct condition consultation and medication inquiry, but also concept queries (e.g., "What is insulin resistance?") and treatment plan consultation (e.g., "What are the postoperative adjuvant chemotherapy regimens for gastric cancer?"). After receiving the query text, the system will first perform basic cleaning: remove meaningless adverbs (e.g., "um" "ah"), correct misspelled words (e.g., correct "double gua" to "double gua" in "high blood sugar and eat double gua"), and perform word segmentation (e.g., split "diabetic nephropathy patients can eat which fruits" into "diabetic nephropathy" "patients" "can eat" "which" "fruits"). These preprocessed query texts will serve as the basis for subsequent extraction of medical entities (e.g., extracting "diabetic nephropathy" and "fruits" from "diabetic nephropathy patients can eat which fruits") and identification of core demands (in this case, dietary consultation, a derivative type of condition consultation), and then converted into semantic vectors through fine-tuned Chinese medical Sentence-BERT model, achieving cross-expression semantic equivalent mapping.

[0023] In the structured processing of Chinese medical knowledge base documents, the content dimension needs to be refined: for disease-related documents, the cause (such as "the main causes of coronary atherosclerotic heart disease include hyperlipidemia and hypertension"), epidemiological data (such as "the prevalence of coronary heart disease in Chinese adults is about 10%"), and differential diagnosis points (such as the differential symptoms from pulmonary embolism) need to be included; for examination-related documents, the indications (such as "the indications for electrocardiogram examination include chest pain and palpitations") and result interpretation criteria (such as "ST segment depression ≥0.1 mV indicates myocardial ischemia") need to be clearly stated. These structured documents are encoded into semantic vectors by the PubMedBERT model and stored in a vector database, serving as the core resource for subsequent BM25 keyword retrieval and vector semantic retrieval, and providing support for quickly recalling relevant document segments for user queries.

[0024] In addition to lung CT and brain MRI, user-uploaded medical images also include ultrasound images (such as abdominal B-mode ultrasound), pathological section images, and X-ray films, accompanied by Chinese explanations that need to include the purpose of the examination (such as "chest CT examination due to chest tightness"), patient's basic information (such as "male, 65 years old, with a 10-year history of hypertension"), and preliminary examination conclusions (such as "B-mode ultrasound suggests the possibility of liver hemangioma"). After these image data are input into the system, they will first undergo standardization processing: adjusting the size to a uniform resolution (such as unifying the CT images taken by different devices to 512x512 pixels), enhancing the contrast of the lesion area (such as highlighting the abnormal signal area in MRI through algorithms), laying the foundation for subsequent segmentation of the lesion area (such as segmenting the nodule from the lung CT) using the optimized U-Net model, extraction of location, size, and other quantitative features, and generation of structured Chinese visual reports such as "a nodule with a diameter of about 8 mm is seen in the upper lobe of the left lung, with clear boundaries."

[0025] In S102, five types of core Chinese medical entities are extracted based on a pre-set Chinese medical NER model, entity associations are modeled through relationship extraction, a Chinese medical knowledge graph containing standard names and Chinese alias attributes is constructed, and a structured knowledge base hub is formed.

[0026] In one implementation, multiple authoritative Chinese medical data sources are integrated and pre-processed to generate pre-processed text data. In the integration of multiple authoritative Chinese medical data sources, in addition to medical textbooks, treatment guidelines, and drug instruction manuals, authoritative medical journal articles (such as research papers on the clinical application of new anti-tumor drugs in the Chinese Journal of Medicine), clinical pathway files (such as the "Community Acquired Pneumonia Clinical Pathway" issued by the National Health Commission), and medical continuing education materials (such as the "Diabetes Complications Diagnosis and Treatment Progress" training courseware organized by medical societies) are also included. These data need to undergo multi-dimensional preprocessing:

[0027] Format unification: not only adjust the font, paragraph format, but also need to convert different formats of documents such as PDF, Word, TXT into a unified text format, extract structured data from tables such as normal blood pressure range tables for different age groups, and recognize text in pictures such as hand-drawn dissection annotations in teaching materials through OCR technology to editable text, ensuring the consistency of data format.

[0028] Redundancy removal: In addition to deleting repeated basic medical knowledge such as "normal human body temperature range", it is also necessary to eliminate the same research data repeatedly cited in multiple papers such as the same clinical trial results of a certain type of drug, advertising information such as irrelevant brand promotion content in drug instructions, and redundant paragraphs unrelated to medical professional knowledge such as editor's preface in teaching materials.

[0029] Error correction: In addition to correcting possible dosage errors in drug instructions such as "3 times a day, 500mg each time" written as "5 times a day, 300mg each time", it is also necessary to correct outdated content in diagnosis and treatment guidelines due to version updates such as obsolete treatment plans in old guidelines, typos in medical terminology such as "heart muscle plug" corrected to "heart muscle infarction", and formula or data errors in literature due to typesetting errors such as symbol errors in drug concentration calculation formula.

[0030] Through the above preprocessing, the standardized text data not only retains the authority of the original data, but also eliminates format differences, redundant information and error content, providing high-quality raw materials for subsequent extraction of five core entities such as diseases and symptoms using the preset Chinese medical NER model, and building a Chinese medical knowledge graph.

[0031] Based on the preset Chinese medical NER model, an entity extraction framework is constructed to generate an entity recognition model. The preset Chinese medical NER model extracts five core Chinese medical entities from the preprocessed text, including disease entities, symptom entities, drug entities, examination entities, and treatment plan entities. The entity extraction framework built by the fine-tuned PCL-MedBERT-NER model is a special tool optimized for Chinese medical text characteristics. Its core is to pre-train medical field knowledge and fine-tune Chinese medical corpus to improve the recognition accuracy of professional terms, aliases and complex expressions. The process of extracting five core entities from preprocessed text requires precise boundary positioning and type judgment based on context semantics:

[0032] Disease entity extraction: Not only identify explicit disease names (e.g. "type 2 diabetes"), but also capture complications (e.g. "diabetic nephropathy"), staging and typing (e.g. "gastric cancer stage IV") and other subtypes. For example, from "chest tightness symptoms occurred in a patient with hypertensive heart disease", the disease entity "hypertensive heart disease" is accurately extracted, rather than just identifying "hypertension" or "heart disease". Symptom entity extraction: Can distinguish typical symptoms (e.g. "polydipsia and polyuria"), accompanying symptoms (e.g. "weakness") and signs (e.g. "hepatosplenomegaly"). For example, from "patient with fever, cough and yellow sputum for 3 days", the symptom entities "fever", "cough" and "yellow sputum" are extracted, and their relevance is determined by contextual association.

[0033] Drug entity extraction: Covers generic names (e.g. "metformin hydrochloride tablets"), trade names (e.g. "Gevayezhi") and dosage forms (e.g. "insulin injection (300U / 3ml)"). For example, from "suggested to take Baiyasi (aspirin enteric-coated tablets 100mg / tablet)", the drug entity "aspirin enteric-coated tablets" and the trade name "Baiyasi" are extracted synchronously, and the specification information is associated. Examination entity extraction: Includes imaging examinations (e.g. "electrocardiogram" "lung CT"), laboratory examinations (e.g. "blood glucose test" "blood routine") and invasive examinations (e.g. "coronary angiography"). For example, from "postoperative monitoring of coagulation function and chest X-ray examination", the examination entities "coagulation function (test)" and "chest X-ray examination" are extracted. Treatment plan entity extraction: Covers drug treatment (e.g. "treatment with metformin hydrochloride tablets"), surgical treatment (e.g. "laparoscopic cholecystectomy"), rehabilitation intervention (e.g. "diabetes diet guidance") and other treatments. For example, from "after diagnosis, chemotherapy combined with targeted therapy, supplemented by nutritional support", the treatment plan entities "chemotherapy combined with targeted therapy" and "nutritional support" are extracted. This framework provides accurate entity basis for subsequent relation extraction and knowledge graph construction through accurate identification of entity boundaries (e.g. distinguishing "diabetes" from "diabetic retinopathy") and fine-grained classification of types.

[0034] The preprocessed text data is input into the entity recognition model, and the entity association data is generated by combining the relation extraction model, wherein the relation extraction process models the medical logical association between entities to clearly define the semantic relationship between entities. After identifying various entities from the preprocessed text by the entity recognition model, the relationship between entities is established by the relation extraction model. For example, from the text, it is identified that "type 2 diabetes" and "polydipsia and polyuria" have a "typical symptom" relationship, and "metformin hydrochloride tablets" and "type 2 diabetes" have an "indication" relationship, and the medical logical association between entities is clearly defined.

[0035] When constructing the initial graph with the extracted five types of core entities as nodes and the medical logical associations between entities as edges, the fine design of entity attributes and relationship types needs to be combined to ensure the structured and semantic integrity of the graph: structured presentation of node information: each entity node not only contains the core definition, but also integrates the basic attributes. For example, the "type 2 diabetes" node needs to cover the definition ("a metabolic disease characterized by insulin resistance and insufficient insulin secretion"), epidemiological data ("the prevalence rate of adults in China is about 11.9%"), pathogenesis ("related to genetics, obesity, and lifestyle"), and other core information; the "metformin hydrochloride tablets" node needs to include drug ingredients ("metformin hydrochloride"), administration route ("oral"), and common dosage forms ("tablets, sustained-release tablets"), etc. basic attributes provide detailed basis for subsequent knowledge association.

[0036] Edge association type and semantic annotation: the edges between entities need to be clearly labeled with specific association types, and the association needs to conform to medical logic. In addition to "typical symptoms" and "indication", it also includes "complications" (such as "type 2 diabetes" and "diabetic nephropathy" connected through this edge), "contraindications" (such as "metformin hydrochloride tablets" and "renal dysfunction" connected through this edge), "examination indications" (such as "type 2 diabetes" and "glycosylated hemoglobin detection" connected through this edge), "treatment options" (such as "type 2 diabetes" and "insulin injection therapy" connected through this edge), etc. For example, the "electrocardiogram" node and the "myocardial infarction" node are connected through the "key examination" edge, which accurately reflects the core role of this examination in disease diagnosis.

[0037] Hierarchical integration of multi-dimensional associations: for complex entity relationships, a hierarchical association structure needs to be constructed. For example, the "hypertension" node and the "stroke" node have a "risk factor" association, while "hypertension" is associated with "atherosclerosis" through the "cause" association, and "atherosclerosis" is associated with "stroke" through the "trigger" association, forming a chain association of "hypertension → atherosclerosis → stroke", which clearly presents the causal chain between diseases. Such an initial graph, through the comprehensiveness of node attributes and the accuracy of edge associations, lays a foundation for subsequent attribute expansion and optimization.

[0038] When expanding and optimizing the attribute of the basic graph structure, the accuracy and practicality of the knowledge base need to be improved from the aspects of attribute dimension expansion and correlation verification around the professional characteristics of the Chinese medical scene. Specifically, the multi-dimensional expansion of node attributes: In addition to the standard name, definition, and Chinese alias, key attributes that adapt to the clinical scene also need to be supplemented. For example, the node of "coronary atherosclerotic heart disease" needs to add attributes such as "typical symptoms" (such as "chest pain, chest tightness"), "high-risk groups" (such as "middle-aged men, hypertension patients"), and "diagnostic gold standard" (such as "coronary angiography"); the node of "penicillin" needs to supplement information such as "pharmacological action" (such as "inhibiting bacterial cell wall synthesis"), "adverse reactions" (such as "anaphylactic shock"), and "dosage form specifications" (such as "80 million units of penicillin sodium for injection"). The expansion of Chinese aliases needs to cover clinical commonly used abbreviations and colloquial names, such as the aliases of "acute myocardial infarction" including "heart attack" and "acute heart attack", and the aliases of "hepatitis B" including "hepatitis B", to ensure that the system can accurately locate entities through alias matching when users use colloquial expressions.

[0039] Precise verification and correction of association relationship: Through cross verification of medical logic rules and authoritative data, eliminate incorrect associations and supplement missing associations. For example, delete the incorrect treatment association between "penicillin" and "viral cold" (because penicillin is ineffective against viruses), and supplement the "sensitive bacteria infection treatment" association between "penicillin" and "streptococcal pharyngitis"; correct the incorrect "indication" association between "nifedipine" and "diabetes", change it to the "first-line treatment drug" association between "nifedipine" and "hypertension", and supplement the "contraindication" association between "nifedipine" and "hypotension". For complex associations, the association strength or conditions need to be marked, such as the "secondary prevention drug" association between "aspirin" and "coronary heart disease" needs to supplement the "daily dose of 75-150mg" usage condition to improve the rigor of knowledge.

[0040] The expanded and optimized structured knowledge base can not only support multi-dimensional understanding of entities through rich attributes, but also ensure the consistency of knowledge logic through precise association relationships, providing reliable support for subsequent semantic disambiguation of user queries (such as distinguishing the different connotations of "coronary heart disease" and "pulmonary heart disease"), precise knowledge matching during retrieval, and factual basis for LLM answer generation.

[0041] S103, extract the medical entity from the user's Chinese query, identify the four core demands of the disease consultation, and convert the Chinese query into a semantic vector using the fine-tuned Chinese medical Sentence-BERT model to realize cross-expression semantic equivalent mapping in the Chinese medical field.

[0042] In one implementation, in the node attribute expansion of the Chinese medical knowledge graph, in addition to the standard name, definition, and Chinese alias, multi-dimensional key attributes need to be supplemented in combination with clinical practice requirements to improve the practicality and scene adaptability of knowledge. Specifically, the attribute expansion of disease nodes: taking "stroke" as an example, in addition to the core attributes, attributes such as "risk factors" (such as "high blood pressure, atrial fibrillation, and smoking"), "first aid measures" (such as "immediately call 120 and keep the airway unobstructed"), and "prognosis" (such as "thrombolytic therapy within 3 hours of onset can significantly improve prognosis") need to be supplemented. These information is directly related to clinical decision-making, for example, when the user queries "what to do after a stroke", the system can quickly return standard treatment recommendations based on the "first aid measures" attribute, avoiding misinformation due to information gaps.

[0043] Attribute expansion of drug nodes: for "insulin injection", attributes such as "route of administration" (such as "subcutaneous injection"), "target population" (such as "type 1 diabetes patients and type 2 diabetes patients with poor oral drug control"), "precautions" (such as "monitor blood sugar after injection to avoid hypoglycemia"), and "drug interactions" (such as "combination with glucocorticoids may reduce efficacy") need to be supplemented. For example, when the user asks "can insulin be used with prednisone?", the system can clearly inform the risk through the "drug interactions" attribute to ensure medication safety.

[0044] Attribute expansion of examination nodes: taking "blood routine" as an example, attributes such as "examination purpose" (such as "judging infection type and anemia degree"), "reference range" (such as "white blood cell count 4-10 x 10 9 / L"), "abnormal result interpretation" (such as "increased neutrophil ratio suggests bacterial infection"), and "pre-examination preparation" (such as "no need to fast") need to be supplemented. When the user queries "what does a high white blood cell count in blood routine mean?", the system can provide professional interpretation in combination with the "abnormal result interpretation" attribute to reduce user misunderstanding of the report. Fine expansion of Chinese aliases: needs to cover clinical colloquialisms, regional nicknames, and abbreviations. For example, the aliases of "chronic obstructive pulmonary disease" include "chronic obstructive pulmonary disease" and "COPD" (Chinese adaptation of English abbreviation); the aliases of "lumbar disc herniation" include "lumbar disc herniation" and "lumbar disc herniation" (association of easily confused terms). For polysemous words, clear differentiation is needed, such as "hyperthyroidism" only corresponds to "hyperthyroidism", avoiding confusion with "hypothyroidism" (hypothyroidism), to ensure that when the user inputs "how to treat hyperthyroidism", the system can accurately match the correct entity.

[0045] Through the above attribute expansion, the knowledge graph node not only contains the basic definition, but also integrates the key information required in the whole process of clinical diagnosis and treatment, which not only meets the deep query needs of professional medical staff, but also provides understandable and practical guidance for ordinary users. At the same time, through the comprehensive coverage of aliases, the retrieval matching problem caused by the diversification of Chinese medical expressions is solved, and solid knowledge support is provided for subsequent semantic disambiguation and accurate question answering.

[0046] Based on the NER model, a medical entity extraction framework is constructed to generate an entity recognition tool, which focuses on extracting medical entities from standardized query texts and accurately capturing core medical elements in the query. Using this tool to extract core medical entities from standardized query texts, such as extracting the disease-related entity "high blood sugar", the symptom entity "dizziness", and the drug entity "metformin" from "high blood sugar and dizziness eat metformin", accurately positioning the medical core elements in the query.

[0047] The standardized query text is input into the intent classifier to generate a core demand label, where the classification process identifies four types of core demands based on text semantics, including disease consultation, medication consultation, concept query, and treatment plan inquiry, to clarify the specific purpose of the user query. The intent classifier identifies the user's core demand based on the text semantics, such as "high blood sugar and dizziness eat metformin", which determines the core demand as "medication consultation" (asking whether metformin is suitable for this situation), and clarifies the user's query purpose.

[0048] The semantic vector conversion tool based on the fine-tuned Chinese medical Sentence-BERT model, its core function is to convert the semantic information of user query into a computable high-dimensional dense vector, providing accurate basis for subsequent semantic matching in the retrieval stage, the specific implementation and application scenarios are as follows: flexible adaptation of input form, this tool supports two input modes. One is the complete query text, such as "Can high blood sugar and dizziness eat metformin?", which directly preserves the semantic integrity of the user's original expression; the second is the "entity + intent" reconstruction sentence, which extracts the core medical entities (such as "high blood sugar", "dizziness", "metformin") and the identified core demand (such as "medication consultation") from the query, and combines them into a structured sentence (such as "[high blood sugar, dizziness, metformin] + medication consultation"). This reconstruction method focuses on key information and reduces the interference of redundant expressions on semantic vectors, especially suitable for long sentence queries or ambiguous expression scenarios.

[0049] The Chinese medical Sentence-BERT model is fine-tuned on large-scale Chinese medical corpus (such as diagnosis and treatment guidelines, doctor-patient question and answer data), and has the ability to deeply understand professional terms, aliases and clinical expressions. For example, for "metformin" and "Gua Hua Zhi" (trade name), "high blood sugar" and "hyperglycemia" (Chinese adaptation of English term), the model can capture their semantic association and ensure that the converted vectors are close in space. The high-dimensional dense vector (usually 768-dimensional or 1024-dimensional) generated by conversion maps the text semantics to a geometric space, and the cosine similarity between vectors can quantify the degree of semantic association. For example, the vector cosine similarity of "Is it safe to take metformin for high blood sugar?" and "How safe is metformin for high blood sugar patients?" is close to 1, while the vector cosine similarity of "What medicine to take for a cold?" tends to 0. This quantitative feature provides a mathematical basis for semantic matching in the subsequent retrieval stage.

[0050] The generated semantic vector will serve as a navigation signal to drive the approximate nearest neighbor search (ANN) of the vector database, quickly recalling documents related to the query semantics. For example, when the user queries "[high blood sugar, metformin] + medication consultation", the vector generated by the tool can accurately match the vectors of relevant documents such as "metformin indications" and "high blood sugar medication contraindications" in the knowledge base, ensuring the semantic relevance of the search results and providing key dense retrieval basis for hybrid retrieval (BM25 + vector retrieval).

[0051] Cross-expression semantic equivalent mapping is achieved through vector space calculation, generating semantic association results. By calculating the cosine similarity of different query vectors, we ensure that semantically equivalent Chinese medical queries are consistently understood, improving the system's ability to handle diverse expressions. For example, the vector cosine similarity of "Can I take metformin for high blood sugar?" and "Can metformin treat high blood sugar?" tends to 1, ensuring that semantically equivalent queries are consistently understood and improving the system's ability to handle diverse expressions.

[0052] S104, generate vectors by encoding Chinese medical documents with PubMedBERT and store them. When receiving Chinese queries, perform BM25 keyword retrieval and vector semantic retrieval in parallel, and generate a preliminary candidate document list through RRF algorithm fusion, taking into account the accuracy of Chinese keywords and semantic association.

[0053] In an embodiment, a text encoding and storage infrastructure is constructed based on a PubMedBERT model to generate an initial encoding storage framework, wherein the initial encoding framework includes a text feature extraction layer, a vector conversion layer, and a storage layer, the feature extraction layer is used to extract Chinese medical entity and relationship features, the vector conversion layer is used to convert entities and relationships into vector representations, and the storage layer is used to store the vector structure. The framework includes a text feature extraction layer, a vector conversion layer, and a storage layer. For example, facing the text "2 type diabetes patients can use hydrochloric acid dimethylguanidine tablets for treatment, and regular blood glucose detection is required", the feature extraction layer extracts the disease entity "2 type diabetes", the drug entity "hydrochloric acid dimethylguanidine tablets", the examination entity "blood glucose detection", and the relationship features such as "treatment" and "regularly". The vector conversion layer converts these entities and relationships into vectors, and the storage layer stores the vector structure in a FAISS or other vector database.

[0054] The preprocessed text data is input into the initial encoding storage framework, and entity recognition algorithms and relationship extraction algorithms are used to extract and model entities and associated relationships to generate intermediate knowledge data. In the extraction process, an entity boundary loss function and a relationship classification loss function are used to improve the extraction accuracy of five types of core Chinese medical entities and associated relationships. At the same time, PubMedBERT encodes the document to generate a semantic vector and stores it in a vector database. Input the preprocessed text "hypertension patients may have dizziness symptoms, can take nifedipine tablets, and are recommended to have an electrocardiogram examination". The entity recognition algorithm extracts entities such as "hypertension", "dizziness", "nifedipine tablets", and "electrocardiogram examination", and the relationship extraction algorithm establishes associations such as "hypertension - may appear - dizziness" and "hypertension - can take - nifedipine tablets". In the extraction process, the entity boundary loss function is used to optimize the boundary recognition accuracy of entities such as "hypertension", and the relationship classification loss function is used to improve the classification accuracy of relationships such as "may appear". At the same time, PubMedBERT encodes the document to generate a semantic vector and stores it in a vector database.

[0055] The intermediate knowledge data is extended and integrated to generate a Chinese medical knowledge graph, wherein in the extension process, standard names, Chinese aliases, and other attributes are added to the entities, and the entity associated relationships are verified and corrected to ensure the integrity and accuracy of the knowledge graph, and a vector retrieval index is established simultaneously to improve the efficiency of subsequent semantic retrieval. The "hypertension" entity is added with attributes such as standard name "essential hypertension", Chinese alias "hypertensive disease", and definition "a clinical syndrome characterized by increased arterial blood pressure in the systemic circulation". The incorrect relationship "nifedipine tablets have a treatment association with diabetes" is corrected to "nifedipine tablets have a treatment association with hypertension". At the same time, an index is established for the vector database, such as an index for "hypertension" related document vectors to improve the matching speed during subsequent semantic retrieval.

[0056] The Chinese medical knowledge graph is stored in a special database to generate a structured knowledge base hub. During storage, the entity and relationship structure is stored in a graph database format, and the document semantic vector is stored in a vector database. The two databases are coordinated and associated indexed to support subsequent hybrid retrieval. When receiving a Chinese medical user query, BM25 sparse retrieval and vector database approximate nearest neighbor semantic retrieval are performed in parallel, and a preliminary candidate document list is generated using a reverse ranking fusion algorithm. The entity and relationship structure of "type 2 diabetes-treatment-metformin hydrochloride tablets" and the like is stored in a graph database, and the semantic vector of the corresponding document is stored in a vector database. The two databases are associated indexed by entity ID. When receiving a user query "Can I take metformin if my blood sugar is high?", BM25 retrieval (matching "high blood sugar" and "metformin" keywords) and vector database approximate nearest neighbor retrieval (recalling semantically relevant documents) are performed in parallel, RRF algorithm is used to calculate the fusion score, and a preliminary candidate document list is generated, such as "Type 2 diabetes treatment guidelines on the applicability of metformin" and the like.

[0057] In one embodiment, a structured prompt template is used to generate a Chinese medical question and answer relevance scoring data set, a lightweight model is trained to score and rank candidate documents, and a Chinese medical evidence set is output to improve evidence accuracy.

[0058] In one embodiment, a structured prompt template is used to generate a Chinese medical question and answer relevance scoring data set, a structured prompt template is designed to guide a powerful general LLM (such as GPT-4) to play the role of a "medical relevance evaluation expert". For example, the prompt template is "Please evaluate the support degree, relevance and information reliability of the document fragment to answer the user's question, output a relevance score in the interval of 0-1, and explain the reasons. User question: 'Can I take metformin if I have high blood sugar and dizziness?' Document fragment: 'Metformin is suitable for type 2 diabetes patients, especially for obese patients, and can lower blood sugar levels.' ". The general LLM outputs a score (such as 0.85) and a reason ("The document fragment mentions that metformin is suitable for lowering blood sugar, which is related to the user's question of high blood sugar, and has high support degree") based on this. This process is performed on more than 100,000 pairs of medical questions and answers (such as "What medicine should I take for high blood pressure?" and various types of antihypertensive drug instruction fragments) to generate a (question, document fragment, relevance score) triplet data set.

[0059] The lightweight model is trained to score and rank candidate documents. Using the generated dataset, a lightweight model based on a 6-layer Transformer architecture (such as TinyBERT) is trained. The model takes as input "user question + [SEP] + candidate document fragment", for example, the input "Can metformin be taken if blood sugar is high and dizzy? [SEP] Metformin is suitable for type 2 diabetes patients, especially obese patients, and can lower blood sugar levels." The top layer of the model outputs a relevance prediction score of 0-1 through a regression layer. The training objective is to minimize the mean square error between the predicted score and the target score in the dataset, and a ListNet ranking loss is introduced to optimize the document ranking performance.

[0060] During online inference, the Top-N (such as N = 20) candidate document fragments obtained by hybrid retrieval fusion are input into the trained lightweight model. For example, for the user question "Can metformin be taken if blood sugar is high and dizzy?", 20 relevant document fragments are input into the model, which outputs their relevance scores (such as 0.85, 0.72, 0.3, etc.). The document fragments are re-ordered in descending order according to the scores, and the top K (such as K = 5) fragments with the highest scores are selected to form a high-quality evidence set, such as document fragments containing "metformin indications" and "blood sugar high medication precautions".

[0061] S106, combined with the semantic of user Chinese query, through the retrieval-ranking-compression pipeline to generate concise context, integrated FlashAttention technology to optimize attention calculation, reduce the memory occupation of Chinese long text processing.

[0062] In an embodiment, the user Chinese query text and the retrieved Chinese long text data are subjected to semantic association analysis and preprocessing to generate associated preprocessed data, wherein the associated preprocessed data includes semantic alignment of the text, filtering of redundant information, and sorting of the retrieval results according to relevance. For the user query "Can metformin be taken if blood sugar is high and dizzy?", combined with the retrieved long text data (such as the chapter on metformin in the "Type 2 Diabetes Diagnosis and Treatment Guidelines" and the full text of the drug instruction), semantic alignment (establishing an association between "high blood sugar" and "high blood sugar" in the document) and redundant information filtering (eliminating irrelevant manufacturer information and packaging specification descriptions in the document) are performed, and the retrieval results are sorted according to the relevance score in the hybrid retrieval stage (such as placing the "metformin indications" related fragment before the "diabetes etiology" fragment), forming the associated preprocessed data.

[0063] A simplified context generation framework is constructed based on a retrieval-ranking-compression pipeline to generate an initial processing framework, wherein the initial processing framework includes a retrieval module, a ranking module, and a compression module, the retrieval module is used to match relevant texts, the ranking module is used to rank by relevance, and the compression module is used to extract core information. The framework includes a retrieval module, a ranking module, and a compression module. For example, the retrieval module recalls long texts related to "medication for high blood sugar" from a knowledge base through vector matching; the ranking module calls a lightweight re-ranking model to reorder the recalled texts according to "semantic relevance to the query" and "information necessity"; and the compression module extracts core information (such as "metformin suitable population" and "contraindications") from the ranked texts through an extractive summarization algorithm (such as BERT-based key sentence extraction) to form an initial processing framework.

[0064] The associated preprocessed data is input into the initial processing framework, and the user query semantic guidance and lightweight re-ranking model are used to evaluate the priority of the text segments to generate an intermediate simplified text, wherein a relevance scoring mechanism and an information density evaluation index are used in the evaluation process to retain high-value content. After inputting the associated preprocessed data (such as the ranked "metformin instruction fragments" and "diabetes diagnosis and treatment guidelines excerpts"), the lightweight re-ranking model evaluates the priority of the text segments guided by the user query semantic ("feasibility of medication for high blood sugar and dizziness"): a relevance scoring mechanism (such as the "metformin reduces blood sugar" segment scoring 0.9, and the "metformin research and development history" segment scoring 0.2) is used in combination with an information density evaluation index (such as the number of key knowledge points contained in a unit of words), to retain high-value content (such as "metformin is suitable for type 2 diabetes patients with high blood sugar, dizziness may be a symptom of hypoglycemia, and medication should be used after monitoring blood sugar") and eliminate low-relevance or repetitive information to generate an intermediate simplified text.

[0065] The FlashAttention technique is integrated into the simplified text to optimize attention computation and generate a target simplified context. During the optimization process, memory access optimization and kernel fusion are used to reduce memory usage and improve processing efficiency when processing long Chinese texts. When processing simplified text (such as long texts containing 5 high-value segments), the FlashAttention technique is integrated to optimize the attention computation of the Transformer model: memory access optimization (avoiding repeated loading of text vectors) and kernel fusion (merging multiple matrix operations) reduce memory usage when processing long texts such as "Precautions for Diabetes Medication + Assessment of Hypoglycemia Symptoms + Contraindications of Metformin" (e.g., reducing the required memory from 8GB to 4GB), while also improving processing speed (e.g., reducing the compression time of 1000 characters from 2 seconds to 0.5 seconds). Finally, a target simplified context is generated (e.g., "Metformin can lower blood sugar and is suitable for patients with type 2 diabetes; dizziness may be a symptom of hypoglycemia; it is recommended to monitor blood sugar before medication; use with caution in patients with renal insufficiency").

[0066] S107 utilizes an optimized U-Net model to segment medical image lesions and generate a structured Chinese visual report. It integrates the report, user Chinese questions, and retrieval knowledge to form a multimodal context, and generates answer information that takes into account both image features and Chinese medical knowledge through LLM inference.

[0067] In one implementation, medical image data and associated text information are preprocessed to generate preprocessed image data and a structured text template. The preprocessing includes enhancing lesion areas and standardizing the dimensions of the images, as well as unifying the format and standardizing the fields of the visual report template. For user-uploaded lung CT images and associated text (e.g., "Patient coughed for a week, CT showed lung shadows"), the images are preprocessed—potential nodule areas are highlighted using a contrast enhancement algorithm, and images from different devices are uniformly adjusted to a standardized size of 512×512 pixels; the visual report template is formatted, setting fixed fields such as "image location, lesion location, size, and morphology" (e.g., "Image found: [location] has [size] [morphology] lesion") to ensure consistent report structure.

[0068] An optimized medical image segmentation model structure is constructed based on a U-Net framework to generate an initial segmentation model, wherein the initial segmentation model includes an encoder, a decoder, and a skip connection, the encoder is used to extract image lesion features, the decoder is used to locate the lesion area, and the skip connection is used to fuse different level features. Through the collaborative design of the encoder, the decoder, and the skip connection, the optimized medical image segmentation model constructed based on the U-Net framework realizes accurate extraction and positioning of the lesion area, and the functions and action mechanisms of each component are as follows. The encoder is used for hierarchical feature extraction, and the encoder is composed of multiple convolutional layers and pooling layers, and the hierarchical feature extraction is realized by gradually reducing the size of the feature map (such as from 512×512→256×256→128×128). Taking a lung CT image as an example, the shallow convolutional layer (such as the first 2 layers) extracts low-level detail features, including the edge texture of the lesion (such as the burr-shaped edge of the nodule), the local density difference (such as the density contrast between the ground glass shadow and the normal lung tissue); the deep convolutional layer (such as the third 4 layers) focuses on high-level semantic features, such as the overall shape of the lesion (circular, irregular), and the spatial relationship with the surrounding tissue (such as close to the pleura). For example, when segmenting a liver cancer MRI image, the encoder can capture key features such as tumor boundary clarity and internal signal uniformity at each level, providing a basis for subsequent positioning.

[0069] The decoder is used for accurate positioning and feature restoration, and the decoder enlarges the size of the feature map through deconvolution operation (such as transpose convolution) step by step, maps the abstract features extracted by the encoder back to the original image space, and realizes accurate coordinate positioning of the lesion. Taking the segmentation of brain glioma in brain MRI as an example, the decoder first receives a 16×16 high-level feature map output by the encoder, and then restores it to 32×32, 64×64 through deconvolution, and finally matches the original image size (256×256). In this process, the decoder combines the spatial position information of the feature map to clearly define the specific pixel range of the lesion in the image (such as the glioma located in the left frontal lobe coordinate interval), and at the same time, the boundary contour is refined to avoid confusion with normal brain tissue.

[0070] The skip connection is used for multi-scale feature fusion. The skip connection directly transmits the feature map of the corresponding level of the encoder to the decoder through cross-layer connection, realizing the complementary fusion of low-level details and high-level semantics. For example, in lung nodule segmentation, the edge texture features (low level) of the shallow layer of the encoder are fused with the nodule contour features (high level) of the corresponding layer in the decoder through the skip connection, which not only retains the fine burrs of the nodule edge (key diagnostic features), but also ensures the accuracy of the overall contour; in the segmentation of X-ray images of fractures, the skip connection can combine the bone texture (low level) with the trend of the fracture line (high level), improving the recognition accuracy of small cracks. This fusion mechanism effectively solves the problem of incomplete feature information of a single level, significantly improving the integrity and accuracy of segmentation. Through the synergistic effect of the above three components, the initial segmentation model can analyze the features layer by layer from complex medical images, accurately locate the lesions, and fuse multi-scale information to optimize the segmentation results, laying the foundation for subsequent generation of structured visual reports.

[0071] The preprocessed image data is input into the initial segmentation model, and the model parameters are iteratively trained using the back propagation algorithm and the loss function to generate the trained segmentation model. In the training process, the dice loss function and the cross-entropy loss function are used to improve the segmentation accuracy of the model for the lesion area. The preprocessed CT image labeled with "left lung nodule" is input, and the model parameters are iteratively optimized using the back propagation algorithm. In the training process, the dice loss function (to reduce the deviation of the segmented area from the actual lesion, such as accurate segmentation of the "nodule edge") and the cross-entropy loss function (to improve the classification accuracy of "nodule" and "normal tissue") are used. After multiple rounds of training, the model can accurately identify lung solid nodules with a diameter of 10 mm or more, and a trained segmentation model is generated.

[0072] The trained segmentation model is applied to medical images to generate structured Chinese visual reports. BM25 sparse retrieval and vector database approximate nearest neighbor semantic retrieval are used to generate a preliminary candidate document list using the inverse ranking fusion algorithm. After inputting the patient's lung CT image into the trained and optimized U-Net model, the system realizes lesion feature extraction and structured report generation through multiple steps, the specific process is as follows: accurate segmentation of the lesion, the model first scans the input lung CT image globally, uses the multi-level features extracted by the encoder (such as the density difference between the nodule and the normal lung tissue, the edge texture), and combines the spatial positioning ability of the decoder to accurately segment the solid nodule area in the upper lobe of the left lung. In this process, the model has boundary recognition ability optimized by the dice loss function, which can clearly distinguish the boundaries between the nodule and the surrounding blood vessels and pleura, ensuring the integrity of the segmented area (such as avoiding missing the small protruding part of the nodule).

[0073] Multi-dimensional feature extraction: For the segmented nodule region, the model automatically extracts key quantitative features and qualitative descriptions: Position feature: Located through the image coordinate system, accurate to the lung lobe partition (such as "left lung upper lobe tip segment"), rather than just labeling "left lung upper lobe"; Size feature: Measure the longest diameter (12mm) and shortest diameter (10mm) of the nodule, and calculate the aspect ratio (1.2) to provide data support for benign and malignant differentiation; Morphology feature: Determine the overall shape of the nodule (circular) through edge detection algorithm, and identify whether there is lobulation (this example has no lobulation); Edge feature: Analyze the edge smoothness (smooth), and record whether there are burrs, pleural traction and other details (this example has no related performance).

[0074] The system integrates the extracted features into a standardized Chinese report according to the preset visual report template (including "image site, lesion feature, measurement data" fixed fields). For example: "Image findings: a solid nodule is visible in the left lung upper lobe tip segment, with a longest diameter of 12mm, a shortest diameter of 10mm, an aspect ratio of 1.2, a circular shape, smooth edges, no lobulation, burrs or pleural traction." The report format is consistent with the "pulmonary nodule image description standard" in the knowledge base, facilitating subsequent semantic alignment with user questions (such as "Does this nodule need surgery?") and retrieved medical knowledge (such as "The probability of a smooth solid nodule less than 15mm being benign is greater than 80%"). This process automatically and standardizes the extraction of image features through the model, converting pixel-level visual information into structured text understandable by machines, providing accurate image basis for multi-modal context integration and LLM reasoning.

[0075] Integrate structured Chinese visual report, user Chinese question and retrieved medical knowledge to generate multi-modal context data, wherein the information is semantically aligned and redundant during the integration process. For the user question "What is the shadow on this CT?", integrate the visual report (the above nodule description) and the retrieved medical knowledge (such as "Pulmonary solid nodules are commonly found in benign adenomas or early lung cancer") to perform semantic alignment (associate "shadow" with "nodule") and redundancy removal (eliminate device parameters unrelated to "nodule properties"), forming a multi-modal context (such as "user asks about the nature of the shadow, the image shows a 12mm circular nodule, and the knowledge base mentions that benign adenomas are mostly smooth edged").

[0076] After inputting the multi-modal context into the LLM, the model generates accurate answers by deep semantic parsing and knowledge fusion, taking into account both image features and medical knowledge. The specific process is as follows: Attention allocation of multi-modal context: The multi-modal context received by the LLM contains three core pieces of information - structured visual report ("left upper lobe 12mm round solid nodule, smooth edge"), user question ("What is the shadow on this CT? Do you need further examination?"), and retrieved medical knowledge (such as "<15mm smooth solid nodule benign probability about 85%, common for adenoma; edge spiculation or lobulation suggests possible malignancy" "Tumor markers CEA, CYFRA21-1 can assist in differentiating benign and malignant lung nodules"). The model dynamically allocates weights through the attention mechanism, focusing on key features in the image ("12mm", "smooth edge") and the user's core demands ("nature judgment", "further examination suggestion"), while associating with matching diagnosis and treatment rules in the knowledge base to ensure that the reasoning is anchored to factual basis.

[0077] LLM constructs reasoning chain based on attention-focused information: First, compare the image features "12mm, round, smooth edge" with the "typical features of benign adenoma" in the knowledge base, and preliminarily determine the benign tendency; second, introduce risk prompts based on the statistical data "<15mm nodules still have 15% risk of malignancy"; finally, based on the "lung nodule diagnosis and treatment guidelines" "diameter >10mm solid nodule suggests further examination", derive the suggestion of tumor marker detection. During the reasoning process, the model automatically checks the logical consistency (such as avoiding the contradictory conclusion "high benign probability but immediate surgery suggestion"), and cites the retrieved guideline content as the basis to suppress "hallucination" generation.

[0078] Beam search decoding strategy is adopted to preferentially select outputs that conform to Chinese medical expression specifications (such as "may be" to reflect uncertainty and avoid absolute judgments). The final answer is based on the integration of reasoning results, optimizing language fluency and clinical practicality: "According to the image features, the left upper lobe 12mm round solid nodule (smooth edge) is consistent with the typical performance of benign adenoma, with a benign probability of about 85%. However, since the diameter of the nodule exceeds 10mm, there is still a certain risk of malignancy (about 15%). It is recommended to further detect tumor markers (such as CEA, CYFRA21-1) to assist in differentiation, and to regularly (3 months later) review CT to observe the changes of the nodule to avoid missing early lung cancer." The answer contains both image feature interpretation and risk assessment and diagnosis and treatment suggestions.

[0079] In one embodiment, as shown in Figure 2 The present application also provides an intelligent medical question and answer device based on hybrid retrieval and light reordering, which comprises:

[0080] The acquisition module 201 is configured to acquire multi-source authoritative Chinese medical data, Chinese medical user input query text, Chinese medical knowledge base documents, and Chinese clinical scene medical image uploaded by a user.

[0081] The processing module 202 is configured to extract five types of core Chinese medical entities based on a preset Chinese medical NER model, model entity association through relation extraction, construct a Chinese medical knowledge graph containing standard names and Chinese alias attributes, form a structured knowledge base hub, extract medical entities from Chinese queries, identify four types of core demands for disease consultation, convert Chinese queries into semantic vectors using a fine-tuned Chinese medical Sentence-BERT model, realize cross-expression semantic equivalent mapping in the Chinese medical field, encode Chinese medical documents using PubMedBERT to generate vectors and store them, perform BM25 keyword retrieval and vector semantic retrieval in parallel when receiving Chinese queries, generate a preliminary candidate document list through RRF algorithm fusion, and give consideration to Chinese keyword accuracy and semantic relevance, generate a Chinese medical question and answer relevance scoring dataset based on a structured prompt template, train a lightweight model to score and sort candidate documents, output a Chinese medical evidence set, and improve evidence accuracy, combine user Chinese query semantics, generate a simplified context through a retrieval-sorting-compression pipeline, integrate FlashAttention technology to optimize attention calculation, and reduce Chinese long text processing memory usage, use an optimized U-Net model to segment medical image lesions and generate structured Chinese visual reports, integrate reports, user Chinese questions, and retrieval knowledge to form a multi-modal context, and generate answer information that takes into account image features and Chinese medical knowledge through LLM reasoning.

[0082] The computer-readable storage medium provided by the above embodiments of the application has the same beneficial effects as the method adopted, run, or implemented by the application program stored therein.

[0083] Each of the embodiments in the present application is described in a related manner, and the same or similar parts of each of the embodiments can be referred to. Each of the embodiments focuses on the differences from other embodiments. In particular, the evaluation of the intelligent medical question and answer method based on hybrid retrieval and lightweight reordering, electronic devices, electronic equipment, and readable storage medium embodiments are basically similar to the above-mentioned embodiments of the intelligent medical question and answer method based on hybrid retrieval and lightweight reordering, so the description is relatively simple, and the relevant parts can be referred to in the above-mentioned embodiments of the intelligent medical question and answer method based on hybrid retrieval and lightweight reordering.

Claims

1. An intelligent medical question answering method based on hybrid retrieval and lightweight reordering, characterized in that, Comprise: Obtain multi-source authoritative Chinese medical data, Chinese medical user input query text, Chinese medical knowledge base documents, and user uploaded Chinese clinical scene medical images; Extract five types of core Chinese medical entities based on a preset Chinese medical NER model, model entity associations through relation extraction, and construct a Chinese medical knowledge graph containing standard names and Chinese alias attributes to form a structured knowledge base hub; Extract medical entities from user Chinese queries, identify four types of core demands for disease consultation, convert Chinese queries into semantic vectors using a fine-tuned Chinese medical Sentence-BERT model, and achieve cross-expression semantic equivalent mapping in the Chinese medical field; Encode Chinese medical documents using PubMedBERT to generate vectors and store them, perform BM25 keyword retrieval and vector semantic retrieval in parallel when receiving Chinese queries, and generate a preliminary candidate document list through RRF algorithm fusion, taking into account the accuracy of Chinese keywords and semantic relevance; Generate a Chinese medical question and answer relevance scoring dataset based on structured prompt templates, train a lightweight model to score and rank candidate documents, and output a Chinese medical evidence set to improve evidence accuracy; Combine user Chinese query semantics, generate a simplified context through a retrieval-ranking-compression pipeline, integrate FlashAttention technology to optimize attention calculation, and reduce memory usage for processing Chinese long text; Use an optimized U-Net model to segment medical image lesions and generate structured Chinese visual reports, integrate reports, user Chinese questions, and retrieval knowledge to form a multi-modal context, and generate answers that consider both image features and Chinese medical knowledge through LLM reasoning.

2. The method of claim 1, wherein, Extract five types of core Chinese medical entities based on a preset Chinese medical NER model, model entity associations through relation extraction, and construct a Chinese medical knowledge graph containing standard names and Chinese alias attributes to form a structured knowledge base hub, including: Integrate and preprocess multi-source authoritative Chinese medical data to generate preprocessed text data; Build an entity extraction framework based on a preset Chinese medical NER model to generate an entity recognition model, where the preset Chinese medical NER model extracts five types of core Chinese medical entities from preprocessed text, including disease entities, symptom entities, drug entities, examination entities, and treatment plan entities; Input preprocessed text data into the entity recognition model and generate entity association data in combination with a relation extraction model, where the relation extraction process models the medical logical associations between entities to clarify the semantic relationships between entities; Construct an initial Chinese medical knowledge graph with entities as nodes and entity relationships as edges to generate a basic graph structure, where nodes contain core information of entities and edges label specific association types between entities; Perform attribute extension and optimization on the basic graph structure to generate a structured knowledge base hub, where the standard name, definition, and Chinese alias attributes of the node are extended, and incorrect associations in the graph are corrected to improve the completeness and accuracy of the knowledge graph, providing support for subsequent semantic disambiguation and answer generation.

3. The method of claim 1, wherein, From the user's Chinese query, extract medical entities, identify four types of core demands for disease consultation, and use fine-tuned Chinese medical Sentence-BERT model to convert Chinese query into semantic vector, realize cross-expression semantic equivalent mapping in Chinese medical field, including: Preprocess the user's input Chinese query text to generate standardized query text, where preprocessing includes tokenization, stop word removal, and special symbol cleaning to ensure the standardization of the query text, laying the foundation for subsequent entity extraction and intent recognition; Build a medical entity extraction framework based on NER model to generate entity recognition tools, which focus on extracting medical entities from standardized query text and accurately capturing core medical elements in the query; Input the standardized query text into the intent classifier to generate core demand labels, where the classification process identifies four types of core demands based on text semantics, including disease consultation, medication consultation, concept query, and treatment plan inquiry, to clarify the specific purpose of the user's query; Build a semantic vector conversion framework based on fine-tuned Chinese medical Sentence-BERT model to generate vector conversion tools, which take complete queries or "entity + intent" reconstructed sentences as input and convert them into high-dimensional dense semantic vectors to realize mathematical representation of semantics; Realize cross-expression semantic equivalent mapping through vector space calculation to generate semantic association results, where the cosine similarity of different query vectors is calculated to ensure that semantically equivalent Chinese medical queries are understood consistently and improve the system's processing capability for diverse expressions.

4. The method of claim 1, wherein, Encode Chinese medical documents using PubMedBERT and store them, perform BM25 keyword retrieval and vector semantic retrieval in parallel when receiving Chinese queries, and generate a preliminary candidate document list through RRF algorithm fusion, taking into account the accuracy of Chinese keywords and semantic relevance, including: Build a text encoding and storage infrastructure based on the PubMedBERT model to generate an initial encoding storage framework, where the initial encoding framework includes a text feature extraction layer, a vector conversion layer, and a storage layer. The feature extraction layer is used to extract Chinese medical entities and relationship features, the vector conversion layer is used to convert entities and relationships into vector representations, and the storage layer is used for structured storage of vectors; Input preprocessed text data into the initial encoding storage framework, extract and model entities and associated relationships using entity recognition algorithms and relationship extraction algorithms to generate intermediate knowledge data, where entity boundary loss functions and relationship classification loss functions are used during the extraction process to improve the extraction accuracy of five types of core Chinese medical entities and associated relationships. At the same time, PubMedBERT is used to encode the document to generate semantic vectors and store them in the vector database; Expand and integrate the intermediate knowledge data to generate a Chinese medical knowledge graph, where properties such as standard names and Chinese aliases are added to entities during the expansion process, and entity association relationships are verified and corrected to ensure the completeness and accuracy of the knowledge graph. Simultaneously, vector retrieval indexes are established to improve the efficiency of subsequent semantic retrieval. The Chinese medical knowledge graph is stored in a special database to generate a structured knowledge base hub. During storage, the entity and relationship structure is stored in a graph database format, the document semantic vector is stored in a vector database, and the two databases are cooperated and associated indexed to support subsequent hybrid retrieval. When receiving a Chinese medical user query, BM25 sparse retrieval and vector database approximate nearest neighbor semantic retrieval are performed in parallel, and a preliminary candidate document list is generated by using a reciprocal ranking fusion algorithm.

5. The method of claim 1, wherein, In combination with the semantic of the user's Chinese query, a simplified context is generated through a retrieval-sorting-compression pipeline, and the FlashAttention technology is integrated to optimize attention calculation and reduce memory occupation in Chinese long text processing, including: The semantic association analysis and preprocessing are performed on the user's Chinese query text and the retrieved Chinese long text data to generate associated preprocessing data, wherein the associated preprocessing data includes semantic alignment and redundant information screening of the text, and the retrieved results are sorted according to relevance; A simplified context generation framework is constructed based on the retrieval-sorting-compression pipeline to generate an initial processing framework, wherein the initial processing framework includes a retrieval module, a sorting module and a compression module, the retrieval module is used to match relevant texts, the sorting module is used to sort according to relevance, and the compression module is used to extract core information; The associated preprocessing data is input into the initial processing framework, the user query semantic guidance and the lightweight reordering model are used to evaluate the priority of the text segments, and intermediate simplified text is generated, wherein the relevance scoring mechanism and information density evaluation index are used in the evaluation process to retain high-value content; The FlashAttention technology is integrated for attention calculation optimization of the intermediate simplified text to generate a target simplified context, wherein the memory access optimization and calculation core fusion are used in the optimization process to reduce the memory occupation in Chinese long text processing and improve the processing efficiency.

6. The method of claim 5, wherein, An optimized U-Net model is used to segment medical image lesions and generate structured Chinese visual reports. The reports, user Chinese questions and retrieval knowledge are integrated to form a multi-modal context, and the LLM inference is used to generate answer information considering image features and Chinese medical knowledge, including: The medical image data and associated text information are preprocessed to generate preprocessed image data and structured text templates, wherein the preprocessing includes lesion region enhancement of the image, standardized size adjustment, format unification of the visual report template and field specification; An optimized medical image segmentation model structure is constructed based on the U-Net framework to generate an initial segmentation model, wherein the initial segmentation model includes an encoder, a decoder and a skip connection, the encoder is used to extract image lesion features, the decoder is used to locate the lesion region, and the skip connection is used to fuse features at different levels; The preprocessed image data is input into the initial segmentation model, and the model parameters are iteratively trained by using a back propagation algorithm and a loss function to generate a trained segmentation model, wherein the dice loss function and the cross-entropy loss function are used in the training process to improve the segmentation accuracy of the model for the lesion region; The trained segmentation model is applied to the medical image to generate a structured Chinese visual report. Integrating structured Chinese visual reports, user Chinese questions and retrieved medical knowledge to generate multi-modal context data, wherein semantic alignment and redundancy removal are performed during the integration process; Inputting the multi-modal context data into the LLM to generate answer information using attention mechanism and generative decoding strategy. 7.An intelligent medical question answering device based on hybrid retrieval and lightweight reordering, configured to implement the method of claim 1.

8. An electronic device, comprising: Comprise: a first processor; and a memory for storing executable instructions of the first processor; wherein the first processor is configured to implement the intelligent medical question answering method based on hybrid retrieval and lightweight reordering of any one of claims 1-6 by executing the executable instructions.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the second processor to implement the intelligent medical question answering method based on hybrid retrieval and lightweight reordering of any one of claims 1-6.

Images