Medical question answering method and system based on multi-stage preference alignment retrieval enhancement

Abstract

The application provides a medical question and answer method and system based on multi-stage preference alignment retrieval enhancement, and belongs to the technical field of natural language processing, and comprises the following steps: performing multi-pass semantic expansion on input medical problems and patient case information, integrating query expressions generated by different paths to obtain a multi-pass input set, inputting the multi-pass input set into a retrieval model to obtain a retrieval evidence set; based on the retrieval evidence set, the medical problems and the patient case information, introducing a preset prompt strategy, using a large language model to analyze the features of the case information to obtain a clinical feature set and iteratively updating the clinical feature set to obtain a retrieval candidate evidence set; screening supporting documents based on the retrieval candidate evidence set; and inputting the supporting documents, the medical problems and the patient medical record information into a model optimized through multi-stage preference learning to generate corresponding medical reasoning processes and question and answer results. The application improves the overall stability and accuracy of the medical question and answer system.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing technology, and in particular relates to a medical question-answering method and system based on multi-stage preference alignment retrieval enhancement. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] The medical setting places much higher demands on the accuracy, reliability, and interpretability of responses than general question-answering tasks. Any output lacking medical basis or with flawed reasoning may pose serious clinical risks.

[0004] To improve the reliability of medical question answering, existing retrieval-enhanced medical question answering methods rely primarily on implicit medical knowledge within the large language model during answer generation and reasoning, neglecting retrieved medical evidence. Furthermore, these methods are prone to introducing non-critical documents into the reasoning process when faced with noise or interfering evidence, affecting the accuracy of the final conclusion. Consequently, even if some methods can produce relatively correct results, issues such as logical jumps, unclear medical causal relationships, or insufficient integration of patient case information may still exist during the question-answering reasoning process. Summary of the Invention

[0005] To overcome the shortcomings of the existing technologies, this invention proposes a medical question-answering method and system based on multi-stage preference alignment retrieval enhancement, which solves the problems of poor document usage and medical reasoning quality caused by factors such as weakened retrieval enhancement mechanisms, introduction of non-critical documents and logical jumps during retrieval.

[0006] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: In a first aspect, this invention discloses a medical question-answering method based on multi-stage preference alignment retrieval enhancement, comprising: The input medical questions and patient case information are subjected to multi-path semantic expansion, which includes a path for extracting structured elements of case information, a path for completing background knowledge of medical questions, and a path for constructing literature associations. The query expressions generated by different paths are integrated to obtain a multi-path input set. The multi-path input set is then input into the retrieval model for retrieval to obtain a set of retrieval evidence. Based on the aforementioned set of retrieval evidence, medical questions, and patient case information, a preset prompting strategy is introduced. A large language model is used to perform feature analysis on the case information to obtain a set of clinical features, which is then iteratively updated to obtain a set of candidate retrieval evidence. Supporting documents are selected based on the aforementioned set of candidate evidence. Based on the input of supporting documents, medical questions, and patient medical records, a large language model optimized through multi-stage preference learning generates corresponding medical reasoning processes and question-and-answer results.

[0007] Secondly, this invention discloses a medical question-answering system based on multi-stage preference alignment retrieval enhancement, comprising: The retrieval enhancement module is used to perform multi-path semantic expansion on the input medical questions and patient case information. The multi-path semantic expansion includes a path for extracting structured elements of case information, a path for completing background knowledge of medical questions, and a path for constructing literature associations. The query expressions generated by different paths are integrated to obtain a multi-path input set. The multi-path input set is then input into the retrieval model for retrieval to obtain a set of retrieval evidence. The feature parsing module is used to analyze the case information based on the retrieved evidence set, medical questions and patient case information, introduce a preset prompting strategy and use a large language model to obtain a set of clinical features and iteratively update it to obtain a set of candidate evidence for retrieval. The document filtering module is used to filter supporting documents based on the search candidate evidence set; The reasoning test module is used to generate corresponding medical reasoning processes and question-and-answer results based on the model optimized through multi-stage preference learning, which is based on the input of supporting documents, medical questions, and patient medical record information.

[0008] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention, on the one hand, introduces a retrieval enhancement mechanism and performs comparative learning on the reasoning behavior of a large language model under unified evidence conditions. This enables the medical question-answering process to explicitly rely on external knowledge to complete reasoning, effectively avoiding responses based solely on the model's internal knowledge, thereby significantly improving the accuracy and reliability of medical question-answering results. On the other hand, it performs multi-path semantic expansion retrieval of medical questions and combines it with clinical feature analysis to construct a broader and more relevant set of retrieval evidence from authoritative medical knowledge bases, and further filters out supporting documents that can support key aspects of medical reasoning. Finally, through deep coupling of retrieval enhancement and multi-stage preference optimization, not only is the final result significantly improved, but the reasoning process is also made more reliable. This method maintains highly reliable reasoning behavior and stable output results in different medical questions and complex retrieval environments, thereby improving the overall stability of the medical question-answering system.

[0009] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0010] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0011] Figure 1 This is a logical block diagram of the medical question-answering method based on multi-stage preference alignment retrieval enhancement in Embodiment 1 of the present invention.

[0012] Figure 2 This is a block diagram of the retrieval enhancement logic in Embodiment 1 of the present invention.

[0013] Figure 3 This is a flowchart of the construction of positive and negative samples in Embodiment 1 of the present invention. Detailed Implementation

[0014] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0015] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0016] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0017] With the development of artificial intelligence technology, medical question-answering systems based on natural language processing have received widespread attention in scenarios such as clinical decision support, medical education, and patient health consultation. In particular, large language models have demonstrated strong capabilities in medical text understanding and generation, making them potentially valuable for answering complex medical questions.

[0018] However, given the high demands of medical scenarios, and to improve the reliability of medical question-answering systems, existing technologies have gradually incorporated external medical knowledge bases. These bases utilize retrieval-enhanced generation methods, incorporating clinical guidelines, medical textbooks, and authoritative literature as supplementary evidence into the model's reasoning process. This approach alleviates, to some extent, the problems of outdated or insufficiently covered knowledge resulting from models relying solely on internal parameter knowledge, providing richer information sources for medical question answering.

[0019] However, existing retrieval-enhanced medical question-answering methods still have many shortcomings in practical applications. First, large language models often fail to fully and stably rely on retrieved medical evidence during answer generation and reasoning. Even when relevant documents are provided, they may still primarily rely on implicit medical knowledge within the model for inference, thus weakening the actual effectiveness of the retrieval enhancement mechanism. Second, since medical retrieval evidence typically contains a large number of documents related to the question topic but lacking key value for specific decisions, and may even contain information supporting erroneous conclusions, existing methods are prone to introducing non-critical documents into the reasoning process when faced with noisy or interfering evidence, affecting the accuracy of the final conclusion. Furthermore, in existing technologies, there is often a lack of strict constraints between the reasoning process generated by the model and the medical evidence it cites, easily leading to inconsistencies in the citation of retrieved documents, such as citing non-existent documents or citing documents whose content cannot support the corresponding medical claims. These problems not only reduce the credibility of the system output but also limit its application in real-world medical scenarios. Furthermore, even if some systems can provide correct question-and-answer results, the reasoning process they generate may still have problems such as logical jumps, unclear medical causal relationships, or insufficient integration of patient case information, making it difficult to meet the requirements of clinical decision-making for high-quality medical reasoning.

[0020] Existing methods for enhancing medical question answering still have shortcomings in terms of document usage and the quality of medical reasoning, and are insufficient to fully meet the comprehensive requirements of security, reliability, and interpretability in real-world medical scenarios. Therefore, a technical solution is urgently needed to address these issues.

[0021] Example 1 In one or more embodiments, a medical question-answering method based on multi-stage preference alignment retrieval enhancement is disclosed, such as... Figure 1 As shown, by constructing high-quality retrieval evidence and optimizing preferences in multiple stages, the large language model can correctly rely on evidence, suppress noise interference, maintain evidence consistency, and conform to the medical reasoning process of the evidence chain constraint in medical question answering. This includes the following steps: Step S100: Define the task.

[0022] The goal of the retrieval-enhanced biomedical question-answering task is to select the correct answer from a set of candidate answers using patient-specific information and retrieved medical evidence. Each instance is defined as follows:

[0023] in, This indicates Patient Case Information (PCI). Indicating medical issues, It is a set of candidate answers, with only one correct option. Search enhancement methods are commonly used to retrieve medical evidence as a basis for selecting the final answer. Formally, a standard search is defined as:

[0024] Where D is the set containing the retrieved medical documents.

[0025] In safety-critical medical question-answering tasks, large language models not only need to predict the correct answer but also ensure that the reasoning process for generating the answer is sufficiently reliable. Therefore, a retrieval-enhanced biomedical question-answering task can be formulated as follows:

[0026] in, Represents a parameterized language model. It is a reasoning process. It is the final answer to the prediction.

[0027] Preferably, in order to ensure the use of evidence and the quality of reasoning in enhanced medical question answering, this embodiment proposes the following four medical evidence constraints (MECs).

[0028] Evidence dependence refers to the model's ability to explicitly base medical decisions on retrieved external evidence, rather than solely relying on implicit medical knowledge encoded in the model parameters. The optimization goal at this stage is to ensure that the retrieved evidence is fully utilized during the reasoning process when relevant medical evidence is provided.

[0029] Noise-resistant evidence refers to the ability of a model to distinguish between supporting medical evidence and irrelevant or confounding evidence in the presence of noisy evidence, prioritizing reasoning based on key supporting evidence to avoid erroneous medical decisions caused by irrelevant information. In retrieval-enhanced medical question answering, some retrieved documents contain content related to the question topic but lacking decision-making value, and may even contain content supporting incorrect answers; these documents are called noisy documents. If the model indiscriminately uses such documents during reasoning, even if the final answer is correct, the medical reasoning process may still be based on non-critical or misleading evidence, thus reducing the reliability of the reasoning.

[0030] Evidence consistency refers to the strict consistency between the explicit use of evidence by the model during reasoning and the actual support provided by the retrieved documents. In retrieval-enhanced medical reasoning, the model may exhibit unfaithful use of evidence, such as citing non-existent documents or citing existing documents that do not support the corresponding medical conclusion. Such evidence inconsistency weakens the credibility of the medical reasoning results.

[0031] Evidence chain constraints refer to the model's ability to generate concise, coherent, and clinically reasonable reasoning. This reasoning explicitly integrates key information from the case, appropriately incorporates retrieved evidence, and proceeds in a progressive and consistent manner, avoiding leaps in reasoning, unfounded assumptions, or clinically unreasonable conclusions.

[0032] These four medical evidence constraints target different capability dimensions in retrieval-enhanced medical reasoning and exhibit a hierarchical dependency in their functional roles. Evidence dependence, as a fundamental requirement, ensures that the model explicitly bases its decisions on retrieved external evidence. Building upon this, noise-resistant evidence guides the model to prioritize key decision-making evidence under interference conditions. Evidence consistency ensures that cited evidence aligns with generated arguments, while the chain of evidence focuses on the overall coherence and clinical plausibility of the reasoning process, further enhancing the reliability of medical reasoning. Since higher-level reasoning quality presupposes basic evidentiary capability, these constraints are introduced sequentially.

[0033] This invention constrains the evidence citation behavior during medical question-and-answer reasoning, ensuring that the reasoning process generated by the model is semantically consistent with the cited medical documents. This avoids situations where non-existent documents are cited or the cited documents do not match the conclusions, thereby improving the traceability of the reasoning process and the credibility of medical evidence.

[0034] To ensure that the retrieval-enhanced reasoning model is consistent with the four evidence criteria (MEC), a high-quality retrieval evidence set is first constructed, and then the sequential evidence constraint DPO is performed.

[0035] This embodiment obtains candidate documents through a multi-path semantic expansion retrieval mechanism and clinical feature parsing, and then filters out supporting documents. These supporting documents serve as a high-quality retrieval evidence set for preference optimization, such as... Figure 2 As shown, the specific steps include S101-103: Step S101: Perform multi-path semantic expansion on the input medical problem and patient case information. The multi-path semantic expansion includes a path for extracting structured elements of case information, a path for completing background knowledge of medical problems, and a path for constructing literature associations. Integrate the query expressions generated by different paths to obtain a multi-path input set. Input the multi-path input set into the retrieval model to obtain a retrieval evidence set.

[0036] Step S1011: Structured Element Extraction Path: This path parses key medical elements with diagnostic value from the original medical problem input and transforms them into structured semantic units.

[0037] Specifically, through pre-defined prompting strategies, the large language model is guided to identify core information in patient medical records, including but not limited to symptoms, physiological indicators, past medical history, medication use, and laboratory test results. This information is then transformed into multiple independent and semantically clear element descriptions. The large language model extracts key elements from the patient's medical record information based on prompts and lists them one by one. By explicitly expressing these key medical elements, the retrieval process can more accurately match diagnostically relevant medical knowledge fragments, thereby improving the relevance of the retrieval results.

[0038] Step S1012: Background Knowledge Completion Path: This path performs semantic expansion of the input medical problem at the knowledge level by introducing medical background information related to the problem to make up for the deficiencies in the knowledge expression of the original input.

[0039] Specifically, leveraging the medical knowledge generation capabilities of a large language model, by providing prompt words, the model supplements with relevant knowledge points related to the problem, thereby constructing a set of potentially related medical concepts, disease mechanisms, diagnostic principles, or treatment strategies around the current problem. Through this process, the retrieval query is no longer limited to surface semantic matching but can cover deeper levels of medical knowledge association, thus improving the breadth and semantic completeness of the retrieval results. This provides the retrieval model with deeper conceptual connections and knowledge context.

[0040] In this embodiment, the retrieval model adopts models such as MedCPT, without specific limitations.

[0041] Step S1013: Literature Association Construction Path: This path establishes the association between the input medical question and the structured medical knowledge system. By generating descriptions of medical literature or guidelines that are semantically related to the question, it enhances the retrieval process's ability to match authoritative knowledge sources.

[0042] Specifically, based on the input medical question as a prompt, the model generates descriptions of relevant medical literature topics or knowledge units, thereby constructing several potentially related medical literature topics or knowledge unit descriptions. This allows the search process to cover relevant content in clinical guidelines, medical textbooks, and research literature. This approach helps improve the alignment between search results and authoritative medical knowledge systems, thus enhancing the credibility of the search results.

[0043] Step S1014: Integrate multi-path semantic expansion retrieval.

[0044] After completing the multi-path semantic expansion retrieval described above, the query expressions generated by different paths are integrated to obtain a multi-path input set, which is then input into the retrieval model to perform retrieval operations and obtain a retrieval evidence set. To balance retrieval accuracy and coverage, differentiated retrieval weights are assigned to queries generated by different paths.

[0045] To improve the coverage and semantic matching accuracy of search results, this embodiment designs a multi-path semantic expansion mechanism. It decomposes and reconstructs the input medical questions and patient case information, expands the original input from different semantic levels, and performs search operations based on the expanded query set.

[0046] Step S102: Based on the retrieved evidence set, medical questions, and patient case information, a preset prompting strategy is introduced, and a large language model is used to perform feature analysis on the case information to obtain a set of clinical features, which is then iteratively updated to obtain a set of candidate retrieved evidence.

[0047] Step S1021: During the t-th round of processing, the set of retrieval evidence obtained in the current round is... Medical issues and patient case information As a combined input, a preset prompting strategy is introduced. This enables large language models to retrieve evidence sets. As a source of relevant knowledge, key information from patient medical records is extracted as a set of clinical features, driving a large language model to perform feature analysis on the case content. This process can be formally represented as follows:

[0048] in, For large language models, Represents the set of clinical features generated in round t, suggesting a strategy. This is used to constrain the form of the model output, focusing it on the expression of clinical information relevant to the problem and organizing it in a structured manner.

[0049] Preferred prompting strategy This includes sources of knowledge, patient case information, and related issues.

[0050] Step S1022: Iteratively update feature parsing.

[0051] To further improve the accuracy and completeness of clinical feature representation, this invention employs an iterative optimization mechanism to progressively update the input semantics. After each round of feature parsing, the clinical feature set generated in the current round is... With medical issues By splicing and merging, new forms of expression can be constructed. This is used to drive the next round of multi-path semantic expansion retrieval and parsing process, and its update method is as follows:

[0052] in, This is a concatenation function used to combine the patient medical record information extracted in the previous stage with the original question to form a new input.

[0053] Subsequently, multi-pathway retrieval and clinical feature analysis are performed based on the updated semantic representation, and this process is repeated. The evaluation index of the results from two consecutive rounds of retrieval is less than a preset threshold. (Right now The iteration process terminates when the number of iterations reaches a preset upper limit T. The final output is a stable set of 32 candidate evidence samples. (i.e., candidate documents). This embodiment uses 32 candidate documents to ensure a balance between selecting a sufficient number of supporting documents and the computational limitations of the language model.

[0054] The patient's medical record information is semantically parsed and processed to construct clinical feature representations related to the current medical problem, thereby improving the matching accuracy and effectiveness of subsequent retrieval evidence.

[0055] Step S103: Filter supporting documents based on the retrieved candidate evidence set.

[0056] Based on the candidate documents, further select supporting documents that are crucial to inferring the correct answer. The selection of supporting documents employs a structured prompting method, where the model selects based on the input instance P, the correct answer a*, and candidate documents. Select appropriate supporting documentation. Specifically, a document is considered supporting documentation if it meets one of the following criteria: (1) This document provides direct medical evidence supporting the correct answer; (2) The document provides the necessary medical background, which is highly matched with the key clinical clues in the question and significantly strengthens the reasoning chain. It should be understood that the necessary medical background is that if the document is removed, the model cannot successfully complete the inference from the key clinical clues to the correct answer. The document covers most of the key clinical clues, for example, more than 70%, and these covered clues are consistent with the core medical decision-making needs required by the question and can significantly strengthen the reasoning chain. Therefore, the document can be considered to be highly matched with the question.

[0057] (3) This document contains decisive information that effectively eliminates one or more incorrect options.

[0058] Finally, several supporting documents are kept for each question. This serves as a unified evidentiary basis for subsequent preference optimization phases:

[0059] in, This is the filtering function.

[0060] This embodiment fixes the number of supporting documents at five, as this number is generally sufficient to cover the key medical evidence needed to infer the correct answer. Using fewer documents may result in insufficient coverage of key evidence, making it difficult to support multi-step clinical reasoning; while using more documents tends to introduce irrelevant information, thereby weakening the key evidence for decision-making.

[0061] This invention suppresses the influence of interfering documents on the inference process, allowing the model to prioritize supporting documents that are highly relevant to medical decisions during inference. This enables the model to stably focus on supporting documents for inference even in the presence of irrelevant or interfering medical documents, significantly enhancing the system's robustness to noisy documents.

[0062] Step S104: Based on the input of the supporting documents, medical questions and patient medical record information, the large language model optimized by multi-stage preference learning generates the corresponding medical reasoning process and question-and-answer results.

[0063] After receiving the input medical question and patient case information, the system calls the retrieval enhancement module to obtain medical retrieval documents (i.e., supporting documents) related to the medical question. Based on the medical retrieval documents, patient case information, and medical question, the system drives the model optimized through multi-stage preference learning to generate the corresponding medical reasoning process and question-and-answer results.

[0064] The multi-stage preference learning optimization includes the evidence dependence preference alignment stage, the noise-resistant evidence preference alignment stage, the evidence consistency preference alignment stage, and the evidence chain constraint preference alignment stage.

[0065] While instruction fine-tuning enables the model to acquire basic answer generation capabilities with retrieval enhancements, it is insufficient to capture complex and mutually constrained requirements. Therefore, this embodiment uses Direct Preference Optimization (DPO) for sequential preference optimization to progressively align the model with the four EMCs.

[0066] Direct preference optimization specifically refers to: for the same input Construct preference pairs ( + , ), where the preference output (positive sample output) is... + Compared to non-preference output (negative sample output) It better meets the medical evidence constraint (MEC). Each preference output contains a complete reasoning process. And an answer prediction Given a parameterized generative model and a reference model ref The objective of direct preference optimization can be expressed as:

[0067] in, This represents the sigmoid function. It is a temperature parameter used to control the intensity of preference comparison; the base of the log can be 2, 10 or e, etc., without specific limitations.

[0068] In this embodiment, the model is optimized from four aspects of medical evidence-constrained MECs: evidence dependence, robust evidence, evidence consistency, and evidence chain constraint. Each MEC is tuned in a dedicated training phase. In all optimization phases, the model is trained based on retrieval with multi-path semantic expansion and the same set of supporting documents. This aims to eliminate confounding factors introduced by evidence variations, allowing the model to learn only reasoning methods and evidence-using behaviors during training.

[0069] Next, we will describe the construction of preference data and the detailed optimization process for each stage.

[0070] Step S1041: Preference dataset construction.

[0071] like Figure 3 As shown, the input construction method is similar for the three medical evidence criteria (MEC), including evidence dependence, evidence consistency, and chain of evidence. Each input... By instruction An example of medical quality assurance = ( , , and a set of fixed supporting documents composition:

[0072] in, It is a prompt used to generate selected preference data consistent with the corresponding MEC.

[0073] For the second MEC, namely noise immunity evidence, its input... Defined as:

[0074] in, To provide a preference for noise-resistant evidence, D This represents a set of noisy documents. Noisy documents that meet the following conditions are selected from the candidate document pool D. candMedium sampling includes: (1) documents that are relevant to the topic of the question but do not support the correct option; (2) documents that primarily support one or more incorrect options; and (3) irrelevant documents.

[0075] Step S1042: Evidence Dependency Preference Alignment Stage, construct evidence dependency preference sample pairs.

[0076] Using evidence dependency constraints, positive samples of evidence dependency are output. 1 + It should reflect evidence-dependent reasoning behavior. The output will only be retained as a positive sample output if the generated reasoning explicitly uses information from supporting documents to support the correct option.

[0077] in, 1 + This is an instruction used to generate positive evidence-dependent samples. Reasoning paths that rely primarily on general medical knowledge or can be completed without referring to provided evidence documents will be excluded. Negative sample output. 1 This is used to describe evidence-independent answering behavior. To generate evidence-dependent negative samples, the model is asked to answer the same medical question but without providing any retrieved documents:

[0078] in, 1 It is a hint used to generate negative samples that depend on evidence.

[0079] Step S1043: Noise-resistant evidence preference alignment stage, constructing noise-resistant evidence preference samples.

[0080] Noise-resistant evidence constraints and noise-resistant evidence output 2 + By providing support documentation only to the model It instructs the generation of a step-by-step medical reasoning process, explicitly specifying the use of supporting evidence to generate the reasoning process. Since there are no noisy documents, these outputs represent reasoning behavior that relies entirely on supporting evidence, implicitly demonstrating their robustness to noise. Therefore, these outputs are retained as positive samples of noise-resistant evidence.

[0081]

[0082] in, 2 + This is a positive sample instruction to generate evidence of robustness against noise. Negative samples... 2 The construction method is: by sending data to the supporting document set D. Add noise to document set D To construct and explicitly constrain the model to reference at least one noisy document as the basis for inference, negative samples of noise-resistant evidence are used:

[0083] in, 2 These are negative sample instructions used to generate noise-resistant evidence. Therefore, the reasoning process includes non-critical or potentially misleading information, a common pattern of reasoning bias under noise injection.

[0084] Step S1044: Evidence consistency preference optimization stage, construct evidence consistency preference sample.

[0085] To ensure consistency of evidence at the sentence level, each sentence referencing a document in the inference output must be appended with a document number, and all sentences are indexed according to their order in the output. These indexes are used only to locate and correct specific document references, while preserving the original reasoning and the final answer. The evidence-consistent inference output is represented as follows:

[0086] Among them, s Indicates the first A reasoning statement, ( ) represents s Referenced document number, It is the total number of statements in the inference output. This is a positive sample hint for consistency of evidence.

[0087] The approach employs automated validation, followed by localized repair, and finally a comparative construction strategy to generate preference output. First, document references in each inference sentence are validated by checking the existence of the referenced document ID and whether the document content semantically supports the corresponding medical claim. This is represented as:

[0088] in, ( ) Indicates the first A reasoning sentence The referenced document, 1 indicates ( ) It exists and is semantically supported. 0 indicates that the reference does not exist or does not support the claim. If Verify( , ( ) If ) = 0, then attempt to repair the reference and... Redirecting to the document that actually supports the claim. If the fix is ​​successful, the corrected sentence and its citations will replace the original sentence, resulting in a fixed reasoning output that fully conforms to the evidence. The corrected output is considered a positive sample, while the original output containing inconsistencies in evidence is considered a negative sample. Therefore, positive samples with a preference for consistency of evidence include two cases:

[0089] Repair=1 indicates successful citation repair. If erroneous citations cannot be reliably repaired, the inference output instance will be discarded. Finally, in addition to the outputs inconsistent with the evidence mentioned above, negative sample outputs are also constructed based on positive samples. Specifically, while keeping the reasoning content and final answer unchanged, this embodiment simulates citation errors under natural conditions, modifying citations in at least one sentence to point to documents that do not support the argument or to non-existent document numbers. In this case, the original output consistent with the evidence is considered a positive sample, while the artificially corrupted output is considered a negative sample. Therefore, the negative sample output with evidence consistency preference includes two cases:

[0090] Step S1045: Evidence chain constraint preference alignment stage, construct evidence chain constraint preference samples.

[0091] According to the chain of evidence constraint, in order to generate positive samples, the reasoning process must simultaneously satisfy the following criterion C: (1) The reasoning process explicitly incorporates key clinical information from the patient’s case, such as symptoms, physical examination results, or examination results; (2) All reasoning steps comply with medical standards; no erroneous medical facts are used; there is no invalid medical causal relationship; and all steps do not violate recognized clinical consensus or guidelines. (3) The overall reasoning process is well-structured and logically coherent, without skipping key intermediate steps or drawing abrupt conclusions. (4) The final option selection naturally stems from the preceding reasoning steps and has a medical basis. Chain of evidence constraint output. Represented as:

[0092] in, This is a positive sample hint for the chain of evidence constraint.

[0093] If the initially generated inference output fails to meet at least one of the above criteria The reference model will then be guided to identify violations and regenerate samples. until the reasoning result meets all criteria. Alternatively, the positive sample IDs may meet the training requirements. The corrected inference output will be retained as a positive sample constrained by the chain of evidence, represented as:

[0094] in, To regenerate the sample, To construct the constraint criteria for the preference sample of the evidence chain constraint.

[0095] Evidence chain-constrained negative samples are generated through unconstrained reasoning hints, enabling the reference model to perform free-form stepwise reasoning based on the same evidence set. Evidence chain-constrained negative samples are represented as follows:

[0096] in, This serves as a negative sample hint for the chain of evidence constraint.

[0097] While such reasoning may appear fluent and clinically sound on the surface, it often suffers from one or more of the following problems: omitting key clinical clues from the patient's case, introducing medical hypotheses that are not supported by evidence, skipping necessary diagnostic or exclusion steps, or exhibiting internally inconsistent and clinically illogical reasoning patterns. Therefore, these outputs are considered samples of problematic reasoning structures or poorly normative use of evidence.

[0098] Step S1046: Preference optimization.

[0099] The four preference optimizations mentioned above are implemented sequentially in a course-like manner, from basic to advanced, with phased training:

[0100] in, ∈{1, 2, 3, 4} represents the training phase. Representation phase DPO target, Dynamically set to stage The proportion of preference data.

[0101] The preferred data proportions are set as follows: Stage 1 consists entirely of preferred data from this stage; Stage 2 adds 25% preferred data from Stage 1, with the remainder being preferred data from this stage; Stage 3 adds 15% data from Stage 1, 10% from Stage 2, with the remainder being preferred data from this stage; Stage 4 adds 10% data from Stage 1, 10% from Stage 2, and 5% from Stage 3, with the remainder being preferred data from this stage. It should be understood that choosing a 75% proportion for the current stage and 25% for previous stages is because this forms a relatively stable anti-forgetting buffer. It prevents severe forgetting due to insufficient replay data, and also avoids suppressing learning in the current stage due to excessive replay data. Furthermore, the more fundamental the ability, the higher the proportion of anti-forgetting data is because fundamental abilities are learned earliest. Training is phased, with later stages continuously introducing new objectives. The earlier the learning is, the further away it is from the current parameter state, the more updates it undergoes, and the easier it is to be diluted.

[0102] In the aforementioned multi-stage preference alignment training process, since different stages optimize for different capability dimensions, directly training with only data from the current stage would cause the model to forget the capabilities acquired in previous stages during subsequent learning processes. Therefore, this invention introduces an anti-forgetting mechanism in the multi-stage preference alignment training process. When training any subsequent preference alignment stage, in addition to using the preference samples corresponding to the current stage, preference samples from previous preference alignment stages are also introduced as replay data for joint training. The proportion of preference samples introduced from different stages is set according to their capability level, ensuring that more fundamental capability samples have a higher proportion in the replay data. This ensures that the model maintains a stable grasp of basic evidence usage capabilities while learning subsequent medical reasoning behaviors.

[0103] As one implementation method, the large language model adopts the Llama-3.1-8B-Instruct model; other commonly used large language models can also be used, such as the Qwen series, other Llama series, GPT series, Deep Seek series, etc.

[0104] This invention guides the model to make step-by-step inferences by combining patient case information with external medical evidence during the medical question-and-answer process. This results in a concise, logically clear, and structurally complete inference chain, reducing inference jumps and the generation of unreasonable medical assumptions, and significantly improving the standardization, interpretability, and clinical rationality of the medical reasoning process.

[0105] As one implementation method, an example of a detailed implementation process using an existing dataset further illustrates the procedure of this method.

[0106] Step S201: Multi-path semantic expansion retrieval mechanism.

[0107] Step S2011: Structured feature extraction path.

[0108] The model uses the following instructions: Please extract several key information points with diagnostic value from the above input. Each point should be a concise and clear description of an independent medical feature. Please output in item form, with each item corresponding to one key piece of information: "Item 1", "Item 2", "Item 3".

[0109] For example, in response to the input: "Electronic Health Record": "A 72-year-old male patient presented with worsening fatigue and abdominal pain over a period of two months. During this time, he also experienced night sweats and shortness of breath on exertion. He has lost 5.6 kg (12 lbs) in the past three months. He had a myocardial infarction three years ago. He has hypertension, diabetes, and chronic bronchitis. His current medications include insulin, aspirin, lisinopril, and salbutamol inhaler…", the "Question": "What is the most appropriate next step in treatment?"

[0110] Model output: "A. Significant wasting symptoms (weight loss, night sweats, fatigue); B. Severe normocytic anemia with thrombocytopenia (Hb 6.4 g / dL, PLT 96,000 / mm³); C. Significant splenomegaly (approximately 6 cm below the left costal margin); D. JAK2 mutation positive, bone marrow aspiration showed extensive fibrosis"; Step S2012: Background knowledge completion path.

[0111] The model uses the following instructions: Please generate medical background information relevant to the current problem to supplement the knowledge base needed to understand it. The output should only contain relevant knowledge points.

[0112] Model output: "Primary myelofibrosis: Commonly seen in the elderly, characterized by extensive bone marrow fibrosis, JAK2 mutation positivity, splenomegaly, and peripheral blood cytopenia. Wasting symptoms (B symptoms): Includes fatigue, night sweats, and weight loss, suggesting high activity of myeloproliferative neoplasms. Aberrant activation of the JAK-STAT pathway: JAK2 mutations lead to abnormal hematopoiesis and inflammatory responses, making it an important therapeutic target. JAK inhibitors (such as ruxolitinib): Used to relieve splenomegaly and systemic symptoms; a first-line symptomatic treatment for patients unsuitable for transplantation. Allogeneic hematopoietic stem cell transplantation: Currently the only potentially curative treatment, but mostly limited to young patients in good overall condition. Supportive care: Includes blood transfusions to treat severe anemia, but does not improve disease progression…" Step S2013: Constructing a path for linking documents.

[0113] The model uses the following instruction: "Please generate several medical literature topics or knowledge entries that are semantically related to this question, to represent the possible directions of medical information involved. The output only contains relevant topic content." Model output: [“Clinical manifestations and diagnostic points of primary myelofibrosis”, “Differential diagnosis and treatment options for myelofibrosis”, “The role of the JAK-STAT signaling pathway in myelofibrosis”, “…”, “Prognostic assessment and treatment decisions for patients with primary myelofibrosis”.

[0114] Step S2014: Multi-path semantic expansion retrieval fusion.

[0115] The structured element extraction path allocates 3 / 4 of the search slots to obtain highly relevant core medical evidence; the background knowledge completion path and the literature association construction path each allocate 1 / 8 of the search slots, and the generated queries are used to supplement relevant background knowledge and expand the semantic scope.

[0116] Step S202: Perform clinical feature analysis using a preset prompting strategy. The preset prompting strategy is as follows: The following is about the sources of knowledge:

[0117] The following is a description of the patient's medical record:

[0118] The following is the question:

[0119] Please analyze and extract the key information related to this issue from the patient's medical record description. Please provide a concise description of each piece of information. For example, "Patient age: 72 years old (elderly) Symptoms: fatigue, worsening abdominal pain, night sweats, shortness of breath after activity, weight loss of 5.6 kg (12 lbs) in 3 months (multiple symptoms) Past medical history: myocardial infarction 3 years ago, hypertension, diabetes, chronic bronchitis (past illnesses) Medications: insulin, aspirin, lisinopril, salbutamol inhaler (current medication) Smoking history: 45 years, half a pack of cigarettes per day (past smoking history) Vital signs: normal (specific vital signs not mentioned) Physical examination results: spleen palpable 6 cm below the left costal margin (splenomegaly) Laboratory test results: + Hemoglobin: 6.4 g / dL (anemia) + Mean corpuscular volume: 85 μm3 (normal) + White blood cell count: 5,200 / mm3 (leukopenia) + Platelet count: 96,000 / mm3..." Please provide only a description. The following is a description:

[0120] Through iterative updates using feature parsing, a stable set of 32 retrieval candidate evidence documents is ultimately output. Using 32 candidate documents ensures a balance between selecting a sufficient number of supporting documents and the computational constraints of the language model.

[0121] Step S203: Use a structured suggestion method to filter 5 supporting documents.

[0122] Step S204: Multi-stage preference learning and reasoning test.

[0123] (1) The evidence dependence preference alignment stage, specifically: The positive sample instruction is: "Below is a medical multiple-choice question and excerpts of relevant documents. Your task is to write a chain of reasoning to answer the question based on these document excerpts."

[0124] The questions are as follows: ; The medical record information is as follows: ; The relevant documents are excerpted below: ; The candidate answers are as follows: ; Please generate the inference chain as follows: Use step-by-step reasoning (e.g., "Step 1, Step 2..."). Clearly cite relevant document excerpts and use [1], [2], [3]... to mark the sources of evidence. (Note: The numbers [1], [2], [3]... in square brackets represent different document excerpt numbers.) Reasoning must be based on the provided document excerpts and no fabricated references may be used.

[0125] Finally, please provide your answers in the following format: Final answer: Please show your reasoning process and the final answer. For example: "Step 1: The question asks for the most sensitive test to detect kidney damage in diabetic patients. The patient has a 15-year history of diabetes and is therefore at risk of diabetic nephropathy. Step 2: According to the document excerpt, the early manifestation of diabetic nephropathy is increased urinary albumin, which can be detected by sensitive detection methods, but cannot be detected by routine urine analysis or test strips. The excerpt [3] states: "The earliest manifestation is increased urinary albumin, which can be detected by sensitive radioimmunoassay... The amount of urinary albumin excretion within 24 hours of increased urinary albumin is called microalbuminuria." Step 3: The excerpt [2] specifically recommends that "for patients with type 1 or type 2 diabetes, it is recommended to have a microalbuminuria test (the ratio of albumin to creatinine in random urine) once a year." It also states that "these low levels of albumin excretion cannot be detected by routine urine protein tests." "This indicates that urine test strips (option D) are not sensitive to early renal impairment. Step 4: Excerpt [1] supports the view that patients with type 1 diabetes begin to excrete small amounts of albumin after 5-10 years, and microalbuminuria is defined as an albumin / creatinine ratio of 30-299 in a random urine sample." Between μg / mg. This corresponds to the urine microalbumin / creatinine ratio (option B). Step 5: Although cystatin C (option A) can estimate glomerular filtration rate (GFR) and may be useful, the excerpt emphasizes that microalbuminuria is the earliest specific marker for screening diabetic nephropathy. The excerpt [5] mentions that some patients have a decrease in GFR without albuminuria, but recommends albuminuria as the focus of early screening. Step 6: Glycated hemoglobin A1C (option C) reflects blood glucose control rather than a direct indicator of renal impairment. Step 7: Therefore, according to the provided documentation, the most sensitive detection method for early renal impairment in diabetic patients is the urine microalbumin / creatinine ratio. Final answer: Urine microalbumin / creatinine ratio.

[0126] The model uses the following instructions to generate negative samples: "The following is a medical multiple-choice question."

[0127] The questions are as follows:

[0128] The medical record information is as follows:

[0129] The candidate answers are as follows:

[0130] Please explain your problem-solving approach following these steps: Please use your clinical knowledge to reason step by step.

[0131] Please show your solution process and final answer. For example: "Step 1. Understand the clinical scenario: The patient has a long history of diabetes, and the doctor is concerned about kidney damage. In diabetes, kidney damage usually begins with microalbuminuria (small amounts of albumin in the urine) and then progresses to significant proteinuria and a decrease in glomerular filtration rate (GFR). Step 2. Review options in conjunction with early diabetic nephropathy: (A) Cystatin C level: This is a serum marker used to estimate GFR and is generally more sensitive than creatinine in detecting a slight decrease in GFR. It can detect early kidney damage, but in diabetic nephropathy, structural damage usually precedes a significant decrease in glomerular filtration rate (GFR). (B) Urinary microalbumin / creatinine ratio (UACR): This test detects small amounts of albumin in the urine (microalbuminuria), which is the earliest clinically detectable sign of diabetic nephropathy. It is recommended that diabetic patients undergo this screening annually, as it can identify kidney damage at an early stage when interventions can slow disease progression." (C) Glycated hemoglobin A1C: Reflects blood glucose control over 2-3 months and is important for diabetes management, but it is not a direct indicator of kidney damage. (D) Urine protein test strips: Detect significant proteinuria (usually >300 mg albumin / day), which is a late-stage manifestation of diabetic nephropathy. It is not sensitive to microalbuminuria. Step 3. Early detection sensitivity comparison: In diabetic patients, kidney damage in diabetic nephropathy begins with glomerular hyperfiltration and increased glomerular permeability to albumin, leading to microalbuminuria (30-300 mg albumin / day). (mg albumin / day). The urine albumin / creatinine ratio (UACR) is specifically designed to detect this early stage. Cystatin C can detect early changes in glomerular filtration rate (GFR), but these changes may appear later in the course of diabetic nephropathy than microalbuminuria. For diabetes screening purposes, guidelines (ADA, KDIGO) recommend UACR as the preferred initial test for detecting early diabetic nephropathy because it is more sensitive than serum markers for the earliest diabetes-related kidney function impairment. Step 4. Conclusion: In routine physical examinations, the most sensitive test for detecting kidney function impairment in diabetic patients is the urine microalbumin / creatinine ratio, as it can identify early diabetic nephropathy before abnormalities appear in serum markers (such as creatinine or cystatin C) and even before abnormal urine test strip results. Final answer: Urine microalbumin / creatinine ratio.

[0132] (2) The noise-resistant evidence preference alignment stage, specifically: The model uses the following instructions to generate negative samples: "Below is a medical multiple-choice question and excerpts of relevant documents. Your task is to write a chain of reasoning to answer the question based on these excerpts."

[0133] The questions are as follows:

[0134] The relevant documents are excerpted below: Excerpt from relevant document 1:

[0135] Excerpt from relevant document 2:

[0136] The answer is as follows:

[0137] Please generate the inference chain as follows: Use step-by-step reasoning (e.g., "Step 1, Step 2..."). Clearly cite relevant document excerpts and use [1], [2], [3]... to mark the sources of evidence. (Note: The numbers [1], [2], [3]... in square brackets represent different document excerpt numbers.) Arguments must be based entirely on excerpts from the provided documents and must not use any fabricated references.

[0138] In the process of argumentation, at least one excerpt from relevant document excerpt 2 must be cited, and its content shall serve as an important basis for the argument.

[0139] Finally, please provide your answers in the following format: Final answer: Please provide your reasoning and final answer. For example: "Step 1: The question asks to find the most sensitive indicator for detecting kidney damage in diabetic patients. This patient has a 15-year history of diabetes, so there is a risk of developing diabetic nephropathy. Step 2: According to the document excerpt, the development of diabetic nephropathy follows a certain pattern. The early kidney involvement is characterized by... Before the appearance of obvious proteinuria, there will be an increase in low-level albumin excretion (microalbuminuria). Excerpt [1] points out that patients with type 1 diabetes begin to excrete small amounts of albumin 5-10 years after the onset of the disease, and the American Diabetes Association (ADA) recommends replacing the old term with "persistent albuminuria". Step 3: Excerpt [2] clearly recommends that patients with type 1 or type 2 diabetes undergo microalbuminuria testing once a year by using the albumin / creatinine ratio in random urine. It points out that routine urine analysis (test strip method) cannot detect these low levels of albuminuria. ... Excerpt 2 of the relevant documents [7] discusses disease screening for asymptomatic individuals, noting that screening tests for diabetes and kidney disease are widely used for early diagnosis and intervention to improve prognosis... Final answer: urinary microalbumin / creatinine ratio.

[0140] (3) Evidence consistency preference optimization stage.

[0141] The verification prompt reads: "You are now an expert in evidence support judgment in the medical field."

[0142] You will receive the following: 1) A set of medical evidence documents:

[0143] 2) Complete arguments generated by the model (may contain multiple sentences and include references such as [number]):

[0144] Your task is: Determine whether the numbers in [ ] in each sentence generated by the model are: (1) It cited the document number of real medical evidence; (2) The content of the cited document is related to the content of the sentence.

[0145] If all sentence references satisfy the above two conditions, then output "SUPPORT".

[0146] If any sentence is not satisfied, output that sentence and the citation used in that sentence. Different sentences are output by line (\\n), and output which condition of the sentence is not satisfied, and also output which sentence it is. Output: Sentence X: ((sentence content)); citation: [number]. (For example, sentence 2, ((sentence content)), citation [3]\\n Sentence 7, ((sentence content)), citation [5]...).

[0147] Output requirements (must be JSON): If all are correct: {{ "output": "SUPPORT" }} If there is an incorrect reference, output as shown in the following example: {{ "output": "Sentence X: ((sentence content)); Quote: [number]" }} Output only the JSON provided above; do not include any additional explanations. By verifying whether the inference process references are correct and outputting the error location and reason when a reference is incorrect, we can provide a basis for the following corrections.

[0148] Repair prompt as follows: "You are a medical evidence citation repair expert."

[0149] You will now receive: 1) A set of medical evidence documents (with numbers):

[0150] 2) Sentences that need to be repaired (where [number] represents the document number):

[0151] Task Objective: The citation in this sentence is incorrect. Please modify only the [numerical] citation portion to ensure it is supported by the document content. The correction method involves finding documents within a medical evidence collection that support the sentence and constructing correct citation numbers. Citation numbers must be real document numbers; fabricated numbers are not allowed. The cited documents should ideally be a "minimum sufficient set," meaning the number of citations should be minimal but sufficient to support the sentence. If no documents support the sentence, output "NOT SUPPORT".

[0152] Output format (must be valid JSON): {{ "rewritten_claim": "The corrected sentence, for example: sentence [3] or "NOT SUPPORT""}} "The corrected thought chain is used as a positive sample, and the original thought chain is a negative sample; the negative sample constructs a prompt: "You are a professional model preference data construction assistant."

[0153] You will receive the following: 1) A set of medical evidence documents (with numbers):

[0154] 2) The reasoning process for model generation:

[0155] Its reasoning and citation of medical evidence documents are correct.

[0156] Your task is to deliberately construct a "reference error" version of the argument citation part generated by the model, as a negative sample for DPO.

[0157] The way a reference is incorrect must satisfy at least one of the following: 1) Reference a non-existent document number (e.g., reference [8], but there are only 1~5 in reality); 2) Citing a real document, but the document's content is unrelated to the sentence; It is not necessary to modify all the cited parts; only some parts need to be modified. However, changes to medical facts are not permitted. The reasoning logic must not be altered. The final answer cannot be changed; Only the referenced parts were modified; Output format: {{ "rejected_output": "Rewrite the above reasoning process as a complete output including the incorrect references." }}” The thought chain constructed in this process is used as a negative sample, while the original thought chain is used as a positive sample.

[0158] For example: "Step 1: The patient is a 47-year-old male smoker with a history of chronic cough, recurrent dyspnea, severe acute exacerbations, a family history of allergic diseases, and a history of occupational exposure as a farmer. Differential diagnosis includes chronic obstructive pulmonary disease (COPD), asthma, and possible occupational lung disease. Chest X-ray is nonspecific, ... Document

[10] states that obstructive airway disease in workers exposed to grain dust (related to agriculture) is almost identical to that in smokers, manifested as decreased FEV1 and FEV1 / FVC ratio. Step 4: Given the presence of an obstructive pattern (low FEV1 / FVC), we evaluate the following options. (A) It is unlikely that FEV1 is normal, as obstruction usually reduces FEV1. (B) It is unlikely that FEF25-75 is increased; in obstructive conditions, mid-expiratory flow is usually decreased rather than increased. Step 5: For obstructive diseases such as COPD, changes in lung volume are characteristic. Document [2] states explicitly: "Lung volume may increase as the severity of the disease progresses, resulting in increases in total lung capacity, functional residual capacity, and residual capacity." "Therefore, for a patient with a 3-year history of severe acute exacerbations, an increase in total lung capacity (TLC) is a possible outcome. Step 6: Option (D) is "a decrease in functional residual capacity (FRC)." This is contrary to the expected change in obstructive disease; FRC usually increases due to gas retention [2]. Step 7: Therefore, based on the clinical presentation of chronic, severe obstructive airway disease (considering smoking history and occupational exposure), it is highly likely to be COPD, and the most likely pulmonary function test result among the options is an increase in TLC. Final answer: Increased total lung capacity (TLC)."

[0159] Revised: "Step 1: The patient is a 47-year-old male smoker with a history of chronic cough, recurrent dyspnea, severe acute exacerbations, a family history of allergic diseases, and a history of occupational exposure as a farmer. Differential diagnosis includes chronic obstructive pulmonary disease (COPD), asthma, and possible occupational lung disease. Chest X-ray is nonspecific... Document [3] states that obstructive airway disease in workers exposed to grain dust (related to agriculture) is almost identical to that in smokers, characterized by decreased FEV1 and FEV1 / FVC ratio. Step 4: Given the presence of an obstructive pattern (low FEV1 / FVC), we evaluate the following options. (A) It is unlikely that FEV1 is normal, as obstruction usually reduces FEV1. (B) It is unlikely that FEF25-75 is increased; in obstructive conditions, mid-expiratory flow is usually decreased rather than increased. Step 5: For obstructive diseases such as COPD, changes in lung volume are characteristic. Document [2] states explicitly: "Lung volume may increase as the severity of the disease progresses, resulting in increases in total lung capacity, functional residual capacity, and residual capacity." "Therefore, for a patient with a 3-year history of severe acute exacerbations, an increase in total lung capacity (TLC) is a possible outcome. Step 6: Option (D) is "a decrease in functional residual capacity (FRC)." This is contrary to the expected change in obstructive disease; FRC usually increases due to gas retention [2]. Step 7: Therefore, based on the clinical presentation of chronic, severe obstructive airway disease (considering smoking history and occupational exposure), it is highly likely to be COPD, and the most likely pulmonary function test result among the options is an increase in TLC. Final answer: C."

[0160] In this example, the original inference sample referenced a document

[10] that did not exist, and the referenced content matched the actual document [3]. The original sample was used as a negative sample, and the modified sample was used as a positive sample. This design enables the model to learn to ensure the authenticity of evidence citations during the retrieval-enhanced medical inference process, thereby preventing evidence forgery.

[0161] (4) Evidence chain constraint preference alignment stage, the following is a positive sample example after correction: "First, extract relevant excerpts from the document and then answer the question. The question is: Which mucolytic agent that can break the disulfide bonds between mucoproteins can also treat overdose? The context discusses various drugs and antidotes. It is necessary to find the mucolytic agents mentioned and their alternative uses." From the excerpt, it is found that [2] "the antidote acetylcysteine, as an alternative to glutathione, can bind to the toxic metabolites produced." and [4] "taking N-acetylcysteine ​​(the safer of the two) within 8-16 hours after acetaminophen overdose has been shown to protect patients from fulminant hepatotoxicity and death." In addition, [5] it is noted that "N-acetylcysteine ​​(NAC) is a clinically effective antidote for acetaminophen overdose and should be taken as early as possible." N-acetylcysteine ​​is known to act as a mucolytic agent by breaking the disulfide bonds in mucus, and the context also confirms that it is used as an antidote for acetaminophen overdose. Therefore, the expectorant described is N-acetylcysteine, which is used to treat acetaminophen overdose. Answer: Acetaminophen.

[0162] After receiving the input medical question and patient case information, the system calls the retrieval enhancement module to obtain medical retrieval documents related to the medical question. Based on the medical retrieval documents, patient case information, and question, the system drives the model optimized through multi-stage preference learning to generate the corresponding medical reasoning process and question-and-answer results.

[0163] The model's mastery of multiple capabilities is verified by evaluating the reasoning process and final question-and-answer results. For the evidence-dependent capability, the accuracy of the generated question-and-answer results is demonstrated by comparing the model with only the first preference learning stage plus the retrieved documents with the model without the first preference learning stage plus the retrieved documents. After the first preference learning stage, the accuracy of the model's question-and-answer results significantly improves, because the model does not possess the data knowledge used in these tests; to answer correctly, it needs to rely on the retrieved evidence. For the noise suppression capability, the similarity of the question-and-answer results is demonstrated by comparing the results after the second preference optimization using only supporting documents with those using supporting documents plus noisy documents. The model with noise suppression capability will produce similar answers using both types of retrieved documents. For the evidence consistency capability, the citation recall (Rec) is illustrated by comparing the results before and after the third preference optimization. This process uses a model specifically designed for judging the correctness of cited documents, NLI, to determine citation behavior. For the evidence chain constraint capability, the simplicity of the reasoning process is demonstrated by statistically analyzing the number of inference chain tokens generated before and after the fourth preference optimization, and the designed scoring rules allow the model to score aspects such as coherence and medical logic.

[0164] The model proposed in this invention achieves superior results compared to other methods on the datasets, as shown in Table 1. Specifically, the proposed method achieves the highest average accuracy of 65.34% on three medical question-answering datasets. This represents an improvement of 3.68 percentage points compared to the current best-performing baseline method, RankCoT (61.66%), and an improvement of 8.79 percentage points compared to the standard RAG (56.55%). This demonstrates that the proposed method effectively improves the overall performance of retrieval-enhanced medical question-answering tasks. From the perspective of method categories, directly using a large model (custom) or adding chained reasoning (CoT) resulted in the weakest overall performance, indicating that relying solely on model parameter knowledge is insufficient to address the professional knowledge requirements in medical question answering. Therefore, it is necessary to introduce external professional knowledge for answering. In contrast, the fine-tuning method outperforms Custom and CoT overall. Among them, the average accuracy of SFT (Chosen) is higher than that of ordinary SFT, indicating that high-quality supervision signals can provide the model with more effective learning objectives, thereby improving performance. DPO (Mix) performs poorly, even lower than some basic methods, indicating that when preference signals from different capability dimensions are directly mixed and optimized without differentiation, the model struggles to learn multiple constraint objectives simultaneously, and there may be mutual interference between different optimization objectives. Therefore, compared to direct mixing optimization, staged preference optimization is more conducive to the model gradually establishing stable evidence use and reasoning capabilities. In RAG variant methods, the emergence of retrieval processing and inference control mechanisms significantly improves performance. However, even so, the introduction of additional components in these methods greatly increases computational cost and time consumption, making them unsuitable for large-scale applications.

[0165] The method of this invention has achieved stable performance improvements in clinical reasoning tasks (MedQA), knowledge-intensive medical question answering tasks (MedMCQA), and basic biomedical tasks (MMLU-Med), demonstrating its effectiveness and applicability in different types of retrieval-enhanced medical question answering scenarios. Compared with other methods, this method not only has higher accuracy, but also stronger reliability due to the inclusion of multiple constraint preferences in the reasoning process to enhance interpretability.

[0166] Table 1. Experimental comparison results of this method with other methods

[0167] It also outperforms other methods on models of different sizes, as shown in Table 2.

[0168] Table 2. Experimental comparison results of this method with other methods on different large language models.

[0169] This method demonstrates robustness across various LLM models. While introducing retrieval augmentation and preference optimization can help supplement expertise and correct inappropriate evidence use and reasoning behavior in small-scale models such as Llama-3.2-1B-Instruct and Qwen2.5-1.5B-Instruct, these methods place demands on the model's inherent capabilities and cannot compensate for the inherent limitations of small-scale models in terms of reasoning ability. These results indicate that this method exhibits strong generalization ability when applied to models with sufficient medical reasoning capabilities.

[0170] Example 2 In one or more embodiments, a medical question-answering system based on multi-stage preference alignment retrieval enhancement is disclosed, specifically including: The retrieval enhancement module is used to perform multi-path semantic expansion on the input medical questions and patient case information. The multi-path semantic expansion includes a path for extracting structured elements of case information, a path for completing background knowledge of medical questions, and a path for constructing literature associations. The query expressions generated by different paths are integrated to obtain a multi-path input set. The multi-path input set is then input into the retrieval model for retrieval to obtain a set of retrieval evidence. The feature parsing module is used to analyze the case information based on the retrieved evidence set, medical questions and patient case information, introduce a preset prompting strategy and use a large language model to obtain a set of clinical features and iteratively update it to obtain a set of candidate evidence for retrieval. The document filtering module is used to filter supporting documents based on the search candidate evidence set; The reasoning test module is used to generate corresponding medical reasoning processes and question-and-answer results based on the model optimized through multi-stage preference learning, which is based on the input of supporting documents, medical questions, and patient medical record information.

[0171] In this embodiment, the retrieval enhancement module includes multi-path semantic expansion retrieval, which reconstructs the input data through multiple paths to improve retrieval coverage. The paths include: a clinical feature expression path (analyzing patient case information, identifying diagnostically significant clinical features, and constructing corresponding semantic expressions); a knowledge completion path (generating relevant medical background knowledge based on question semantics to expand the knowledge dimension of the input); and a topic association path (constructing medical topic descriptions related to the question to enhance alignment with the medical knowledge system). Subsequently, the set of semantic expressions generated by the above multi-path methods is input into the dense retrieval model MedCPT, and a retrieval operation is performed in the medical knowledge base to obtain the top-k relevant medical documents, forming an initial evidence set. Based on this, the feature parsing module performs clinical feature parsing, analyzing the patient case information to generate structured clinical feature representations, and fusing them with the original question to form updated semantic expressions. These updated expressions drive the next round of multi-path semantic expansion retrieval and clinical feature parsing. This process is repeated to gradually improve the relevance and completeness of the retrieval results. Subsequently, supporting documents are screened, and a key reasoning support document set is constructed from the candidate document set. Documents that can provide effective support for the medical reasoning process are selected and used as evidence input in the subsequent model training and reasoning process.

[0172] The inference testing module is used to verify the capabilities of the preference-aligned model after completing multi-stage preference learning.

[0173] It also includes a multi-stage preference learning module for phased preference alignment training of the large language model under the condition of supporting documents output by the retrieval enhancement module, guiding the model to form reasoning behavior that meets the requirements of evidence-driven medical decision-making. Based on a unified and fixed set of supporting documents, the multi-stage preference learning module progressively optimizes the model according to multiple preset preference learning stages, enabling the model to align different dimensions of medical reasoning capabilities at different stages. In the first preference learning stage, the model is guided to explicitly rely on external medical evidence for reasoning during medical question-and-answering, suppressing the behavior of answering solely based on the model's internal knowledge. In the second preference learning stage, the model is guided to prioritize medical evidence highly relevant to medical decisions in the presence of noisy or non-critical medical evidence, thereby improving the model's robustness to noisy evidence. In the third preference learning stage, the model's citation behavior of medical evidence during reasoning is constrained, ensuring that the reasoning process generated by the model is semantically consistent with the cited medical documents, avoiding inconsistencies or fabricated evidence. In the fourth preference learning stage, the quality of the model's medical reasoning is further optimized, enabling the model to generate a concise, coherent reasoning process that conforms to medical logic, based on the combination of patient case information and medical evidence.

[0174] The descriptions of each embodiment in the above embodiments have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0175] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A medical question-answering method based on multi-stage preference alignment retrieval enhancement, characterized in that, include: The input medical questions and patient case information are subjected to multi-path semantic expansion, which includes a path for extracting structured elements of case information, a path for completing background knowledge of medical questions, and a path for constructing literature associations. The query expressions generated by different paths are integrated to obtain a multi-path input set. The multi-path input set is then input into the retrieval model for retrieval to obtain a set of retrieval evidence. Based on the aforementioned set of retrieval evidence, medical questions, and patient case information, a preset prompting strategy is introduced. A large language model is used to perform feature analysis on the case information to obtain a set of clinical features, which is then iteratively updated to obtain a set of candidate retrieval evidence. Supporting documents are selected based on the aforementioned set of candidate evidence. Based on the input of supporting documents, medical questions, and patient medical record information, the model, optimized through multi-stage preference learning, generates the corresponding medical reasoning process and question-and-answer results. The multi-stage preference learning optimization includes a sequentially executed evidence dependency preference alignment stage, a noise-resistant evidence preference alignment stage, an evidence consistency preference alignment stage, and an evidence chain constraint preference alignment stage.

2. The medical question-answering method based on multi-stage preference alignment retrieval enhancement as described in claim 1, characterized in that, The evidence dependency preference alignment stage involves constructing evidence dependency preference sample pairs, including: Evidence-dependent positive sample output: in, It is a reasoning process. It is the final answer to the prediction. 1 + It is an instruction used to generate positive samples with evidence dependence. For supporting documentation, P For example, ref For reference model; Evidence-dependent negative sample output: in, 1 It is a hint used to generate negative samples that depend on evidence.

3. The medical question-answering method based on multi-stage preference alignment retrieval enhancement as described in claim 1, characterized in that, The noise-resistant evidence preference alignment stage involves constructing noise-resistant evidence preference sample pairs, including: Positive samples of evidence for noise resistance: in, 2 + It is a positive sample instruction to generate evidence of robustness to noise. It is a reasoning process. It is the final answer to the prediction. For supporting documentation, P For example, ref For reference model; Negative samples of evidence for noise resistance: in, 2 It is a negative sample instruction used to generate noise-resistant evidence, D This represents a set of noisy documents.

4. The medical question-answering method based on multi-stage preference alignment retrieval enhancement as described in claim 1, characterized in that, The evidence consistency preference alignment stage involves constructing evidence consistency preference sample pairs, including: Positive samples of evidence consistency preference: Repair=1 indicates that the citation repair was successful. For the reasoning output of evidence consistency, To fix the inference output, s Indicates the first A reasoning statement, ( ) represents s Referenced document number, For document citation verification, ( ) Indicates the first A reasoning sentence Referenced documents; Negative samples of evidence consistency preference: in, Construct negative samples for positive samples and output them.

5. The medical question-answering method based on multi-stage preference alignment retrieval enhancement as described in claim 1, characterized in that, The evidence chain constraint preference alignment stage involves constructing evidence chain constraint preference sample pairs, including: A positive sample constrained by a chain of evidence is represented as: The negative sample constrained by the chain of evidence is represented as: in, Output as a chain of evidence constraint. It is a reasoning process. To regenerate the sample, To construct the constraint criteria for the chain of evidence constraint preference sample, This serves as a negative sample hint for the chain of evidence constraint. It is the final answer to the prediction. For supporting documentation, P For example, ref This is a reference model.

6. The medical question-answering method based on multi-stage preference alignment retrieval enhancement as described in claim 1, characterized in that, The specific process of feature analysis for case information is as follows: In the t-th round of processing, the set of retrieved evidence and medical questions obtained in the current round are analyzed. and patient case information As a combined input, a preset prompting strategy is introduced. This drives the large language model to perform feature analysis on the case content: in, Let t represent the set of clinical features generated in round t. As a suggestion strategy, In order to retrieve the evidence set, c For patient medical record information, For medical issues, For large language models.

7. The medical question-answering method based on multi-stage preference alignment retrieval enhancement as described in claim 1, characterized in that, A document is considered a supporting document if it meets one of the following conditions: (1) This document provides direct medical evidence supporting the correct answer; (2) The document provides medical background that matches the key clinical clues in the question and significantly strengthens the reasoning chain; (3) This document contains decisive information that effectively eliminates one or more incorrect options.

8. The medical question-answering method based on multi-stage preference alignment retrieval enhancement as described in claim 1, characterized in that, The specific set of clinical features is as follows: An iterative optimization mechanism is employed to progressively update the input semantics. After each round of feature parsing, the clinical feature set generated in the current round is... With medical issues By splicing and merging, new forms of expression can be constructed. This is used to drive the next round of multi-path semantic expansion retrieval and parsing process.

9. A medical question-answering system based on multi-stage preference alignment retrieval enhancement, characterized in that, include: The retrieval enhancement module is used to perform multi-path semantic expansion on the input medical questions and patient case information. The multi-path semantic expansion includes a path for extracting structured elements of case information, a path for completing background knowledge of medical questions, and a path for constructing literature associations. The query expressions generated by different paths are integrated to obtain a multi-path input set. The multi-path input set is then input into the retrieval model for retrieval to obtain a set of retrieval evidence. The feature parsing module is used to analyze the case information based on the retrieved evidence set, medical questions and patient case information, introduce a preset prompting strategy and use a large language model to obtain a set of clinical features and iteratively update it to obtain a set of candidate evidence for retrieval. The document filtering module is used to filter supporting documents based on the search candidate evidence set; The reasoning test module is used to generate corresponding medical reasoning processes and question-and-answer results based on the model optimized through multi-stage preference learning, which is based on the input of the supporting documents, medical questions and patient medical records. The multi-stage preference learning optimization includes a sequentially executed evidence dependency preference alignment stage, a noise-resistant evidence preference alignment stage, an evidence consistency preference alignment stage, and an evidence chain constraint preference alignment stage.

Images