A clinical thinking evaluation method and system based on thought chain reasoning and an electronic device

Abstract

The application provides a clinical thinking evaluation method and system based on thought chain reasoning and an electronic device, relates to the technical field of artificial intelligence assisted medical education and clinical ability evaluation, and comprises the following steps: S1, collecting interactive dialogue records between a student end and a virtual patient end, and constructing an inquiry time axis comprising a plurality of time nodes, the time nodes carrying diagnosis hypotheses submitted by the student; S2, inputting the interactive dialogue records into a main evaluation large language model, the main evaluation large language model sequentially performing fact extraction, clinical mapping and quantitative tracing before outputting a quantitative score; by prepositioning a thought chain reasoning engine in the scoring link and forcing the main evaluation large language model to adopt the analysis priority strategy of fact extraction, clinical mapping and quantitative tracing, each time of scoring and deducting of the large language model has a clear logical basis and a visualized reasoning graph, and the black box scoring state of the existing scheme is fundamentally changed.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence-assisted medical education and clinical competence assessment technology, and in particular to a clinical thinking assessment method, system and electronic device based on thought chain reasoning. Background Technology

[0002] Clinical reasoning is the core cognitive process by which physicians collect and analyze clinical information and make diagnostic and treatment decisions during diagnosis and treatment. It is a key ability to measure the success of medical students' transition from theory to clinical practice and is also one of the core objectives of medical education assessment. Traditional clinical reasoning assessment mainly relies on objective structured clinical examinations and subjective evaluations by supervising physicians. Objective structured clinical examinations require the organization of standardized patients, setting up examination stations, and arranging examiners, which is time-consuming, labor-intensive, and difficult to conduct frequently. Subjective evaluations by supervising physicians are greatly influenced by individual experience, and assessment standards are difficult to unify among different teachers. Furthermore, feedback is significantly delayed, often requiring a comprehensive evaluation to be given only after a round of rotations, failing to provide targeted feedback when students reveal erroneous diagnostic reasoning. In recent years, clinical reasoning assessment schemes based on large language models have been introduced into the field of medical education, attempting to improve the objectivity and timeliness of assessments through automated scoring. However, after long-term empirical research by the applicant, it was found that existing schemes still have the following three core technical defects that have not been properly addressed in the high-security scenario of clinical reasoning assessment: First, the evaluation process is essentially a black-box scoring system, severely lacking interpretability. Existing solutions generally adopt a zero-sample direct scoring model, where the consultation text is used as input, and a large language model outputs the final scores for each dimension all at once. This black-box model has two specific drawbacks: firstly, students cannot understand the specific logic behind the deductions, and instructors find it difficult to trace and appeal the scores, rendering the teaching feedback mechanism ineffective; secondly, the large language model is prone to drastic score fluctuations in different inference iterations, leading to potentially very different scores for the same consultation text in two evaluations, seriously affecting the stability and credibility of the evaluation.

[0003] Secondly, it neglects the dynamic adaptability of clinical reasoning. Real clinical reasoning is a dynamic process of continuously updating hypotheses based on incremental information. For example, after inquiring about the patient's chief complaint and obtaining key positive signs, the student needs to promptly adjust their initial diagnostic hypothesis and shift towards a higher-priority differential diagnosis. Similarly, after receiving results from auxiliary examinations such as electrocardiograms, the student should be able to promptly overturn preconceived preliminary diagnoses based on abnormal indicators. However, existing solutions often use a static checklist-style scoring based on the total amount of information collected at the end of the consultation, failing to track and quantify the trajectory of students' diagnostic hypothesis adjustments in response to new information. Therefore, they cannot truly reflect students' key abilities in dynamic clinical reasoning.

[0004] Third, a single large language model carries the risk of medical illusions, while introducing full human review would negate the efficiency advantages of AI assessment; it's difficult to reconcile these two challenges. Medical assessment has extremely high safety requirements, and a single general-purpose large language model inevitably has occasional instances of medical illusions or logical inconsistencies. For example, it might fail to identify cases where students are missed in diagnosing life-threatening conditions like acute myocardial infarction or pulmonary embolism, resulting in inflated scores that could lead to serious teaching accidents or even clinical safety hazards. If this deficiency is remedied by full human review, AI assessment degenerates into traditional human assessment, losing its efficiency benefits.

[0005] In summary, how to simultaneously achieve the following three goals in clinical reasoning assessment driven by large language models: first, to make the scoring process white-box, interpretable, and traceable; second, to accurately capture and quantify the adjustments to diagnostic hypotheses made by students in response to incremental information; and third, to significantly reduce the risk of missing critical errors while maintaining second-level feedback efficiency without introducing full manual review. These are the technical problems that urgently need to be solved in this field. Summary of the Invention

[0006] The purpose of this invention is to provide a clinical reasoning assessment method, system, and electronic device based on thought chain reasoning to solve the above-mentioned technical problems.

[0007] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: A clinical reasoning assessment method based on thought chain reasoning, the method comprising the following steps: S1. Collect the interactive dialogue records between the student terminal and the virtual patient terminal, and construct a consultation timeline containing several time nodes, with the diagnostic hypotheses submitted by the student at each time node. S2. Input the interactive dialogue record into the main assessment big language model. Before outputting the quantitative score, the main assessment big language model performs fact extraction, clinical mapping and quantitative tracing in sequence to generate a diagnostic reasoning graph and output a multi-dimensional scoring vector. S3. Based on the consultation timeline, the deviation of the diagnostic hypotheses submitted by students at adjacent time points is compared and the dynamic adaptability index is obtained according to the magnitude and direction of the deviation. The dynamic adaptability index is then incorporated into the multi-dimensional scoring vector. S4. Identify fatal errors in interactive dialogue records based on the medical guideline knowledge base, and output a fatal error level sequence according to the preset multi-level classification criteria; S5. Call the secondary validation language model to perform blind evaluation of the interactive dialogue record, and compare the output of the main evaluation language model with that of the secondary validation language model. If the two models agree, issue an evaluation report; otherwise, trigger the circuit breaker and transfer the case to expert review. S6. Weight the multi-dimensional scoring vectors according to the department to which the case belongs, and generate an assessment report that includes the reasoning process.

[0008] Preferably, constructing a consultation timeline includes: Add timestamps to the multimodal interaction data in the interactive dialogue records, and use the diagnostic hypothesis events actively submitted by students as time node anchors to generate a consultation timeline; Multimodal interaction data includes at least text-based question-and-answer sessions between students and virtual patients, student-initiated auxiliary examinations and medical orders, and diagnostic hypothesis events submitted by students.

[0009] Preferably, fact extraction, clinical mapping, and quantitative traceability include: Fact extraction involves extracting key positive and key negative signs from the interactive dialogue record, and inferences and scoring are prohibited at this step. The clinical mapping loads the examination checklist for the current case from the medical guideline knowledge base, compares the extracted signs with the examination checklist, marks the omissions, and compares the student's actual consultation order with the recommended order in the examination checklist, marking the logical breakpoints. Quantitative tracing is based on omission points and logical breakpoints. It deducts items from the multi-dimensional scoring vector in the corresponding dimensions one by one, and organizes the extracted signs, omission points and logical breakpoints into a traceable diagnostic reasoning map.

[0010] Preferably, deviation from quantification comparisons includes: For adjacent time node pairs in the consultation timeline, based on the target diagnostic direction of the current case in the medical guide knowledge base, the deviation degree and deviation direction of the two diagnostic hypothesis sets are determined. Bonus points are awarded when the deviation exceeds the first threshold and the deviation direction points towards the target diagnostic direction. When the deviation exceeds the first threshold but the direction of deviation deviates from the target diagnostic direction, points will be deducted. When the deviation is less than or equal to the second threshold, no increase or decrease is made; The scores or deductions generated by all adjacent time points in the consultation timeline are accumulated according to their weights to obtain a dynamic adaptive index.

[0011] Preferably, the multi-level classification criteria include at least: a level for missed diagnosis of fatal diseases, a level for neglect of important symptoms, a level for emergency treatment of errors, and a level for failure to identify drug contraindications; each fatal error in the fatal error level sequence carries its own level and severity rating.

[0012] Preferably, the outputs of the primary evaluation large language model and the secondary validation large language model are compared, including: While the primary evaluation language model generates preliminary evaluation results, the secondary verification language model is asynchronously called to perform back-to-back blind evaluation. The input of the secondary verification language model does not include the output of the primary evaluation language model. When the primary evaluation language model and the secondary validation language model agree on the fatal error and the difference in the overall score is within the preset tolerance range, the two models are considered to have the same judgment result and an evaluation report is automatically issued; otherwise, the evaluation report is intercepted, the dialogue segment that triggered the divergence and the corresponding evaluation dimension are highlighted, and the case to be reviewed is pushed to the expert review queue. After the experts completed their review, the revised scores given by the experts were used to cover the original preliminary assessment results, and the cases to be reviewed were included in the difficult case corpus for secondary verification and iterative fine-tuning of the large language model.

[0013] Preferably, the multi-dimensional scoring vector includes the dimensions of information collection completeness, information collection logic, differential diagnostic thinking, treatment plan rationality, communication skills, and doctor-patient relationship establishment, with dynamic adaptive indicators incorporated as components into the differential diagnostic thinking dimension. The generation of the assessment report also includes determining the level of evidence-based medicine for the student's final diagnosis and treatment plan, and incorporating the determination results into the assessment report.

[0014] A clinical reasoning assessment system based on thought chain reasoning, used to implement a clinical reasoning assessment method based on thought chain reasoning, includes: The data acquisition unit collects interactive dialogue records between the student's end and the virtual patient's end, and constructs a consultation timeline containing several time nodes; The thought chain reasoning engine unit drives the main assessment language model to perform fact extraction, clinical mapping, and quantitative tracing in sequence before outputting the quantitative score, so as to generate a diagnostic reasoning graph and output a multi-dimensional scoring vector. The dynamic adaptive assessment unit compares the deviation of diagnostic hypotheses at adjacent time points based on the consultation time axis and incorporates the obtained dynamic adaptive indicators into the multi-dimensional scoring vector. The fatal error identification unit identifies fatal errors in interactive dialogue records based on a medical guideline knowledge base and outputs a fatal error level sequence according to a preset multi-level classification standard. The dual-model cross-validation unit calls the secondary validation large language model to perform blind evaluation of the interactive dialogue records, and performs consistency comparison between the output of the main evaluation large language model and the secondary validation large language model. When the two models are consistent, the evaluation report is released; when the two models are inconsistent, the circuit breaker is triggered and the process is transferred to expert review. The report generation unit weights the multi-dimensional scoring vectors according to the department to which the case belongs, and generates an assessment report that includes the reasoning process.

[0015] Preferably, the dual-model cross-validation unit also includes a feedback write-back module, which is used to write back the expert's corrected score to the iterative corpus of the secondary validation large language model after the expert has completed the review.

[0016] An electronic device, characterized in that it includes a memory and a processor, the memory storing a computer program executable on the processor, the processor executing the computer program to implement the method as described in any one of claims 1 to 7.

[0017] The beneficial effects of this invention are: 1. By placing the reasoning engine at the beginning of the scoring process and forcing the main assessment language model to adopt a priority strategy of fact extraction, clinical mapping, and quantitative tracing, every point added or deducted by the language model has a clear logical basis and a visualized reasoning graph, changing the black-box scoring status of the existing scheme. Each deduction can be traced back to a specific omission or logical break, improving the interpretability of the assessment process and the persuasiveness of teaching, and enabling students and teachers to conduct effective feedback interaction based on the reasoning graph.

[0018] 2. By constructing a consultation timeline and quantifying the deviation of diagnostic hypotheses at adjacent time points, a dynamic adaptive assessment mechanism is incorporated. This allows the assessment to not only focus on the final result but also to deeply depict the student's thought process in adjusting diagnostic hypotheses based on incremental information. This mechanism is more consistent with the actual clinical reasoning process, and is particularly effective in identifying students who, although ultimately making the correct diagnosis, have rigid thought processes, as well as those whose initial diagnoses are incorrect but who can correct them promptly based on new information. This results in an assessment that more closely reflects actual clinical competence.

[0019] 3. A dual-model cross-validation mechanism, combining a primary evaluation model's comprehensive assessment across all dimensions with a secondary validation model's back-to-back asynchronous blind review, along with a consistency-based circuit breaker strategy, is employed. When both models agree, an evaluation report is automatically released within seconds; when there is significant discrepancy, the system automatically circuit breaks and initiates expert review. This mechanism retains the strong reasoning capabilities of a large-scale general-purpose language model while maintaining a medical safety baseline through a medium-sized, medically-oriented fine-tuned model. Without requiring full manual review, it significantly reduces the rate of missed fatal errors, while simultaneously improving evaluation efficiency to second-level response compared to traditional manual evaluation methods, effectively balancing safety and timeliness.

[0020] 4. By incorporating the weighting of specialty assessments and the determination of the level of evidence-based medicine into the final report generation process, the assessment results possess both specialty specificity and evidence-based characteristics. This makes the assessment suitable for teaching scenarios in multiple specialties, such as cardiology, respiratory medicine, gastroenterology, neurology, pediatrics, and emergency medicine, and has good scalability.

[0021] 5. Through the feedback and write-back mechanism of the expert review interface unit, the expert decision results are written back to the iterative corpus of the secondary verification large language model as a supervision signal, forming a data closed loop of evaluation, review, and retraining. This enables the system's evaluation capability to continuously evolve over time and maintain a steady improvement in evaluation quality over the long term. Attached Figure Description

[0022] Figure 1 The overall architecture diagram of the clinical thinking assessment system based on thought chain reasoning provided in the embodiments of the present invention; Figure 2 A flowchart illustrating the overall process of the clinical reasoning assessment method based on thought chain reasoning provided in this embodiment of the invention; Figure 3 The thought chain reasoning graph provided in the embodiments of the present invention; Figure 4 This is a schematic diagram of the dynamic adaptive thought trajectory tracking and diagnosis hypothesis deviation provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the dual-model cross-validation and AI-expert fusion circuit breaker mechanism provided in an embodiment of the present invention. Detailed Implementation

[0023] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Preferred embodiments of the invention are shown in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the invention.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0025] Specific embodiments of the present invention are described below with reference to the accompanying drawings.

[0026] Example 1 Please see Figure 2 This embodiment provides a clinical reasoning assessment method based on thought chain reasoning, the steps of which are as follows: Step 1: Data Acquisition and Timeline Construction; The purpose of this step is to extract structured data from the multimodal interactions between students and virtual patients that can be used for subsequent dynamic assessment.

[0027] In this embodiment, the data acquisition unit collects multimodal interaction data between the student terminal and the virtual patient terminal at a configurable polling period (preferably between 10 milliseconds and 500 milliseconds).

[0028] Multimodal interactive data includes at least: text-based question-and-answer content input by students; auxiliary examination instructions initiated by students on the client, such as accessing electrocardiograms, requesting blood routine tests, and requesting imaging examinations; diagnostic and treatment instructions issued by students in the medical order module, such as issuing nitroglycerin tests and issuing antiplatelet therapy; diagnostic hypothesis events submitted by students in the staged diagnostic hypothesis submission dialog box; and results such as physical signs, tests, and images fed back by the virtual patient end based on the case script.

[0029] All collected multimodal interaction data are stamped with a system-level timestamp to ensure the traceability of the time sequence. Based on this, the consultation timeline construction sub-unit uses the student's proactively submitted stage-by-stage diagnostic hypothesis events as time node anchors to generate a directed consultation timeline containing several time nodes.

[0030] Each time node must carry at least the following attributes: node timestamp; the type and content of the medical condition information obtained at this node (including its hierarchical marker in the medical guideline knowledge base, such as chief complaint level, present illness level, past medical history level, auxiliary examination level, result report level, etc.); and the set of diagnostic hypotheses submitted by the student at this node.

[0031] In a preferred implementation, this step further includes a dialogue normalization sub-step: normalizing the multi-turn dialogue into a question-and-answer pairing structure; and standardizing the mapping of clinical entities such as symptoms, signs, examination items, and medications using named entity recognition technology, so that they have a unified semantic identifier under different expressions (e.g., between students' colloquial expressions and guideline-standard expressions), so that they can be reused in subsequent units. Named entity recognition can be implemented using a pre-trained medical named entity recognition model or a few-shot prompting method based on a large language model.

[0032] Step two, prioritize scoring based on the thought chain analysis; please refer to [link / reference]. Figure 3 This step abandons the existing approach of directly inputting the consultation text into the large language model and having the model output the final score all at once. Instead, it adopts an analysis-first strategy, which forces the main evaluation large language model to perform three inference sub-steps in sequence before outputting the final score: fact extraction, clinical mapping, and quantitative tracing.

[0033] ① Fact Extraction: This sub-step is implemented as follows: After preprocessing the complete interactive dialogue record, it is input into the main evaluation language model. A pre-defined analysis priority prompt template instructs the main evaluation language model to extract only the key positive and negative physical signs already obtained by the student, based on directly verifiable evidence from the dialogue. The analysis priority prompt template explicitly prohibits the main evaluation language model from making any inferences or scoring in this sub-step to avoid probability generation bias from the language model contaminating subsequent reasoning.

[0034] ② Clinical mapping: This sub-step is implemented by loading the examination checklist for the current case's disease from the medical guideline knowledge base. The medical guideline knowledge base is organized into a multi-level atlas structure based on disease, symptoms, examination items, and contraindications. Its content comes from current clinical practice guidelines, expert consensus, and evidence-based medicine databases, and can be reviewed and updated monthly or quarterly by clinical experts. After loading the examination checklist, the key positive and negative signs extracted from the facts are compared item by item with the examination checklist, marking any omissions not collected by the student. Simultaneously, the student's actual consultation order is compared with the recommended order in the examination checklist, and significant order mismatches (e.g., skipping the present illness and directly proceeding to auxiliary examinations before confirming the chief complaint) are marked as logical breakpoints.

[0035] ③ Quantitative traceability: This sub-step is implemented as follows: Six dimensions are used for evaluation: completeness of information collection, logicality of information collection, differential diagnostic thinking, rationality of treatment plan, communication skills, and establishment of doctor-patient relationship. A maximum score is pre-set for each dimension (in this embodiment, the default configuration is 25, 20, 20, 10, 10, and 15 points, for a total of 100 points). Based on each specific deduction item in the omissions and logical breaks marked by clinical mapping, the maximum score is algebraically subtracted in the corresponding dimension to obtain a six-dimensional scoring vector. Simultaneously, the key positive and negative signs obtained from fact extraction, along with the omissions and logical breaks obtained from clinical mapping, are serially organized into a traceable diagnostic reasoning graph and output. This diagnostic reasoning graph not only serves as an internal data structure for downstream units to reuse but can also be rendered as a visual view by the report generation unit and presented on the client interfaces of students, teachers, and experts, thereby achieving white-box and visualization of the evaluation process.

[0036] In a preferred implementation, the main evaluation large language model can be a general-purpose large language model with a parameter size not less than a first size threshold, which is preferably one hundred billion in this embodiment. The main evaluation large language model can be deployed on a local server cluster with graphics processor acceleration capabilities, or it can be invoked through a remote interface; this invention does not limit this.

[0037] Step 3, time-axis-based dynamic adaptability assessment; please refer to [link / reference]. Figure 4 The processing object of this step is the consultation timeline constructed in step one. Its core task is to compare the deviation of the diagnostic hypotheses submitted by the student at adjacent time nodes in a deviation metric, and to determine whether the student's diagnostic thinking is evolving in a reasonable direction.

[0038] The specific processing procedure for this step is as follows: For any pair of adjacent time nodes in the consultation timeline, take the set of diagnostic hypotheses submitted by the student at the previous and next time nodes respectively.

[0039] Based on the target diagnostic direction for the current case in the medical guideline knowledge base, the degree to which the two sets of diagnostic hypotheses approach the target diagnostic direction is compared to determine the deviation direction.

[0040] The degree of deviation is determined by comparing the differences between two sets of diagnostic hypotheses. When the deviation is greater than a preset first threshold and the direction of deviation is towards the target diagnostic direction, a dynamic adaptive bonus is given. In this embodiment, the specific value of the bonus can be configured to be between 30% and 50% of the full score of the corresponding component at the adjacent time point. When the deviation is greater than the first threshold but the direction of deviation is away from the target diagnostic direction, a corresponding dynamic adaptive deduction is given. When the deviation is less than or equal to a preset second threshold, it is determined that the student has not made a significant adjustment to the diagnostic hypothesis between two adjacent time points, and no increase or decrease is given.

[0041] The scores or deductions generated by all adjacent time nodes in the consultation timeline are accumulated according to their weights to obtain a dynamic adaptive index.

[0042] Those skilled in the art will understand that there are various equivalent implementations for quantifying the degree of difference between two sets of diagnostic hypotheses, including but not limited to quantification based on the relative proportion of difference between sets; quantification based on ontological distance of diagnostic codes, i.e., calculating the semantic distance between two diagnoses along the hierarchical tree of the clinical medical diagnostic coding system; graph structure distance quantification based on the differential diagnosis subgraphs corresponding to the diagnoses; and vector space distance quantification based on the clinical feature vectors corresponding to the diagnoses. All of the above equivalent implementations fall within the protection scope of this invention.

[0043] The obtained dynamic adaptability index is incorporated as a component into the differential diagnostic thinking dimension of the six-dimensional scoring vector. In the default configuration of this embodiment, its proportion in the full score of the differential diagnostic thinking dimension can be configured to any percentage between 30% and 50% to fully reflect the weight of dynamic reasoning ability in clinical thinking assessment. Furthermore, the output of this step not only participates in the final score but also serves as the data source for the dynamic adaptive thinking trajectory diagram. During the report generation stage, it is rendered as a visual view with the consultation timeline as the horizontal axis, allowing students to intuitively see the evolution of their diagnostic hypotheses at each time point.

[0044] Step 4, extracting fatal error features at multiple levels; this step is executed in parallel with step 2, aiming to conduct an independent red line check on the consultation process from a security perspective.

[0045] The specific process in this step involves identifying fatal errors in the complete interactive dialogue record based on a medical guideline knowledge base and using preset multi-level classification standards.

[0046] A multi-level classification standard includes at least four levels: The first level is the missed diagnosis of fatal diseases; this level is used to identify whether students have failed to identify highly fatal diseases that could lead to the patient’s death or serious consequences in the short term. Typical situations include, but are not limited to: missed diagnosis of acute myocardial infarction, that is, when students are faced with typical clinical manifestations of chest pain accompanied by sweating and radiating to the left upper limb, they fail to include acute myocardial infarction in the differential diagnosis; Misdiagnosis of pulmonary embolism occurs when students are presented with a combination of high-risk features, such as shortness of breath, recent long-distance travel history, and lower extremity swelling, and pulmonary embolism is not included in the differential diagnosis. Missed diagnosis of aortic dissection, that is, when students face tearing pain accompanied by a difference in blood pressure on both sides, they fail to include aortic dissection in the differential diagnosis; Missed stroke diagnosis refers to the failure to recognize stroke and initiate the corresponding emergency procedures when students are faced with high-risk symptoms such as sudden hemiplegia, speech disorders, and facial paralysis.

[0047] The second level is the neglect of important symptoms; this level is used to identify whether students have not paid enough attention to the high-alert symptoms in the patient's chief complaint, typically including but not limited to neglect of chest pain, difficulty breathing, and altered consciousness.

[0048] The third level is emergency handling of errors. This level is used to identify whether students have taken incorrect or delayed actions when faced with an emergency. Typical situations include, but are not limited to, failing to identify an emergency that requires immediate attention, failing to initiate emergency procedures in a timely manner, or using medications that should be avoided in the context of the emergency.

[0049] Level 4: Failure to identify drug contraindications. This level is used to identify whether students have overlooked drug contraindications when developing treatment plans. Typical cases include, but are not limited to, failure to identify drug incompatibilities or contraindications for special populations (such as pregnant women, breastfeeding women, people with liver or kidney dysfunction, children, and the elderly).

[0050] The fatal error identification unit assigns a severity rating to each identified fatal error according to its level and the severity of the condition, and outputs it to step five in the form of a fatal error rating sequence as one of the inputs for dual-model cross-validation.

[0051] In a preferred implementation, the fatal error identification unit can be implemented based on a hybrid architecture that combines rule templates and a large language model. The rule templates are used to achieve high recall for typical fatal errors, while the large language model is used to achieve refined discrimination for context-dependent complex fatal errors.

[0052] Step 5: False positive prevention measures based on dual-model cross-validation; please refer to [link / reference]. Figure 5This step includes four specific sub-steps: concurrent blind evaluation, consistency circuit breaker comparison, highlighting of disputed nodes and transfer to review, and writing back the review results.

[0053] ① Concurrent blind evaluation sub-step: While the main evaluation language model generates the preliminary evaluation results, the secondary verification language model, which has been fine-tuned by the medical domain corpus, is asynchronously called to perform back-to-back blind evaluation of the complete interactive dialogue record.

[0054] The secondary validation large language model performs only two lightweight tasks: first, it makes fatal error judgments on the complete interactive dialogue transcript; second, it makes interval predictions on the scores of the core dimensions. The input of the secondary validation large language model does not include the output of the main evaluation large language model, thus ensuring that the judgments of the two models on the same diagnostic text are independent and blinded, avoiding the influence of one model's judgment on the other, and thus preserving the value of the two models as independent sources of evidence.

[0055] ② Consistency Circuit Breaking Comparison Sub-step: The arbitration service sub-unit receives the outputs of the primary evaluation language model and the secondary verification language model, and performs two consistency comparisons: Compare 1 to determine whether the two models agree on fatal errors, that is, whether the two models identify the same set of fatal errors; Comparison 2: Determine whether the difference in the overall scores of the two models is within a preset tolerance range. In this embodiment, the tolerance range is preferably any percentage between 5% and 15% of the overall score output by the secondary verification large language model.

[0056] The two models are considered to have the same judgment result only if both Comparison 1 and Comparison 2 pass. The arbitration service subunit then outputs an automatic release signal to the report generation unit, causing the evaluation report to be automatically released to the student terminal with a delay of up to one second. If either Comparison 1 or Comparison 2 fails, the arbitration service subunit outputs a circuit breaker signal to the report generation unit, blocking the release of the evaluation report.

[0057] ③ Highlighting disputed nodes and transferring to the review sub-step; For intercepted evaluation reports, the arbitration service sub-unit visually highlights the specific dialogue segments that triggered the disagreement in the complete interactive dialogue record, and highlights the evaluation dimensions that caused the disagreement between the two models.

[0058] Cases awaiting review, after being highlighted, are seamlessly pushed to the expert review queue via the expert review interface unit. The expert review queue is preferably implemented based on a web-based work order system, which supports sorting by the severity of the dispute, automatic routing to the corresponding expert by department (e.g., cardiovascular disputes are routed to cardiology teaching staff, neurological emergency disputes are routed to the head of the neurology teaching and research department), and allows experts to modify scores and write comments online.

[0059] ④ Review Result Write-back Sub-step: After the experts complete their review and ruling on the cases to be reviewed, the revised scores and qualitative conclusions are written back to the report generation unit. The expert rulings overwrite the original main assessment language model scores, and the student end is notified to update. At the same time, the case to be reviewed is automatically included in the difficult example corpus as a high-value sample with expert annotations, used for the next round of fine-tuning of the language model in secondary validation, forming a data closed loop of assessment, review, and retraining, enabling the system's assessment capabilities to continuously evolve over time.

[0060] In one preferred implementation, the secondary validation large language model can be obtained by supervised fine-tuning or parameter-efficient fine-tuning using a general-purpose base large language model with a parameter scale in the tens of billions, and using medical domain corpora (including but not limited to electronic medical records, medical textbooks, evidence-based medicine databases, clinical guideline texts, etc.). This results in higher domain fit in the medical field and superior performance in suppressing medical hallucinations compared to general-purpose large language models. In another implementation, the invention can be extended to a multi-model voting mechanism consisting of three or more large language models to further enhance robustness against fatal error identification.

[0061] Step Six: Specialty Specificity and Evidence-Based Medicine Level Assessment; The purpose of this step is to enhance the specialty specificity and evidence-based characteristics of the assessment results, making the assessment report adaptable to the teaching requirements of different departments. The specific processing procedure for this step is as follows: Based on the department to which the case belongs, the corresponding specialty assessment weights are applied to weight the six-dimensional scoring vector. In this embodiment, the specialty assessment weights can be configured differently for different departments such as internal medicine, surgery, emergency medicine, pediatrics, obstetrics and gynecology, neurology, cardiology, respiratory medicine, and gastroenterology. For example, in the emergency department scenario, the weights of the information gathering logic dimension and the fatal error identification dimension are relatively increased, while the weight of the communication skills dimension is relatively decreased, to match the special characteristics of emergency care where rescue comes first and consultation follows; in the pediatric scenario, the weights of the communication skills dimension and the doctor-patient relationship establishment dimension are relatively increased, to match the high sensitivity of pediatric patients and their families to doctor-patient communication.

[0062] The system assesses the level of evidence-based medicine for the final diagnosis and treatment plan proposed by the student. This assessment is based on the current evidence-based medicine grading system, which categorizes evidence into at least four levels: A, B, C, and D. Level A corresponds to strong evidence from multiple high-quality randomized controlled trials or authoritative guidelines; Level B corresponds to moderately strong evidence from a single high-quality randomized controlled trial or multiple observational studies; Level C corresponds to weak evidence from small-sample studies or expert consensus; and Level D corresponds to very weak evidence based solely on case reports or clinical experience. The system adjusts the final score for the rationality dimension of the treatment plan based on the level of evidence cited in the student's proposed diagnosis and treatment plan.

[0063] Based on the weighted six-dimensional scoring vector, diagnostic reasoning graph, fatal error level sequence, and evidence-based medicine evidence level determination results, the report generation unit generates a personalized improvement report containing the reasoning process. The personalized improvement report includes, but is not limited to, the following sections: total score and six-dimensional radar chart; a visualization of the diagnostic reasoning graph (white background, black text, clearly showing key positive signs, key negative signs, omissions, and logical breaks); a dynamic adaptive thought trajectory diagram (with the consultation timeline as the horizontal axis, marking the evolution of diagnostic hypotheses at each time point); a list of highlighted fatal errors and corresponding references to treatment guidelines; personalized improvement suggestions and links to extended learning resources; if the case has undergone expert review, it will also display the expert's decision and its differences from the initial AI judgment.

[0064] Example 2 Please see Figure 1 This embodiment provides a clinical reasoning assessment system based on thought chain reasoning, used to implement the method in Embodiment 1. From a hardware perspective, the clinical reasoning assessment system can be composed of several servers and network devices; from a software functional perspective, the clinical reasoning assessment system includes a data acquisition unit, a thought chain reasoning engine unit, a dynamic adaptive assessment unit, a fatal error identification unit, a dual-model cross-validation unit, an expert review interface unit, and a report generation unit.

[0065] The data acquisition unit's function and implementation are consistent with step one of Example 1; the thought chain reasoning engine unit includes a fact extraction submodule, a clinical mapping submodule, and a quantitative tracing submodule, and its function and implementation are consistent with step two of Example 1; the dynamic adaptive assessment unit includes a diagnostic hypothesis comparison submodule and a dynamic adaptive assignment submodule, and its function and implementation are consistent with step three of Example 1; the fatal error identification unit's function and implementation are consistent with step four of Example 1; the dual-model cross-validation unit includes a main large model interface, a secondary validation model interface, and an arbitration service subunit, and its function and implementation are consistent with step five of Example 1; the expert review interface unit's function and implementation are consistent with the third and fourth sub-steps of step five of Example 1; and the report generation unit's function and implementation are consistent with step six of Example 1.

[0066] In a preferred hardware deployment, the data acquisition unit, report generation unit, and expert review interface unit are deployed on a web application server, allowing students, teachers, and experts to access them via browsers or mobile clients. The thought chain inference engine unit, dynamic adaptive evaluation unit, fatal error identification unit, and dual-model cross-validation unit are deployed on an AI inference server with GPU acceleration capabilities. The medical guideline knowledge base and difficult example corpus are deployed on a database server with high availability. The units communicate with each other via application programming interfaces (APIs) and asynchronous task scheduling is achieved through message queues.

[0067] Those skilled in the art will understand that the physical implementation of the above-described units is not limited to a single server, but can also be deployed as distributed microservices on local servers, private clouds, or public clouds, and scheduled through a pre-defined application programming interface gateway. Each unit can also be implemented using any combination of general-purpose processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and servers with graphics processing unit (GPU) acceleration capabilities.

[0068] Example 3 This embodiment further illustrates the actual operation of the method in Embodiment 1 using a typical emergency scenario. The case involves a 55-year-old male patient who presented to the emergency department with a chief complaint of sudden chest pain accompanied by acid reflux for two hours. The patient had a ten-year history of type 2 diabetes with poor blood sugar control. The patient's father died of acute myocardial infarction, and there was no family history of gastrointestinal tumors. At the time of this visit, the electrocardiogram showed ST segment elevation of 0.3 mV in leads V1-V4.

[0069] Student A interacts with a virtual patient during a consultation: At the initial stage of the consultation (the first point in time), student A collected the chief complaint (sudden chest pain accompanied by acid reflux) and some current medical history (worsening after meals and failure of acid-suppressing drugs to relieve it), and submitted a preliminary diagnosis hypothesis to the system as gastroesophageal reflux disease.

[0070] During the in-depth consultation phase (second time point), Student A further inquired about the patient's past medical history and family history (learning that the patient's father died of acute myocardial infarction and that the patient himself had a history of diabetes), and requested an electrocardiogram (obtaining the crucial result of ST segment elevation in leads V1-V4). At this point, Student A submitted a revised diagnostic hypothesis to the system: gastroesophageal reflux disease and acute coronary syndrome should be differentiated concurrently.

[0071] During the thought process evolution stage (the third time point), after synthesizing the above information, Student A proactively modified the diagnostic hypothesis to acute coronary syndrome in the system and issued instructions for nitroglycerin testing, as well as examinations such as myocardial enzyme spectrum and coagulation function.

[0072] The execution process of the method is as follows: The data acquisition unit constructs a consultation timeline containing the above three time nodes. The thinking chain reasoning engine unit forces the main assessment large language model to perform fact extraction (extracting key positive signs such as the nature of chest pain, radiation location, family history of coronary heart disease, ST segment elevation on electrocardiogram, and key negative signs such as no dysphagia and no hematemesis), clinical mapping (comparing the above facts with the checklist of cardiovascular guidelines, no significant omissions were found; comparing the order of students' consultations with the order recommended by the guidelines, no significant logical breaks were found), and quantitative tracing (giving specific scores in each dimension: 24 points for completeness of information collection, 19 points for logicality of information collection, 15 points for basic differential diagnostic thinking, etc.), and forming a diagnostic reasoning map.

[0073] The dynamic adaptability assessment unit identified that between the first and second time points, Student A's diagnostic hypothesis was adjusted from gastroesophageal reflux disease (GERD) to a differential diagnosis of GERD and acute coronary syndrome (ACS), a significant deviation, but the direction of deviation was approaching the target diagnostic direction (ACS), earning a bonus of 3 points for dynamic adaptability. Between the second and third time points, Student A's diagnostic hypothesis further converged to ACS, a significant deviation, but the direction of deviation continued to approach the target diagnostic direction, earning a bonus of 2 points for dynamic adaptability. The total dynamic adaptability score is 5 points, which are added to the differential diagnosis thinking dimension.

[0074] The fatal error identification unit did not detect any fatal errors at any level. In the dual-model cross-validation unit, the primary evaluation language model gave a comprehensive score of 92, while the secondary validation language model gave a range prediction of 90 to 95. The two models were consistent in their judgment of "no fatal errors," and the difference in comprehensive scores was within the tolerable range. The arbitration service subunit output an automatic release signal, and the report generation unit pushed the evaluation report to the student's end within 2.8 seconds.

[0075] The evaluation report clearly states that Student A achieved a perfect score in the differential diagnostic thinking dimension, demonstrating excellent dynamic adaptability. Based on a family history of coronary heart disease and electrocardiogram characteristics, Student A was able to promptly overturn the initial hypothesis of a gastrointestinal disease and accurately steer the diagnosis towards cardiovascular emergencies. However, the report also noted a slight deduction in Student A's communication skills dimension (the questioning of the nature of pain was somewhat mechanical, and insufficient attention was paid to the patient's emotions), and provided suggestions for improvement.

[0076] Example 4 This example uses a typical respiratory medicine scenario to further illustrate the circuit breaker and expert review process of the dual-model cross-validation mechanism. The case involves a 38-year-old female patient with a body mass index of 31 and a history of oral contraceptive use for one and a half years (hidden in the medical record). She had experienced consecutive long-haul intercontinental flights in the past two weeks and presented with sudden respiratory distress accompanied by swelling of the left lower extremity.

[0077] During the consultation with the virtual patient, Student B collected basic respiratory symptoms (dyspnea, cough nature, presence or absence of wheezing), but failed to inquire about recent long-distance travel history, lower limb swelling, history of oral contraceptive use, or request examinations for deep vein thrombosis such as lower limb vascular ultrasound. Instead, Student B directly diagnosed the patient as having an acute exacerbation of bronchial asthma and prescribed salbutamol nebulization.

[0078] During the execution of the method of this invention, the primary evaluation large language model, after reasoning through the thought chain, gives a comprehensive score of 80 points based on its affirmative recognition of respiratory symptoms, and the list of fatal errors is empty. However, the secondary validation large language model, during asynchronous blind evaluation, has higher sensitivity to the high-risk feature combination of dyspnea, obese women, oral contraceptives, and long-distance flight due to its fine-tuning of medical corpus. Thus, it determines that the list of fatal errors includes a missed diagnosis of pulmonary embolism and predicts a comprehensive score range of 40 to 55 points.

[0079] The arbitration service subunit performed a consistency comparison: Comparison 1 failed because the primary evaluation language model determined there were no fatal errors, while the secondary verification language model determined there were fatal errors; Comparison 2 also failed because the difference in the overall score far exceeded the tolerance range. The arbitration service subunit immediately output a circuit breaker signal, intercepting the 80-point evaluation report that was originally scheduled to be automatically released; at the same time, it visually highlighted the disputed nodes triggered in the complete interactive dialogue record, including specific dialogue segments such as not inquiring about long-haul flight history, not inquiring about lower limb swelling, and not paying attention to oral contraceptive use, as well as the two evaluation dimensions of differential diagnosis thinking and fatal error identification.

[0080] The highlighted cases awaiting review were pushed to the review queue of the head of the Department of Respiratory Medicine through the expert review interface unit. The expert reviewed the complete dialogue record, the discrepancies between the two models, and the marked disputed nodes in the work order system on the expert's end. Based on the comprehensive judgment, the expert concluded that the judgment of the secondary validation large language model was valid (this case did indeed have a serious risk of missed diagnosis of pulmonary embolism). The expert manually corrected the score of this case to failing (40 points) and issued a strong warning feedback to student B regarding the differential diagnosis of pulmonary embolism, requiring student B to relearn the differential diagnosis process for dyspnea and the list of high-risk features for pulmonary embolism.

[0081] The expert's decision is bidirectionally written back through the feedback write-back module: on the one hand, Student B's final evaluation report is updated to 40 points, with a strong warning from the expert attached; on the other hand, this case is included in the difficult case corpus and used as a high-value sample in the next round of secondary verification of the large language model iteration and fine-tuning, so as to further improve the system's ability to identify this type of case.

[0082] As can be seen from this embodiment, the dual-model cross-validation and circuit breaker mechanism provided by the present invention can construct a second safety valve in the case of occasional medical hallucinations in the main evaluation large language model, in the form of asynchronous blind evaluation, effectively preventing the inflated scores of the main evaluation large language model from flowing to the student end, thereby maintaining the safety bottom line of medical teaching evaluation while preserving the second-level feedback efficiency.

[0083] Example 5 This embodiment illustrates the specific implementation of the method of the present invention in terms of specialist assessment weighting and fatal error identification using a neurological emergency scenario. The case involves a 72-year-old male patient with a 15-year history of hypertension (irregular blood pressure control) and a 5-year history of atrial fibrillation (irregular use of anticoagulants). He presented to the neurology department with the chief complaint of sudden right-sided limb weakness accompanied by slurred speech for 40 minutes.

[0084] Student C interacts with a virtual patient during a consultation: At the first critical moment, student C collected the chief complaint, the mode of onset (sudden onset), and the characteristics of the symptoms (weakness in the right limbs, drooping of the mouth, and dysarthria), and quickly submitted a preliminary diagnostic hypothesis: acute ischemic stroke.

[0085] At the second time point, Student C inquired about the patient's history of underlying diseases such as hypertension, atrial fibrillation, and diabetes, as well as their history of anticoagulant medication use, and learned that the patient did not take anticoagulant medication regularly. At this point, Student C submitted a revised diagnostic hypothesis: acute ischemic stroke (most likely cardioembolic).

[0086] At the third time point, student C requested a plain MRI scan of the head with diffusion-weighted imaging, which yielded the key examination result of "acute infarction lesion in the blood supply area of ​​the left middle cerebral artery", and a preliminary medical order for intravenous thrombolysis was issued (because the patient's onset time was within the 4.5-hour window).

[0087] The execution process of the method is as follows: The data acquisition unit constructs a consultation timeline containing the above three time nodes. The thinking chain reasoning engine unit forces the main assessment large language model to perform three-step reasoning. In the fact extraction stage, key positive signs such as sudden onset, unilateral limb weakness, facial drooping, dysarthria, irregular anticoagulation, and MRI showing acute infarction are extracted, as well as key negative signs such as no obvious headache and no loss of consciousness. In the clinical mapping stage, no significant omissions were found, but it was found that Student C's inquiry about contraindications to thrombolysis was relatively brief (the history of recent surgery and gastrointestinal bleeding were not clearly inquired about). In the quantitative tracing stage, one point was deducted in the dimension of information collection completeness and one point was deducted in the dimension of treatment plan rationality.

[0088] The dynamic adaptability assessment unit identified that the evolution of the diagnostic hypothesis of student C between the three time points was relatively reasonable (from stroke to cardioembolic ischemic stroke, and then to thrombolytic ischemic stroke within the 4.5-hour window), and awarded four points for dynamic adaptability.

[0089] The fatal error identification unit did not detect a fatal error, but found that Student C's inquiry about contraindications for thrombolysis was incomplete. This suggestion was included in the report as a non-fatal improvement suggestion.

[0090] During the specialist-specific assessment phase, the system identified this case as a neurological emergency, applying neurological assessment weights. The weights for the logicality of information gathering and the identification of fatal errors were relatively increased (because the time window for neurological emergencies is extremely critical; any delay in information gathering could lead to missing the thrombolysis window), while the weight for communication skills was relatively decreased. Student C's final weighted score was 89 points, earning a "Good" rating.

[0091] Within three seconds, the report generation unit pushed the evaluation report to the student's end. The report clearly pointed out that: Student C's diagnostic approach was clear and he was able to seize the opportunity for thrombolysis within the time window, but the inquiry into the contraindications for thrombolysis needed to be more complete. At the same time, through a visualized diagnostic reasoning graph, the complete reasoning chain from the chief complaint to the final diagnosis was clearly displayed, allowing Student C to intuitively see the strengths and areas for improvement of his approach.

[0092] Example 6 This embodiment illustrates the implementation of the method of the present invention in the dimensions of communication skills and doctor-patient relationship establishment using a pediatric scenario. The case involves a seven-year-old boy with a history of allergic rhinitis and asthma (diagnosed three years ago). He was accompanied by his mother to the pediatric department for treatment. The child experienced wheezing and shortness of breath after strenuous exercise that morning, which was not completely relieved after treatment with a home inhaler. The child's mother was anxious and repeatedly asked whether there was any danger to his life.

[0093] Student Ding interacts with a virtual patient (a child and his mother) during a consultation.

[0094] At the first point in time, student Ding collected the chief complaint (wheezing, shortness of breath), triggering factors (strenuous exercise), and information on the use of home medications, and submitted a preliminary diagnostic hypothesis: acute exacerbation of bronchial asthma.

[0095] At the second time point, student Ding inquired about his past asthma history, allergy history, family history of asthma, and recent upper respiratory tract infection, thus completing the medical history and maintaining the original diagnostic hypothesis.

[0096] At the third time point, student Ding applied for pulmonary function tests and blood oxygen saturation monitoring, obtained the corresponding objective test results, and was given a prescription for short-acting β2 receptor agonist combined with glucocorticoid nebulized inhalation.

[0097] The method's execution process was as follows: After the thought chain reasoning engine unit completed three steps of reasoning, it scored highly in dimensions such as information collection completeness, information collection logic, and differential diagnostic thinking. However, the clinical mapping phase revealed that: throughout the consultation process, student Ding failed to respond to the child's mother's anxiety, neither proactively comforting her nor clearly explaining the prognosis of an acute asthma attack; furthermore, the student did not adjust the language style for the seven-year-old child, using numerous technical terms, which the child clearly did not understand during the interaction.

[0098] Accordingly, in the quantitative tracing phase, three points were deducted from the communication skills dimension (out of a maximum of ten points), and four points were deducted from the doctor-patient relationship establishment dimension (out of a maximum of fifteen points). The specific basis for the deductions was clearly marked in the diagnostic reasoning graph. No significant adjustments to diagnostic assumptions were found in the dynamic adaptive assessment unit, therefore no points were added or deducted for the differential diagnostic thinking dimension.

[0099] The fatal error identification unit did not detect any fatal errors. In the dual-model cross-validation unit, the primary evaluation language model and the secondary validation language model made the same judgment on fatal errors. The difference in the overall score was within the tolerance range. The arbitration service subunit output an automatic release signal.

[0100] During the specialist-specific assessment phase, the system identified this case as belonging to a pediatric setting and applied pediatric assessment weights: the weight of the communication skills dimension was increased from the default 10% to 15%, and the weight of the doctor-patient relationship establishment dimension was increased from the default 15% to 20%, to match the clinical characteristics of pediatric patients and their families' high sensitivity to doctor-patient communication. After weighting, student Ding's final score was 74 points, receiving a moderate evaluation.

[0101] The evaluation report clearly states that student Ding has solid clinical diagnostic skills and is able to diagnose and manage acute asthma attacks in children in a standardized manner; however, there is still considerable room for improvement in communication skills and the establishment of doctor-patient relationships, specifically in insufficient response to the anxiety of family members and failure to adjust language style to children; the report also includes links to extended learning resources on key points of pediatric doctor-patient communication for student Ding's reference.

[0102] Example 7 This embodiment illustrates the specific implementation of the method of the present invention in dynamic adaptive assessment at multiple time points using a gastroenterology scenario. The case involves a 46-year-old female patient who presented to the gastroenterology department with a chief complaint of right upper quadrant abdominal pain for three hours, accompanied by nausea and vomiting. The patient had a five-year history of gallstones (without surgery) and no history of hepatitis. At the time of this visit, her body temperature was 38.2 degrees Celsius and Murphy's sign was positive.

[0103] The consultation process in this case involved four time points to fully demonstrate the application of the dynamic adaptive assessment mechanism in long-term consultation.

[0104] At the first point in time, student E collected the chief complaint and the trigger (onset after eating greasy food) and submitted a preliminary diagnostic hypothesis: acute gastroenteritis.

[0105] At the second time point, student E further inquired about the patient's medical history (learning that the patient had a five-year history of gallstones) and conducted a detailed physical examination simulation of the abdominal pain (learning that there was tenderness in the right upper quadrant and a positive Murphy's sign). The student then submitted a revised diagnostic hypothesis: acute cholecystitis was the most likely diagnosis, while acute gastroenteritis was the least likely diagnosis.

[0106] At the third time point, Student E requested blood routine and liver function tests, and obtained key test results showing elevated white blood cell count and mildly elevated alanine aminotransferase. He then submitted a further revised diagnostic hypothesis: possible acute cholecystitis complicated by mild liver function impairment.

[0107] At the fourth time point, Student E requested an abdominal ultrasound examination, which yielded key findings including gallbladder wall thickening, multiple gallstones in the gallbladder, and perigastrium effusion. The final diagnostic hypothesis was submitted: acute cholecystitis (gallstone-related) with concurrent infection.

[0108] The method's execution process is as follows: the data acquisition unit constructs a consultation timeline containing the four time nodes mentioned above. After completing three steps of reasoning, the thought chain reasoning engine unit provides high scores across each dimension.

[0109] The dynamic adaptability assessment unit performs deviation quantification comparisons on three pairs of adjacent time points between the four time points: Between the first and second time points, the diagnostic hypothesis is highly likely to change from acute gastroenteritis to acute cholecystitis, with a significant deviation and the direction of deviation approaching the target diagnostic direction, thus receiving two bonus points for dynamic adaptability; Between the second and third time points, the diagnostic hypothesis is further refined by adding a detailed judgment of concurrent liver function impairment, with a small deviation, and no changes are made; Between the third and fourth time points, the diagnostic hypothesis is further refined to cholecystitis with gallstones and concurrent infection, with a significant deviation and the direction of deviation continuing to approach the target diagnostic direction, thus receiving two bonus points for dynamic adaptability. The total dynamic adaptability index is four points, which are added to the differential diagnostic thinking dimension.

[0110] The fatal error identification unit did not detect any fatal errors; the dual-model cross-validation unit determined that the scores were consistent and the difference in the overall scores was within the tolerance range. The arbitration service subunit output an automatic release signal, and the report generation unit pushed the evaluation report to the student's end within 2.5 seconds.

[0111] The evaluation report, through a dynamic adaptive thinking trajectory visualization, clearly demonstrates the evolution of Student E's diagnostic hypothesis at four time points: from acute gastroenteritis to cholecystitis with gallstones and concurrent infection, showcasing strong clinical dynamic reasoning ability. The report also points out that Student E proposed standardized anti-infection treatment and surgical consultation recommendations, earning a high score in the treatment plan rationality dimension.

[0112] Example 8 This embodiment describes the visualization presentation of the intermediate products and final personalized improvement report output by the present invention. The visualization presentation includes at least the following types of views: Firstly, the diagnostic reasoning graph view. This view presents the fact extraction results, clinical mapping results, and quantitative tracing results generated by the thought chain reasoning engine unit in a tree structure or directed acyclic graph structure, allowing students and teachers to intuitively see the specific evidence corresponding to each deduction. The root node of this view is the disease category of the case, the middle nodes are the branches of the guideline checklist, and the leaf nodes are the key positive signs, key negative signs, and missed points collected by the student. Each missed point is marked with its corresponding deduction dimension and deduction value, so that every point added or deducted in the assessment process can be traced back to the original evidence in this view.

[0113] Secondly, the Dynamic Adaptive Thinking Trajectory View. This view uses the consultation timeline as the horizontal axis and the evolution of diagnostic hypotheses as the vertical axis. It marks the diagnostic hypotheses submitted by the student at each time point as nodes on the timeline, connecting adjacent nodes with arrows indicating the degree and direction of deviation. When the deviation is large and the direction points towards the target diagnosis, the arrow is presented as a bold solid line; when the deviation is large but the direction deviates from the target diagnosis, the arrow is presented as a bold dashed line; when the deviation is small, the arrow is presented as a thin, ordinary line. Students can use this view to intuitively review their clinical dynamic reasoning process.

[0114] Third, the Fatal Error Highlight View; this view presents each fatal error identified by the fatal error identification unit in red, along with the corresponding dialogue segment. Each fatal error is followed by its level, severity rating, and relevant clinical guideline reference, enabling students to clearly understand what they missed, why it was fatal, and how to rectify it.

[0115] Fourth, the six-dimensional radar chart; this view visualizes the scores of the six assessment dimensions in the form of a radar chart, allowing students to see their relative strengths and weaknesses in each dimension at a glance, which helps students to make up for their weaknesses in a targeted manner.

[0116] All the above visualizations are presented in a structured, downloadable format with black text on a white background on the student, teacher, and expert ends. In actual system operation, these visualizations not only aid in teaching feedback but also provide verifiable evidence of the system's operation in the event of evaluation disputes or potential infringement conflicts. This allows the method steps, parameter values, unit division, and dual-model circuit breaker triggering conditions claimed in this invention to be objectively verified through interface screenshots and log records.

[0117] Example 9 This invention also provides an electronic device, which includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements all or part of the steps of the clinical reasoning assessment method based on thought chain reasoning in Embodiment 1. The electronic device can be a desktop computer, server, workstation, dedicated clinical teaching all-in-one machine, etc. The processor can be a central processing unit, or other general-purpose processors, digital signal processors, application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.; the memory can be a read-only memory, random access memory, flash memory, erasable programmable read-only memory, electrically erasable programmable read-only memory, registers, hard disks, removable hard disks, optical disks, or other read-write storage media.

[0118] This invention also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements all or part of the steps of the clinical reasoning assessment method based on thought chain reasoning in Embodiment 1. The computer-readable storage medium includes, but is not limited to, disks, optical discs, read-only memory, random access memory, flash memory, and cloud storage media.

[0119] Example 10 Those skilled in the art, based on the technical content disclosed in the embodiments of the present invention, may make any one or more equivalent modifications without departing from the essence of the present invention: First, the primary evaluation large language model in the embodiment is replaced by another general large language model with equivalent parameter size or structure, and the secondary verification large language model in the embodiment is replaced by another large language model with equivalent parameter size or structure fine-tuned from medical domain corpus.

[0120] Secondly, the diagnostic hypothesis difference quantification method in the embodiments can be replaced by any equivalent distance metric suitable for characterizing semantic differences, such as ontology distance, graph editing distance, or vector space distance.

[0121] Third, adjust the number of components and the full score configuration of each component in the six-dimensional scoring vector in the embodiment. For example, expand the six dimensions to eight dimensions (add dimensions such as the depth of medical history collection and the standardization of physical examination), or adjust the full score ratio of each dimension according to special teaching requirements.

[0122] Fourth, a multi-model concurrent voting mechanism is constructed using three or more large language models, which proportionally expands the dual-model cross-validation mechanism in the embodiment. For example, a majority vote or weighted voting method is used to determine the final consistency result.

[0123] Fifth, the method and system of this invention can be applied to any teaching scenario related to medical history taking and reasoning, such as nursing education, oral medicine education, traditional Chinese medicine diagnostics education, veterinary medicine education, and general practitioner training.

[0124] Sixth, without affecting the essential technical effect of the present invention, non-substantial local adjustments can be made to the execution order of each step, such as slightly advancing or delaying the execution timing of the fatal error identification step.

[0125] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0126] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A clinical reasoning assessment method based on thought chain reasoning, characterized in that, The method includes the following steps: S1. Collect the interactive dialogue records between the student terminal and the virtual patient terminal, and construct a consultation timeline containing several time nodes, with the diagnostic hypotheses submitted by the student at each time node. S2. Input the interactive dialogue record into the main assessment big language model. Before outputting the quantitative score, the main assessment big language model performs fact extraction, clinical mapping and quantitative tracing in sequence to generate a diagnostic reasoning graph and output a multi-dimensional scoring vector. S3. Based on the consultation timeline, the deviation of the diagnostic hypotheses submitted by students at adjacent time points is compared and the dynamic adaptability index is obtained according to the magnitude and direction of the deviation. The dynamic adaptability index is then incorporated into the multi-dimensional scoring vector. S4. Identify fatal errors in interactive dialogue records based on the medical guideline knowledge base, and output a fatal error level sequence according to the preset multi-level classification criteria; S5. Call the secondary validation language model to perform blind evaluation of the interactive dialogue record, and compare the output of the main evaluation language model with that of the secondary validation language model. If the two models agree, issue an evaluation report; otherwise, trigger the circuit breaker and transfer the case to expert review. S6. Weight the multi-dimensional scoring vectors according to the department to which the case belongs, and generate an assessment report that includes the reasoning process.

2. The clinical reasoning assessment method based on thought chain reasoning according to claim 1, characterized in that, Constructing a consultation timeline includes: Add timestamps to the multimodal interaction data in the interactive dialogue records, and use the diagnostic hypothesis events actively submitted by students as time node anchors to generate a consultation timeline; Multimodal interaction data includes at least text-based question-and-answer sessions between students and virtual patients, student-initiated auxiliary examinations and medical orders, and diagnostic hypothesis events submitted by students.

3. The clinical reasoning assessment method based on thought chain reasoning according to claim 1, characterized in that, Fact extraction, clinical mapping, and quantitative traceability include: Fact extraction involves extracting key positive and key negative signs from the interactive dialogue record, and inferences and scoring are prohibited at this step. The clinical mapping loads the examination checklist for the current case from the medical guideline knowledge base, compares the extracted signs with the examination checklist, marks the omissions, and compares the student's actual consultation order with the recommended order in the examination checklist, marking the logical breakpoints. Quantitative tracing is based on omission points and logical breakpoints. It deducts items from the multi-dimensional scoring vector in the corresponding dimensions one by one, and organizes the extracted signs, omission points and logical breakpoints into a traceable diagnostic reasoning map.

4. The clinical reasoning assessment method based on thought chain reasoning according to claim 1, characterized in that, Deviation from metric comparisons includes: For adjacent time node pairs in the consultation timeline, based on the target diagnostic direction of the current case in the medical guide knowledge base, the deviation degree and deviation direction of the two diagnostic hypothesis sets are determined. Bonus points are awarded when the deviation exceeds the first threshold and the deviation direction points towards the target diagnostic direction. When the deviation exceeds the first threshold but the direction of deviation deviates from the target diagnostic direction, points will be deducted. When the deviation is less than or equal to the second threshold, no increase or decrease is made; The scores or deductions generated by all adjacent time points in the consultation timeline are accumulated according to their weights to obtain a dynamic adaptive index.

5. A clinical reasoning assessment method based on thought chain reasoning according to claim 1, characterized in that, The multi-level classification criteria include at least: a level for missed diagnosis of fatal diseases, a level for neglect of important symptoms, a level for emergency treatment of errors, and a level for failure to identify drug contraindications; each fatal error in the fatal error level sequence carries its own level and severity rating.

6. The clinical reasoning assessment method based on thought chain reasoning according to claim 1, characterized in that, The outputs of the primary evaluation large language model and the secondary validation large language model are compared and include: While the primary evaluation language model generates preliminary evaluation results, the secondary verification language model is asynchronously called to perform back-to-back blind evaluation. The input of the secondary verification language model does not include the output of the primary evaluation language model. When the primary evaluation language model and the secondary validation language model agree on the fatal error and the difference in the overall score is within the preset tolerance range, the two models are considered to have the same judgment result and an evaluation report is automatically issued; otherwise, the evaluation report is intercepted, the dialogue segment that triggered the divergence and the corresponding evaluation dimension are highlighted, and the case to be reviewed is pushed to the expert review queue. After the experts completed their review, the revised scores given by the experts were used to cover the original preliminary assessment results, and the cases to be reviewed were included in the difficult case corpus for secondary verification and iterative fine-tuning of the large language model.

7. The clinical reasoning assessment method based on thought chain reasoning according to claim 1, characterized in that, The multi-dimensional scoring vector includes the dimensions of information collection completeness, information collection logic, differential diagnostic thinking, treatment plan rationality, communication skills, and doctor-patient relationship establishment. Dynamic adaptive indicators are incorporated as components into the differential diagnostic thinking dimension. The generation of the assessment report also includes determining the level of evidence-based medicine for the student's final diagnosis and treatment plan, and incorporating the determination results into the assessment report.

8. A clinical reasoning assessment system based on thought chain reasoning, used to implement the clinical reasoning assessment method based on thought chain reasoning as described in any one of claims 1 to 7, characterized in that, include: The data acquisition unit collects interactive dialogue records between the student's end and the virtual patient's end, and constructs a consultation timeline containing several time nodes; The thought chain reasoning engine unit drives the main assessment language model to perform fact extraction, clinical mapping, and quantitative tracing in sequence before outputting the quantitative score, so as to generate a diagnostic reasoning graph and output a multi-dimensional scoring vector. The dynamic adaptive assessment unit compares the deviation of diagnostic hypotheses at adjacent time points based on the consultation time axis and incorporates the obtained dynamic adaptive indicators into the multi-dimensional scoring vector. The fatal error identification unit identifies fatal errors in interactive dialogue records based on a medical guideline knowledge base and outputs a fatal error level sequence according to a preset multi-level classification standard. The dual-model cross-validation unit calls the secondary validation large language model to perform blind evaluation of the interactive dialogue records, and performs consistency comparison between the output of the main evaluation large language model and the secondary validation large language model. When the two models are consistent, the evaluation report is released; when the two models are inconsistent, the circuit breaker is triggered and the process is transferred to expert review. The report generation unit weights the multi-dimensional scoring vectors according to the department to which the case belongs, and generates an assessment report that includes the reasoning process.

9. The system according to claim 8, characterized in that, The dual-model cross-validation unit also includes a feedback write-back module, which is used to write back the experts' corrected scores to the iterative corpus of the secondary validation large language model after the experts have completed their review.

10. An electronic device, characterized in that, It includes a memory and a processor, the memory storing a computer program that can run on the processor, and the processor executing the computer program to implement the method as described in any one of claims 1 to 7.

Images