Method and system for evaluating clinical inquiry quality

Abstract

The application provides a clinical inquiry quality evaluation method and system. The method comprises: acquiring real-time doctor-patient inquiry data; analyzing the doctor-patient inquiry data in real time, and constructing a current inquiry clinical semantic context; generating a question list based on the clinical semantic context through a hybrid evaluation model, wherein the hybrid evaluation model integrates a rule engine and a machine learning model; determining whether the contents of the question list are missed in the doctor-patient inquiry data; and generating and outputting quality evaluation feedback information for the inquiry according to the determination result, wherein the quality evaluation feedback information comprises a quantitative quality score and / or an inquiry supplement suggestion. The application realizes complete collection, intelligent supplement and interpretable feedback of inquiry information by fusing a medical knowledge graph, a rule engine and a machine learning model to perform real-time semantic analysis and dynamic quality evaluation on the doctor-patient inquiry process.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and medical information processing technology, specifically to a method and system for assessing the quality of clinical consultations. Background Technology

[0002] In clinical practice, patient history taking is a crucial step for doctors to obtain information about a patient's condition and form a preliminary diagnostic approach. The quality of the history taking directly affects the completeness of medical records and the accuracy of diagnostic decisions. However, in current medical practice, doctors' history taking still relies heavily on personal experience and subjective judgment. Limited by time pressure, case complexity, and the doctor's attention allocation, information omissions, disorganized questioning order, or logical inconsistencies often occur during the history taking, leading to incomplete medical records (such as the subjective, objective, assessment, and planning information sections in a SOAP structure). These problems not only increase the risk of misdiagnosis and missed diagnosis but also expose the widespread phenomenon of inconsistent quality in medical documentation, especially among young doctors or those working at the grassroots level.

[0003] To improve the efficiency and standardization of patient consultations, some medical institutions have introduced electronic medical records (EHR) systems or template-based data entry tools. These systems often guide doctors through pre-set templates or rules, but lack flexibility and semantic understanding capabilities, failing to dynamically adjust questions or key points based on the patient's specific condition. Doctors often need to repeatedly switch between, select, and input data within the system interface, increasing administrative burden and weakening the quality of communication between doctors and patients. Over-reliance on template-based recording can also lead to mechanical consultations, making it difficult for the system to detect hidden symptoms or key clues in the patient's description, resulting in missing information and biased clinical judgment.

[0004] On the other hand, most existing methods for evaluating the quality of medical consultations are post-hoc analyses. Systems typically perform structured extraction and scoring of the generated medical record text, but this offline assessment model lacks real-time feedback capabilities, failing to dynamically identify omissions or prompt doctors to ask additional questions during the consultation process. Furthermore, current scoring mechanisms are mostly based on superficial indicators such as word count, time, or keyword matching, failing to incorporate semantic-level analysis using medical knowledge systems. They also struggle to assess the rationality and completeness of the consultation from a clinical logic perspective, resulting in quality assessment results lacking clinical guidance significance.

[0005] In recent years, medical knowledge graphs have been widely used in clinical decision support, but their application in consultation scenarios remains relatively superficial. Existing systems mostly use knowledge graphs for static knowledge retrieval or entity recognition, failing to deeply integrate them with real-time natural language dialogue semantics, thus hindering the semantic modeling and logical evaluation of the consultation process. Furthermore, customized knowledge graphs for different specialties, medical theoretical systems, or regional clinical characteristics are still incomplete, limiting the system's adaptability and professionalism in specific clinical scenarios.

[0006] Furthermore, current medical history analysis technologies largely focus on single-modal text data, failing to fully integrate multimodal medical information such as voice and images. This lack of multimodal information makes it difficult for systems to comprehensively capture the context and characteristics of the patient's condition during the consultation, affecting the accuracy of quality assessments. In addition, when faced with sudden new diseases or public health emergencies, existing systems often cannot update relevant knowledge or adjust consultation priorities in a timely manner, lacking dynamic response capabilities based on the latest disease databases. On the other hand, existing models have weak mechanisms for continuous learning, making it difficult for them to self-optimize based on actual clinical feedback, resulting in assessment results that are difficult to maintain in terms of long-term accuracy and reliability.

[0007] In summary, existing technologies in the field of medical consultation quality assessment still suffer from problems such as insufficient detection of missing information, rigid templates, lack of real-time feedback, weak multimodal fusion capabilities, and poor model self-evolution. How to achieve dynamic understanding of clinical semantics during the consultation process and perform intelligent reasoning and real-time quality assessment based on medical knowledge systems has become a pressing technical challenge in the field of medical artificial intelligence. Summary of the Invention

[0008] This invention provides a clinical consultation quality assessment method and system based on medical knowledge graphs and a hybrid assessment model. It acquires real-time consultation data through speech recognition and speaker separation, constructs a clinical semantic context using a medical knowledge graph, and generates a list of mandatory and implicit questions using a hybrid assessment mechanism integrating an integrated rule engine and a machine learning model. This allows for dynamic monitoring and quality feedback during the consultation process, enabling real-time identification and completion of missing information. The system can output quality assessment results in a visual format or automatically generate structured medical records, significantly reducing doctors' paperwork burden and improving the completeness and consistency of consultations. Furthermore, this invention supports the fusion analysis of multimodal data such as medical images and achieves continuous model optimization through expert annotation and interactive correction mechanisms. This constructs an intelligent, interpretable, and self-evolving system for consultation quality assessment, effectively overcoming the shortcomings of existing technologies in terms of flexibility, real-time performance, and semantic reasoning depth.

[0009] In a first aspect, the present invention provides a method for assessing the quality of clinical consultations, characterized in that the method comprises: Obtain real-time doctor-patient consultation data; Based on real-time analysis of the doctor-patient consultation data using a medical knowledge graph, a clinical semantic context for the current consultation is constructed. Based on the clinical semantic context, a question list is generated through a hybrid evaluation model, wherein the hybrid evaluation model integrates a rule engine and a machine learning model. The question list includes a list of mandatory questions generated by the rule engine based on the medical knowledge graph for the current clinical semantic context, and a list of implicit questions inferred by the machine learning model based on the medical knowledge graph and the clinical semantic context. Determine whether any items from the question list are missing from the doctor-patient consultation data; Based on the judgment results, quality assessment feedback information for the consultation is generated and output, including quantitative quality scores and / or consultation gap-filling suggestions.

[0010] Secondly, the present invention also provides a clinical consultation quality assessment system, characterized in that the system comprises: Input device, used to acquire real-time doctor-patient consultation data; An analysis device is used to analyze the doctor-patient consultation data in real time based on a medical knowledge graph and construct the clinical semantic context of the current consultation. A generation device is used to generate a question list based on the clinical semantic context using a hybrid evaluation model, wherein the hybrid evaluation model integrates a rule engine and a machine learning model, and the question list includes a list of mandatory questions generated by the rule engine based on the medical knowledge graph for the current clinical semantic context, and a list of implicit questions inferred by the machine learning model based on the medical knowledge graph and the clinical semantic context. A judgment device is used to determine whether any content of the question list is missing from the doctor-patient consultation data; An output device is used to generate and output quality assessment feedback information for the consultation based on the judgment result, the quality assessment feedback information including a quantitative quality score and / or consultation supplementation suggestions.

[0011] The clinical consultation quality assessment method and system provided by this invention: First, by using speech recognition and speaker separation technology, voice data during the doctor-patient consultation process can be acquired in real time. Combined with medical knowledge graphs, semantic layer analysis is performed to construct a clinical semantic context, enabling dynamic quality assessment and immediate feedback of the consultation process. This overcomes the shortcomings of existing technologies that can only perform post-event analysis and cannot promptly detect missed problems.

[0012] Second, this invention generates a list of mandatory questions and a list of implicit questions based on a hybrid assessment model. It can automatically identify key information that doctors may have missed during consultations and provide prompts for supplementary questions, making medical records more consistent with clinical logical structures and promoting the complete presentation of consultation content within the SOAP framework.

[0013] Third, this invention can automatically generate electronic medical record reports (including SOAP cases) that conform to standardized formats and output quality feedback in a visual manner during the consultation process, which greatly reduces the workload of doctors in medical record entry and organization, and improves consultation efficiency and doctor-patient communication quality.

[0014] Fourth, this invention utilizes the semantic mapping between medical entities and clinical logical relationships in a medical knowledge graph, combined with the collaborative work of a rule engine and a machine learning model, to ensure the interpretability of the evaluation results and to possess dynamic reasoning and adaptive capabilities, thus flexibly responding to complex and ever-changing consultation scenarios.

[0015] Fifth, this invention not only processes voice and text data, but also combines multimodal data such as medical images. It extracts visual medical features through image recognition and integrates them into the clinical semantic context, thereby achieving a more comprehensive understanding and assessment of the patient's condition.

[0016] Sixth, this invention enables the hybrid evaluation model to continuously update parameters and optimize performance during actual use through supervised learning based on expert-annotated samples and reinforcement learning mechanisms based on user feedback, forming a self-evolving closed loop of "evaluation-feedback-optimization".

[0017] Seventh, it supports the construction of customized medical knowledge graphs for different clinical specialties, medical theoretical systems and regional diagnosis and treatment practices, which can flexibly adapt to the quality assessment needs of different medical institutions and consultation scenarios. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of the clinical consultation quality assessment method provided in the embodiments of the present invention; Figure 2 This is a block diagram of the clinical consultation quality assessment system provided in the embodiments of the present invention. Detailed Implementation

[0020] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Invention Overview As mentioned above, this invention provides a clinical consultation quality assessment method and system. By integrating medical knowledge graphs, rule engines, and machine learning models, it performs real-time semantic analysis and dynamic quality assessment of the doctor-patient consultation process, thereby achieving complete collection of consultation information, intelligent supplementation, and interpretable feedback.

[0022] Exemplary methods Figure 1 This is a flowchart of a clinical consultation quality assessment method provided in an embodiment of the present invention. This embodiment includes the following steps: S101: Obtain real-time doctor-patient consultation data.

[0023] Automatic Speech Recognition (ASR) converts the audio stream of doctor-patient (or physician-patient) dialogue into a text transcript in real time. The ASR can employ models such as OpenAI Whisper, Google Speech-to-Text, or others with medical semantic recognition capabilities, and can be trained on medical-specific corpora to improve the accuracy of recognizing medical terminology, drug names, and regional accents. This process achieves high-precision transcription while preserving medical semantic features, ensuring the accuracy of subsequent semantic analysis.

[0024] Secondly, speaker diarization technology is used to differentiate roles and segment sentences in the text dialogue records. This process identifies the speeches of the doctor and patient and breaks down sentences semantically, constructing a multi-turn question-and-answer structure with temporal sequence and logical connection. This step not only distinguishes the speaking roles but also makes comprehensive judgments based on acoustic features (such as tone and speech rate) and linguistic features, ensuring the accuracy of role recognition and dialogue segmentation.

[0025] Compared with traditional speech transcription tools, this invention can simultaneously achieve speaker role recognition and medical semantics preservation, thereby ensuring the contextual consistency of input data and the integrity of clinical expression.

[0026] In addition, doctors or patients can upload or collect medical images related to the consultation through terminal devices (such as medical assistants, mobile applications, etc.), including but not limited to photos of skin lesions, tongue images, radiographic images, or other visual examination results. These image data are received synchronously and correlated temporally and semantically with the text dialogue records to achieve multimodal information fusion, providing a more comprehensive input foundation for subsequent understanding of the patient's condition and analysis of consultation quality.

[0027] S102: Analyze the doctor-patient consultation data in real time to construct the clinical semantic context of the current consultation.

[0028] Preferably, the doctor-patient consultation data is analyzed in real time based on a medical knowledge graph to construct the clinical semantic context of the current consultation.

[0029] In a preferred embodiment of the present invention, the construction of the clinical semantic context is based on a medical knowledge graph, thereby enabling higher recognition accuracy and a more complete assessment of the consultation process by utilizing the semantic relationships between medical entities.

[0030] However, the core idea of ​​this invention lies in the semantic parsing and quality assessment generation mechanism of doctor-patient consultation data, and it is not limited to relying on the implementation of medical knowledge graphs.

[0031] In other embodiments, the clinical semantic context can also be generated through rule templates, statistical language models, deep semantic models, or other medical knowledge base construction methods; although such alternative methods are slightly lower in terms of semantic coverage and reasoning accuracy, they can still achieve the same technical objectives and basic technical effects as the present invention.

[0032] Therefore, the scope of protection of this invention is not limited to the implementation scheme based on medical knowledge graphs.

[0033] The medical knowledge graph is a customized knowledge graph for a specific medical field.

[0034] The specific medical fields mentioned are divided based on different medical theoretical systems (such as traditional Chinese medicine and Western medicine), clinical specialties (such as internal medicine, dermatology and psychology), or regional medical practices (such as clinical habits and terminology in Hong Kong and Taiwan) to adapt to diverse clinical scenarios.

[0035] The customized knowledge graph includes medical entities specific to the specific medical field (such as symptoms, diseases, medications, examination items, etc.) and the clinical logical relationships between these medical entities, such as the association between symptoms and medical history, the correspondence between causes and diagnoses, and the logical connection between examination results and treatment plans.

[0036] For example, in the field of Traditional Chinese Medicine (TCM), the medical knowledge graph can be constructed based on the TCM theoretical system, covering core knowledge elements such as meridian theory, visceral differentiation, and the theory of qi, blood, and body fluids. Medical entities in this graph can include "symptoms" (such as cough, bitter taste in the mouth, and yellow tongue coating), "syndromes" (such as excessive liver fire, qi deficiency, and blood stasis), "prescriptions" (such as Longdan Xiegan Tang and Sijunzi Tang), and "treatment methods" (such as clearing heat and purging fire, and invigorating qi and promoting blood circulation). The clinical logical relationships between entities can be represented as a "symptom-syndrome-treatment-prescription" association path, thereby supporting the system in automatically inferring potential diagnostic directions or treatment strategies based on TCM semantic understanding when analyzing doctor-patient consultations, achieving semantic-level auxiliary evaluation of the quality of TCM consultations.

[0037] For example, in the fields of psychology or psychiatry, knowledge graphs can include emotional states (such as anxiety, depression, and irritability), cognitive symptoms (such as inattention and slowed thinking), behavioral manifestations (such as insomnia and social avoidance), psychological scale indicators (such as PHQ-9 and GAD-7 scores), and corresponding diagnostic entities (such as depressive disorders, anxiety disorders, and obsessive-compulsive disorder). Their clinical logical relationships can be reflected in a multi-layered associative path of "emotional / behavioral symptoms—scale results—diagnosis—treatment plan," thereby supporting the system in automatically identifying potential psychological states, inferring risk levels, and assisting in assessing the comprehensiveness and clinical logical rationality of the doctor's consultation when analyzing psychological consultation dialogues.

[0038] The medical entities include local terminology corresponding to the regional medical practices to support accurate understanding of local languages ​​or professional terms. For example, doctors or patients in different regions may use different colloquialisms or dialects to describe the same symptoms; incorporating these expressions into the knowledge graph can ensure the accuracy of entity recognition and semantic analysis.

[0039] By constructing customized knowledge graphs for specific medical fields, the semantic understanding capabilities and diagnostic adaptability of medical knowledge graphs in different clinical scenarios can be significantly improved. On the one hand, by integrating multiple medical theoretical systems and regional linguistic features, medical knowledge graphs can accurately identify and interpret the diverse expressions used by different doctors and patients during consultations, thereby reducing information omissions caused by semantic ambiguity or dialect differences. On the other hand, based on the knowledge organization structure of medical entities and their clinical logical relationships, when analyzing consultation content, it can not only understand surface-level linguistic information but also perform clinical reasoning and semantic association inference. Thus, this customized knowledge graph, while ensuring the accuracy of semantic understanding during consultations, also improves the system's scalability and cross-domain applicability, taking into account the intelligent integration needs of traditional Chinese medicine, Western medicine, and medical practices in different regions.

[0040] Specifically, firstly, entity recognition is performed on the doctor-patient consultation data to extract medical entities. More specifically, a Named Entity Recognition (NER) model can be used, based on pre-trained models in the medical field such as BioClinicalBERT, MedBERT, or RoBERTa-Med, to perform semantic-level recognition and annotation on the input doctor-patient consultation data. Through this model, multiple types of medical entities can be automatically extracted from the natural language dialogue between doctors and patients, including but not limited to symptoms (such as cough, chest tightness, dizziness), diseases (such as hypertension, diabetes, bronchitis), time information (such as duration of illness, onset time), medical history (such as past medical history, surgical history), medications (such as taking aspirin, antibiotics), examination items (such as blood routine, electrocardiogram, imaging examinations), and risk factors (such as smoking, drinking, family history, etc.).

[0041] During the extraction process, the model can combine contextual semantics for discrimination, distinguishing between subjective information (patient's self-reported symptoms and feelings) and objective information (doctor's observations and examination results), as well as current symptoms and past medical history, thereby ensuring the accuracy and clinical relevance of medical entity recognition. For semantically ambiguous cases such as dialects, vague expressions, or elliptical sentences, semantic completion and disambiguation can be achieved through contextual attention mechanisms and word vector similarity calculations, improving robustness to natural dialogue. Subjective information includes the chief complaint and secondary complaints; the chief complaint refers to the primary symptom, and the secondary complaint refers to the minor symptom, both derived from the patient's self-report.

[0042] Subsequently, based on the extracted medical entities, the clinical logical relationships between pre-stored medical entities in the medical knowledge graph are queried to obtain the association relationships between the medical entities, thereby establishing semantic connections between medical entities. Specifically, causal, concurrent, or subordinate relationships related to the medical entity can be retrieved in the medical knowledge graph, such as the possible association between "fever" and "infectious disease," the logical connection between "dizziness" and "anemia or low blood pressure," and the risk mapping between "medication records" and "adverse drug reactions."

[0043] On the other hand, based on the pre-stored medical entities and clinical logical relationships in the medical knowledge graph, the intent recognition model analyzes the statements in the doctor-patient consultation data to obtain the clinical intent corresponding to each statement. The intent recognition model can determine whether a doctor or patient is stating symptoms, describing medical history, providing life or psychological information, or proposing treatment suggestions or intervention plans in a certain statement. For example, if a patient says, "I've had insomnia for the past two weeks," the intent recognition model can determine that the statement expresses subjective information about symptoms; if a doctor asks, "Do you have palpitations or difficulty breathing?" it is identified as an inquiry intent to obtain objective information.

[0044] After completing intent identification, each statement is categorized into the SOAP framework's categories of Subjective (S, patient's self-reported symptoms and feelings), Objective (O, doctor's observations and examination results), Assessment (A, analysis and diagnosis of symptoms and examination information), and Plan (P, proposed treatment or intervention plan) based on its clinical intent. This yields the SOAP classification results, thereby achieving a structured representation of the consultation content. For example, the patient's self-reported "insomnia" statement is categorized into the Subjective category, the doctor's recorded blood pressure measurement results into the Objective category, psychological assessment conclusions or traditional Chinese medicine diagnosis and treatment conclusions into the Assessment category, and the recommended medication or psychological intervention plan into the Plan category.

[0045] Based on the relationships between the medical entities and / or the SOAP classification results, the clinical semantic context is constructed to provide a standardized and computable semantic foundation for subsequent question list generation and consultation quality assessment.

[0046] Specifically, in constructing the clinical semantic context, the system first identifies the intent of each doctor-patient consultation statement and categorizes it into the subjective (S), objective (O), assessment (A), or planning (P) categories of the SOAP framework based on clinical intent, thereby providing a sentence-level structured representation of the consultation content. Subsequently, the medical entities identified in each statement (such as symptoms, medical history, diseases, medications, examination items, and risk factors) are embedded in a medical knowledge graph, and the clinical semantic context is established through the pre-existing clinical logical relationships between medical entities.

[0047] For example, in a psychological consultation, the patient describes symptoms such as "anxiety," "insomnia," and "loss of appetite," while the doctor observes "social withdrawal" and "loss of interest." The system first identifies the intent behind each doctor-patient dialogue statement, categorizing the patient's self-reported symptoms into subjective information (S), the doctor's observed behaviors into objective information (O), the preliminary judgment into assessment information (A), and future intervention measures into planning information (P), thus achieving a statement-level SOAP structured classification representation. Subsequently, the medical entities identified in each statement (such as symptoms and behavioral characteristics like "anxiety," "insomnia," "loss of appetite," "social withdrawal," and "loss of interest") are embedded into a psychological knowledge graph, establishing connections through pre-existing clinical logical relationships between entities. For example, the subjective information S node "anxiety" is connected to the S node "insomnia" through the "may cause" relationship, and simultaneously connected to the A node "mild depressive tendency" through the "risk warning" relationship; the objective information O nodes "social withdrawal" and "loss of interest" are connected to the same A node through the "support / reinforcement" relationship. In this process, each medical entity retains its statement-level SOAP classification results and embeds its relationships with other medical entities, ultimately forming a multi-level, structured clinical semantic context, providing a standardized and computable semantic foundation for subsequent question list generation and consultation quality assessment.

[0048] Furthermore, the medical images (such as photos of skin lesions, tongue images, traditional Chinese medicine pulse imaging, or psychometric charts) are analyzed using image recognition models to obtain visual medical features; then, based on a medical knowledge graph, the visual medical features are integrated into the clinical semantic context to achieve the fusion of multimodal information.

[0049] For example, in a dermatology consultation, a patient uploaded photos of skin lesions, and the doctor described the morphology and location of the lesions during the consultation. First, an image recognition model was used to analyze the medical image, extracting the visual medical features of the lesions, such as the area, color, shape, and distribution of the erythema. Then, the system associated these visual medical features with medical entities in a medical knowledge graph. For example, "erythema, ring-shaped distribution" was mapped to the relevant symptom node for "psoriasis vulgaris," and connections were established with entities such as the patient's subjective information, medical history, and examination results through clinical logical relationships in the medical knowledge graph. Finally, the image features were integrated into the clinical semantic context, forming a multimodal information fusion with the text consultation content.

[0050] S103: Based on the clinical semantic context, a question list is generated through a hybrid assessment model, wherein the hybrid assessment model integrates a rule engine and a machine learning model, and the question list includes a list of mandatory questions generated by the rule engine for the current clinical semantic context and a list of implicit questions inferred by the machine learning model based on the clinical semantic context.

[0051] Preferably, the question list includes a list of mandatory questions generated by the rule engine based on the medical knowledge graph for the current clinical semantic context, and a list of implicit questions inferred by the machine learning model based on the medical knowledge graph and the clinical semantic context.

[0052] The rules engine ensures the standardization of the consultation process and essential questions, while the machine learning model enhances the ability to reason about individual patient situations.

[0053] A rule engine is a module that statically matches the input current clinical semantic context based on predefined clinical rules, guidelines, or knowledge graph logic. It ensures that mandatory questions and standard procedures are executed during the consultation process and can automatically generate a fixed or standardized list of questions under specific conditions. Specifically, the rule engine statically verifies the clinical semantic context against the clinical logical relationships in the medical knowledge graph to automatically generate the list of mandatory questions. Mandatory questions are information that must be obtained according to clinical treatment standards. For example, in psychological counseling, if a patient exhibits anxiety and insomnia, the rule engine will generate questions such as "Do you have a history of hypertension?" and "Do you experience palpitations?" These questions are deterministic, highly interpretable, but relatively fixed, and must be asked in any similar consultation scenario.

[0054] Machine learning models, trained on extensive historical consultation data, can dynamically infer potential implicit questions or assessment conclusions based on clinical semantic context and clinical logical relationships within a medical knowledge graph. Their role is to identify questions not explicitly stated in the rule base but potentially necessary based on patient characteristics. Through this machine learning model, based on clinical logical relationships within the medical knowledge graph and the clinical semantic context, a list of implicit questions to be asked is dynamically inferred. Implicit questions typically possess personalized characteristics, depending on the patient's current symptoms and the clinical semantic context. For example, in the same psychological consultation, if a patient reports anxiety, insomnia, and decreased appetite, and the doctor observes social withdrawal and decreased interest, the machine learning model might infer questions such as "How has your recent life stress been?" or "Have you experienced persistent negative emotions?" While these questions are not standard mandatory questions, combined with semantic context, they are of significant reference value in assessing the patient's psychological state and developing intervention plans.

[0055] The hybrid assessment model of this invention generates a question list that enables comprehensive coverage and personalized supplementation of consultation information. On the one hand, the mandatory questions generated by the rule engine ensure that no information required by clinical standards is omitted, improving the standardization and security of the consultation. On the other hand, the implicit questions dynamically inferred by the machine learning model can provide personalized prompts based on the patient's specific symptoms and semantic context, helping doctors to promptly identify potential key issues or high-risk information, thereby improving the completeness of the consultation and the scientific nature of treatment decisions. Furthermore, this method achieves the structuring and computability of the question list, providing a reliable data foundation for consultation gap filling, real-time quality assessment, and intelligent auxiliary decision-making, reducing human error and improving clinical efficiency and medical quality.

[0056] S104: Determine whether any content from the question list is missing from the doctor-patient consultation data.

[0057] The questions in the question list are semantically matched with the doctor-patient consultation data. Semantic matching can be achieved based on natural language processing techniques, such as using embedded vector representation, sentence similarity calculation, or pre-trained language models to compare the questions with the doctor-patient consultation data, thereby determining whether the clinical information referred to by the question has been covered in the consultation process.

[0058] If no semantically matching item is found for a question in the question list within the text content, the question is considered omitted. For example, in a psychological counseling session, the question list might include "How has your recent life stress been?" If no description related to life stress appears in the doctor-patient consultation data, the question is determined not to have been asked and is thus marked as omitted. This judgment step identifies missing information in the consultation process, providing a basis for generating supplementary questions and quality assessments.

[0059] S105: Based on the judgment result, generate and output quality assessment feedback information for the consultation, the quality assessment feedback information including quantitative quality score and / or consultation supplementation suggestions.

[0060] For example, the dashboard displays a "Consultation Quality Score: 4.2 (out of 5)," providing a clear picture of the overall consultation quality. It also offers prompts for further questions, such as "Suggested follow-up questions: Is there night sweats?" or "Family history not recorded?" Furthermore, it issues high-risk warnings when potentially critical symptoms are identified. This allows doctors to monitor the consultation quality in real-time and adjust their questioning strategies or record content accordingly, thereby improving the completeness of the consultation and the safety of treatment.

[0061] The quantitative quality score is used to objectively reflect the completeness, logic, and clinical rationality of the consultation. The consultation gap-filling suggestions are used to prompt doctors to ask follow-up questions or supplement records on key issues that were not covered or may have been missed.

[0062] For example, in a clinical consultation for suspected cardiovascular symptoms, semantic analysis and medical knowledge graph reasoning based on patient-doctor consultation data generate quality assessment feedback. The quantitative quality score includes sub-scores and a comprehensive score for subjective information (S), objective information (O), assessment information (A), and planning information (P), with the comprehensive score being 3.5 / 5 (3.5 out of 5 in this case). This score reflects the overall quality level of the current consultation and displays the scoring basis and improvement suggestions in a visual dashboard categorized by SOAP. Subjective Information (S) score: 4 / 5, indicating that the doctor has collected relatively complete information on the core symptoms, including the nature, duration, and triggers of the pain, and has also obtained information on major risk factors (such as hypertension and smoking history). However, it is noted that questions about accompanying symptoms (such as difficulty breathing and sweating) and family history or allergies were omitted, and it is recommended to supplement these in subsequent consultations.

[0063] Objective Information (O) score: 2.5 / 5, indicating that although the doctor ordered routine laboratory tests, the basic physical examination (such as heart rate and lung auscultation) was not fully recorded, resulting in incomplete objective data. The dashboard prompts "It is recommended to supplement the physical examination record" and highlights it in yellow.

[0064] Assessment Information (A) Score: 2 / 5. The analysis found that the doctor had not yet formed a clear preliminary diagnosis, such as angina or gastroesophageal reflux. The clinical reasoning chain was not fully reflected, thus indicating "lack of clear clinical assessment basis".

[0065] Plan Information (P) score: 4 / 5. The doctor performed well in terms of follow-up treatment plan, and was able to clearly list the steps of "examination → adjustment of plan → follow-up reminder". However, it was also noted that "the significance of the examination was not explained to the patient, and health education was insufficient". It is recommended to strengthen patient communication in future consultations.

[0066] This step can synchronously generate and display real-time quality assessment feedback information for the consultation during the doctor-patient consultation process, based on the judgment results, in the form of a visual dashboard. This dashboard integrates multiple functions, including a scoring mechanism, a prompting mechanism, and high-risk alerts. The scoring mechanism displays the overall quality of the consultation, the prompting mechanism alerts to uncovered or potentially omitted issues, and the high-risk alert warns of potential critical symptoms, helping doctors to promptly identify information gaps and conduct supplementary questioning.

[0067] This step can also be performed after the doctor-patient consultation process is completed, based on the judgment results, to generate and output complete quality assessment feedback information for the consultation, such as an automatically generated electronic medical record report, which includes SOAP, DAP, GIRP, PIE, POMR and / or SOAPIER medical records.

[0068] Among them, SOAP (Subjective-Objective-Assessment-Plan) medical records are an internationally recognized structured medical record format that records the complete diagnosis and treatment process through four parts: subjective information, objective information, assessment and planning.

[0069] DAP (Data-Assessment-Plan) medical records are a commonly used record-keeping model in the fields of psychological counseling and nursing, which simplifies the recording by dividing the data into three parts: data, assessment, and plan.

[0070] GIRP (Goal-Intervention-Response-Plan) records are used in behavioral health and rehabilitation settings, emphasizing treatment goals, interventions, patient responses, and follow-up plans.

[0071] The PIE (Problem-Intervention-Evaluation) case study uses problems as its core structure, highlighting the correspondence between problems, interventions, and effectiveness evaluations.

[0072] POMR (Problem-Oriented Medical Record) is a medical record system based on a patient's problem list, combined with a comprehensive recording method that includes medical history, examinations, assessments, and plans.

[0073] SOAPIER Medical Records (Subjective-Objective-Assessment-Plan-Intervention Evaluation-Revision is an extended version based on SOAP, often used in nursing records, to further include intervention, assessment, and revision content.

[0074] By being compatible with multiple medical record formats, the quality assessment feedback mechanism of this invention can adaptively output structured electronic medical records that meet the requirements according to the recording standards of different medical institutions or medical specialties, thereby improving the system's versatility and clinical usability.

[0075] For example, a SOAP medical record might state: A 55-year-old male patient presents with chest pain lasting one week, occurring during exercise and radiating to the left shoulder. He denies any accompanying sweating symptoms (subjective information). Physical examination reveals a blood pressure of 150 / 90 mmHg, a body mass index (BMI) of 28, and an electrocardiogram has been scheduled (objective information). Based on the above subjective and objective information, a preliminary assessment suggests stable angina and gastroesophageal reflux disease (GERD). The patient has high cardiovascular risk factors, including smoking and hypertension (assessment information). The treatment plan includes lipid testing, an exercise stress test, and recommendations for smoking cessation. A follow-up appointment is scheduled for one week later (planning information).

[0076] The SOAP medical records are automatically structured and output by an SOAP medical record generator into an electronic medical record (EHR) system compatible format (such as HL7 / FHIR standard) for easy archiving. The dashboard can use graphics, colors, or lists to indicate the content covered, missing information, and potential hidden problems during the consultation, intuitively reflecting the quality of the consultation, facilitating doctors to adjust their questioning strategies in a timely manner, and supporting subsequent medical record organization and clinical review.

[0077] As an optional embodiment, this embodiment takes a traditional Chinese medicine consultation scenario as an example to illustrate the application method and technical effect of the intelligent consultation quality assessment and consultation assistance of the present invention in the actual doctor-patient communication process.

[0078] First, speech recognition and text conversion are performed on the consultation process between the TCM doctor and the patient to obtain doctor-patient consultation data. Below is a sample of the processed dialogue: Doctor: What time do you usually go to bed? Patient: It's probably between 12 and 1 o'clock now.

[0079] Doctor: So you only went to sleep at 1 a.m.? But you should actually be asleep before 11 p.m. Do you have trouble falling asleep? Patient: It will take about fifteen minutes.

[0080] Doctor: Fifteen minutes isn't too long. Did you wake up during the night? Did you get up to urinate at night? Patient: No, I don't urinate at night, I sleep through the night.

[0081] Doctor: Do you have more trouble falling asleep in the afternoon? You usually wake up at 7:30, right? Here, put your hands here. (During examination) Do you stay up so late because you're used to it, or is it because of work that you can't sleep? Patient: I don't know... I probably just wanted to play on my phone.

[0082] Doctor: Oh, so if you want healthy hair, here are a few things to keep in mind: First, go to bed early; second, maintain a balanced diet; and third, have a healthy constitution. You're prone to bloating and stomach pain, which is actually a problem with your spleen and stomach, meaning you're not absorbing nutrients well. So, first, avoid cold drinks and fatty, fried foods, like fried noodles or rice. Once your spleen and stomach function improves, you'll be able to absorb the nutrients you eat. It's actually quite simple: each meal should include a staple food like rice or noodles, plus protein such as chicken, fish, pork, or beef, and some vegetables. Cooking methods should be light; avoid thick sauces and frying. This will help improve your condition. Your pulse is weak; it doesn't feel strong when pressed. This might reflect some liver and kidney deficiency, which is causing hair loss. Come on, let me look at the back… this side too, is the back also losing hair? Yes, it's mainly a problem on the top of your head; the hair on the top is indeed quite thin. Would you like acupuncture treatment? Patient: Let's do it together next time.

[0083] Doctor: I'll prescribe the medication for you now. When will you come back for your follow-up appointment? Patient: Anything is fine... next week? Is this week okay? Doctor: You can come in this week on Thursday evening or Friday morning. Which time would you like to schedule? Patient: Thursday evening.

[0084] Doctor: Okay, I'll prescribe three herbs for you. Would you like to decoct the herbs yourself or make them into powder? Patient: Are there big differences between these two? Doctor: Decoction is more effective, but if you want something more convenient, you can also choose powder, depending on your needs.

[0085] Patient: Then let's use the powdered medicine.

[0086] Doctor: Okay, I'll prescribe three doses of the powder for you. Shall I write the diagnosis for you? Patient: Okay.

[0087] Doctor: See you next time. Remember to pay attention to your lifestyle habits and try to follow the advice I just gave: eat breakfast early, avoid cold drinks, and go to bed early at night.

[0088] It automatically completes speech transcription, semantic understanding, and entity recognition, labeling symptom information (medical entities) such as "delayed sleep onset", "normal nocturia", "gastric distension and pain", and "hair loss" as subjective information (S), and observation content such as "weak pulse" and "sparse hair on the top of the head" as objective information (O), and establishes the association relationship of medical entities in the medical knowledge graph.

[0089] When a doctor is detected asking a patient about hair loss, suggestions for further questions are generated in real time based on a medical knowledge graph and semantic context. These suggestions appear as prompts on the side of the screen, as follows: When the current topic is hair loss, the system will automatically suggest the following: When did it start? Has it suddenly worsened recently? Is there a family history of hair loss? Do you experience itchy scalp, dandruff, or pain? Have you recently lost weight, experienced stress, had a fever, or given birth? Are you taking any medications (birth control pills, antidepressants, weight loss pills, etc.)? Are your menstrual cycles regular? When the topic "stomach bloating and pain" is detected, the following suggestions are provided: How long has the stomach pain lasted? When did the pain begin? Have you experienced acid reflux, vomiting, black stools, or weight loss? Have you been taking painkillers or caffeine for a long time? General health and life history: past medical history (thyroid, anemia), alcohol consumption, smoking, exercise habits, stress and emotional state. Through the aforementioned real-time prompting mechanism, dynamic suggestions for supplementary questions are provided during the physician consultation process, helping physicians identify missing information during the consultation and improving the completeness of the consultation and the coverage of treatment.

[0090] After the consultation, a structured electronic medical record report is automatically generated based on the clinical semantic context, as shown in the example below: The tongue is of moderate size, pale red in color, and has a thin, white, and slightly dry coating. The pulse is weak. Nocturia is normal. The patient has a fair appetite, but prefers cold and greasy foods, and experiences abdominal distension after meals. Menstrual history: Normal menstrual cycle, no dysmenorrhea. Specialized Internal Medicine Diagnosis (Disease Name): Insomnia, Spleen and Stomach Qi Deficiency Differential diagnosis: Liver and kidney yin deficiency with spleen and stomach damp-heat The treatment principle is to nourish the liver and kidneys, strengthen the spleen, and resolve dampness. Acupuncture (System-generated suggestion, for reference only) Treatment principle: Nourish the heart and calm the mind, strengthen the spleen and benefit the stomach. Acupoints: Taichong (LR3), Sanyinjiao (SP6), Zusanli (ST36), Zhongwan (CV12). Explanation: Taichong (LR3) is the Yuan-Source point of the Liver Meridian, which can soften the liver and subdue yang; Sanyinjiao (SP6) nourishes kidney yin, nourishes blood and calms the mind; Zusanli (ST36) strengthens the spleen and benefits qi; Zhongwan (CV12) promotes the flow of qi in the middle jiao, resolves dampness and harmonizes the stomach. The patient's sleep was affected by mobile phone dependence and irregular lifestyle. Doctor's advice: Avoid cold drinks and fatty or fried foods; go to bed early (before midnight); eat a balanced diet; and pay attention to hair care. Traditional Chinese Medicine Prescription (System-generated suggestions, confirmed by a physician after review): Prescription Name: Zhi Bai Di Huang Wan (Modified Zhi Bai Di Huang Wan) Ingredients: Rehmannia glutinosa (processed), Cornus officinalis, Dioscorea opposita, Alisma plantago-aquatica, Paeonia suffruticosa, Poria cocos, Anemarrhena asphodeloides, Phellodendron chinense, Citrus reticulata, Pinellia ternata. Main Functions: Rehmannia glutinosa nourishes kidney yin; Anemarrhena asphodeloides and Phellodendron chinense clear heat and reduce fire. Contraindications: Suitable for those with liver and kidney yin deficiency combined with spleen and stomach damp-heat; contraindicated for those with spleen and stomach deficiency-cold and loose stools. Precautions: Avoid spicy and greasy foods; monitor sleep improvement and stool consistency.

[0091] As can be seen from this embodiment, the present invention realizes a closed-loop application of speech recognition, semantic understanding, knowledge graph reasoning and real-time auxiliary prompts in the consultation scenario. This not only improves the integrity and standardization of consultation, but also automatically generates structured electronic medical record documents, realizing an integrated process of intelligent consultation and medical record generation.

[0092] As another optional embodiment, this embodiment takes a traditional Chinese medicine consultation scenario as an example to illustrate the specific application of the present invention in consultation quality scoring and feedback generation.

[0093] First, speech recognition and text conversion are performed on the consultation process between the TCM doctor and the patient to obtain doctor-patient consultation data. Below is a sample of the processed dialogue: Doctor: How have you been lately? Patient: Lately, I've been feeling a bit bloated, in my lower abdomen. It's also... a bit sore and achy, like a dull ache.

[0094] Doctor: Yes, yes.

[0095] Patient: Also, I feel like my period is coming earlier each month. Last month it came on September 25th.

[0096] Doctor: Yes, last time it was also four or five days apart? Well, actually your cycle has always been about 28 to 30 days.

[0097] Patient: Hmm.

[0098] Doctor: Yes, so you see, 28 to 35 days is actually within the normal range. So, since it's been 25 days since your last visit, theoretically, it shouldn't be this early. If it's too early, it might indicate that your body's regulatory function is a little weak. It would be best to come back in two or three more days.

[0099] Patient: If I come on the 23rd of this month, will that be okay? Doctor: The 23rd, yes, that's right. Since today is Sunday, the 19th, it would be best if it came four or five days later. But sometimes you can't exactly control when it comes; your body will adjust itself based on its condition, emotions, and diet. So don't be too rigid about when you should come, otherwise you'll put a lot of pressure on yourself.

[0100] Patient: Hmm.

[0101] Doctor: Yes, it's better to let nature take its course. The most important thing is not to make things too dramatic, like it coming a week or two early, or it coming continuously and uncontrollably. Hmm, are you sleeping well now? Patient: It's pretty good, because I'm not working in Hong Kong this month, so maybe... Doctor: Where did you go to work? Patient: I went to Macau.

[0102] Doctor: Oh.

[0103] Patient: Yes, OK. They're doing some telemarketing work now, things like beauty treatments. We'll help them out.

[0104] Doctor: Yes, it seems you're not too tired. You probably have something to do, and I think you're in a better mood.

[0105] Patient: OK, of course, because I don't have to think about so many things.

[0106] Doctor: Yes, it's something to hold onto.

[0107] Patient: I basically don't have much time... I just work the days I have to. I think I've been sleeping okay for the past two weeks, I'm doing pretty well, yeah, overall I seem to be better.

[0108] Doctor: Good, OK, then keep it up.

[0109] Patient: But I'm afraid it will get worse in a day or two, so I'm... well, so I'll take care of it first. I'll also come see you to get some medicine. Also, I don't know if it's because of the changing seasons, but I feel my throat is dry when I wake up in the morning, right? It's dry and sore, not to the point of being very painful, but it feels like there's some sticky discharge. Drinking some warm water makes it feel better. It will gradually disappear, but it's about the same every morning.

[0110] Doctor: Yes, I understand. It's possible, because the weather is already in autumn, and it will get cooler and windy soon.

[0111] Patient: Yes, I was so irritated by the typhoon that I felt like I was going crazy.

[0112] Doctor: Yes, yes, it's so strange, the weather is really unusual.

[0113] Patient: It's the end of October.

[0114] Doctor: Yes, everyone's body may not be able to adapt very well.

[0115] Patient: Okay, actually, the reason I'm picking up some medicine today is because I'm going on a trip next week, and I'd like to come back on the morning of my trip.

[0116] Doctor: Oh, okay, okay.

[0117] Patient: Because I want to get some medicine so I can take it with me when I travel.

[0118] Doctor: Okay, okay, no problem, no problem. Do you want acupuncture today? Patient: Not today, because I'm staying at... Doctor: You're not doing moxibustion either? Patient: Yes, because some of the tastes are inconvenient, I... Doctor: Oh, so do you do smokeless moxibustion? Patient: Uh, no need, the kind with a strong smell, and no smoke either... Yes, because I might have three meetings today.

[0119] Doctor: Oh, okay, no problem. My goal was to see you and get you checked out. Okay, no problem. Can I prescribe your medication now? Patient: Yes.

[0120] Doctor: After you finish this medicine, come back for a follow-up visit in five days.

[0121] Patient: What kind of medicine is this? Do I also take it during my period? Doctor: Yes, that's right, you can take this medicine during your period.

[0122] Patient: Oh.

[0123] Doctor: Yes, that's right.

[0124] Patient: Well, I don't think it's bad.

[0125] Doctor: We have a bit of qi deficiency.

[0126] Patient: Oh.

[0127] Doctor: Yes, you have Qi deficiency. It's probably because you've been busy with work lately, which has caused you to become weak.

[0128] Patient: Okay, so the medicine you prescribed for me at the beginning of the month, I really... well, I felt like I wasn't getting enough energy, and my breathing was all over the place, but it helped a bit. It's just that I didn't have time to come over today.

[0129] Doctor: You'll feel better after taking it.

[0130] Patient: Yes.

[0131] Doctor: Because the medicine was tailor-made for you. You've been with me for so long, I know your constitution very well.

[0132] Patient: Yes, so I also take these medications... Doctor: Okay, okay. Do you still want to go for massages and the like? Patient: Oh, no, I don't have time.

[0133] Doctor: No time? That's okay, as long as you're okay. Also, it (referring to the previous treatment) helped you. It's just that you might not be able to come to us, but we can do it for you later. When you come back later, okay? Patient: OK, okay.

[0134] Objective information collected showed that the tongue was wide with teeth marks on the sides, and its movement was acceptable without cracks; the tongue body was pale red with a slightly dark color and the surface was moist; the coating was thin and white, and slightly thicker at the root.

[0135] After semantic understanding and clinical entity recognition processing, medical entities (such as "abdominal distension", "dull pain", "early menstruation", "dry and sore throat") are automatically extracted and clinically semantically labeled and classified for subsequent quality analysis and structured medical record generation.

[0136] Subsequently, the physician's consultation process is quantitatively scored. Combining medical knowledge graphs and scoring models, a quantitative quality scoring table for consultation structure and completeness is automatically generated, as shown in the example below:

[0137] Overall quantitative results Average score of consultation coverage (Total score 23 / 40) 2.9 / 5.0 Rating – Slightly below average (Grade C) A quantitative quality scoring mechanism generates a comprehensive consultation score (e.g., 2.9 / 5.0) immediately after the consultation, and provides a grade rating (e.g., Grade C) based on the results, helping physicians intuitively understand the quality level of the consultation. Simultaneously, targeted professional interpretations and improvement suggestions can be provided based on the sub-scores. For example, if the consultation structure demonstrates completeness in daily health maintenance and traditional Chinese medicine conditioning, but does not meet clinical quality control standards, the consultation should cover at least three systems (gynecology, urology, and endocrinology) for gynecological complaints (abdominal pain and menstrual irregularities). The physician-patient interaction is natural, establishing a good doctor-patient relationship.

[0138] After the consultation, a structured electronic medical record report is automatically generated based on the clinical semantic context, as shown in the example below: Chief complaint and symptoms: Dry and sore throat with a feeling of mucus, occurring every morning upon waking, relieved by drinking warm water; mild lower abdominal bloating and discomfort. Secondary complaint and symptoms: Menstruation started earlier, on September 23rd or 25th, with a shorter cycle than before. The tongue is broad with teeth marks on the sides, and it is relatively mobile without cracks; the tongue body is pale red with a slightly dark color and the surface is moist; the coating is thin and white, and slightly thicker at the root. My menstrual cycle is normally 28 to 30 days, but recently it has started earlier, around September 23 or 25. Specialized Gynecology Diagnosis (Disease Name): Premenstrual Syndrome Differentiation of Qi Deficiency Syndrome Treatment principle: Tonify Qi and regulate menstruation Acupuncture (System-generated suggestion, for reference only) Treatment principle: Tonify Qi and regulate menstruation, unblock Qi flow. Acupoints: Zusanli (ST36), Qihai (CV6), Guanyuan (CV4), Sanyinjiao (SP6). The patient's mood is stable and sleep is generally good, though there are occasional fluctuations due to work stress; recently, the patient went to Macau to work in beauty telemarketing. The doctor advised to let nature take its course and to return for a follow-up appointment when menstruation resumes; to return for a follow-up appointment five days later; and to carefully observe any changes in the body while taking the medication.

[0139] This embodiment demonstrates that the present invention can automatically achieve multi-dimensional quality quantification of the consultation process based on natural language dialogue, including indicators such as consultation coverage, structural integrity, and the rationality of clinical reasoning. Simultaneously, it can automatically generate electronic medical record reports that conform to medical document standards, realizing intelligent processing from voice input to structured output, and improving the efficiency of physician consultation quality assessment and recording.

[0140] The present invention also includes training the hybrid evaluation model, the specific steps of which are as follows: First, multiple sets of real doctor-patient consultation data were acquired as training samples. These training samples cover different medical specialties, disease types, and consultation scenarios to ensure that the hybrid evaluation model can learn rich and diverse clinical consultation semantics and logical relationships.

[0141] For each set of training samples, medical experts manually score the samples based on multiple predefined evaluation dimensions, generating standard scoring labels. Evaluation dimensions may include indicators such as completeness of the consultation, logical coherence, and clinical rationality, ensuring that the model's learning objectives align with clinical practice. Through this supervised learning process, the hybrid evaluation model gradually learns how to combine consultation semantics, SOAP classification, and knowledge graph logical reasoning to generate accurate and structured consultation quality assessments.

[0142] Then, using the training samples as input and the corresponding standard rating labels as supervision targets, the hybrid evaluation model is trained so that it can automatically output quality evaluation feedback information that is similar to the evaluation standards of medical experts based on the consultation.

[0143] Furthermore, the present invention also includes a user feedback loop mechanism. When a user corrects or supplements the automatically generated quality assessment feedback information during use, the mechanism receives the user's correction operations on the quality assessment feedback information and uses this corrected data for subsequent model fine-tuning. This mechanism ensures that the model not only relies on historical data but can also continuously learn by combining actual clinical usage scenarios.

[0144] Based on the aforementioned correction operations, the hybrid evaluation model is trained using reinforcement learning or transfer learning. Reinforcement learning guides the model to optimize its consultation logic through a reward mechanism, while transfer learning can quickly adapt to new consultation scenarios or disease characteristics, thereby improving the model's adaptability and intelligence in different clinical environments. This allows the present invention to continuously optimize its consultation logic and language model in real-world usage scenarios.

[0145] Through the above training and optimization mechanisms, the hybrid assessment model has the ability to self-evolve. It can update risk identification, problem list and consultation process in a timely manner as new diseases emerge, making up for the shortcomings of static and difficult-to-iterate models in the existing technology. It can achieve long-term optimization and dynamic improvement of consultation quality assessment, and provide doctors with continuous and reliable intelligent assistance support.

[0146] Exemplary System Accordingly, embodiments of the present invention also provide a clinical consultation quality assessment system. Figure 2 This is a block diagram of the clinical consultation quality assessment system provided in an embodiment of the present invention, such as... Figure 2 As shown, the system 100 provided in this embodiment includes: Input device 101, which is used to acquire real-time doctor-patient consultation data; Analysis device 102 is used to analyze the doctor-patient consultation data in real time and construct the clinical semantic context of the current consultation. The generation device 103 is used to generate a question list based on the clinical semantic context using a hybrid assessment model, wherein the hybrid assessment model integrates a rule engine and a machine learning model, and the question list includes a list of mandatory questions generated by the rule engine for the current clinical semantic context and a list of implicit questions inferred by the machine learning model based on the clinical semantic context. The judgment device 104 is used to determine whether the contents of the question list are missing from the doctor-patient consultation data; The output device 105 is used to generate and output quality assessment feedback information for the consultation based on the judgment result. The quality assessment feedback information includes a quantitative quality score and / or consultation supplementation suggestions.

[0147] The input device 101 includes: A unit used to convert audio streams of doctor-patient conversations into text records in real time through automatic speech recognition. A unit used to perform role differentiation and sentence segmentation processing on the text dialogue record using speaker separation technology to obtain the doctor-patient consultation data.

[0148] The analysis device 102 is also used to analyze the doctor-patient consultation data in real time based on a medical knowledge graph, and to construct the clinical semantic context of the current consultation. The question list includes a list of mandatory questions generated by the rule engine based on the medical knowledge graph for the current clinical semantic context, and a list of implicit questions inferred by the machine learning model based on the medical knowledge graph and the clinical semantic context.

[0149] The medical knowledge graph is a customized knowledge graph for a specific medical field; The specific medical field mentioned is a field defined based on different medical theoretical systems, clinical specialties, or regional diagnosis and treatment practices; The customized knowledge graph contains medical entities specific to the specific medical field and the clinical logical relationships between these medical entities; The medical entity includes local terminology expressions corresponding to the regional medical practices.

[0150] The analysis device 102 includes: A unit used to perform entity recognition on the doctor-patient consultation data and extract medical entities; A unit used to query the clinical logical relationships between pre-stored medical entities in the medical knowledge graph based on the medical entity, and to obtain the association relationship of the medical entity.

[0151] The analysis device 102 includes: A unit used to analyze statements in the doctor-patient consultation data based on the pre-stored medical entities and clinical logical relationships in the medical knowledge graph, and to obtain the clinical intent corresponding to each statement through an intent recognition model; A unit used to classify each statement into the SOAP framework's subjective information, objective information, assessment information, or planning information category based on its clinical intent, thereby obtaining SOAP classification results.

[0152] The analysis device 102 includes: A unit used to construct the clinical semantic context based on the associations of the medical entities and / or the SOAP classification results.

[0153] The generating device 103 includes: A unit used to statically verify the clinical semantic context against the clinical logical relationships in the medical knowledge graph through the rule engine, and generate the list of mandatory questions; A unit for dynamically inferring a list of implicit questions to be asked based on the clinical logical relationships in the medical knowledge graph and the clinical semantic context using the machine learning model.

[0154] The output device 105 includes: The real-time output unit 106 is used to synchronously generate and output quality assessment feedback information for the consultation based on the judgment result during the doctor-patient consultation process. The end output unit 107 is used to generate and output quality assessment feedback information for the consultation based on the judgment result after the doctor-patient consultation process is completed.

[0155] The real-time output unit 106 is also used to generate and display in real time quality assessment feedback information for the consultation in the form of a visual dashboard, based on the judgment result during the doctor-patient consultation process. The quality assessment feedback information also includes electronic medical record reports, which include SOAP, DAP, GIRP, PIE, POMR and / or SOAPIER medical records; The end output unit 107 is also used to generate and output an electronic medical record report for the consultation based on the judgment result after the consultation process is completed.

[0156] The input device 101 further includes: A unit for acquiring medical images related to the consultation; The analysis device 102 further includes: A unit for analyzing the medical images using an image recognition model to obtain visual medical features; Units used to integrate the visual medical features into the clinical semantic context based on medical knowledge graphs.

[0157] The system 100 also includes: Units used to acquire multiple sets of real doctor-patient consultation data as training samples; This unit is used to generate standard rating labels by having medical experts manually score each set of training samples based on multiple predefined evaluation dimensions. This unit is used to train the hybrid evaluation model by taking the training samples as input and the corresponding standard rating labels as supervision targets, so that the hybrid evaluation model can automatically output quality evaluation feedback information that is similar to the evaluation standards of medical experts based on medical consultation.

[0158] The system 100 also includes: A unit for receiving correction operations from users regarding the quality assessment feedback information; A unit for training the hybrid evaluation model using reinforcement learning based on the correction operation.

[0159] It should be noted that although the operation of the clinical consultation quality assessment method of the present invention is described in a specific order in the accompanying drawings, this does not require or imply that the operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0160] Furthermore, although several devices, units, or modules of the clinical consultation quality assessment system have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more modules described above can be embodied in a single module. Conversely, the features and functions of a single module described above can be further divided and embodied by multiple modules.

[0161] While the spirit and principles of the invention have been described with reference to several specific embodiments, it should be understood that the invention is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined for benefit; such division is merely for ease of description. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

[0162] This invention provides: 1. A method for assessing the quality of clinical consultations, characterized in that the method comprises: Obtain real-time doctor-patient consultation data; The doctor-patient consultation data is analyzed in real time to construct the clinical semantic context of the current consultation. Based on the clinical semantic context, a question list is generated through a hybrid assessment model, wherein the hybrid assessment model integrates a rule engine and a machine learning model, and the question list includes a list of mandatory questions generated by the rule engine for the current clinical semantic context and a list of implicit questions inferred by the machine learning model based on the clinical semantic context. Determine whether any items from the question list are missing from the doctor-patient consultation data; Based on the judgment results, quality assessment feedback information for the consultation is generated and output, including quantitative quality scores and / or consultation gap-filling suggestions.

[0163] 2. The clinical consultation quality assessment method according to item 1, characterized in that the step of acquiring real-time doctor-patient consultation data includes: The audio stream of doctor-patient dialogue is converted into a text dialogue record in real time through automatic speech recognition. The doctor-patient consultation data is obtained by using speaker separation technology to distinguish roles and segment sentences in the text dialogue records.

[0164] 3. The clinical consultation quality assessment method described in item 2 is characterized in that the step of analyzing the doctor-patient consultation data in real time and constructing the clinical semantic context of the current consultation specifically includes: Based on real-time analysis of the doctor-patient consultation data using a medical knowledge graph, a clinical semantic context for the current consultation is constructed. The question list includes a list of mandatory questions generated by the rule engine based on the medical knowledge graph for the current clinical semantic context, and a list of implicit questions inferred by the machine learning model based on the medical knowledge graph and the clinical semantic context. The medical knowledge graph is a customized knowledge graph for a specific medical field; The specific medical field mentioned is a field defined based on different medical theoretical systems, clinical specialties, or regional diagnosis and treatment practices; The customized knowledge graph contains medical entities specific to the specific medical field and the clinical logical relationships between these medical entities; The medical entity includes local terminology expressions corresponding to the regional medical practices.

[0165] 4. The clinical consultation quality assessment method described in item 3 is characterized by the following steps: Based on real-time analysis of the doctor-patient consultation data using a medical knowledge graph, the steps for constructing the clinical semantic context of the current consultation include: Entity recognition is performed on the aforementioned doctor-patient consultation data to extract medical entities; Based on the medical entity, query the pre-stored clinical logical relationships between medical entities in the medical knowledge graph to obtain the association relationship of the medical entity.

[0166] 5. The clinical consultation quality assessment method according to item 3 or 4, characterized in that the step of constructing the clinical semantic context of the current consultation based on real-time analysis of the doctor-patient consultation data using a medical knowledge graph includes: Based on the pre-stored medical entities and clinical logical relationships in the medical knowledge graph, the statements in the doctor-patient consultation data are analyzed through an intent recognition model to obtain the clinical intent corresponding to each statement. Each statement is categorized into the subjective information, objective information, assessment information, or planning information category within the SOAP framework based on its clinical intent, thus obtaining the SOAP classification results.

[0167] 6. The clinical consultation quality assessment method according to item 5, characterized in that the step of constructing the clinical semantic context of the current consultation based on real-time analysis of the doctor-patient consultation data using a medical knowledge graph includes: The clinical semantic context is constructed based on the association relationships of the medical entities and / or the SOAP classification results.

[0168] 7. The clinical consultation quality assessment method according to item 1, characterized in that the step of generating a question list based on the clinical semantic context using a hybrid assessment model includes: The rule engine is used to statically verify the clinical semantic context against the clinical logical relationships in the medical knowledge graph, and generate the list of essential questions. Using the machine learning model, based on the clinical logical relationships in the medical knowledge graph, a list of implicit questions that should be asked is dynamically inferred according to the clinical semantic context.

[0169] 8. The clinical consultation quality assessment method according to item 1, characterized in that the step of generating and outputting quality assessment feedback information for the consultation based on the judgment result includes: During the doctor-patient consultation process, based on the judgment results, quality assessment feedback information for the consultation is generated and output simultaneously; and / or After the doctor-patient consultation process is completed, quality assessment feedback information for the consultation is generated and output based on the judgment results.

[0170] 9. The clinical consultation quality assessment method according to item 8 is characterized in that, during the doctor-patient consultation process, the step of simultaneously generating and outputting quality assessment feedback information for the consultation based on the judgment result specifically comprises: During the doctor-patient consultation process, based on the judgment results, quality assessment feedback information for the consultation is generated synchronously and displayed in real time in the form of a visual dashboard; The quality assessment feedback information also includes electronic medical record reports, which include SOAP, DAP, GIRP, PIE, POMR and / or SOAPIER medical records; The specific steps for generating and outputting quality assessment feedback information for the consultation after the doctor-patient consultation process has ended, based on the judgment results, are as follows: After the doctor-patient consultation process is completed, an electronic medical record report is generated and output based on the judgment results.

[0171] 10. The clinical consultation quality assessment method according to item 1, characterized in that the step of obtaining real-time doctor-patient consultation data further includes: Acquire medical images related to the consultation; The steps of constructing the clinical semantic context of the current consultation based on real-time analysis of the doctor-patient consultation data using a medical knowledge graph also include: The medical images are analyzed using an image recognition model to obtain visual medical features. The visual medical features are integrated into the clinical semantic context based on a medical knowledge graph.

[0172] 11. The clinical consultation quality assessment method according to item 1, characterized in that the method further includes: Multiple sets of real doctor-patient consultation data were obtained as training samples; For each set of training samples, medical experts manually score the samples based on multiple predefined evaluation dimensions to generate standard score labels. Using the training samples as input and the corresponding standard rating labels as supervision targets, the hybrid evaluation model is trained so that it can automatically output quality evaluation feedback information that is similar to the evaluation standards of medical experts based on medical consultation.

[0173] 12. The clinical consultation quality assessment method according to item 1, characterized in that the method further includes: Receive correction operations from users regarding the quality assessment feedback information; Based on the correction operation, the hybrid evaluation model is trained using reinforcement learning.

[0174] 13. A clinical consultation quality assessment system, characterized in that the system comprises: Input device, used to acquire real-time doctor-patient consultation data; An analysis device is used to analyze the doctor-patient consultation data in real time and construct the clinical semantic context of the current consultation. A generation device is used to generate a list of questions based on the clinical semantic context using a hybrid assessment model, wherein the hybrid assessment model integrates a rule engine and a machine learning model, and the list of questions includes a list of mandatory questions generated by the rule engine for the current clinical semantic context and a list of implicit questions inferred by the machine learning model based on the clinical semantic context. A judgment device is used to determine whether any content of the question list is missing from the doctor-patient consultation data; An output device is used to generate and output quality assessment feedback information for the consultation based on the judgment result, the quality assessment feedback information including a quantitative quality score and / or consultation supplementation suggestions.

[0175] 14. The clinical consultation quality assessment system according to item 13, characterized in that the input device comprises: A unit used to convert audio streams of doctor-patient conversations into text records in real time through automatic speech recognition. A unit used to perform role differentiation and sentence segmentation processing on the text dialogue record using speaker separation technology to obtain the doctor-patient consultation data.

[0176] 15. The clinical consultation quality assessment system according to item 14, characterized in that the analysis device is further used to analyze the doctor-patient consultation data in real time based on a medical knowledge graph to construct the clinical semantic context of the current consultation; The question list includes a list of mandatory questions generated by the rule engine based on the medical knowledge graph for the current clinical semantic context, and a list of implicit questions inferred by the machine learning model based on the medical knowledge graph and the clinical semantic context. The medical knowledge graph is a customized knowledge graph for a specific medical field; The specific medical field mentioned is a field defined based on different medical theoretical systems, clinical specialties, or regional diagnosis and treatment practices; The customized knowledge graph contains medical entities specific to the specific medical field and the clinical logical relationships between these medical entities; The medical entity includes local terminology expressions corresponding to the regional medical practices.

[0177] 16. The clinical consultation quality assessment system according to item 15, characterized in that the analysis device comprises: A unit used to perform entity recognition on the doctor-patient consultation data and extract medical entities; A unit used to query the clinical logical relationships between pre-stored medical entities in the medical knowledge graph based on the medical entity, and to obtain the association relationship of the medical entity.

[0178] 17. The clinical consultation quality assessment system according to claim 15 or 16, characterized in that the analysis device comprises: A unit used to analyze statements in the doctor-patient consultation data based on the pre-stored medical entities and clinical logical relationships in the medical knowledge graph, and to obtain the clinical intent corresponding to each statement through an intent recognition model; A unit used to classify each statement into the SOAP framework's subjective information, objective information, assessment information, or planning information category based on its clinical intent, thereby obtaining SOAP classification results.

[0179] 18. The clinical consultation quality assessment system according to item 17, characterized in that the analysis device comprises: A unit used to construct the clinical semantic context based on the associations of the medical entities and / or the SOAP classification results.

[0180] 19. The clinical consultation quality assessment system according to item 13, characterized in that the generating device comprises: A unit used to statically verify the clinical semantic context against the clinical logical relationships in the medical knowledge graph through the rule engine, and generate the list of mandatory questions; A unit for dynamically inferring a list of implicit questions to be asked based on the clinical logical relationships in the medical knowledge graph and the clinical semantic context using the machine learning model.

[0181] 20. The clinical consultation quality assessment system according to item 13, characterized in that the output device comprises: A real-time output unit is used to synchronously generate and output quality assessment feedback information for the consultation based on the judgment result during the doctor-patient consultation process. The end output unit is used to generate and output quality assessment feedback information for the consultation based on the judgment result after the doctor-patient consultation process is completed.

[0182] 21. The clinical consultation quality assessment system according to item 20, wherein the real-time output unit is further used to generate and display in real time quality assessment feedback information for the consultation in the form of a visual dashboard, based on the judgment result during the doctor-patient consultation process. The quality assessment feedback information also includes electronic medical record reports, which include SOAP, DAP, GIRP, PIE, POMR and / or SOAPIER medical records; The end output unit is also used to generate and output an electronic medical record report for the consultation based on the judgment result after the consultation process is completed.

[0183] 22. The clinical consultation quality assessment system according to item 13, characterized in that the input device further includes: A unit for acquiring medical images related to the consultation; The analytical apparatus further includes: A unit for analyzing the medical images using an image recognition model to obtain visual medical features; Units used to integrate the visual medical features into the clinical semantic context based on medical knowledge graphs.

[0184] 23. The clinical consultation quality assessment system according to item 13, characterized in that the system further includes: Units used to acquire multiple sets of real doctor-patient consultation data as training samples; This unit is used to generate standard rating labels by having medical experts manually score each set of training samples based on multiple predefined evaluation dimensions. This unit is used to train the hybrid evaluation model by taking the training samples as input and the corresponding standard rating labels as supervision targets, so that the hybrid evaluation model can automatically output quality evaluation feedback information that is similar to the evaluation standards of medical experts based on the consultation.

[0185] 24. The clinical consultation quality assessment system according to item 13, characterized in that the system further comprises: A unit for receiving correction operations from users regarding the quality assessment feedback information; A unit for training the hybrid evaluation model using reinforcement learning based on the correction operation.

Claims

1. A method for assessing the quality of clinical consultations, characterized in that, The method includes: Obtain real-time doctor-patient consultation data; The doctor-patient consultation data is analyzed in real time to construct the clinical semantic context of the current consultation. Based on the clinical semantic context, a question list is generated through a hybrid assessment model, wherein the hybrid assessment model integrates a rule engine and a machine learning model, and the question list includes a list of mandatory questions generated by the rule engine for the current clinical semantic context and a list of implicit questions inferred by the machine learning model based on the clinical semantic context. Determine whether any items from the question list are missing from the doctor-patient consultation data; Based on the judgment results, quality assessment feedback information for the consultation is generated and output, including quantitative quality scores and / or consultation gap-filling suggestions.

2. The clinical consultation quality assessment method according to claim 1, characterized in that, The steps for obtaining real-time doctor-patient consultation data include: The audio stream of doctor-patient dialogue is converted into a text dialogue record in real time through automatic speech recognition. The doctor-patient consultation data is obtained by using speaker separation technology to distinguish roles and segment sentences in the text dialogue records.

3. The clinical consultation quality assessment method according to claim 2, characterized in that, The specific steps for analyzing the doctor-patient consultation data in real time and constructing the clinical semantic context of the current consultation are as follows: Based on real-time analysis of the doctor-patient consultation data using a medical knowledge graph, a clinical semantic context for the current consultation is constructed. The question list includes a list of mandatory questions generated by the rule engine based on the medical knowledge graph for the current clinical semantic context, and a list of implicit questions inferred by the machine learning model based on the medical knowledge graph and the clinical semantic context. The medical knowledge graph is a customized knowledge graph for a specific medical field; The specific medical field mentioned is a field defined based on different medical theoretical systems, clinical specialties, or regional diagnosis and treatment practices; The customized knowledge graph contains medical entities specific to the specific medical field and the clinical logical relationships between these medical entities; The medical entity includes local terminology expressions corresponding to the regional medical practices.

4. The clinical consultation quality assessment method according to claim 3, characterized in that, The steps for constructing the clinical semantic context of the current consultation based on real-time analysis of the doctor-patient consultation data using a medical knowledge graph include: Entity recognition is performed on the aforementioned doctor-patient consultation data to extract medical entities; Based on the medical entity, query the pre-stored clinical logical relationships between medical entities in the medical knowledge graph to obtain the association relationship of the medical entity.

5. The clinical consultation quality assessment method according to claim 3 or 4, characterized in that, The steps for constructing the clinical semantic context of the current consultation based on real-time analysis of the doctor-patient consultation data using a medical knowledge graph include: Based on the pre-stored medical entities and clinical logical relationships in the medical knowledge graph, the statements in the doctor-patient consultation data are analyzed through an intent recognition model to obtain the clinical intent corresponding to each statement. Each statement is categorized into the subjective information, objective information, assessment information, or planning information category within the SOAP framework based on its clinical intent, thus obtaining the SOAP classification results.

6. The clinical consultation quality assessment method according to claim 5, characterized in that, The steps for constructing the clinical semantic context of the current consultation based on real-time analysis of the doctor-patient consultation data using a medical knowledge graph include: The clinical semantic context is constructed based on the association relationships of the medical entities and / or the SOAP classification results.

7. The clinical consultation quality assessment method according to claim 1, characterized in that, Based on the aforementioned clinical semantic context, the steps for generating a question list using a hybrid assessment model include: The rule engine is used to statically verify the clinical semantic context against the clinical logical relationships in the medical knowledge graph, and generate the list of essential questions. Using the machine learning model, based on the clinical logical relationships in the medical knowledge graph, a list of implicit questions that should be asked is dynamically inferred according to the clinical semantic context.

8. The clinical consultation quality assessment method according to claim 1, characterized in that, The step of generating and outputting quality assessment feedback information for the consultation based on the judgment result includes: During the doctor-patient consultation process, based on the judgment results, quality assessment feedback information for the consultation is generated and output simultaneously; and / or After the doctor-patient consultation process is completed, quality assessment feedback information for the consultation is generated and output based on the judgment results.

9. The clinical consultation quality assessment method according to claim 8, characterized in that, The specific steps for generating and outputting quality assessment feedback information for the consultation based on the judgment result during the doctor-patient consultation process are as follows: During the doctor-patient consultation process, based on the judgment results, quality assessment feedback information for the consultation is generated synchronously and displayed in real time in the form of a visual dashboard; The quality assessment feedback information also includes electronic medical record reports, which include SOAP, DAP, GIRP, PIE, POMR and / or SOAPIER medical records; The specific steps for generating and outputting quality assessment feedback information for the consultation after the doctor-patient consultation process has ended, based on the judgment results, are as follows: After the doctor-patient consultation process is completed, an electronic medical record report is generated and output based on the judgment results.

10. A clinical consultation quality assessment system, characterized in that, The system includes: Input device, used to acquire real-time doctor-patient consultation data; An analysis device is used to analyze the doctor-patient consultation data in real time and construct the clinical semantic context of the current consultation. A generation device is used to generate a list of questions based on the clinical semantic context using a hybrid assessment model, wherein the hybrid assessment model integrates a rule engine and a machine learning model, and the list of questions includes a list of mandatory questions generated by the rule engine for the current clinical semantic context and a list of implicit questions inferred by the machine learning model based on the clinical semantic context. A judgment device is used to determine whether any content of the question list is missing from the doctor-patient consultation data; An output device is used to generate and output quality assessment feedback information for the consultation based on the judgment result, the quality assessment feedback information including a quantitative quality score and / or consultation supplementation suggestions.

Images