Intelligent evaluation and feedback system for simulating patient teaching based on dialogue analysis

Abstract

The invention discloses an intelligent evaluation and feedback system for simulating patient teaching based on dialogue analysis, and relates to the technical field of medical education and artificial intelligence. The system comprises a dialogue data acquisition module, an NLP analysis module, a machine learning and deep learning model module, an evaluation and scoring module, a feedback and improvement suggestion module and a data storage and teaching management module. The system is based on technologies of natural language processing, voice recognition, machine learning and the like, analyzes the conversation between students and simulated patients in real time, comprehensively evaluates clinical communication, illness state judgment and treatment decision ability, realizes real-time feedback on the basis, and has the advantages of simple structure, convenient operation and high efficiency. Personalized evaluation reports and suggestions are provided, and decision basis is provided for teaching improvement.

Description

Technical Field

[0001] The present invention relates to the field of medical education and artificial intelligence technology, and in particular to an intelligent evaluation and feedback system for simulated patient teaching based on dialogue analysis. Background Art

[0002] Medical simulation teaching plays a key role in modern medical education, providing medical students with a risk-free practice environment and helping them develop clinical skills, communication, and teamwork abilities. Current medical simulation teaching evaluations mostly rely on teacher observation, student self-evaluation and mutual evaluation, and questionnaires. While these methods can reflect learning outcomes, they have significant flaws: teacher observation is easily influenced by subjective factors, student self-evaluation and mutual evaluation are easily interfered with by interpersonal relationships, questionnaire feedback is not timely, and information is not in-depth. Furthermore, existing systems struggle to quantitatively assess complex abilities such as clinical thinking and communication skills, making it impossible to provide targeted improvement suggestions. Existing simulation teaching methods, including manikin and virtual simulation software, can restore clinical scenarios to a certain extent, but they also suffer from high costs, are limited by time and space, are limited by venues, and suffer from a single type of simulated disease, making them ineffective in adapting to the rapid development of medical technology.

[0003] With the development and popularization of information technology, the role of artificial intelligence technology in learning style analysis, intelligent question answering, and teaching evaluation has gradually become prominent. Can artificial intelligence be fully integrated with traditional medical simulation teaching evaluation methods, and can machine learning and natural language processing technologies be fully applied to solve the problems in existing medical simulation teaching? Summary of the Invention

[0004] The purpose of the present invention is to overcome the shortcomings of the existing technology and provide an intelligent evaluation and feedback system for simulated patient teaching based on dialogue analysis, so as to solve the limitations of the existing evaluation system and the problems of strong subjectivity and low accuracy in traditional evaluation.

[0005] The object of the present invention is achieved through the following technical solutions: An intelligent evaluation and feedback system for simulated patient teaching based on conversation analysis, comprising: The conversation data acquisition module includes a microphone array and a voice processing submodule. The microphone array is used to collect the user's voice data. The voice processing submodule is used to receive the voice data collected by the microphone array, convert the voice data into text data in real time using ASR technology, and pre-process the voice data. The pre-processed text data is formatted and stored; The NLP analysis module includes a logical analysis submodule, a grammatical analysis submodule, and a sentiment analysis submodule. The logical analysis submodule is used to perform semantic understanding and logical relationship analysis on text data, extract medical entities, and evaluate logical coherence. The grammatical analysis submodule is used to check the grammatical structure of text data and identify grammatical errors or non-standard expressions. The sentiment analysis submodule is used to quantify empathy expressions using a sentiment dictionary and a Bi-LSTM model, analyze emotional tendencies in conversations, and evaluate whether the user's language expression meets the standards for doctor-patient communication. The machine learning and deep learning model module is used to conduct deep learning and modeling of conversation texts, capturing the contextual information, semantic relationships, and emotional tendencies in the conversations, and achieving a comprehensive assessment of students' questioning abilities. The evaluation and scoring module includes a scoring algorithm submodule and a scoring result storage submodule. The scoring algorithm submodule is used to integrate the output results of the NLP analysis module and the machine learning and deep learning model module to score the conversation content in each dimension according to the preset scoring criteria; the scoring result storage submodule is used to store the scoring results of each dimension in a structured manner; The feedback and improvement suggestion module includes a feedback report submodule and an improvement suggestion submodule. The feedback report submodule is used to generate a feedback report containing scoring details and overall evaluation based on the scoring results; the improvement suggestion submodule is used to provide targeted improvement suggestions based on the scoring results and analysis data; The data storage and teaching management module includes a database storage submodule and a teaching management submodule. The database storage submodule is used to store, query, count and analyze user historical conversation data, analysis results, scoring results and feedback reports; the teaching management submodule is used to provide a data management interface for data management and analysis.

[0006] Furthermore, the use of ASR technology to convert voice data into text data in real time and pre-processing the data specifically includes: Using a speech recognition model based on a deep neural network, the speech data collected from the microphone array is converted into text data in real time; Use the Kalman filter algorithm to perform noise reduction and dereverberation processing on text data; Perform word segmentation and tagging operations on text data.

[0007] Furthermore, the semantic understanding and logical relationship analysis of text data, extraction of medical entities and evaluation of logical coherence include: Parse the conversation text to obtain core medical information, and use the BERT-Base model to extract medical information entities from the core medical information. Medical information entities include symptoms, medical history, and diagnosis information. The medical information entities are analyzed by dependency parsing technology and the logical coherence of the text data is evaluated.

[0008] Furthermore, the deep learning and modeling of the conversation text captures the contextual information, semantic relationships, and emotional tendencies in the conversation, thereby achieving a comprehensive assessment of the students' questioning ability, specifically including: The selected deep learning model is trained using a large amount of labeled teaching data, including conversation records between simulated patients and students, teacher ratings, and student self-feedback information; Optimize deep learning models through cross-validation, regularization, and dropout techniques; The feature data of the conversation text is extracted and normalized, and the normalized features are input into the optimized deep learning model to capture the contextual information, semantic relationships and emotional tendencies in the conversation, and output the evaluation results of the students' medical consultation ability, which includes clinical communication ability, disease diagnosis ability and treatment decision-making ability.

[0009] Furthermore, the output results of the comprehensive NLP analysis module and the machine learning and deep learning model module are used to score the conversation content in various dimensions according to preset scoring criteria, specifically including: Set corresponding medical skills teaching scoring standards and evaluation indicators based on ACGME core competencies; Through the AI ​​scoring model, the user's performance in questioning logic, medical history collection completeness, and communication skills is quantified to obtain automatic scoring results; According to the scoring calibration mechanism, the automatic scoring results are combined with the teacher's manual review and scoring results to perform a comprehensive scoring.

[0010] Furthermore, the feedback report is presented in the form of text description and chart display, and the content of the feedback report includes improvement of voice expression, optimization of consultation / assessment strategies, detailed score comparison and display of the user's progress; the improvement suggestions include suggestions on logical expression, grammatical correction, and emotional adjustment.

[0011] Furthermore, the database storage submodule adopts MongoDB distributed database.

[0012] Furthermore, the teaching management submodule is also used to provide teachers with system adjustment functions, which include modifying scoring standards and adding new teaching cases.

[0013] The beneficial effects of the present invention are: 1) Improve the quality of medical skills teaching: Through intelligent assessment, standardized objective scoring is provided to reduce the subjective errors of human scoring; AI scoring is combined with teacher review to ensure that the evaluation results are accurate and reliable, which helps to optimize skill teaching plans. 2) Enhance students' medical consultation and communication skills: Based on NLP technology, the user's medical consultation logic, language expression, empathy ability, etc. are analyzed to help them improve their doctor-patient communication skills; automatically generated personalized feedback reports guide users to improve their medical consultation / assessment strategies and optimize their medical history collection capabilities. 3) Real-time and accurate feedback mechanism: Users can obtain analysis results immediately after the consultation, quickly adjust their learning direction, and improve learning efficiency; feedback content, mainly in the form of charts, scores, text analysis, etc., is intuitive and clear, promoting user understanding and improvement. 4) Improve the efficiency and scalability of medical training: Through large-scale automatic evaluation, the grading workload of teachers is reduced, and teaching resources can be allocated more efficiently; the distance learning function can be used for online medical training or MOOC courses to expand the scope of learning; at the same time, the system can also be used for classroom teaching in colleges and universities, standardized medical training, continuing medical education and other courses, and is adapted to different disciplines (such as internal medicine, surgery, pediatrics, etc.).

[0014] 5) Data-driven teaching optimization: By recording users' historical consultation / assessment data, analyzing learning progress, and adjusting teaching plans individually; the visualization of teaching data can help teachers identify common problems, optimize course design, and improve overall teaching effectiveness. 6) Reduce teaching costs and improve resource utilization: Through AI automated scoring, the reliance on manual evaluation is reduced, reducing teaching labor costs; the limited number of intelligent simulated patient resources can be used more efficiently, improving the cost-effectiveness of teaching. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a structural diagram of an intelligent evaluation and feedback system for simulated patient teaching based on conversation analysis. DETAILED DESCRIPTION

[0016] The following will clearly and completely describe the technical solutions of the present invention in conjunction with the embodiments. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work shall fall within the scope of protection of the present invention.

[0017] See Figure 1 , the present invention provides a technical solution: An intelligent evaluation and feedback system for simulated patient teaching based on conversation analysis, e.g. Figure 1 Shown, including: The conversation data acquisition module includes a microphone array and a voice processing submodule. The microphone array is used to collect the user's voice data. The voice processing submodule is used to receive the voice data collected by the microphone array, use ASR technology to convert the voice data into text data in real time, and pre-process it, and format and store the pre-processed text data.

[0018] The ASR technology is used to convert speech data into text data in real time and pre-process the data, specifically including: Using a speech recognition model based on a deep neural network, the speech data collected from the microphone array is converted into text data in real time; Use the Kalman filter algorithm to perform noise reduction and dereverberation processing on text data; Perform word segmentation and tagging operations on text data.

[0019] The conversation data acquisition module uses a highly sensitive microphone array to collect the conversation content between the user and the intelligent simulated patient, obtaining conversation voice data, which is then processed and formatted for storage by the voice processing submodule to facilitate subsequent NLP analysis and scoring.

[0020] The NLP analysis module includes a logic analysis submodule, a grammatical analysis submodule, and a sentiment analysis submodule. The logic analysis submodule is used to perform semantic understanding and logical relationship analysis on text data, extract medical entities, and evaluate logical coherence; the grammatical analysis submodule is used to check the grammatical structure of text data and identify grammatical errors or non-standard expressions; the sentiment analysis submodule is used to quantify empathy expression through a sentiment dictionary and a Bi-LSTM model, analyze the emotional tendencies in the conversation, and evaluate whether the user's language expression meets the doctor-patient communication standards (such as empathy, humanistic care, etc.).

[0021] Preferably, the performing of semantic understanding and logical relationship analysis on text data, extracting medical entities and evaluating logical coherence includes: Parse the conversation text to obtain core medical information, and use the BERT-Base model to extract medical information entities from the core medical information. Medical information entities include symptoms, medical history, and diagnosis information. The medical information entities are analyzed by dependency parsing technology and the logical coherence of the text data is evaluated.

[0022] In a preferred embodiment, the syntax analysis submodule uses a syntactic parser (such as Stanford Parser) to check the grammatical structure of the text and identify grammatical errors or non-standard expressions.

[0023] The machine learning and deep learning model module is used to perform deep learning and modeling on the conversation text, capture the contextual information, semantic relationships and emotional tendencies in the conversation, and achieve a comprehensive assessment of the student's medical consultation ability. Preferably, the deep learning and modeling of the conversation text, capture the contextual information, semantic relationships and emotional tendencies in the conversation, and achieve a comprehensive assessment of the student's medical consultation ability, specifically includes: The selected deep learning model is trained using a large amount of labeled teaching data, including conversation records between simulated patients and students, teacher ratings, and student self-feedback information; Optimize deep learning models through cross-validation, regularization, and dropout techniques to reduce overfitting and improve the generalization ability of the model; The feature data of the conversation text is extracted and normalized, and the normalized features are input into the optimized deep learning model to capture the contextual information, semantic relationships and emotional tendencies in the conversation, and output the evaluation results of the students' medical consultation ability, which includes clinical communication ability, disease diagnosis ability and treatment decision-making ability.

[0024] In the above process, the deep learning models selected include LSTM, Bi-LSTM, BERT, GPT, etc., such as using convolutional neural networks to analyze video data to evaluate operational standardization, and using recurrent neural networks to analyze conversation text to evaluate communication skills; the purpose of normalizing feature data is to eliminate the dimensional differences between different features and improve the convergence speed and stability of the deep learning model.

[0025] The evaluation and scoring module includes a scoring algorithm submodule and a scoring result storage submodule. The scoring algorithm submodule is used to integrate the output results of the NLP analysis module and the machine learning and deep learning model module to score the conversation content in each dimension according to the preset scoring criteria; the scoring result storage submodule is used to store the scoring results of each dimension in a structured manner; Furthermore, the output results of the comprehensive NLP analysis module and the machine learning and deep learning model module are used to score the conversation content in various dimensions according to preset scoring criteria, specifically including: Set corresponding medical skills teaching scoring standards and evaluation indicators based on ACGME core competencies; The AI ​​scoring model quantifies the user's performance in terms of questioning logic (whether standard procedures are followed), medical history collection completeness (whether key information is omitted), and communication skills (such as tone, empathy, professionalism, and humanistic qualities), and automatically generates a scoring result. According to the scoring calibration mechanism, the automatic scoring results are combined with the teacher's manual review and scoring results to conduct a comprehensive scoring to ensure the accuracy of the evaluation results.

[0026] The feedback and improvement suggestion module includes a feedback report submodule and an improvement suggestion submodule. The feedback report submodule is used to generate a feedback report containing scoring details and overall evaluation based on the scoring results; the improvement suggestion submodule is used to provide targeted improvement suggestions based on the scoring results and analysis data.

[0027] The feedback report is presented in both text and charts, using a visual interface to help users intuitively understand the feedback. The feedback report includes improvements to speech expression (such as speaking speed, clarity, and politeness), optimization of interview / assessment strategies (such as adding open-ended questions and improving the logic of history collection), detailed score comparisons, and a display of the user's progress. Suggestions for improvement include suggestions for logical expression, grammatical correction, and emotional adjustment. The generated personalized feedback report can provide targeted learning suggestions to help users improve their interview / assessment skills.

[0028] The data storage and teaching management module includes a database storage submodule and a teaching management submodule. The database storage submodule is used to store, query, compile statistics, and analyze historical user conversation data, analysis results, scoring results, and feedback reports; the teaching management submodule provides a data management interface for data management and analysis. Preferably, the database storage submodule uses the MongoDB distributed database to support large-scale data query, statistics, and analysis operations.

[0029] The teaching management submodule is also used to provide teachers with system adjustment functions, including modifying scoring standards and adding new teaching cases.

[0030] The data storage and teaching management module records the user's learning progress, facilitating subsequent teaching management and data analysis. It also classifies and indexes the stored data, supporting teachers to adjust teaching strategies to ensure the flexibility and adaptability of the system.

[0031] The present invention is different from the prior art in the following ways: 1. Evaluation Method: This invention utilizes an AI-driven intelligent evaluation method based on the evaluation indicators of medical skills teaching standards. It uses technologies such as automatic speech recognition (ASR), natural language processing (NLP), and machine learning to objectively and quantitatively analyze student consultation conversations. This, combined with manual review by teachers, ensures the accuracy and reliability of evaluation results and reduces the subjective errors associated with human scoring. Traditional medical simulation teaching evaluations rely primarily on manual scoring, which is heavily influenced by subjectivity and experience, and is susceptible to human factors, leading to certain subjective errors in the scoring results.

[0032] 2. Key Technologies: This invention utilizes intelligent conversation analysis as its core technology. It uses automated speech recognition (ASR) to convert spoken conversations into text. Natural language processing (NLP) technology then deeply analyzes the text, analyzing students' language logic and emotional expression. This allows for a comprehensive and accurate assessment of students' conversational skills during consultations, providing more objective and robust support for teaching. Traditional simulation-based teaching assessments rely primarily on teachers observing students during consultations and then conducting a comprehensive evaluation. This method requires significant teacher effort and time, and it struggles to provide a comprehensive and in-depth analysis of students' conversational skills.

[0033] 3. Feedback Method: Based on intelligent assessment results, this system automatically generates personalized intelligent reports, presenting feedback in various formats, such as charts and text analysis. This report intuitively demonstrates students' performance in areas such as interview logic, medical history collection completeness, and communication skills. It also provides targeted improvement suggestions based on individual student circumstances, thereby more effectively improving students' interview and communication skills. In contrast, traditional feedback methods often involve teachers providing qualitative feedback based on observational records. These feedback are often simplistic and limited by teaching time. They lack comprehensive and personalized improvement suggestions, making it difficult for students to gain clear directions and methods for improvement.

[0034] 4. Teaching Efficiency: This invention uses an efficient, intelligent assessment process to quickly and accurately evaluate student performance during consultations. This significantly reduces the teacher's grading workload, optimizes resources, shortens evaluation time, and improves teaching efficiency. This method can meet the needs of large-scale medical simulation teaching, while also providing strong support for remote medical education and online medical training, and enhancing the scalability of medical training. Traditional manual assessment methods, on the other hand, require teachers to pay full attention to observe, record, and evaluate each student's performance during the entire process. This leads to high grading workload and low efficiency, limiting the scale of simulation teaching.

[0035] 5. Applicability: This invention has remote intelligent support capabilities. Students can interact with intelligent simulated patients through an online teaching platform. The system automatically completes evaluation and feedback. At the same time, teachers can intervene and adjust the system through a management platform. It is not restricted by time and space, and is applicable to a variety of teaching scenarios and different disciplines. It can fully utilize teaching resources and improve the flexibility and coverage of teaching. Traditional simulation teaching evaluation methods are mainly limited by the on-site teaching scene and teaching duration. Teachers need to observe students' consultation process in real time on site. It is difficult to adapt to emerging teaching models such as remote teaching, which limits the scope of teaching resource utilization and the innovation of teaching models.

[0036] 6. Big Data Driven: This invention is based on the concept of big data-driven teaching optimization. The system can record students' historical consultation data, track and analyze their long-term learning progress, and through data mining and analysis techniques, identify students' learning problems and development trends at different stages. This provides teachers with personalized teaching suggestions, helps them optimize teaching plans, and improves overall teaching effectiveness, achieving a shift from traditional short-term, multiple single-shot scoring to long-term learning optimization. Traditional assessment methods, however, typically focus only on students' performance in a single consultation and lack tracking and analysis of the student's long-term learning process. This makes it difficult to identify students' learning characteristics and problems at different stages, and cannot provide strong data support for optimizing teaching plans.

[0037] The foregoing description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein and should not be construed as excluding other embodiments. Rather, the present invention can be used in various other combinations, modifications, and environments and can be modified within the scope of the concept described herein through the above teachings or techniques or knowledge in the relevant field. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention are intended to be protected by the appended claims.

Claims

1. An intelligent evaluation and feedback system for simulated patient teaching based on dialogue analysis, characterized in that: include: The conversation data acquisition module includes a microphone array and a voice processing submodule. The microphone array is used to collect the user's voice data. The voice processing submodule is used to receive the voice data collected by the microphone array, convert the voice data into text data in real time using ASR technology, and pre-process the voice data. The pre-processed text data is formatted and stored; The NLP analysis module includes a logical analysis submodule, a grammatical analysis submodule, and a sentiment analysis submodule. The logical analysis submodule is used to perform semantic understanding and logical relationship analysis on text data, extract medical entities, and evaluate logical coherence. The grammatical analysis submodule is used to check the grammatical structure of text data and identify grammatical errors or non-standard expressions. The sentiment analysis submodule is used to quantify empathy expressions using a sentiment dictionary and a Bi-LSTM model, analyze the emotional tendencies in the conversation, and assess whether the user's language expression meets the standards for doctor-patient communication. The machine learning and deep learning model module is used to conduct deep learning and modeling of conversation texts, capturing the contextual information, semantic relationships, and emotional tendencies in the conversations, and achieving a comprehensive assessment of students' questioning abilities. The evaluation and scoring module includes a scoring algorithm submodule and a scoring result storage submodule. The scoring algorithm submodule is used to integrate the output results of the NLP analysis module and the machine learning and deep learning model module to score the conversation content in each dimension according to the preset scoring criteria; the scoring result storage submodule is used to store the scoring results of each dimension in a structured manner; The feedback and improvement suggestion module includes a feedback report submodule and an improvement suggestion submodule. The feedback report submodule is used to generate a feedback report containing scoring details and overall evaluation based on the scoring results; The improvement suggestion submodule is used to provide targeted improvement suggestions based on the scoring results and analysis data; The data storage and teaching management module includes a database storage submodule and a teaching management submodule. The database storage submodule is used to store, query, count and analyze user historical conversation data, analysis results, scoring results and feedback reports; the teaching management submodule is used to provide a data management interface for data management and analysis.

2. The intelligent evaluation and feedback system for simulated patient teaching based on conversation analysis according to claim 1, characterized in that: The ASR technology is used to convert speech data into text data in real time and pre-process the data, specifically including: Using a speech recognition model based on a deep neural network, the speech data collected from the microphone array is converted into text data in real time; Use the Kalman filter algorithm to perform noise reduction and dereverberation processing on text data; Perform word segmentation and tagging operations on text data.

3. The intelligent evaluation and feedback system for simulated patient teaching based on conversation analysis according to claim 1, characterized in that: The semantic understanding and logical relationship analysis of text data, extraction of medical entities and evaluation of logical coherence include: Parse the conversation text to obtain core medical information, and use the BERT-Base model to extract medical information entities from the core medical information. Medical information entities include symptoms, medical history, and diagnosis information. The medical information entities are analyzed by dependency parsing technology and the logical coherence of the text data is evaluated.

4. The intelligent evaluation and feedback system for simulated patient teaching based on conversation analysis according to claim 1, characterized in that: The deep learning and modeling of the conversation text captures the contextual information, semantic relationships, and emotional tendencies in the conversation, enabling a comprehensive assessment of students' questioning abilities. Specifically, the following are included: The selected deep learning model is trained using a large amount of labeled teaching data, including conversation records between simulated patients and students, teacher ratings, and student self-feedback information; Optimize deep learning models through cross-validation, regularization, and dropout techniques; The feature data of the conversation text is extracted and normalized, and the normalized features are input into the optimized deep learning model to capture the contextual information, semantic relationships and emotional tendencies in the conversation, and output the evaluation results of the students' medical consultation ability, which includes clinical communication ability, disease diagnosis ability and treatment decision-making ability.

5. The intelligent evaluation and feedback system for simulated patient teaching based on conversation analysis according to claim 1, characterized in that: The output results of the comprehensive NLP analysis module and the machine learning and deep learning model module are used to score the conversation content in various dimensions according to the preset scoring criteria, including: Set corresponding medical skills teaching scoring standards and evaluation indicators based on ACGME core competencies; Through the AI ​​scoring model, the user's performance in questioning logic, medical history collection completeness, and communication skills is quantified to obtain automatic scoring results; According to the scoring calibration mechanism, the automatic scoring results are combined with the teacher's manual review and scoring results to perform a comprehensive scoring.

6. The intelligent evaluation and feedback system for simulated patient teaching based on dialogue analysis according to claim 1, characterized in that: The feedback report is presented in the form of text description and chart display. The content of the feedback report includes improvement of voice expression, optimization of consultation / assessment strategy, detailed score comparison and display of user's progress; the improvement suggestions include suggestions on logical expression, grammatical correction and emotional adjustment.

7. The intelligent evaluation and feedback system for simulated patient teaching based on conversation analysis according to claim 1, characterized in that: The database storage submodule adopts MongoDB distributed database.

8. The intelligent evaluation and feedback system for simulated patient teaching based on dialogue analysis according to claim 1, characterized in that: The teaching management submodule is also used to provide teachers with system adjustment functions, including modifying scoring standards and adding new teaching cases.

Images