Virtual standardized patient construction method based on multi-modal large model

By constructing a virtual standardized patient through a multimodal large model, the problems of single case generation, insufficient emotional simulation and poor cross-department adaptability in the existing system are solved. Dynamic case generation and emotional simulation are realized, and the multidisciplinary applicability and authenticity of teaching are improved.

CN120705466APending Publication Date: 2025-09-26WEST CHINA HOSPITAL SICHUAN UNIV
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202510580411.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-05-07
Publication Date
2025-09-26

AI Technical Summary

Technical Problem

The existing virtual standardized patient system has difficulty in dynamically generating complex cases, has limited emotional simulation capabilities, and has weak cross-department adaptability, and cannot meet the needs of multidisciplinary and multi-stage teaching.

Method used

A multimodal large model is used to construct a virtual standardized patient. Through multimodal data preprocessing, an interdisciplinary adaptation framework and a hybrid expert model architecture, a virtual standardized patient with emotions is generated. Combined with the large model inference engine for dynamic interactive response, dynamic adjustment of emotional intensity and interdisciplinary adaptation are achieved.

Benefits of technology

The virtual standardized patient system can be dynamically adjusted based on real-time interaction, has a complex multimodal database and emotional characteristics, and is cross-disciplinary. It can comprehensively assess students and improve the timeliness and authenticity of teaching.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120705466A_ABST
    Figure CN120705466A_ABST
Patent Text Reader

Abstract

The invention relates to the related field of medical education, in particular to a virtual standardized patient construction method based on a multi-modal large model. Comprising a multi-modal data preprocessing module, an interdisciplinary adaptation framework, a hybrid expert model architecture and a dynamic interaction response system. The hybrid expert model architecture performs fusion reasoning on a multi-modal case library of the multi-modal data preprocessing module and scene parameters of the interdisciplinary adaptation framework to generate a virtual standardized patient model with emotions; the dynamic interactive response system asks questions for students according to the virtual standardized patient model of the hybrid expert model, and identifies answers and interactions of the students; the answers and interaction data of the students are transmitted to the virtual standardized patient model of the mixed expert model architecture for reasoning, and the students are further questioned according to the reasoning result of the mixed expert model architecture until the examination is finished; the virtual standardized patient generated by the method has a real emotional state of the patient, interaction is more real, and interaction which better conforms to character features is carried out according to reasoning of student questions.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and medical education technology, and in particular to a method for constructing a virtual standardized patient based on a multimodal large model. Background Art

[0002] In traditional medical education, standardized patients (SP) provide medical students with clinical practice opportunities by playing the role of patients or family members through real-life roles; however, real-life SPs have problems such as high training costs, poor repeatability, and strong scenario limitations; with the development of artificial intelligence technology, virtual standardized patients (VSP) have gradually become an alternative; existing VSP systems are mostly based on rule engines or small-scale knowledge bases, which make it difficult to dynamically generate complex cases, and their emotional simulation capabilities are limited, and they cannot meet the needs of multidisciplinary and multi-stage teaching; for example, the standardized patient in patent CN202010639937.9 has a single disease evolution path: it relies on a preset case library and cannot be dynamically adjusted according to real-time interaction; and the emotional simulation dimension is insufficient: it only supports basic emotional labels, and lacks delicate physiological and psychological reaction linkage; cross-departmental adaptability is weak: the teaching needs of different departments (such as anxious family members in pediatrics and chronic disease patients in geriatrics) require independent development of models, and the reusability is poor. Summary of the Invention

[0003] The present invention aims to provide a method for constructing a virtual standardized patient based on a multimodal large model, which is characterized by comprising:

[0004] Multimodal data preprocessing module: integrates clinical data, medical knowledge graph, and case emotional characteristics to generate a multimodal case database;

[0005] Interdisciplinary Adaptive Framework: Configure the department type, case difficulty, and emotional intensity of the teaching that requires assessment, and generate assessment scenario parameters;

[0006] Hybrid Expert Model Architecture: This architecture matches the scenario parameters of the multimodal case database and the interdisciplinary adaptation framework to generate emotionally engaged virtual standardized patients. Combined with a large model inference engine, it questions students and infers their responses.

[0007] Dynamic Interactive Response System: Based on the hybrid expert model reasoning, the system asks students questions with set emotional intensity, identifies their answers and interactions, and transmits their answers and interaction data to the virtual standardized patient model of the hybrid expert model architecture for reasoning;

[0008] The hybrid expert model architecture integrates and infers the multimodal case library of the multimodal data preprocessing module and the scenario parameters of the interdisciplinary adaptation framework to generate a virtual standardized patient model with emotions; the dynamic interactive response system generates a digital human image with strong emotions based on the virtual standardized patient model of the hybrid expert model to ask questions to students and recognize the students' answers and interactions; the students' answers and interaction data are transmitted to the virtual standardized patient model of the hybrid expert model architecture for reasoning, and further questions are asked to the students based on the reasoning results of the hybrid expert model architecture until the end of the assessment.

[0009] Optionally, the multimodal data preprocessing module specifically integrates the following steps:

[0010] S10. Clinical data collection: Integrate structured data from electronic medical records, imaging reports, and treatment guidelines, combined with unstructured data from interview recordings and physical examination videos, to construct a multimodal case corpus;

[0011] S11. Knowledge Graph Construction: Identify a multimodal case corpus and, based on standard terminology in the medical language system, map the data from the multimodal case corpus into a relational graph of disease-symptom list-vital signs-treatment plan-department type-case difficulty. This knowledge graph is then constructed to support complex logical reasoning.

[0012] S12. Case emotional feature annotation: Introducing a psychological scale to grade the emotional states of the symptom list in the multimodal case corpus, and combining it with physiological signal data of physical signs to construct a multi-dimensional emotional feature library;

[0013] S13. Multimodal case database: The multimodal case corpus, knowledge graph, and multi-dimensional emotional feature library collected from clinical data are merged and integrated to form a multimodal case database.

[0014] Optionally, the specific configuration steps of the interdisciplinary adaptation framework are as follows:

[0015] S20. Scenario Configuration: The teacher sets parameters for department type, case difficulty, and emotional intensity based on the teaching stage requirements, and generates scenario parameters for the virtual patient based on the set parameters.

[0016] Optionally, the specific steps for building the hybrid expert model architecture are as follows:

[0017] S30. Match the scenario parameters of the interdisciplinary adaptation framework with the multimodal case database of the multimodal data preprocessing module; generate a virtual standardized patient with emotions through the large model tool;

[0018] S31. The virtual standardized patient generates a basic case and initial state based on the dynamic case generation logic; sends it to the dynamic interactive response system and asks questions to the student;

[0019] S32. Based on the students' responses and interactions to the questions in the dynamic interactive response system, the virtual standardized patient makes inferences, generates further instructions, and sends them to the dynamic interactive response system.

[0020] Optionally, the specific steps of the dynamic interactive response system are as follows:

[0021] S40. Display the questions of the virtual standardized patient in step S31 to the students for them to answer;

[0022] S41. Recognize the student's voice, image, and video responses and transmit them to a virtual standardized patient for reasoning.

[0023] S42. Send the further instructions of the virtual standardized patient in step S32 to the student for response, and repeat step S41 until the response is completed.

[0024] Optionally, the hybrid expert model architecture also includes an incremental learning mechanism, which combines the scenario parameters of the virtual patient with the case data of different external medical institutions with privacy removed, thereby improving the simulation capabilities of rare diseases and complex cases.

[0025] Optionally, the dynamic interactive response system further includes an emotion expression module; the instructions of the virtual standardized patient with emotions generated by the emotion computing model are converted into a digital human image with emotional voice, facial expressions and physiological signals.

[0026] Optionally, the dynamic interactive response system also includes a physical examination linkage mechanism; the student's response and interaction also includes the operation signal of the simulation teaching equipment, which identifies the student's operation signal of the simulation teaching equipment and sends it to the virtual standardized patient for judgment.

[0027] Optionally, the specific steps of the dynamic case generation logic are as follows:

[0028] S311: Matching scenario parameters with data in the multimodal case database; generating basic cases;

[0029] S312: Initialize the condition status: check the virtual standardized patient's vital signs, symptom list, and emotional state; and select triggering complication types and aggravating conditions based on difficulty;

[0030] S313: Output the basic case and initial status and ask questions to students;

[0031] Optionally, the emotion expression module has the following specific steps:

[0032] Emotional Expression Module: Based on the emotional intensity set by the interdisciplinary adaptation framework, the set emotional intensity is simulated by a speech synthesis module, a facial expression generation module, and a physiological signal simulation module. The speech synthesis module adjusts the speech rate and fundamental frequency according to the emotional intensity parameter to generate a speech output signal. The facial expression generation module adjusts the 3D facial mesh and expression coefficients according to the emotional intensity parameter to generate a digital human facial output signal. The physiological signal simulation module adjusts the respiratory rate and heart rate according to the emotional intensity parameter to generate a physiological signal output signal. The speech synthesis module, facial expression generation module, and physiological signal simulation module are integrated to simulate a digital human with emotions.

[0033] The technical solutions provided by the embodiments of the present application may have the following beneficial effects:

[0034] 1. The virtual standardized patient constructed by the present invention can be constructed based on preset cases and knowledge graphs, with a more complex multimodal database, more realistic emotional characteristic simulation, and the assessment is more in line with actual conditions.

[0035] 2. The present invention can be configured across disciplines and, based on large-scale model reasoning, can provide a more comprehensive assessment of students.

[0036] 3. The incremental learning mechanism of the present invention can be adjusted dynamically in real time, rather than relying solely on a preset case library, and is more timely. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 This is a framework flow chart of a method for constructing a virtual standardized patient based on a multimodal large model according to the present invention;

[0039] Figure 2 This is a flow chart of a multimodal data preprocessing module of a method for constructing a virtual standardized patient based on a multimodal large model of the present invention;

[0040] Figure 3 A flow chart of a hybrid expert model architecture for a method for constructing a virtual standardized patient based on a multimodal large model according to the present invention;

[0041] Figure 4 A flow chart of a dynamic interactive response system for a method of constructing a virtual standardized patient based on a multimodal large model according to the present invention;

[0042] Figure 5 A dynamic case generation logic flow chart for a method for constructing a virtual standardized patient based on a multimodal large model according to the present invention;

[0043] Figure 6 This is a flow chart of the emotion expression module of the method for constructing a virtual standardized patient based on a multimodal large model of the present invention. DETAILED DESCRIPTION

[0044] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0045] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0046] The present invention is further described below with reference to specific embodiments.

[0047] like Figure 1-6 This is a flow chart of the main body of the method for constructing a virtual standardized patient based on a multimodal large model of the present invention; Figure 1-6 As shown, a virtual standardized patient based on a multimodal large model includes the following:

[0048] Multimodal data preprocessing module: integrates clinical data, medical knowledge graph, and case emotional characteristics to generate a multimodal case database;

[0049] Interdisciplinary Adaptive Framework: Configure the department type, case difficulty, and emotional intensity of the teaching that requires assessment, and generate assessment scenario parameters;

[0050] Hybrid Expert Model Architecture: This architecture matches the scenario parameters of the multimodal case database and the interdisciplinary adaptation framework to generate emotionally engaged virtual standardized patients. Combined with a large model inference engine, it questions students and infers their answers.

[0051] Dynamic Interactive Response System: Based on the hybrid expert model reasoning, the system asks students questions with set emotional intensity, identifies their answers and interactions, and transmits their answers and interaction data to the virtual standardized patient model of the hybrid expert model architecture for reasoning;

[0052] The hybrid expert model architecture integrates and infers the multimodal case library of the multimodal data preprocessing module and the scenario parameters of the interdisciplinary adaptation framework to generate a virtual standardized patient model with emotions; the dynamic interactive response system generates a digital human image with strong emotions based on the virtual standardized patient model of the hybrid expert model to ask questions to students and recognize the students' answers and interactions; the students' answers and interaction data are transmitted to the virtual standardized patient model of the hybrid expert model architecture for reasoning, and further questions are asked to the students based on the reasoning results of the hybrid expert model architecture until the end of the assessment.

[0053] In the method provided in the embodiment of the present application, a virtual standardized patient with emotional characteristics is constructed, which can not only rely on a preset multimodal case library, but also is connected to a large model reasoning engine such as DeepSeek-R1, which can reason about students' questions; it can understand students' questions and conduct more realistic questions and answers in a targeted manner, rather than relying on a fixed answer library; it is closer to the actual medical consultation situation, and the emotional intensity and department type are set in a targeted manner, so that the questioning process is more in line with the patient's emotional characteristics such as anxiety and impatience, enriching students' real experience.

[0054] Specifically, the multimodal data preprocessing module has the following fusion steps:

[0055] S10. Clinical data collection: Integrate structured data from electronic medical records, imaging reports, and treatment guidelines, combined with unstructured data from interview recordings and physical examination videos, to construct a multimodal case corpus;

[0056] S11. Knowledge Graph Construction: Identify a multimodal case corpus and, based on standard terminology in the medical language system, map the data from the multimodal case corpus into a relational graph of disease-symptom list-vital signs-treatment plan-department type-case difficulty. This knowledge graph is then constructed to support complex logical reasoning.

[0057] S12. Case emotional feature annotation: Introducing a psychological scale to grade the emotional states of the symptom list in the multimodal case corpus, and combining it with physiological signal data of physical signs to construct a multi-dimensional emotional feature library;

[0058] S13. Multimodal case database: The multimodal case corpus, knowledge graph, and multi-dimensional emotional feature library collected from clinical data are merged and integrated to form a multimodal case database.

[0059] It should be noted that the construction of sentiment features and knowledge graphs in the multimodal data preprocessing module can manually label the multimodal case corpus in the early stage, and then use the large model tool to simulate the labeled part. After the simulation training is completed, the tool completed by the simulation training will automatically label the multimodal case corpus, which is more intelligent and efficient than the preset corpus.

[0060] Specifically, the specific configuration steps of the interdisciplinary adaptation framework are as follows:

[0061] S20. Scenario Configuration: The teacher sets scenario parameters such as department type, case difficulty, and emotional intensity based on the teaching stage requirements, and generates scenario parameters for the virtual patient based on the set parameters;

[0062] Specifically, the hybrid expert model architecture has the following fusion steps:

[0063] S30. Match the scenario parameters of the interdisciplinary adaptation framework with the multimodal case database of the multimodal data preprocessing module; generate a virtual standardized patient with emotions through the large model tool;

[0064] S31. The virtual standardized patient generates a basic case and initial state based on the dynamic case generation logic; sends it to the dynamic interactive response system and asks questions to the student;

[0065] S32. Based on the students' responses and interactions to the questions in the dynamic interactive response system, the virtual standardized patient makes inferences, generates further instructions, and sends them to the dynamic interactive response system.

[0066] Specifically, the specific steps of the dynamic interactive response system are as follows:

[0067] S40. Display the questions of the virtual standardized patient in step S31 to the students for them to answer;

[0068] S41. Recognize the student's voice, image, and video responses and transmit them to a virtual standardized patient for reasoning.

[0069] S42. Send the further instructions of the virtual standardized patient in step S32 to the student for response, and repeat step S41 until the response is completed.

[0070] It should be noted that the virtual standardized patient with emotions of the present invention can selectively answer students' questions according to the set emotions during the interaction with students; and urge students to interact. The virtual standardized patient with emotions is more consistent with the patient's condition in reality.

[0071] Specifically, the hybrid expert model architecture also includes an incremental learning mechanism that combines scenario parameters of virtual patients with external case data from different medical institutions, removing privacy information, to enhance simulation capabilities for rare and complex diseases. Because each hospital has limited data on rare and complex diseases, the incremental learning mechanism allows for combined training with data from more institutions, increasing students' learning efficiency for rare and uncommon diseases.

[0072] Specifically, the dynamic interactive response system also includes an emotion expression module; the instructions of the virtual standardized patient with emotions generated by the emotion computing model are converted into a digital human image with emotional voice, facial expressions and physiological signals.

[0073] Specifically, the dynamic interactive response system also includes a physical examination linkage mechanism; students' responses and interactions also include operating signals from the simulation teaching equipment, which are recognized and sent to the virtual standardized patient for assessment. This not only allows for assessment of students' questioning skills, but also their operational skills.

[0074] Specifically, the steps of the dynamic case generation logic are as follows:

[0075] S311: Matching scenario parameters with data in the multimodal case database; generating basic cases;

[0076] S312: Initialize the condition status: check the virtual standardized patient's vital signs, symptom list, and emotional state; and select triggering complication types and aggravating conditions based on difficulty;

[0077] S313: Output the basic case and initial status and ask questions to students;

[0078] Specifically, the emotion expression module has the following steps:

[0079] Emotional expression module: According to the emotional intensity set by the interdisciplinary adaptation framework, the set emotional intensity is divided into a speech synthesis module, a facial expression generation module, and a physiological signal simulation module for simulation; the speech synthesis module adjusts the speech rate and speech fundamental frequency according to the emotional intensity parameter to generate a speech output signal; the facial expression generation module adjusts the 3D facial mesh and expression coefficient according to the emotional intensity parameter to generate a digital human facial output signal; the physiological signal simulation module adjusts the breathing frequency and heart rate according to the emotional intensity parameter; generates a physiological signal output signal; the speech synthesis module, facial expression generation module, and physiological signal simulation module are integrated to simulate a digital human image with emotions.

[0080] It should be noted that the speech synthesis module is based on the Tacotron2 model, and combines the emotional parameters fundamental frequency and speaking rate to generate speech that conforms to the emotional state; the digital human rendering of the facial expression generation module: generates 3D facial expressions through the 3D deformation model 3DMM of the Blanz-Vetter model, and maps the emotional calculation results in real time; the physiological signal simulation module performs simulation: adjusts the virtual patient's vital sign parameters such as respiratory rate and heart rate according to the emotion type.

[0081] For example, the teacher sets the assessment as: pediatric consultation, the case difficulty is a moderate cold, and the emotional characteristics are an anxious and impatient mother;

[0082] According to the set scenario parameters and the relevant case corpus of the multimodal data preprocessing module, a virtual standardized patient with the image of a mother with a fast speaking speed and an anxious face is generated.

[0083] Virtual standardized patients were extracted to generate base cases, with higher body temperatures;

[0084] Student question: "How many days has the child had this symptom?"

[0085] The virtual standardized patient system triggers a knowledge graph query and generates an answer based on the case: "Body temperature has been elevated for two days." The virtual standardized patient is currently experiencing accelerated speech and red eyes.

[0086] The student's body temperature was measured at 39°C through the physical examination linkage mechanism;

[0087] The mother of the virtual standardized patient urged the students to proceed to the next step of the examination.

[0088] The student verbally comforts the virtual standardized patient, and the student's lungs and psychology are listened to; and further diagnosis is performed.

[0089] The collected voice and operations of students’ responses are automatically evaluated, and the data is saved for teachers to review in the background and make targeted suggestions.

[0090] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0091] Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for constructing a virtual standardized patient based on a multimodal large model, characterized in that: include: Multimodal data preprocessing module: integrates clinical data, medical knowledge graph, and case emotional characteristics to generate a multimodal case database; Interdisciplinary Adaptive Framework: Configure the department type, case difficulty, and emotional intensity of the teaching that requires assessment, and generate assessment scenario parameters; Hybrid Expert Model Architecture: This architecture matches the scenario parameters of the multimodal case database and the interdisciplinary adaptation framework to generate emotionally engaged virtual standardized patients. Combined with a large model inference engine, it questions students and infers their responses. Dynamic Interactive Response System: Based on the hybrid expert model reasoning, the system asks students questions with set emotional intensity, identifies their answers and interactions, and transmits their answers and interaction data to the virtual standardized patient model of the hybrid expert model architecture for reasoning; The hybrid expert model architecture integrates and infers the multimodal case library of the multimodal data preprocessing module and the scenario parameters of the interdisciplinary adaptation framework to generate a virtual standardized patient model with emotions; the dynamic interactive response system generates a digital human image with strong emotions based on the virtual standardized patient model of the hybrid expert model to ask questions to students and recognize the students' answers and interactions; the students' answers and interaction data are transmitted to the virtual standardized patient model of the hybrid expert model architecture for reasoning, and further questions are asked to the students based on the reasoning results of the hybrid expert model architecture until the end of the assessment.

2. A method for constructing a virtual standardized patient based on a multimodal large model according to claim 1, characterized in that The specific fusion steps of the multimodal data preprocessing module are as follows: S10. Clinical data collection: Integrate structured data from electronic medical records, imaging reports, and treatment guidelines, combined with unstructured data from interview recordings and physical examination videos, to construct a multimodal case corpus; S11. Knowledge Graph Construction: Identify a multimodal case corpus and, based on standard terminology in the medical language system, map the data from the multimodal case corpus into a relational graph of disease-symptom list-vital signs-treatment plan-department type-case difficulty. This knowledge graph is then constructed to support complex logical reasoning. S12. Case emotional feature annotation: Introducing a psychological scale to grade the emotional states of the symptom list in the multimodal case corpus, and combining it with physiological signal data of physical signs to construct a multi-dimensional emotional feature library; S13. Multimodal case database: The multimodal case corpus, knowledge graph, and multi-dimensional emotional feature library collected from clinical data are merged and integrated to form a multimodal case database.

3. A method for constructing a virtual standardized patient based on a multimodal large model according to claim 1, characterized in that The specific configuration steps of the interdisciplinary adaptation framework are as follows: S20. Scenario configuration: The teacher sets the parameters of department type, case difficulty, and emotional intensity according to the needs of the teaching stage, and generates scenario parameters of the virtual patient based on the set parameters.

4. A method for constructing a virtual standardized patient based on a multimodal large model according to claim 1, characterized in that The specific steps of building the hybrid expert model architecture are as follows: S30. Match the scenario parameters of the interdisciplinary adaptation framework with the multimodal case database of the multimodal data preprocessing module; generate a virtual standardized patient with emotions through the large model tool; S31. The virtual standardized patient generates a basic case and initial state based on the dynamic case generation logic; sends it to the dynamic interactive response system and asks questions to the student; S32. Based on the responses and interactions of the students to the questions in the dynamic interactive response system, the virtual standardized patient makes inferences, generates further instructions, and sends them to the dynamic interactive response system.

5. A method for constructing a virtual standardized patient based on a multimodal large model according to claim 4, characterized in that The specific steps of the dynamic interactive response system are as follows: S40. Display the questions of the virtual standardized patient in step S31 to the students for them to answer; S41. Recognize the student's voice, image, and video responses and transmit them to a virtual standardized patient for reasoning. S42. Send further instructions of the virtual standardized patient in step S32 to the student for response, and repeat step S41 until the response is completed.

6. A method for constructing a virtual standardized patient based on a multimodal large model according to claim 3, characterized in that : The hybrid expert model architecture also includes an incremental learning mechanism, which combines the scenario parameters of virtual patients with case data from different external medical institutions with privacy removed, thereby improving the simulation capabilities of rare diseases and complex cases.

7. The method for constructing a virtual standardized patient based on a multimodal large model according to claim 5, characterized in that :The dynamic interactive response system also includes an emotional expression module; the instructions of the virtual standardized patient with emotions generated by the emotional computing model are converted into a digital human image with emotional voice, facial expressions and physiological signals.

8. The method for constructing a virtual standardized patient based on a multimodal large model according to claim 5, characterized in that :The dynamic interactive response system also includes a physical examination linkage mechanism; the students' responses and interactions also include the operation signals of the simulation teaching equipment, which identify the students' operation signals of the simulation teaching equipment and send them to the virtual standardized patients for judgment.

9. The method for constructing a virtual standardized patient based on a multimodal large model according to claim 4, characterized in that The specific steps of the dynamic case generation logic are as follows: S311: Matching scenario parameters with data in the multimodal case database; generating basic cases; S312: Initialize the condition status: check the virtual standardized patient's vital signs, symptom list, and emotional state; and select triggering complication types and aggravating conditions based on difficulty; S313: Output basic cases and initial conditions and ask students questions.

10. A method for constructing a virtual standardized patient based on a multimodal large model according to claim 7, characterized in that The specific steps of the emotion expression module are as follows: Emotional expression module: According to the emotional intensity set by the interdisciplinary adaptation framework, the set emotional intensity is divided into a speech synthesis module, a facial expression generation module, and a physiological signal simulation module for simulation; the speech synthesis module adjusts the speech rate and speech fundamental frequency according to the emotional intensity parameter to generate a speech output signal; the facial expression generation module adjusts the 3D facial mesh and expression coefficient according to the emotional intensity parameter to generate a digital human facial output signal; the physiological signal simulation module adjusts the breathing frequency and heart rate according to the emotional intensity parameter to generate a physiological signal output signal; the speech synthesis module, the facial expression generation module, and the physiological signal simulation module are integrated to simulate a digital human image with emotions.

Citation Information

Patent Citations

  • Inquiry training evaluation method and device based on standardized patients

    CN111861161A