Medical teaching system and method based on virtual reality technology

Abstract

The invention relates to the technical field of medical teaching, in particular to a medical teaching system and method based on a virtual reality technology. Comprising a virtual training module, an intelligent evaluation module and a dynamic teaching module. A personal action reference model of a student is established through the virtual training module, the intelligent evaluation module compares operation actions of the student and position information of a surgical instrument with the personal action reference model, and comprehensive evaluation is performed on the student by calculating a geometric similarity score and combining with time complexity of an operation sequence of the student, so that the operation accuracy of the student is improved. The dynamic teaching module is used for setting a plurality of emergency situations in an operation, adding a virtual training process of the student, capturing facial micro-expressions of the student, combining heart rate monitoring data, judging an emotional state of the student and adopting corresponding guidance and adjustment according to the emotional state of the student, so that the student can quickly adapt to the emergency situations; and the comprehensive strength of students is improved.

Description

Technical Field

[0001] This invention relates to the field of medical teaching technology, and more specifically, to a medical teaching system and method based on virtual reality technology. Background Technology

[0002] The ultimate goal of medical education is clinical practice. Traditional medical education mainly relies on printed textbooks and supplemented by hands-on teaching, which makes it difficult to provide learners with efficient guidance. This is especially true for surgical teaching, where the scarcity of resources leads to fewer learning opportunities. Existing medical schools do not have enough resources to combine theory and practice in their teaching. However, with the rapid development of virtual reality technology, the simulated environment is becoming more realistic, the perception system is becoming more sophisticated, and the user's immersive experience is becoming deeper.

[0003] Currently, medical teaching systems based on virtual reality technology have many shortcomings, such as insufficient immersion and interaction precision, inability to accurately simulate the subtle tactile sensations of surgical instruments and the physical characteristics of human tissues, reliance on subjective judgment for teaching effectiveness evaluation, and a lack of objective and real-time evaluation system. Furthermore, unexpected situations are inevitable during surgery, such as massive bleeding or a sudden drop in heart rate. To address these issues and allow for the introduction of unexpected situations during virtual learning, and to provide appropriate guidance based on the learner's emotional state, enabling them to quickly adapt to these situations and improve their overall competence, we propose a medical teaching system and method based on virtual reality technology. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of virtual reality technology in medical teaching systems, such as insufficient immersion and interaction precision, reliance on subjective judgment for teaching effectiveness evaluation, lack of an objective and real-time evaluation system, and lack of scenarios involving unexpected situations during surgery. In order to increase the number of unexpected situations during virtual learning and to take corresponding guidance measures based on the student's psychological state at the time, the invention aims to help students quickly adapt to unexpected situations and improve their overall capabilities.

[0005] To achieve the above objectives, the present invention provides a medical teaching system based on virtual reality technology, including a virtual training module, an intelligent assessment module, and a dynamic teaching module;

[0006] The virtual training module establishes a personal motion benchmark model for trainees and collects their operational movements, eye movement trajectories, and the positional information of surgical instruments during surgical procedures.

[0007] The intelligent assessment module compares the trainee's operational actions and the position information of surgical instruments with the individual action benchmark model. By calculating the geometric similarity score and combining it with the time complexity of the trainee's operation sequence, the module comprehensively assesses the trainee and determines the trainee's learning status. If the trainee does not meet the standard, the trainee returns to the virtual training module to continue training. If the trainee meets the standard, the trainee enters the dynamic teaching module to learn about unexpected situations.

[0008] The dynamic teaching module sets up multiple scenarios of unexpected situations during surgery and incorporates them into the virtual training process for trainees. By capturing the trainees' facial micro-expressions and combining them with heart rate monitoring data, the module judges the trainees' emotional state. If the trainees are confused, the module guides them through step-by-step instruction and visual guidance. If the trainees are overly nervous, the module adjusts the pressure parameters of the virtual scene to alleviate their tension. Once the trainees' state stabilizes, the difficulty level is gradually increased.

[0009] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0010] 1. This medical teaching system and method based on virtual reality technology establishes a personal action benchmark model for trainees through a virtual training module. By using eye tracking and electromyography signal acquisition, it records the trainees' eye movement trajectory, muscle exertion pattern, and action timing characteristics when performing standard operations. It also collects the trainees' operational actions, eye movement trajectory, and surgical instrument position information during surgical operations, quantifies individual operational characteristics, establishes personalized evaluation benchmarks, and achieves standardized comparison, laying the foundation for subsequent evaluation.

[0011] 2. The intelligent assessment module compares the trainee's operational actions and the position information of surgical instruments with the individual action benchmark model. By calculating the geometric similarity score and combining it with the time complexity of the trainee's operation sequence, the module conducts a comprehensive assessment of the trainee, determines the trainee's learning status, analyzes the trainee's strengths and weaknesses in different operations based on the trainee's learning status, formulates the trainee's next learning goals, provides personalized learning for the trainee, and improves learning efficiency.

[0012] 3. The dynamic teaching module sets up multiple scenarios of unexpected situations during surgery, incorporating virtual training processes for trainees. By capturing trainees' facial micro-expressions and combining them with heart rate monitoring data, the module assesses their emotional state. If a trainee is confused, the module provides step-by-step guidance and visual instruction. If a trainee is overly nervous, the module adjusts the virtual scenario's pressure parameters to alleviate their anxiety. Once the trainee's state stabilizes, the difficulty level is gradually increased. This cultivates trainees' ability to handle unexpected situations during surgery. By providing corresponding guidance and adjustments based on the trainee's emotional state, the module helps trainees quickly adapt to unexpected situations and improves their overall skills.

[0013] As a further improvement to this technical solution, the virtual training module uses eye tracking to locate the pupil and collect electromyographic signals, recording the trainee's eye movement trajectory, muscle exertion pattern, and action timing characteristics when performing standard operations, and establishing the trainee's personal action benchmark model.

[0014] As a further improvement to this technical solution, the virtual training module uses a personal action benchmark model to analyze the strengths and weaknesses of trainees in different operations, classifies trainees into talent stratification, and assists in the formulation of targeted training programs.

[0015] The beneficial effect of adopting the above-mentioned further improvements is that different trainees have natural differences in physical conditions (such as limb length and muscle strength) and operating habits. For example, trainees with longer arms may have a larger range of motion when performing a certain surgical step, and the intensity of electromyographic signals may also vary due to differences in muscle development. The benchmark model incorporates these physiological differences into the evaluation dimension by collecting data such as individual electromyographic characteristics and movement trajectories, avoiding misjudgment by using a uniform standard. For example, it will not directly judge the operation as non-standard simply because a trainee's "range of motion deviates from the standard model by 5%", but will judge whether it is a reasonable fluctuation based on their individual benchmark.

[0016] As a further improvement to this technical solution, the intelligent evaluation module compares the key points of the trainee's operating equipment trajectory, action posture vector, and spatial distance measurement with the individual benchmark model, quantifies the spatial form of the operating action, extracts geometric features, and calculates the geometric similarity score.

[0017] As a further improvement to this technical solution, the intelligent evaluation module extracts time features by segmenting the action sequence and calculating the time complexity of the student's operation sequence. It evaluates time efficiency by assessing the redundancy of the student's operation steps, the number of repeated actions, and the duration of abnormal pauses.

[0018] The beneficial effect of adopting the above-mentioned further improvements is that, in traditional teaching, teachers may judge whether the "amplitude of movement is in place" by visual inspection, but the intelligent module transforms the operation form into quantifiable geometric parameters through the coordinates of key points of the trajectory (such as the inflection point and vertex of the movement trajectory of the tip of the surgical instrument) and spatial distance measurement (such as the deviation value of the needle distance from the reference model during suturing).

[0019] By clearly identifying the specific dimensions of operational shortcomings (whether it is insufficient spatial precision or unreasonable time allocation), actionable improvement directions can be obtained. Data can replace experience, achieving an upgrade from "fuzzy evaluation" to "scientific guidance," ultimately improving the quality and efficiency of skills training. This assessment model can effectively shorten the growth cycle from "novice" to "expert," especially in fields such as medicine and industry where operational precision and timeliness are extremely important.

[0020] As a further improvement to this technical solution, the intelligent evaluation module uses a weighted fusion algorithm to calculate a comprehensive evaluation score of geometric similarity and time complexity, and uses different colors to mark the difference between the student's trajectory and the benchmark trajectory, thus visualizing the student's operational deviation.

[0021] The beneficial effect of adopting the above-mentioned further improvements is that the combination of weighted fusion algorithm and visualization technology essentially upgrades the operation evaluation from "abstract numerical judgment" to "concrete visual cognition". By visually seeing "where is wrong" and "how serious is the mistake" through color, actions can be corrected in real time, shortening the trial and error cycle. Visual tools replace tedious data explanations, allowing communication between teachers and students to focus on specific color-marked areas, thus improving guidance efficiency.

[0022] As a further improvement to this technical solution, the dynamic teaching module sets up multiple scenarios of sudden emergencies during surgery, including ruptured blood vessels and massive bleeding, organ perforation and leakage of contents, and arrhythmia and sudden changes in vital signs.

[0023] As a further improvement to this technical solution, the dynamic teaching module sets the student's confusion, tension, and focus emotional characteristics based on the student's personal action benchmark model, and correlates them with heart rate data.

[0024] As a further improvement to this technical solution, when there is a contradiction between facial expression and heart rate data, the dynamic teaching module initiates a secondary verification mechanism, triggers voice interaction inquiry, and makes auxiliary judgments based on voice tone characteristics. If the contradiction persists, the heart rate data takes priority.

[0025] The beneficial effects of adopting the above-mentioned further improvements are that emotional fluctuations during operation can be intuitively perceived through heart rate data, and students can learn how to regulate tension and confusion while maintaining focus, forming a self-regulation ability of "physiological signal-action correction". Using physiological indicators as a bridge, abstract emotional states are transformed into interventionist teaching variables, realizing three-dimensional guidance of "observing actions, measuring heart rate, and regulating emotions". Especially in high-pressure operation scenarios (such as surgery and special operations), this assessment model can effectively improve the psychological adaptability and operational stability of trainees.

[0026] Humans can actively control facial expressions, but physiological signals such as heart rate are more difficult to manipulate. When the two contradict each other, secondary verification can be performed by supplementing non-voluntary physiological signals (such as skin conductance and respiratory rate) or by combining the operational scenario (such as sudden changes in heart rate when an operational error occurs) to determine the authenticity of the emotion.

[0027] The second objective of this invention is to provide a medical teaching method based on virtual reality technology, including any one of the above-mentioned medical teaching systems based on virtual reality technology, comprising the following steps:

[0028] S1. Establish a personal motion benchmark model for trainees through the virtual training module, and collect trainees' operational movements, eye movement trajectories, and position information of surgical instruments during surgical operations;

[0029] S2. Using the intelligent assessment module, the trainee's operation actions and the position information of surgical instruments are compared with the individual action benchmark model. By calculating the geometric similarity score and combining it with the time complexity of the trainee's operation sequence, a comprehensive assessment of the trainee is conducted to determine the trainee's learning status.

[0030] S3. By setting up multiple emergency scenarios during surgery through the dynamic teaching module, and incorporating them into the virtual training process for trainees, the system can judge the trainees' emotional state by capturing their facial micro-expressions and combining them with heart rate monitoring data.

[0031] In addition to the objectives, features, and advantages described above, the present invention has other objectives, features, and advantages. The invention will now be described in further detail with reference to the figures. Attached Figure Description

[0032] Figure 1 This is an overall flowchart of the present invention;

[0033] Figure 2 This is a flowchart of the method of the present invention.

[0034] The meanings of the labels in the diagram are as follows:

[0035] 100. Virtual Training Module; 200. Intelligent Assessment Module; 300. Dynamic Teaching Module. Detailed Implementation

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] Currently, the immersion and interaction precision of virtual reality medical teaching systems are insufficient. The evaluation of teaching effectiveness relies on subjective judgment and lacks an objective, real-time evaluation system. There is also a lack of scenarios involving unexpected situations during surgery. In order to increase the number of unexpected situations during students' virtual learning and to take corresponding guidance measures based on the students' psychological state at the time, so as to help students quickly adapt to unexpected situations and improve their overall capabilities.

[0038] Therefore, this invention proposes to establish a personal action benchmark model for trainees through a virtual training module, and to compare the trainees' operational actions and the position information of surgical instruments with the personal action benchmark model through an intelligent evaluation module. By calculating the geometric similarity score and combining it with the time complexity of the trainees' operation sequence, the module comprehensively evaluates the trainees and determines their learning status. The dynamic teaching module sets up multiple scenarios of sudden situations during surgery and incorporates them into the trainees' virtual training process. By capturing the trainees' facial micro-expressions and combining them with heart rate monitoring data, the module determines the trainees' emotional state.

[0039] Specifically as follows:

[0040] like Figure 1 As shown, one of the objectives of this invention is to provide a medical teaching system based on virtual reality technology, including a virtual training module 100, an intelligent assessment module 200, and a dynamic teaching module 300.

[0041] The virtual training module 100 establishes a personal motion baseline model for trainees and collects their operational movements, eye movement trajectories, and the positional information of surgical instruments during surgical procedures.

[0042] In traditional teaching, students' actions are often compared with "standard action templates," but individual physiological differences (such as hand-eye coordination and muscle control) are ignored. Personal models can compare students' action data with their own baselines to judge operational deviations. For example, if a student's eye scan rate is 20% higher than their own baseline during suturing, it may indicate that their attention is distracted. Traditional standard templates are difficult to identify such individual fluctuations.

[0043] In order to better establish the trainee’s personal movement benchmark model, the virtual training module 100 uses eye tracking to locate pupils and collect electromyographic signals, recording the trainee’s eye movement trajectory, muscle force pattern, and movement timing characteristics when performing standard operations, and establishing the trainee’s personal movement benchmark model.

[0044] Using an infrared light source and a high frame rate camera, it supports pupil and corneal reflection (PCCR) tracking technology. Three sets of infrared LED light sources (wavelength 850nm) are set in the eye tracker, distributed in a 120-degree ring. The three-dimensional coordinates of the pupil are calculated by the position difference of the reflected light spots from different light sources.

[0045] The main light source (intensity 500 lux) is used for pupil boundary detection, and the auxiliary light source (intensity 300 lux) is used for corneal reflection point (CR point) localization;

[0046] Using 8-channel surface electromyography electrodes, which are attached to key muscle groups such as the biceps brachii and deltoid, trainees perform 3 repetitions of standard operations (such as holding a scalpel to cut), each movement lasting 5-8 seconds, with a 10-second rest interval. Electromyography signals (magnification 1000, bandpass filter 20-500Hz), IMU data (angular velocity range ±2000 degrees / s), and movement video are recorded simultaneously.

[0047] The above operations are used to collect eye-tracking data and electromyographic signal data from trainees, record the trainees' eye movement trajectory, muscle exertion patterns, and movement timing characteristics when performing standard operations, and establish a personal movement benchmark model for trainees.

[0048] In order to better identify the strengths and weaknesses of trainees, a personal action benchmark model is used to analyze the strengths and weaknesses of trainees in different operations, thereby stratifying trainees and assisting in the development of targeted training programs.

[0049] Horizontal comparison:

[0050] Operational scenarios are categorized by difficulty (basic / advanced / complex) and type (detailed operation / procedural operation / emergency handling). For example, in medical training, "suture," "hemostasis," and "instrument assembly" are considered different scenarios.

[0051] Competency Radar Chart Generation: Generate radar charts for each trainee in various scenarios, including geometric accuracy, time efficiency, and emotional stability, to visually display strengths (e.g., top 10% in geometric accuracy in "fine suturing") and weaknesses (e.g., time efficiency below the benchmark in "emergency hemostasis").

[0052] Vertical comparison:

[0053] Historical data comparison: The progress curve of the learner in the same operation is displayed through the timeline (such as the improvement of geometric similarity between the 1st and 10th operation), and learning bottlenecks are identified (such as the error rate of a certain step is always higher than the baseline value).

[0054] Based on the targeted training design for those with insufficient spatial accuracy, VR simulators are used for three-dimensional spatial positioning training (such as moving the device along a virtual trajectory and displaying the deviation value in real time).

[0055] For those with low time efficiency, process decomposition: break down complex operations into sub-steps and train the time control of each step (e.g., set the standard time for the "equipment change" step to 3 seconds and optimize it through timing training).

[0056] By quantifying differences from multiple dimensions such as geometry, time, and emotion, subjective experience biases are avoided. The tiers and plans are adjusted in real time as trainees progress, adapting to personalized growth paths. Targeted training reduces ineffective investment, making strengths more prominent and weaknesses quickly compensated for. It is especially suitable for medical fields that require high precision of movement.

[0057] In addition, the intelligent assessment module 200 compares the trainee's operation actions and the position information of surgical instruments with the individual action benchmark model. By calculating the geometric similarity score and combining it with the time complexity of the trainee's operation sequence, the module comprehensively assesses the trainee and determines the trainee's learning status. If the trainee does not meet the standard, the trainee returns to the virtual training module 100 to continue training. If the trainee meets the standard, the trainee enters the dynamic teaching module 300 to learn about unexpected situations.

[0058] In order to better extract geometric features, the intelligent evaluation module 200 compares the key points of the trainee's operating equipment trajectory, action posture vector and spatial distance measurement with the personal benchmark model, quantifies the spatial shape of the operating action, extracts geometric features, and calculates the geometric similarity score.

[0059] Key points of the instrument trajectory: The real-time coordinate sequence of the instrument during the trainee's operation is simplified into a set of key points (such as trajectory turning points and velocity change points), and compared with the key points of the benchmark model;

[0060] Action posture vector: Transforms muscle force patterns (electromyographic signals) into multidimensional vectors. For example, the electromyographic characteristics of clamp holding action can be represented as [biceps brachii signal intensity, finger extensor signal intensity, wrist flexor signal intensity];

[0061] Spatial distance measurement: Calculate the Euclidean distance between the trainee's operation trajectory and the reference trajectory to measure the geometric similarity of the trajectory (such as the degree of deviation between the stitching path and the standard path);

[0062] Trajectory similarity (40%): The Dynamic Time Warping (DTW) algorithm is used to align the trainee's equipment trajectory with the baseline trajectory in the time dimension and calculate the curve overlap.

[0063] Pose similarity (30%): The angle between the trainee's electromyographic feature vector and the reference vector is calculated using cosine similarity (the smaller the angle, the more similar the trainee).

[0064] Key point deviation (30%): Calculate the coordinate deviation of key positions (such as needle entry point) during operation. If the deviation exceeds the threshold, points will be deducted (e.g., 5 points will be deducted for every millimeter when the deviation is >2mm).

[0065] In order to better extract time feature data, the intelligent evaluation module 200 extracts time features by using the action segmentation time sequence and the time complexity of calculating the student's operation sequence. It evaluates time efficiency by the redundancy of the student's operation steps, the number of repeated actions, and the duration of abnormal pauses.

[0066] Action segmentation and timing: Decompose the operation into sub-steps (such as "disinfection → draping → incision → hemostasis"), and record the time consumed by each sub-step and the connection time between steps;

[0067] Time complexity calculation: The time efficiency is evaluated by the redundancy of the steps in the student's operation sequence, the number of repeated actions, and the duration of abnormal pauses. For example, if a student repeatedly adjusts the position of the hemostat in the hemostasis step, the time taken for this step will exceed twice that of the baseline model, and the time complexity will increase significantly.

[0068] Complexity index:

[0069] Time Deviation Rate = |Student Time - Benchmark Median| / Benchmark Median × 100%

[0070] Abnormal pause count: The number of times the operation speed is 0 and lasts for more than 0.5 seconds;

[0071] Step redundancy: The number of times the same sub-step is executed repeatedly;

[0072] In order to better calculate the comprehensive evaluation score, the intelligent evaluation module 200 uses a weighted fusion algorithm to calculate the comprehensive evaluation score of geometric similarity and time complexity, and uses different colors to mark the difference between the student's trajectory and the benchmark trajectory to visualize the student's operation deviation.

[0073] Spatial deviation visualization: In the virtual scene, use different colors to mark the difference between the trainee's trajectory and the baseline trajectory (e.g., red line segments represent paths with a deviation of >1mm), and mark the deviations at key positions;

[0074] Time Deviation Analysis: Time-series heatmaps are used to display time anomalies in each stage of the operation. For example, in the "vascular anastomosis" step, intervals where the time exceeds the baseline by 30% are highlighted in red, and the points of sudden rise in electromyographic signals are associated with these anomalies.

[0075] Targeted feedback is generated based on the evaluation results:

[0076] If the geometric similarity is low but the time complexity is normal: the message "The suture path deviates from the standard, and it is recommended to focus on practicing needle entry angle control" will be displayed.

[0077] If the time complexity is high but the geometric similarity meets the standard: the message "The efficiency of step connection is insufficient, and the rhythm can be optimized by segmented timing training" will be displayed.

[0078] By combining historical data from the individual baseline model, the trainee's current operation is compared with past performance. For example, "The similarity of the suture trajectory this time has improved by 15% compared with last week, but the time spent on the hemostasis step has increased by 20%, and the proficiency in using hemostatic forceps needs to be strengthened."

[0079] In addition, the dynamic teaching module 300 sets up multiple scenarios of unexpected situations during surgery and incorporates them into the virtual training process for trainees. By capturing the trainees' facial micro-expressions and combining them with heart rate monitoring data, the module judges the trainees' emotional state. If the trainees are confused, the module guides them through step-by-step instruction and visual guidance. If the trainees are overly nervous, the module adjusts the pressure parameters of the virtual scene to alleviate their tension. Once the trainees are in a stable state, the difficulty level is gradually increased.

[0080] In order to better set up emergency scenarios, the dynamic teaching module 300 sets up multiple emergency scenarios during surgery, including blood vessel rupture and massive bleeding, organ perforation and leakage of contents, and arrhythmia and sudden changes in vital signs.

[0081] Blood vessel rupture and massive bleeding: The fluid dynamics model is used to calculate the bleeding volume and blood pressure changes in real time (e.g., blood pressure decreases exponentially as the bleeding volume decreases). In the virtual scene, the blood jet trajectory is driven by virtual gravity and pressure field, and at the same time triggers interactive feedback such as gauze compression and hemostat clamping (pressure feedback disappears when hemostasis is successful).

[0082] Organ perforation and contents leakage: A collision detection mechanism using a physics engine is employed. When the force applied by the instrument exceeds the tissue's tolerance threshold (e.g., the tensile strength of the esophageal wall is 1.2 N / mm²), the perforation will be detected. 2 When the perforation is triggered, virtual gas / liquid (such as amniotic fluid) is released, and the corresponding odor is released through an odor simulation device;

[0083] Heart rate arrhythmia and sudden changes in vital signs: The virtual monitor (displaying parameters such as ECG, blood pressure, and SpO2) is linked with the physiological drive model. When the surgical operation triggers a stress response (such as cardiac traction), abnormal vital sign curves are generated in real time (such as ventricular premature beats on ECG). Trainees need to simultaneously administer medication and adjust the operation.

[0084] In order to better define the emotional characteristics of trainees, the dynamic teaching module 300 sets the trainees' confused, tense, and focused emotional characteristics based on their personal movement baseline models and correlates them with heart rate data.

[0085] Confused emotional characteristics: Corrugator supercilii muscle contraction (AU1+AU2) lasts for ≥10 seconds, accompanied by ptosis (AU4) and gaze deviation;

[0086] Characteristics of anxiety: pupil diameter change >20% (based on baseline), unnatural contraction of the zygomaticus major muscle;

[0087] Focus on emotional characteristics: raised eyebrows (AU5), decreased blinking frequency (<15 times / minute);

[0088] After the trainee puts on the device, the system automatically collects the heart rate data of the first 3 minutes at rest and calculates the mean baseline heart rate (HR) and the standard deviation of HRV (SD).

[0089] Confusion state: Heart rate fluctuation range <10%HR, but HRV decrease >30%SD (reflecting increased cognitive load);

[0090] Stressful state: Heart rate >120 beats / minute or >130% HR, with SD <50ms (reflecting autonomic nervous system disorder);

[0091] In order to better handle the situation where facial expressions and heart rate data conflict, the dynamic teaching module 300 activates a secondary verification mechanism when there is a conflict between facial expressions and heart rate data. It triggers a voice interaction inquiry and uses voice tone features to make an auxiliary judgment. If the conflict persists, the heart rate data takes priority.

[0092] When there is a discrepancy between facial expression and heart rate data, subjective feedback from the learner is obtained through voice interaction. If the learner responds effectively within 10 seconds (voice clarity > 80%), tone feature analysis is initiated. If there is no response or the response is invalid, the interface text prompts and voice repeats the question (up to 3 times) to avoid misjudgment due to environmental noise.

[0093] Heart rate, as an objective response of the autonomic nervous system, is regulated by the subconscious and is difficult to actively fake. Facial expressions, on the other hand, may exhibit "performative expressions" due to subjective factors such as social pressure and deliberate concealment. For example, a participant might force themselves to smile to avoid being perceived as "nervous," but their elevated heart rate reveals their true stress response.

[0094] The second objective of this invention is to provide a medical teaching method based on virtual reality technology, including any of the above-mentioned medical teaching systems based on virtual reality technology, such as... Figure 2 As shown, it includes the following steps:

[0095] S1. Establish a personal motion benchmark model for trainees through the virtual training module 100, and collect trainees' operational actions, eye movement trajectories, and position information of surgical instruments during surgical operations;

[0096] S2. Using the intelligent evaluation module 200, the trainee's operation actions and the position information of the surgical instruments are compared with the individual action benchmark model. By calculating the geometric similarity score and combining it with the time complexity of the trainee's operation sequence, a comprehensive evaluation of the trainee is conducted to determine the trainee's learning status.

[0097] S3. The dynamic teaching module 300 sets up multiple scenarios of unexpected situations during surgery and incorporates them into the virtual training process for trainees. By capturing the trainees' facial micro-expressions and combining them with heart rate monitoring data, the system can determine the trainees' emotional state.

[0098] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A medical teaching system based on virtual reality technology, characterized in that: It includes a virtual training module (100), an intelligent evaluation module (200), and a dynamic teaching module (300); The virtual training module (100) establishes a personal action benchmark model for trainees and collects the trainees' operational actions, eye movement trajectories, and position information of surgical instruments during surgical procedures. The intelligent evaluation module (200) compares the student's operation actions and the position information of the surgical instruments with the personal action benchmark model. By calculating the geometric similarity score and combining the time complexity of the student's operation sequence, it conducts a comprehensive evaluation of the student and determines the student's learning status. If the student does not meet the standard, it returns to the virtual training module (100) to continue training. If the student meets the standard, it enters the dynamic teaching module (300) to learn about emergencies. The dynamic teaching module (300) sets up multiple scenarios of sudden situations during surgery and incorporates them into the virtual training process of trainees. By capturing the trainees' facial micro-expressions and combining them with heart rate monitoring data, the module judges the trainees' emotional state. If the trainees are confused, the module guides them through step-by-step instruction and visual guidance. If the trainees are overly nervous, the module adjusts the virtual scene pressure parameters to alleviate their tension. Once the trainees' state stabilizes, the difficulty is gradually increased.

2. The medical teaching system based on virtual reality technology according to claim 1, characterized in that: The virtual training module (100) uses eye tracking to locate pupils and collect electromyographic signals, recording the trainee's eye movement trajectory, muscle exertion pattern, and action timing characteristics when performing standard operations, and establishing the trainee's personal action benchmark model.

3. The medical teaching system based on virtual reality technology according to claim 2, characterized in that: The virtual training module (100) uses a personal action benchmark model to analyze the strengths and weaknesses of trainees in different operations, classifies trainees into talent stratification, and assists in the formulation of targeted training programs.

4. The medical teaching system based on virtual reality technology according to claim 1, characterized in that: The intelligent evaluation module (200) compares the key points of the trainee's operating equipment trajectory, action posture vector and spatial distance measurement with the personal benchmark model, quantifies the spatial form of the operating action, extracts geometric features, and calculates the geometric similarity score.

5. The medical teaching system based on virtual reality technology according to claim 1, characterized in that: The intelligent evaluation module (200) extracts time features by segmenting action timing and calculating the time complexity of student operation sequences, and evaluates time efficiency by the redundancy of student operation steps, the number of repeated actions, and the duration of abnormal pauses.

6. The medical teaching system based on virtual reality technology according to claim 1, characterized in that: The intelligent evaluation module (200) uses a weighted fusion algorithm to calculate a comprehensive evaluation score of geometric similarity and time complexity, and uses different colors to mark the difference between the student's trajectory and the baseline trajectory, thus visualizing the student's operational deviation.

7. The medical teaching system based on virtual reality technology according to claim 1, characterized in that: The dynamic teaching module (300) sets up multiple emergency scenarios during surgery, including ruptured blood vessels and massive bleeding, organ perforation and leakage of contents, and arrhythmia and sudden changes in vital signs.

8. The medical teaching system based on virtual reality technology according to claim 1, characterized in that: The dynamic teaching module (300) sets the student's confused emotional characteristics, tense emotional characteristics, and focused emotional characteristics based on the student's personal movement benchmark model, and associates them with heart rate data.

9. The medical teaching system based on virtual reality technology according to claim 8, characterized in that: When there is a contradiction between facial expression and heart rate data, the dynamic teaching module (300) initiates a secondary verification mechanism, triggers voice interaction inquiry, and makes auxiliary judgments based on voice tone characteristics. If the contradiction persists, the heart rate data takes priority.

10. A method for implementing medical teaching based on virtual reality technology, comprising the medical teaching system based on virtual reality technology as described in any one of claims 1-9, characterized in that: Includes the following steps: S1. Establish a personal action benchmark model for trainees through the virtual training module (100) and collect the trainees' operation actions, eye movement trajectories and position information of surgical instruments during the surgical operation; S2. Using the intelligent evaluation module (200), the student's operation actions and the position information of the surgical instruments are compared with the individual action benchmark model. By calculating the geometric similarity score and combining it with the time complexity of the student's operation sequence, the student is comprehensively evaluated and the student's learning status is determined. S3. Multiple emergency scenarios during surgery are set up through the dynamic teaching module (300), and the virtual training process of the trainees is added. By capturing the trainees' facial micro-expressions and combining them with heart rate monitoring data, the trainees' emotional state is judged.

Images