Man-machine cooperative medical teaching interaction method and system
Through the human-machine collaborative medical teaching interaction method, virtual reality and deep learning are used to solve the problems of insufficient practical opportunities and insufficient emergency response training in traditional medical teaching methods, personalized teaching and emergency response capabilities are improved, and a teaching closed loop is formed. Continuously optimize teaching results.
Patent Information
- Application Number
- CN202411449183.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-17
- Publication Date
- 2024-11-15
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Traditional medical teaching methods have problems such as insufficient practical opportunities, uneven teaching resources, and difficulty in real-time feedback. Especially in complex medical operations and emergency response training, it is difficult to meet the personalized needs of learners and improve emergency response capabilities.
Adopt human-machine collaborative medical teaching interaction method, collect data through sensors, combine virtual reality and augmented reality technology, introduce high-precision biomechanical models, use deep learning models to build situational operation simulation, provide real-time feedback and customized suggestions, and dynamically adjust teaching Content and difficulty, supporting remote collaboration and supervision.
Personalized teaching has been realized, learners' emergency response ability and the pertinence and effectiveness of complex medical operations have been improved, and a closed-loop teaching has been formed, continuously optimized teaching content, and adapted to the needs of different learners.
Smart Images

Figure CN118969227A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of human-computer interaction technology, and in particular to a human-computer collaborative medical teaching interaction method and system. Background Art
[0002] With the continuous development of medical technology, medical education and training are also constantly improving. However, traditional medical teaching methods have some limitations, such as insufficient practice opportunities, uneven distribution of teaching resources, and difficulty in real-time feedback. These problems have greatly affected the effectiveness of medical education, especially in the training of high-end medical skills. In recent years, the rapid development of human-computer collaborative technology has brought new opportunities to the field of medical teaching. By introducing artificial intelligence and virtual reality technology, it can provide medical students and doctors with a more interactive and immersive learning experience.
[0003] However, the inventors have discovered that this technical solution still has at least the following defects: First, existing medical teaching interaction systems usually rely on static teaching content and a single feedback mechanism, which makes it difficult to meet the learners' personalized needs in complex medical operations.
[0004] Second, the traditional system is insufficient in emergency response capability training and cannot effectively simulate emergencies in real operations, limiting learners' practical ability in emergency response. Summary of the invention
[0005] Based on the above objectives, the present invention provides a human-computer collaborative medical teaching interaction method and system.
[0006] The human-computer collaborative medical teaching interaction method includes the following steps: S1, data collection: collect learners’ physiological data, operational data and environmental data through sensors and medical devices, and transmit them to the central processing unit, which performs preliminary filtering and standardization on the data; S2, virtual simulation and interaction: combining virtual reality and augmented reality technologies to provide an immersive teaching environment, introducing high-precision biomechanical models to simulate the real physical properties of human tissues. When learners operate in a virtual environment, the biomechanical models provide real-time feedback and adjust the tissue response in the virtual environment; S3, deep learning-driven situational operation simulation and emergency response assessment: Use deep learning models to build situational operation simulations, introduce emergency situations into the teaching process, evaluate learners' emergency response capabilities, and provide customized improvement suggestions; S4, intelligent analysis and evaluation: The central processing unit conducts in-depth analysis of the preliminarily processed data, evaluates the learner's operation accuracy, speed and robustness, and generates a personalized evaluation report based on historical data; S5, interactive feedback and guidance: Based on intelligent analysis and evaluation results, multi-level feedback is generated to provide operation suggestions, error correction and psychological state adjustment in visual, auditory and tactile forms, and dynamically adjust teaching content and difficulty according to learners' real-time performance; S6, data reflow and optimization: After learners complete training, all interaction data, feedback data and performance data are stored in the central database. The database data is analyzed regularly to identify common problems and personalized needs and optimize teaching content; S7, Remote Collaboration and Supervision: Supports remote teaching and collaboration, allowing medical experts to supervise and guide learners' operations in real time through remote connections, and dynamically adjust teaching strategies based on learners' performance.
[0007] Optionally, S1 includes: S11, physiological data collection: collect physiological data through biosensors installed on the learner, including heart rate, blood pressure and respiratory rate, and transmit the data to the central processing unit in real time; S12, operation data collection: using operation sensors installed on teaching equipment and instruments to collect operation data, and transmitting the collected data to a central processing unit; S13, environmental data collection: monitoring environmental data through environmental sensors, collecting environmental data in real time and transmitting it to the central processing unit; S14, data processing: The central processing unit performs preliminary filtering on the collected physiological data, operation data and environmental data, and performs standardization on the data.
[0008] Optionally, S2 includes: S21, virtual teaching environment creation: first, virtual reality and augmented reality technologies are combined to generate a highly simulated immersive virtual teaching environment. Learners enter the virtual environment by wearing virtual reality head-mounted display devices or using augmented reality glasses; S22, Introduction of biomechanical model: Introduce biomechanical models based on real anatomical data and biomechanical principles in the virtual teaching environment, introduce different layers of human tissue, and simulate the real physical properties of tissue. The biomechanical model includes the dynamic response of tissue deformation, damage and healing under external forces; S23, real-time feedback generation: when learners perform surgical operations in a virtual environment, feedback information is generated in real time by collecting learners' operation data and combining them with the calculation of the biomechanical model, generating corresponding tissue responses and presenting the cutting depth and bleeding of tissues in the virtual environment. If the operation is inappropriate, feedback warnings will be provided to prompt learners to adjust the operation; S24, tissue response adjustment: While the learner is operating, the tissue response in the virtual environment will be dynamically adjusted to closely match the learner's operating actions, and the parameters in the environment will be adjusted according to the learner's operating performance.
[0009] Optionally, the S22 includes: S221, anatomical data acquisition and processing: First, high-resolution anatomical data of real patients or anatomical samples are acquired through medical imaging technology, and the data is preprocessed to generate human tissue structure models at different levels; S222, biomechanical property setting: on the constructed three-dimensional anatomical model, assign corresponding physical properties to each layer of tissue based on the biomechanical principles, and define the mechanical behavior of each tissue under external force through physical parameters; S223, interactive coupling of organizational levels: interactive coupling of organizational models at different levels, so that in a virtual environment, operations on external organizations affect the state of their lower-level organizations, and their coupling relationships are determined through numerical simulation technology; S224, Dynamic response mechanism establishment: In the virtual teaching environment, a dynamic response mechanism is introduced for the biomechanical model. When learners apply external forces (such as cutting and compression), the deformation, damage and subsequent healing process of the tissue are calculated in real time.
[0010] Optionally, S3 includes: S31, Situational data collection and preprocessing: by collecting a large amount of medical operation data, including historical data of normal operation scenarios and emergency situations, the data is preprocessed, including data cleaning, normalization and feature extraction; S32, Convolutional Neural Network (CNN) model construction and training: Based on the preprocessed data, a convolutional neural network (CNN) model is constructed to extract spatial features during the operation through multiple convolutional layers and pooling layers to identify the learner's hand movements and device usage during the operation; S321, convolution layer design: The convolution layer performs convolution operations on the input data through filters to extract local features, assuming that the input is an image representation of the operation steps; S322, pooling layer design: After the convolution layer, the feature dimension reduction is performed through the pooling layer. The commonly used pooling method is maximum pooling; S323, model training: The CNN model is trained through back propagation algorithm and gradient descent optimization, and the loss function adopts cross-extraction loss function; S33, situational operation simulation construction: using the trained model, construct a variety of situational operation simulations in the virtual teaching environment. The simulation scenarios include normal operation steps in surgery and emergency situations, and dynamically adjust the development of simulation scenarios; S34, Emergency Response Ability Assessment: During the simulation, analyze the learner's operating behavior in real time, especially the response in emergency situations, compare the learner's operation with the standard operation, evaluate its emergency response ability, and calculate the learner's emergency response ability score ; S35, customized improvement suggestion generation generates customized improvement suggestions for learners, adjusts simulation scenarios in subsequent training, and helps learners gradually improve their emergency response capabilities.
[0011] Optionally, S4 includes: S41, data integration and feature extraction: The data after preliminary processing, including the learner's operation data, physiological data and environmental data, are integrated, and the data are processed using feature extraction algorithms to extract features related to operation accuracy, speed and robustness; S42, Operation Accuracy Analysis: Evaluate the learner's operation accuracy by setting a benchmark operation path and comparing the learner's operation to quantify its operation accuracy; S43, operation speed analysis: analyze the learner's operation speed. The operation speed analysis includes calculating the completion time of each operation step, the total operation time, and the time distribution of each stage in the operation process, comparing the data with the standard operation speed, and evaluating the learner's reaction speed and execution efficiency during the operation process; S44, Operational Robustness Analysis: Assess the learner's operational robustness by analyzing the learner's force control, movement consistency, and fluctuations during the operation.
[0012] Optionally, S5 includes: S51, multi-level feedback generation: Generate feedback information at different levels for learners, and divide the feedback information into immediate feedback, stage feedback and overall feedback; S52, visual feedback presentation: Provide visual feedback to learners through the display in the virtual teaching environment. The visual feedback content will be dynamically updated according to the learner's operation, so that the learner can improve the operation area; S53, auditory feedback delivery: auditory feedback is provided to learners through headphones, and the volume and tone of the auditory feedback are adjusted according to the learner's current state; S54, tactile feedback provision: delivering tactile feedback during operation to the learner through a tactile feedback device; S55, Real-time performance monitoring and content adjustment: Real-time monitoring of learners' performance in training, and dynamic adjustment of teaching content and difficulty based on the monitoring results; S56, Mental state adjustment guidance: Based on the learner's mental state monitoring data (such as through facial expressions, heart rate variability), provide corresponding psychological adjustment suggestions.
[0013] Optionally, the S55 includes: S551, Operation accuracy monitoring: Use sensors and data acquisition equipment to monitor learners’ operation accuracy in real time during training, analyze data in real time, and identify learners’ errors in operation; S552, operation speed monitoring: monitor the learner's operation speed during training through time measurement tools, record the completion time of each operation step, the reaction time during the operation process and the overall task execution speed, and compare the data with the standard operation time; S553, Psychological state monitoring: Monitor the learner's psychological state in real time through sensors, analyze physiological and behavioral data, evaluate whether the learner shows signs of negative emotions during training, and analyze the data in combination with operational performance; S554, data integration and analysis: Integrate the above monitoring data and conduct comprehensive analysis. Through the overall evaluation of the learner's operation accuracy, speed and psychological state, a real-time performance evaluation report is generated to automatically adjust the teaching content and difficulty of the current training.
[0014] Optionally, S6 includes: S61, Data storage and archiving: After learners complete training, all relevant data, including interaction data, feedback data, and performance data, are stored in a central database to form a complete historical record; S62, Data cleaning and preprocessing: Regularly clean and preprocess the data stored in the central database; S63, Common Problem Identification: Through data mining and statistical analysis techniques, regularly analyze the data in the central database to identify common problems that multiple learners encounter during training, classify and prioritize the common problems, and provide a basis for optimizing teaching content; S64, Personalized needs analysis: By analyzing each learner’s historical data, identifying their individual needs, generating personalized learning path suggestions for each learner based on the data, and adjusting the difficulty and rhythm of the teaching content; S65, Optimization of teaching content: redesign or adjust the difficulty of specific training modules, increase or decrease the training frequency of certain operation steps, and improve the feedback mechanism.
[0015] The human-computer collaborative medical teaching interaction system is used to implement the above-mentioned human-computer collaborative medical teaching interaction method, and includes the following modules: The data acquisition module collects the learner's physiological data, operation data and environmental data through sensors and medical equipment, and transmits them to the central processing unit, which performs preliminary filtering and standardization on the data; The virtual simulation and interaction module combines virtual reality and augmented reality technologies to provide an immersive teaching environment, introduces a high-precision biomechanical model, and simulates the real physical properties of human tissue. When learners operate in the virtual environment, the biomechanical model provides real-time feedback and adjusts the tissue response in the virtual environment; The deep learning-driven situational operation simulation and emergency response assessment module uses a deep learning model to build a situational operation simulation system, introduces emergency situations into the teaching process, assesses learners' emergency response capabilities, and provides customized improvement suggestions; The intelligent analysis and evaluation module uses the central processing unit to conduct in-depth analysis of the preliminarily processed data, evaluate the learner's operation accuracy, speed and robustness, and generate a personalized evaluation report in combination with historical data; The interactive feedback and guidance module generates multi-level feedback based on intelligent analysis and evaluation results, provides operation suggestions, error correction and psychological state adjustment in visual, auditory and tactile forms, and dynamically adjusts teaching content and difficulty according to learners' real-time performance; The data reflow and optimization module stores all interaction data, feedback data and performance data in a central database, regularly analyzes database data, identifies common problems and personalized needs, and optimizes teaching content; The remote collaboration and supervision module supports remote teaching and collaboration, allowing medical experts to supervise and guide learners' operations in real time through remote connections, and dynamically adjust teaching strategies based on learners' performance.
[0016] Beneficial effects of the present invention: The present invention, by introducing a deep learning-driven situational operation simulation system, enables the teaching process to dynamically adapt to the learner's actual operation performance and psychological state, thus achieving personalized teaching. This innovation significantly improves the pertinence and effectiveness of teaching content.
[0017] The present invention, through virtual reality and augmented reality technology, combined with a high-precision biomechanical model, provides learners with a highly immersive virtual operating environment, in which the physical properties of human tissues such as elasticity, hardness and responsiveness are realistically simulated. Learners can obtain feedback on the interaction between instruments and tissues in real surgery through the model. Through the situational operation simulation driven by the deep learning model, especially in emergency situations (such as equipment failure, surgical complications, etc.), the learner's operation behavior is dynamically evaluated, and sudden emergency situations can be generated in real time, and the learner's response ability in these situations can be evaluated. By evaluating the learner's reaction time, the accuracy of operational decisions, and the effectiveness of treatment measures, the system generates personalized evaluation reports and improvement suggestions for each learner, which not only improves the learner's operation ability in emergency scenarios, but also provides them with a tailored training path to help them have higher coping ability and confidence when facing complex medical situations.
[0018] The present invention continuously analyzes and improves teaching content through real-time data reflux and optimization mechanism, forming a closed-loop feedback system. This dynamic adjustment and continuous optimization capability ensures long-term improvement of teaching effect and adapts to the needs of different learners. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on the drawings without paying creative work.
[0020] Figure 1 A schematic diagram of a human-machine collaborative medical teaching interaction method according to an embodiment of the present invention; Figure 2 Schematic diagram of a human-machine collaborative medical teaching interaction system according to an embodiment of the present invention. DETAILED DESCRIPTION
[0021] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. At the same time, it is explained here that in order to make the embodiments more detailed, the following embodiments are the best and preferred embodiments, and those skilled in the art may also adopt other alternatives to implement some known technologies; and the accompanying drawings are only for more specific description of the embodiments, and are not intended to specifically limit the present invention.
[0022] It should be noted that the references to "one embodiment", "embodiment", "exemplary embodiments", "some embodiments" and the like in the specification indicate that the embodiments described may include specific features, structures or characteristics, but not every embodiment may include the specific features, structures or characteristics. In addition, when a specific feature, structure or characteristic is described in conjunction with an embodiment, it should be within the knowledge of a person skilled in the art to implement such feature, structure or characteristic in conjunction with other embodiments (whether or not explicitly described).
[0023] In general, a term can be understood, at least in part, from its use in context. For example, depending, at least in part, on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in the singular sense, or can be used to describe a combination of features, structures, or characteristics in the plural sense. Additionally, the term "based on" can be understood as not necessarily intended to convey an exclusive set of factors, but can instead, depending, at least in part, on the context, allow for the presence of other factors that are not necessarily explicitly described.
[0024] like Figure 1 As shown, the human-computer collaborative medical teaching interaction method includes the following steps: S1, data collection: collect learners’ physiological data, operational data and environmental data through sensors and medical devices, and transmit them to the central processing unit, which performs preliminary filtering and standardization on the data; S2, virtual simulation and interaction: combining virtual reality and augmented reality technologies to provide an immersive teaching environment for learners to practice medical operations, introducing high-precision biomechanical models to simulate the real physical properties of human tissues. When learners operate in a virtual environment, they receive real-time feedback based on the biomechanical model and adjust the tissue response in the virtual environment to improve learners' surgical skills; S3, deep learning-driven situational operation simulation and emergency response assessment: Use deep learning models to build situational operation simulations, introduce emergency situations into the teaching process, evaluate learners' emergency response capabilities, and provide customized improvement suggestions to help learners improve their response capabilities when facing emergencies; S4, intelligent analysis and evaluation: The central processing unit conducts in-depth analysis of the preliminarily processed data, evaluates the learner's operation accuracy, speed and robustness, and generates a personalized evaluation report based on historical data; S5, interactive feedback and guidance: Based on intelligent analysis and evaluation results, multi-level feedback is generated to provide operation suggestions, error correction and psychological state adjustment in visual, auditory and tactile forms, and dynamically adjust teaching content and difficulty according to learners' real-time performance; S6, data reflux and optimization: After learners complete training, all interaction data, feedback data and performance data are stored in the central database. The database data is regularly analyzed to identify common problems and personalized needs, optimize teaching content, and form a closed teaching loop; S7, Remote Collaboration and Supervision: Supports remote teaching and collaboration, allowing medical experts to supervise and guide learners' operations in real time through remote connections, and dynamically adjust teaching strategies based on learners' performance.
[0025] S1 includes: S11, physiological data collection: collect physiological data through biosensors installed on the learner, including heart rate, blood pressure and respiratory rate, and transmit the data to the central processing unit in real time; S12, operation data collection: using operation sensors installed on teaching equipment and instruments to collect operation data, and transmitting the collected data to a central processing unit; S13, environmental data collection: monitoring environmental data through environmental sensors, collecting environmental data in real time and transmitting it to the central processing unit; S14, data processing: The central processing unit performs preliminary filtering on the collected physiological data, operational data, and environmental data, and standardizes the data to ensure the consistency and reliability of the data.
[0026] S2 includes: S21, virtual teaching environment creation: First, virtual reality and augmented reality technologies are combined to generate a highly simulated immersive virtual teaching environment. Learners enter the virtual environment by wearing virtual reality headsets or using augmented reality glasses, and experience the real medical operation scene as if they were there in person; S22, Introduction of biomechanical model: Introduce a biomechanical model based on real anatomical data and biomechanical principles in the virtual teaching environment, introduce different layers of human tissue, and simulate the real physical properties of tissue. The biomechanical model includes the dynamic response of tissue deformation, damage and healing under external forces, so that learners can feel feedback similar to the real surgical process during the operation; S23, real-time feedback generation: when learners perform surgical operations in a virtual environment, feedback information is generated in real time by collecting learners' operation data and combining them with the calculation of the biomechanical model, generating corresponding tissue responses and presenting the cutting depth and bleeding of tissues in the virtual environment. If the operation is inappropriate, feedback warnings will be provided to prompt learners to adjust the operation; S24, tissue response adjustment: While the learner is operating, the tissue response in the virtual environment will be dynamically adjusted to closely match the learner's operating actions, and the parameters in the environment will be adjusted according to the learner's operating performance.
[0027] S22 includes: S221, anatomical data acquisition and processing: First, high-resolution anatomical data of real patients or anatomical samples are acquired through medical imaging technology, and the data is preprocessed to generate human tissue structure models at different levels; S222, biomechanical property setting: on the constructed three-dimensional anatomical model, assign corresponding physical properties to each layer of tissue based on the biomechanical principles, and define the mechanical behavior of each tissue under external force through physical parameters; S223, interactive coupling of tissue levels: interactive coupling of tissue models at different levels, so that in the virtual environment, the operation of external tissues affects the state of the underlying tissues (such as muscle and fat). The coupling relationship is determined by numerical simulation technology, so that learners can feel the real interactive effects between the various layers of tissues when operating; S224, establishment of dynamic response mechanism: In the virtual teaching environment, a dynamic response mechanism is introduced for the biomechanical model. When learners apply external force (such as cutting, compression), the deformation, damage and subsequent healing process of the tissue are calculated in real time, and the calculation results are dynamically displayed in the virtual environment, so that learners can see the impact of surgical operations on tissues in real time.
[0028] S3 includes: S31, Contextual Data Collection and Preprocessing: By collecting a large amount of medical operation data, including historical data of normal operation scenarios and emergency situations, the data is preprocessed, including data cleaning, normalization, and feature extraction, to provide high-quality input data for building deep learning models; S32, Convolutional Neural Network (CNN) model construction and training: Based on the preprocessed data, a convolutional neural network (CNN) model is constructed to extract spatial features during the operation through multiple convolutional layers and pooling layers to identify the learner's hand movements and device usage during the operation; S321, convolution layer design: The convolution layer performs convolution operation on the input data through filters to extract local features. Assuming that the input is an image representation of the operation steps, it is expressed as: ; in, For the The convolution kernels are located at The output at For the input image at position The pixel value at For the The weight of the convolution kernel, is the bias, is the activation function; S322, pooling layer design: After the convolution layer, the feature dimension reduction is performed through the pooling layer. The commonly used pooling method is maximum pooling, which is expressed as: ; in, is the feature value after pooling, It is the feature map output by the convolution layer; S323, model training: The CNN model is trained through back propagation algorithm and gradient descent optimization. The loss function adopts the cross-extraction loss function, which is expressed as: ; in, are model parameters, is the true label, Predict output for the model; S33, situational operation simulation construction: using the trained model, construct a variety of situational operation simulations in the virtual teaching environment. The simulation scenarios include normal operation steps in surgery and emergency situations, and dynamically adjust the development of simulation scenarios; S34, Emergency Response Ability Assessment: During the simulation, analyze the learner's operating behavior in real time, especially the response in emergency situations, compare the learner's operation with the standard operation, evaluate its emergency response ability, and calculate the learner's emergency response ability score , expressed as: ; in, is the standard reaction time, is the learner's actual reaction time, For action accuracy, To ensure the effectiveness of the response measures, is the weight coefficient; S35, customized improvement suggestion generation generates customized improvement suggestions for learners, adjusts simulation scenarios in subsequent training, and helps learners gradually improve their emergency response capabilities.
[0029] S4 includes: S41, data integration and feature extraction: The data after preliminary processing, including the learner's operation data, physiological data and environmental data, are integrated, and the data are processed using feature extraction algorithms to extract features related to operation accuracy, speed and robustness; S42, Operation Accuracy Analysis: Evaluate the learner's operation accuracy by setting a benchmark operation path and comparing the learner's operation to quantify its operation accuracy; S43, operation speed analysis: analyze the learner's operation speed. The operation speed analysis includes calculating the completion time of each operation step, the total operation time, and the time distribution of each stage in the operation process, comparing the data with the standard operation speed, and evaluating the learner's reaction speed and execution efficiency during the operation process; S44, Operational Robustness Analysis: Assess the learner's operational robustness by analyzing the learner's force control, movement consistency, and fluctuations during the operation.
[0030] S5 includes: S51, multi-level feedback generation: Generate feedback information at different levels for learners, and divide the feedback information into immediate feedback, stage feedback and overall feedback; S52, visual feedback presentation: Provide visual feedback to learners through the display in the virtual teaching environment. The visual feedback content will be dynamically updated according to the learner's operation, so that the learner can improve the operation area; S53, auditory feedback delivery: provide auditory feedback to learners through headphones. The volume and tone of the auditory feedback will be adjusted according to the learner's current state to ensure the effectiveness and comfort of information transmission; S54, tactile feedback provision: delivering tactile feedback during operation to the learner through a tactile feedback device; S55, Real-time performance monitoring and content adjustment: Real-time monitoring of learners' performance in training, and dynamic adjustment of teaching content and difficulty based on the monitoring results; S56, Mental state adjustment guidance: Based on the learner's mental state monitoring data (such as through facial expressions, heart rate variability), provide corresponding psychological adjustment suggestions.
[0031] S55 includes: S551, Operation accuracy monitoring: Use sensors and data acquisition equipment to monitor learners’ operation accuracy in real time during training, analyze data in real time, and identify learners’ errors in operation; S552, operation speed monitoring: monitor the learner's operation speed during training through time measurement tools, record the completion time of each operation step, the reaction time during the operation process and the overall task execution speed, and compare the data with the standard operation time; S553, Psychological state monitoring: Monitor the learner's psychological state in real time through sensors, analyze physiological and behavioral data, evaluate whether the learner shows signs of negative emotions during training, and analyze the data in combination with operational performance; S554, data integration and analysis: Integrate the above monitoring data and conduct comprehensive analysis. Through the overall evaluation of the learner's operation accuracy, speed and psychological state, a real-time performance evaluation report is generated to automatically adjust the teaching content and difficulty of the current training.
[0032] S6 includes: S61, Data storage and archiving: After learners complete training, all relevant data, including interaction data, feedback data, and performance data, are uniformly stored in a central database to form a complete historical record for subsequent analysis and query; S62, Data cleaning and preprocessing: Regularly clean and preprocess the data stored in the central database; S63, Common Problem Identification: Through data mining and statistical analysis techniques, regularly analyze the data in the central database to identify common problems that multiple learners encounter during training, classify and prioritize the common problems, and provide a basis for optimizing teaching content; S64, Personalized needs analysis: By analyzing each learner's historical data, identifying their individual needs, generating personalized learning path suggestions for each learner based on the data, and adjusting the difficulty and rhythm of the teaching content to better meet the needs of different learners; S65, Optimization of teaching content: redesign or adjust the difficulty of specific training modules, increase or decrease the training frequency of certain operation steps, improve the feedback mechanism to better support the learners' learning process, and introduce new teaching strategies or tools to improve teaching effectiveness and ensure continuous improvement and optimization of teaching content, so as to continuously improve learners' skill level and learning experience; Through detailed steps, the system can effectively store and analyze learners' training data, identify common problems and personalized needs, and continuously optimize teaching content based on the analysis results, forming a teaching closed-loop system with dynamic adjustment and feedback.
[0033] like Figure 2 As shown, the human-computer collaborative medical teaching interaction system is used to implement the above-mentioned human-computer collaborative medical teaching interaction method, including the following modules: The data acquisition module collects learners' physiological data, operational data and environmental data through sensors and medical equipment, and transmits them to the central processing unit, which performs preliminary filtering and standardization on the data; The virtual simulation and interaction module combines virtual reality and augmented reality technologies to provide an immersive teaching environment for learners to practice medical operations. It introduces a high-precision biomechanical model to simulate the real physical properties of human tissue. When learners operate in a virtual environment, the biomechanical model provides real-time feedback and adjusts the tissue response in the virtual environment to improve learners' surgical skills. The deep learning-driven situational operation simulation and emergency response assessment module uses deep learning models to build a situational operation simulation system, introduces emergency situations into the teaching process, evaluates learners' emergency response capabilities, and provides customized improvement suggestions to help learners improve their response capabilities when facing emergencies; The intelligent analysis and evaluation module uses the central processing unit to conduct in-depth analysis of the preliminarily processed data, evaluate the learner's operation accuracy, speed and robustness, and generate personalized evaluation reports based on historical data; The interactive feedback and guidance module generates multi-level feedback based on intelligent analysis and evaluation results, provides operation suggestions, error correction and psychological state adjustment in visual, auditory and tactile forms, and dynamically adjusts teaching content and difficulty according to learners' real-time performance; The data return and optimization module stores all interaction data, feedback data and performance data in the central database, regularly analyzes database data, identifies common problems and personalized needs, optimizes teaching content, and forms a closed teaching loop; The remote collaboration and supervision module supports remote teaching and collaboration, allowing medical experts to supervise and guide learners' operations in real time through remote connections, and dynamically adjust teaching strategies based on learners' performance.
[0034] The present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention. In order to make the public have a thorough understanding of the present invention, specific details are described in detail in the following preferred embodiments of the present invention, but those skilled in the art can fully understand the present invention without detailed description. In addition, in order to avoid unnecessary confusion about the essence of the present invention, well-known methods, processes, procedures, components and circuits are not described in detail.
[0035] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. Improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. Human-computer collaborative medical teaching interaction method, It is characterized in that The following steps are involved: S1, data collection: collect learners’ physiological data, operational data and environmental data through sensors and medical devices, and transmit them to the central processing unit, which performs preliminary filtering and standardization on the data; S2, virtual simulation and interaction: combining virtual reality and augmented reality technologies to provide an immersive teaching environment, introducing high-precision biomechanical models to simulate the real physical properties of human tissues. When learners operate in a virtual environment, the biomechanical models provide real-time feedback and adjust the tissue response in the virtual environment; S3, deep learning-driven situational operation simulation and emergency response assessment: Use deep learning models to build situational operation simulations, introduce emergency situations into the teaching process, evaluate learners' emergency response capabilities, and provide customized improvement suggestions; S4, intelligent analysis and evaluation: The central processing unit conducts in-depth analysis of the preliminarily processed data, evaluates the learner's operation accuracy, speed and robustness, and generates a personalized evaluation report based on historical data; S5, interactive feedback and guidance: Based on intelligent analysis and evaluation results, multi-level feedback is generated to provide operation suggestions, error correction and psychological state adjustment in visual, auditory and tactile forms, and dynamically adjust teaching content and difficulty according to learners' real-time performance; S6, data reflow and optimization: After learners complete training, all interaction data, feedback data and performance data are stored in the central database. The database data is analyzed regularly to identify common problems and personalized needs and optimize teaching content; S7, Remote Collaboration and Supervision: Supports remote teaching and collaboration, allowing medical experts to supervise and guide learners' operations in real time through remote connections, and dynamically adjust teaching strategies based on learners' performance.
2. The human-computer collaborative medical teaching interaction method according to claim 1, It is characterized in that The S1 includes: S11, physiological data collection: collect physiological data through biosensors installed on the learner, including heart rate, blood pressure and respiratory rate, and transmit the data to the central processing unit in real time; S12, operation data collection: using operation sensors installed on teaching equipment and instruments to collect operation data, and transmitting the collected data to a central processing unit; S13, environmental data collection: monitoring environmental data through environmental sensors, collecting environmental data in real time and transmitting it to the central processing unit; S14, data processing: The central processing unit performs preliminary filtering on the collected physiological data, operation data and environmental data, and performs standardization on the data.
3. The human-computer collaborative medical teaching interaction method according to claim 1, It is characterized in that The S2 includes: S21, virtual teaching environment creation: first, virtual reality and augmented reality technologies are combined to generate a highly simulated immersive virtual teaching environment. Learners enter the virtual environment by wearing virtual reality head-mounted display devices or using augmented reality glasses; S22, Introduction of biomechanical model: Introduce biomechanical models based on real anatomical data and biomechanical principles in the virtual teaching environment, introduce different layers of human tissue, and simulate the real physical properties of tissue. The biomechanical model includes the dynamic response of tissue deformation, damage and healing under external forces; S23, real-time feedback generation: when learners perform surgical operations in a virtual environment, feedback information is generated in real time by collecting learners' operation data and combining them with the calculation of the biomechanical model, generating corresponding tissue responses and presenting the cutting depth and bleeding of tissues in the virtual environment. If the operation is inappropriate, feedback warnings will be provided to prompt learners to adjust the operation; S24, tissue response adjustment: While the learner is operating, the tissue response in the virtual environment will be dynamically adjusted to closely match the learner's operating actions, and the parameters in the environment will be adjusted according to the learner's operating performance.
4. The human-computer collaborative medical teaching interaction method according to claim 3, It is characterized in that The S22 includes: S221, anatomical data acquisition and processing: First, high-resolution anatomical data of real patients or anatomical samples are acquired through medical imaging technology, and the data is preprocessed to generate human tissue structure models at different levels; S222, biomechanical property setting: on the constructed three-dimensional anatomical model, assign corresponding physical properties to each layer of tissue based on the biomechanical principles, and define the mechanical behavior of each tissue under external force through physical parameters; S223, interactive coupling of organizational levels: interactive coupling of organizational models at different levels, so that in a virtual environment, operations on external organizations affect the state of their lower-level organizations, and their coupling relationships are determined through numerical simulation technology; S224, Dynamic response mechanism establishment: In the virtual teaching environment, a dynamic response mechanism is introduced for the biomechanical model. When learners apply external force, the deformation, damage and subsequent healing process of the tissue are calculated in real time.
5. The human-computer collaborative medical teaching interaction method according to claim 1, It is characterized in that The S3 includes: S31, Situational data collection and preprocessing: by collecting a large amount of medical operation data, including historical data of normal operation scenarios and emergency situations, the data is preprocessed, including data cleaning, normalization and feature extraction; S32, Convolutional neural network model construction and training: Based on the preprocessed data, a convolutional neural network model is constructed to extract spatial features during the operation through multiple convolutional layers and pooling layers to identify the learner's hand movements and device usage during the operation; S321, convolution layer design: The convolution layer performs convolution operations on the input data through filters to extract local features, assuming that the input is an image representation of the operation steps; S322, pooling layer design: After the convolution layer, the feature dimension reduction is performed through the pooling layer. The commonly used pooling method is maximum pooling; S323, model training: The CNN model is trained through back propagation algorithm and gradient descent optimization, and the loss function adopts cross-extraction loss function; S33, situational operation simulation construction: using the trained model, construct a variety of situational operation simulations in the virtual teaching environment. The simulation scenarios include normal operation steps in surgery and emergency situations, and dynamically adjust the development of simulation scenarios; S34, Emergency Response Ability Assessment: During the simulation, analyze the learner's operating behavior in real time, especially the response in emergency situations, compare the learner's operation with the standard operation, evaluate its emergency response ability, and calculate the learner's emergency response ability score ; S35, customized improvement suggestion generation generates customized improvement suggestions for learners, adjusts simulation scenarios in subsequent training, and helps learners gradually improve their emergency response capabilities.
6. The human-machine collaborative medical teaching interaction method according to claim 1, It is characterized in that The S4 includes: S41, data integration and feature extraction: The data after preliminary processing, including the learner's operation data, physiological data and environmental data, are integrated, and the data are processed using feature extraction algorithms to extract features related to operation accuracy, speed and robustness; S42, Operation Accuracy Analysis: Evaluate the learner's operation accuracy by setting a benchmark operation path and comparing the learner's operation to quantify its operation accuracy; S43, operation speed analysis: analyze the learner's operation speed. The operation speed analysis includes calculating the completion time of each operation step, the total operation time, and the time distribution of each stage in the operation process, comparing the data with the standard operation speed, and evaluating the learner's reaction speed and execution efficiency during the operation process; S44, Operational Robustness Analysis: Assess the learner's operational robustness by analyzing the learner's force control, movement consistency, and fluctuations during the operation.
7. The human-computer collaborative medical teaching interaction method according to claim 1, It is characterized in that The S5 includes: S51, multi-level feedback generation: Generate feedback information at different levels for learners, and divide the feedback information into immediate feedback, stage feedback and overall feedback; S52, visual feedback presentation: Provide visual feedback to learners through the display in the virtual teaching environment. The visual feedback content will be dynamically updated according to the learner's operation, so that the learner can improve the operation area; S53, auditory feedback delivery: auditory feedback is provided to learners through headphones, and the volume and tone of the auditory feedback are adjusted according to the learner's current state; S54, tactile feedback provision: delivering tactile feedback during operation to the learner through a tactile feedback device; S55, Real-time performance monitoring and content adjustment: Real-time monitoring of learners' performance in training, and dynamic adjustment of teaching content and difficulty based on the monitoring results; S56, psychological state adjustment guidance: Based on the learner's psychological state monitoring data, provide corresponding psychological adjustment suggestions.
8. The human-machine collaborative medical teaching interaction method according to claim 7, It is characterized in that The S55 includes: S551, Operation accuracy monitoring: Use sensors and data acquisition equipment to monitor learners’ operation accuracy in real time during training, analyze data in real time, and identify learners’ errors in operation; S552, operation speed monitoring: monitor the learner's operation speed during training through time measurement tools, record the completion time of each operation step, the reaction time during the operation process and the overall task execution speed, and compare the data with the standard operation time; S553, Psychological state monitoring: Monitor the learner's psychological state in real time through sensors, analyze physiological and behavioral data, evaluate whether the learner shows signs of negative emotions during training, and analyze the data in combination with operational performance; S554, data integration and analysis: Integrate the above monitoring data and conduct comprehensive analysis. Through the overall evaluation of the learner's operation accuracy, speed and psychological state, generate a real-time performance evaluation report and automatically adjust the teaching content and difficulty of the current training.
9. The human-computer collaborative medical teaching interaction method according to claim 1, It is characterized in that The S6 includes: S61, Data storage and archiving: After learners complete training, all relevant data, including interaction data, feedback data, and performance data, are stored in a central database to form a complete historical record; S62, Data cleaning and preprocessing: Regularly clean and preprocess the data stored in the central database; S63, Common Problem Identification: Through data mining and statistical analysis techniques, regularly analyze the data in the central database to identify common problems that multiple learners encounter during training, classify and prioritize the common problems, and provide a basis for optimizing teaching content; S64, Personalized needs analysis: By analyzing each learner’s historical data, identifying their individual needs, generating personalized learning path suggestions for each learner based on the data, and adjusting the difficulty and rhythm of the teaching content; S65, Optimization of teaching content: redesign or adjust the difficulty of specific training modules, increase or decrease the training frequency of certain operation steps, and improve the feedback mechanism.
10. A human-computer collaborative medical teaching interaction system, used to implement a human-computer collaborative medical teaching interaction method as claimed in any one of claims 1 to 9, It is characterized in that Includes the following modules: The data acquisition module collects the learner's physiological data, operation data and environmental data through sensors and medical equipment, and transmits them to the central processing unit, which performs preliminary filtering and standardization on the data; The virtual simulation and interaction module combines virtual reality and augmented reality technologies to provide an immersive teaching environment, introduces a high-precision biomechanical model, and simulates the real physical properties of human tissue. When learners operate in the virtual environment, the biomechanical model provides real-time feedback and adjusts the tissue response in the virtual environment; The deep learning-driven situational operation simulation and emergency response assessment module uses a deep learning model to build a situational operation simulation system, introduces emergency situations into the teaching process, assesses learners' emergency response capabilities, and provides customized improvement suggestions; The intelligent analysis and evaluation module uses the central processing unit to conduct in-depth analysis of the preliminarily processed data, evaluate the learner's operation accuracy, speed and robustness, and generate a personalized evaluation report in combination with historical data; The interactive feedback and guidance module generates multi-level feedback based on intelligent analysis and evaluation results, provides operation suggestions, error correction and psychological state adjustment in visual, auditory and tactile forms, and dynamically adjusts teaching content and difficulty according to learners' real-time performance; The data reflow and optimization module stores all interaction data, feedback data and performance data in a central database, regularly analyzes database data, identifies common problems and personalized needs, and optimizes teaching content; The remote collaboration and supervision module supports remote teaching and collaboration, allowing medical experts to supervise and guide learners' operations in real time through remote connections, and dynamically adjust teaching strategies based on learners' performance.
Citation Information
Patent Citations
Multi-person interaction clinical nursing teaching method based on mixed reality
CN117911211A
Intelligent medical education and training system
CN118039108A
Intelligent teaching model construction and application of ultrasonic AI technology in medical undergraduate teaching
CN118608339A
Cited By
Virtual environment-based advanced training course resource adaptation method
CN119417670A
Ear-nose-throat operation teaching method and system based on virtual reality
CN119494761A
Man-machine cooperation operation training interaction method and system and operation robot
CN119541304A
Emergency and critical disease nursing training system and method based on vr
CN119942873A
Corneal contact lens fitting intelligent teaching method based on virtual simulation technology
CN120032550A
