Virtual simulation learning interaction system for nursing in obstetrics and gynecology department

Abstract

The utility model relates to the technical field of medical virtual simulation teaching, in particular to a virtual simulation learning interaction system for nursing in the obstetrics and gynecology department, which comprises a main control processor, a physiological parameter acquisition array, a scene self-adaptive controller and a multi-sensory feedback actuator, and innovatively deeply integrates a BOPPPS teaching model and a virtual simulation technology. The main control processor realizes teaching process control and data processing; the physiological parameter acquisition array is used for accurately collecting operation data; the scene adaptive controller adjusts the scene in real time according to the medical operation data of the user; and the feedback output module gives multi-sensory feedback. According to the utility model, the teaching process is optimized based on the BOPPPS model, the practical ability of a learner is improved through a highly immersive simulation environment, and the problems of insufficient teaching resources, limited practical opportunities, inaccurate effect evaluation and the like of traditional obstetrics and gynecology nursing teaching are effectively solved.

Description

Technical Field

[0001] The utility model relates to the technical field of medical virtual simulation teaching, in particular to an obstetrics and gynecology nursing virtual simulation learning interactive system. Background Art

[0002] With the rapid development of information technology and artificial intelligence, virtual simulation technology has gradually demonstrated its unique advantages in medical education, bringing profound changes to traditional teaching models. Traditionally, a significant gap existed between theoretical knowledge transfer and practical clinical practice. Although students mastered a wealth of basic medical theory, they struggled to flexibly apply it in real-world situations. Furthermore, in specialized fields such as obstetrics and gynecology nursing, which require high practical and operational skills, practical teaching often faces constraints such as limited teaching resources, safety risks, and ethical considerations, further hindering effective teaching. Virtual simulation technology addresses this shortcoming. By creating immersive digital virtual scenarios, students can conduct highly realistic practical training in a risk-free environment, systematically mastering operational procedures, protocols, and emergency response techniques. This significantly enhances their clinical practice and overall adaptability, meeting the high standards and requirements of practical teaching in modern medical education.

[0003] However, the existing virtual simulation learning system for obstetrics and gynecology nursing still has many areas that need to be improved. First, in terms of teaching design, the existing system has insufficient integration of advanced teaching models, making it difficult to achieve a coordinated improvement in process optimization and teaching effectiveness. Secondly, the means of interaction are relatively simple, mostly relying on traditional operating tools such as the mouse and keyboard, which makes it difficult to satisfy the user's sense of immersion and experience in real interactive simulation. In addition, the current system is still weak in data perception and feedback mechanisms, lacking refined data collection and evaluation and analysis methods for the entire process of student operations. Feedback information is often delayed and generalized, making it difficult to achieve dynamic scene adjustments based on the individual abilities and learning progress of students. These limitations restrict the implementation of the concept of "teaching students in accordance with their aptitude" and are not conducive to the phased advancement and continuous improvement of students' skill levels. Utility Model Content

[0004] The purpose of the utility model is to provide an obstetrics and gynecology nursing virtual simulation learning interactive system to solve the problems of single interaction mode, inaccurate data feedback and fixed scene complexity in the prior art.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] Main control processor;

[0007] The physiological parameter acquisition array is connected to the main control processor via the SPI bus;

[0008] The scene adaptive controller is connected to the main control processor via Ethernet;

[0009] A multi-sensory feedback actuator is connected to a scene adaptive controller; it includes: a dynamic resistance glove and a temperature simulator; the dynamic resistance glove is used to output adjustable mechanical resistance to simulate the operational resistance of obstetric and gynecological case scenarios; the temperature simulator is used to simulate changes in the surface temperature of the mother's body.

[0010] Using the above technical solution, the physiological parameter acquisition array in this solution includes:

[0011] The pressure sensing matrix has its output connected to the analog input port of the ADC conversion circuit, and the ADC conversion circuit transmits data to the matrix input port of the main control processor via the SPI data bus;

[0012] The heart rate sensor has its output connected to the differential amplifier circuit and transmitted to the gradient input port of the main control processor through the SPI data bus;

[0013] The infrared motion capture device converts the analog signal of the operator's gesture spatial trajectory into a digital signal through the ADC conversion circuit, and transmits it to the matrix input port of the main control processor through the SPI data bus.

[0014] Using the above technical solution, the scene adaptive controller in this solution includes:

[0015] Feature extraction chip, used to calculate trainees' operational skill deficiency indicators in real time;

[0016] Edge decision-making unit, used to execute reinforcement learning strategies to generate scenario adjustment instructions.

[0017] The above technical solution is adopted, and the solution also includes a startup module, which includes:

[0018] A touch input unit, used for inputting preset case characteristics by touch;

[0019] The voice recognition unit is used to wake up the system based on keywords.

[0020] Adopting the above technical solution, the main control processor in this solution includes:

[0021] FPGA accelerator, used to receive the pressure distribution matrix digital signal, the trainee's heart rate signal and the gesture spatial trajectory digital signal transmitted by the physiological parameter acquisition array;

[0022] The control chip is used to execute the state machine control of the BOPPPS teaching protocol and generate multi-dimensional skill evaluation vectors based on the signals collected by the FPGA accelerator.

[0023] The above technical solution is adopted, in which the three-dimensional scene engine includes a graphics rendering engine, which is used to construct a three-dimensional virtual scene library for obstetrics and gynecology nursing, and realize the simulation of vaginal cervical medication, delivery care, and obstetric emergency operation scenes.

[0024] Using the above technical solution, the 3D scene engine in this solution also includes:

[0025] The main communication channel is connected to the scene adaptive controller via Ethernet;

[0026] The secondary communication channel is connected to the startup module through the SPI bus.

[0027] Due to the adoption of the above technical solution, the present invention has achieved the following technical advancements compared to the prior art:

[0028] This utility model significantly improves the interactivity and adaptability of the virtual simulation teaching system by introducing a multimodal data perception and teaching case scenario adjustment mechanism. Compared with the traditional static interaction method that relies only on preset scenarios, the system uses multi-source physiological parameter acquisition equipment such as pressure sensor matrix, heart rate sensor and infrared motion capture to achieve high-dimensional real-time perception of the operator's behavioral details and physiological reactions during the operation process; through the collaborative processing of FPGA accelerator and control chip, a skill evaluation vector of the student's operation performance is constructed, so that the system has the ability to understand and judge the student's status in real time. This architecture not only improves the timeliness and accuracy of feedback, but also provides a data basis for the intelligent adjustment of teaching strategies, breaking through the bottleneck of delayed feedback and single information dimension of traditional systems.

[0029] The present invention integrates a scene adaptive control mechanism based on deep reinforcement learning into the system, which enables the teaching content and intervention strategy to be dynamically adjusted according to the student's current operating status and skill deficiencies. The edge decision-making unit uses a preset reinforcement learning strategy to generate scene adjustment instructions and sensory feedback control signals, achieving real-time adjustment of parameters such as difficulty level, operating resistance, and temperature feedback, greatly enhancing the teaching system's adaptability to different learning stages and individual differences. Especially in complex scenarios such as emergency simulation, operational error correction, and emotional regulation, the system can automatically guide students to perform corrective training or rest suggestions, enhancing the immersive experience and practical training effect, significantly surpassing the rigid response capabilities of traditional rule-based control systems, and reflecting the technological progress of intelligent and humanized teaching design. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The present invention will be further described below with reference to the accompanying drawings.

[0031] Figure 1 This is a structural diagram of the obstetrics and gynecology nursing virtual simulation learning interactive system of the present invention;

[0032] Figure 2 This is a schematic diagram of the circuit structure of the main control processor of the present utility model.

[0033] In the figure: 1. Main control processor; 2. 3D scene engine; 3. Startup module; 4. Physiological parameter acquisition array; 5. Scene adaptive controller; 6. Multi-sensory feedback actuator. DETAILED DESCRIPTION

[0034] The present invention is further described in detail below with reference to the embodiments:

[0035] Example 1

[0036] like Figure 1-Figure 2 As shown, the utility model provides an obstetrics and gynecology nursing virtual simulation learning interactive system, comprising:

[0037] Main control processor 1 includes an FPGA accelerator, which receives digital signals of the pressure distribution matrix, the student's heart rate signal, and the spatial trajectory of the hand gestures from the physiological parameter acquisition array 4. A control chip is used to execute the state machine control of the BOPPPS teaching protocol and generate multidimensional skill assessment vectors based on the signals collected by the FPGA accelerator. In this embodiment, the control chip uses an STM32F746ZGT6 chip, and the FPGA accelerator uses a Xilinx XC7A100T accelerator. These two interact via high-speed data, with the control chip responsible for state machine control of the BOPPPS process and the FPGA for parallel processing of multi-sensor data streams. Main control processor 1 is connected to the physiological parameter acquisition array 4 via an SPI bus and to the 3D scene engine 2 via Ethernet.

[0038] In this embodiment, the FPGA accelerator serves as a high-speed parallel computing platform, receiving digital pressure distribution signals, heart rate signals, and gesture trajectory data from the physiological parameter acquisition array. The FPGA utilizes its parallel logic array to perform preprocessing operations on these three types of data, including feature extraction, data cleaning, normalization, and time series mapping. The FPGA then generates standardized multidimensional status data packets, reducing system response latency.

[0039] The control chip loads the state machine logic of the BOPPPS teaching model, using the six-stage evaluation process of "introduction - objectives - pre-test - engagement - post-test - summary" as a reference. Based on status data transmitted by the FPGA, the chip calculates the student's performance score in real time during each teaching stage. Using a weighted fusion algorithm, it constructs a multi-dimensional skill assessment vector covering metrics such as operational accuracy, reaction time, physiological stability, and situational adaptability.

[0040] 3D Scene Engine 2: This includes a graphics rendering engine used to build a 3D virtual scene library for obstetrics and gynecology care, enabling simulations of vaginal and cervical medication application, labor and delivery care, and obstetric emergency care. In this embodiment, 3D Scene Engine 2 uses an NVIDIA RTX graphics card and the Unity3D 2022 LTS engine to build a 3D virtual scene library for obstetrics and gynecology care, providing real-time rendering of simulated scenes for vaginal and cervical medication application, labor and delivery care, and obstetric emergency care.

[0041] The 3D scene engine has a communication link, including:

[0042] The primary communication channel is used to transmit scene adjustment commands to the scene adaptation controller via Ethernet. The secondary communication channel is used to interact with the startup module via the SPI bus to generate preset virtual case scenarios. The 3D scene engine generates the initial case operation virtual scene based on the preset scene digital signal output by the startup module.

[0043] Startup Module 3: This module includes a touch input unit that receives user input of preset case characteristics and generates a preset scenario digital signal. A voice recognition unit wakes up the system based on keywords. In this embodiment, the touch input unit uses a capacitive touch screen, and the voice recognition unit uses the iFlytek XFS5152CE chip.

[0044] Physiological Parameter Acquisition Array 4: This array comprises a pressure sensor matrix, a heart rate sensor, and an infrared motion capture device, responsible for real-time acquisition of user medical operation data and conversion into interactive signals. The pressure sensor matrix detects the force distribution of the practitioner and generates a pressure distribution matrix; the heart rate sensor records the practitioner's heart rate curve; and the infrared motion capture device records the spatial trajectory of the practitioner's gestures. It also includes an ADC conversion circuit for converting the collected user medical operation data into interactive signals. In this embodiment, the pressure sensor matrix uses an FSR402 thin-film pressure sensor matrix, the heart rate sensor uses a photoplethysmography detector, and the infrared motion capture device uses an Intel RealSense infrared camera. Each sensor group is connected to the ADC conversion circuit via an SPI bus, acquiring real-time user operation data, converting it into interactive signals, and transmitting them to the virtual simulation engine.

[0045] In this embodiment, the pressure sensor matrix is placed on the operating platform or the inner surface of the operating glove. When the trainee performs operations such as pressing, pulling, pushing, or pulling, the sensor elements in different areas sense the applied force in real time and generate a pressure distribution matrix in the form of analog signals. This analog signal is first transmitted to the ADC circuit, where it is sampled with high precision and converted into a digital signal. This signal is then transmitted to the matrix input port of the main control processor 1 via the SPI bus.

[0046] The heart rate sensor, worn on the trainee's fingertip or wrist, uses photoplethysmography (PPE) technology to capture a raw analog waveform of heart rate changes over time. This analog signal first enters a differential amplifier circuit for noise suppression and signal enhancement, and then is input via the SPI bus to the gradient input port of the main control processor, enabling dynamic sensing of emotions and stress levels.

[0047] Infrared motion capture devices are positioned around the operating space to capture the three-dimensional trajectory of the trainee's hands. The captured infrared reflection signals are converted into analog position data through hardware processing and filtering. These signals are then converted to digital data using an ADC circuit and transmitted via the SPI bus to the matrix input port of the main control processor, enabling quantitative recording of the motion path, amplitude, and accuracy.

[0048] Scenario Adaptive Controller 5: This includes a feature extraction chip for real-time calculation of the trainee's operational skill deficit indicators and an edge decision-making unit for executing reinforcement learning strategies to generate scenario adjustment instructions. The edge decision-making unit uses a built-in reinforcement learning algorithm to dynamically adjust the complexity of the virtual scene based on user operation data.

[0049] In this embodiment, the feature extraction chip uses an FPGA accelerator, model Xilinx Zynq-7000; the scene adaptive controller 5 uses the NVIDIA Jetson Nano development board, which is connected to the central control system via Gigabit Ethernet, runs a reinforcement learning model based on the PPO algorithm, analyzes user operation data in real time, and generates scene parameter adjustment instructions.

[0050] The input interface of the scene adaptive controller 5 receives the multi-dimensional state vector output by the main control processor 1; the feature extraction chip calculates the trainee's operation skill deficiency index in real time based on the input multi-dimensional state vector; the edge decision unit generates scene adjustment instructions and sensory feedback signals based on the trainee's operation skill deficiency index and the preset execution reinforcement learning strategy;

[0051] In this embodiment, the multi-dimensional state vector output by the main control processor 1 includes a pressure matrix, a heart rate curve, and a gesture trajectory signal. The feature extraction chip extracts key skill deficiency indicators based on the trainee's operation information, mainly including:

[0052] Pressure Deviation: The pressure matrix obtained by the pressure sensor can determine whether the student's force distribution during operation is uniform. If the force is too high or too low, the system will mark it as a skill deficiency indicator.

[0053] Heart rate fluctuations: The heart rate sensor measures the trainee's heart rate. Drastic fluctuations in the trainee's heart rate indicate significant emotional fluctuations during the operation, potentially leading to anxiety or tension, which can affect their performance.

[0054] Gesture trajectory error: The trainee's gesture trajectory is recorded using an infrared motion capture device. If the gesture trajectory deviates significantly from the standard path, the system will mark it as low operation accuracy, which is used as an indicator of skill deficiency.

[0055] The feature extraction chip calculates features extracted from various dimensions to form a comprehensive indicator of operational skill deficiencies, including operational accuracy, stability, response latency, and heart rate fluctuations. This indicator is further used to generate learning feedback and decision-making instructions for students.

[0056] The generation process of scene adjustment instructions and sensory feedback signals is dynamically adjusted by the edge decision unit in the scene adaptive controller 5 according to the trainee's operational skill deficiency indicators and the preset reinforcement learning strategy. The specific process is as follows:

[0057] After the trainee completes the operation, the skill deficiency indicator calculated by the feature extraction chip is input into the edge decision-making unit. Based on this indicator and the reinforcement learning strategy, the decision-making unit analyzes the trainee's performance in the current operation and makes an appropriate judgment. If the trainee's operating ability deviates from the standard, the system generates scene adjustment instructions and sensory feedback signals.

[0058] Scene adjustment command generation:

[0059] Based on the trainees' skill deficiency indicators, the edge decision-making unit will adjust the complexity and difficulty of the virtual simulation scenario.

[0060] If the trainee's operation accuracy is low, the system will automatically adjust the task difficulty, simplify the operation steps or reduce interference factors in the operation, so that the trainee can complete the task better.

[0061] If the student's emotions fluctuate greatly, the system may reduce the scene pressure and ease the difficulty of operation to help the student gradually adapt.

[0062] The scene adjustment instructions are then transmitted to the 3D scene engine 2 through the edge decision unit, which is responsible for rendering and updating the virtual scene so that the trainees face an environment that is more suitable for their current skill level and emotional state.

[0063] Sensory feedback signal generation:

[0064] The edge decision unit generates sensory stimulation based on the trainee's feedback signals to further enhance the trainee's immersive experience. Sensory feedback signals include:

[0065] Tactile feedback: If the student operates improperly, the dynamic resistance glove's PWM duty cycle will adjust to increase the glove's mechanical resistance, helping the student better understand the feedback of incorrect operation. If the student applies too much pressure, the glove will increase resistance to indicate excessive force.

[0066] Temperature feedback: The temperature simulator changes the simulated body temperature based on the trainee's performance. If the trainee's performance is close to the preset standard, the simulator may adjust to a more stable body temperature. Conversely, the system may use a sudden temperature change to indicate that the trainee has made an improper operation in certain steps.

[0067] Feedback execution:

[0068] The generated scene adjustment instructions and sensory feedback signals are transmitted through the output of the edge decision unit to the 3D scene engine 2 and the multi-sensory feedback actuator 6. The 3D scene engine is responsible for updating the virtual scene to ensure that the trainee is always in an environment that meets their learning needs. The multi-sensory feedback actuator, on the other hand, provides the trainee with instant tactile and temperature feedback based on the sensory feedback signals, enhancing their operational experience.

[0069] Multi-sensory feedback actuator 6: This includes a display screen, a dynamic resistance glove, and a temperature simulator, which are used to output operation results and simulate resistance and temperature feedback. The display screen is used to simultaneously display the operation interface and teaching points; the dynamic resistance glove is used to simulate the resistance of the instrument operation; and the temperature control module is used to simulate changes in the mother's body surface temperature. In this embodiment, the dynamic resistance glove uses the Dexta Robotics Dexmo integrated dynamic resistance glove, and the temperature control device uses the TEC1-12706 semiconductor cooler. The dynamic resistance glove adjusts the PWM duty cycle of the resistance glove based on sensory feedback signals to linearly adjust the mechanical resistance. The temperature simulator changes the cooler activation state based on sensory feedback signals to adjust the glove temperature over a short period of time.

[0070] The main control processor 1 implements BOPPPS process control in the following ways:

[0071] Bridge-in stage: The system is awakened by the voice recognition unit and the 3D scene engine 2 is called to load the preset scene;

[0072] Objective stage: Based on the learning objectives, users input disease characteristics or preset operation scenarios through the touch screen to generate scenario-matching instructions;

[0073] Pre-assessment stage: obtaining the user's basic skill data through the physiological parameter acquisition array 4 and generating the initial difficulty coefficient;

[0074] Participatory Learning stage: The scene adaptive controller 5 modifies scene events based on real-time operation data;

[0075] Post-assessment stage: The main control processor 1 generates a skill assessment report by combining the recorded user operation success rate, response time, and error type;

[0076] Summary stage: Multisensory feedback actuator 6 presents a visual report.

[0077] The following is a detailed description of the working principle of the obstetrics and gynecology nursing virtual simulation learning interactive system.

[0078] The obstetrics and gynecology nursing virtual simulation learning interactive system provided by this utility model realizes intelligent simulation training based on the BOPPPS teaching process by integrating multi-source sensing technology, FPGA parallel processing and reinforcement learning decision-making mechanism. The system consists of a main control processor, a three-dimensional scene engine, a physiological parameter acquisition array, a startup module, a scene adaptive controller and a feedback output module. First, the trainee activates the preset case scene through touch input or voice recognition of the startup module, and the three-dimensional scene engine renders the corresponding operating environment. When the trainee performs nursing operations in the virtual environment, the pressure sensor matrix, heart rate sensor and infrared motion capture device synchronously collect signals such as force distribution, heart rate curve and movement trajectory, convert them into digital data and transmit them to the main control processor. The FPGA accelerator extracts features, cleans and normalizes these signals to generate a standardized multi-dimensional state vector; the control chip combines the teaching stage logic to evaluate the trainee's operation accuracy, stability, reaction speed and emotional changes, and form a skill evaluation vector. After the scene adaptive controller receives the vector, the feature extraction chip calculates the operation skill defect index, and the edge decision unit generates scene adjustment instructions and sensory feedback signals based on the reinforcement learning algorithm, which are transmitted to the three-dimensional engine and multi-sensory feedback module respectively, adjusting the simulation difficulty and feedback intensity in real time, and improving the students' practical ability in a closed loop.

[0079] The advantages of this technical solution are: the system integrates the BOPPPS teaching concept, FPGA efficient data processing, Unity3D virtual simulation technology, and reinforcement learning intelligent control mechanism, with significant innovative advantages. First, the FPGA and STM32 control chip collaboratively process multimodal physiological signals to achieve millisecond-level operation evaluation and feedback response. Second, through feature extraction and reinforcement learning, personalized scene adjustment strategies are dynamically generated to support adaptive adjustment of teaching difficulty. Third, a high-fidelity three-dimensional engine and multi-sensory feedback tactile resistance and temperature changes provide an immersive learning experience, effectively improving skill mastery and situational response capabilities, breaking through the venue, resource, and risk limitations of traditional nursing training.

[0080] The above generally describes the present invention in detail. However, it is obvious to those skilled in the art that modifications or improvements may be made to the present invention. Therefore, modifications or improvements that do not depart from the spirit of the present invention are within the scope of protection of the present invention.

Claims

1. A virtual simulation learning interactive system for obstetrics and gynecology nursing, characterized by: include: Main control processor (1); The physiological parameter acquisition array (4) is connected to the main control processor (1) via the SPI bus. A scene adaptive controller (5) is connected to the main control processor (1) via Ethernet; A multi-sensory feedback actuator (6) is connected to the scene adaptive controller; it comprises: a dynamic resistance glove and a temperature simulator; the dynamic resistance glove is used to output adjustable mechanical resistance to simulate the operation resistance of a maternity case scene; the temperature simulator is used to simulate the change of the mother's body surface temperature.

2. The obstetrics and gynecology nursing virtual simulation learning interactive system according to claim 1, characterized in that: The physiological parameter acquisition array (4) includes: A pressure sensing matrix, the output of which is connected to the analog input port of an ADC conversion circuit, which transmits data to the matrix input port of the main control processor (1) via an SPI data bus; A heart rate sensor, the output end of which is connected to a differential amplifier circuit and transmitted to a gradient input port of the main control processor (1) via an SPI data bus; The infrared motion capture device converts the analog signal of the operator's gesture space trajectory into a digital signal through an ADC conversion circuit, and transmits the digital signal to the matrix input port of the main control processor (1) through an SPI data bus.

3. The obstetrics and gynecology nursing virtual simulation learning interactive system according to claim 1 is characterized by: The scene adaptive controller (5) comprises: Feature extraction chip, used to calculate trainees' operational skill deficiency indicators in real time; Edge decision-making unit, used to execute reinforcement learning strategies to generate scenario adjustment instructions.

4. The obstetrics and gynecology nursing virtual simulation learning interactive system according to claim 1, characterized in that: It also includes a startup module (3), which includes: A touch input unit, used for inputting preset case characteristics by touch; The voice recognition unit is used to wake up the system based on keywords.

5. The obstetrics and gynecology nursing virtual simulation learning interactive system according to claim 1 is characterized by: The main control processor (1) comprises: An FPGA accelerator is used to receive the pressure distribution matrix digital signal, the trainee's heart rate signal and the gesture space trajectory digital signal transmitted by the physiological parameter acquisition array (4); The control chip is used to execute the state machine control of the BOPPPS teaching protocol and generate a multi-dimensional skill evaluation vector based on the signal collected by the FPGA accelerator.

6. The obstetrics and gynecology nursing virtual simulation learning interactive system according to claim 4, characterized in that: It also includes a three-dimensional scene engine (2), which includes a graphics rendering engine and is used to construct a three-dimensional virtual scene library for obstetrics and gynecology nursing, so as to realize the simulation of vaginal cervical medication, delivery care, and obstetric emergency operation scenes.

7. The obstetrics and gynecology nursing virtual simulation learning interactive system according to claim 6, characterized in that: The three-dimensional scene engine (2) further includes: A main communication channel connected to the scene adaptive controller (5) via Ethernet; The secondary communication channel is connected to the startup module (3) via the SPI bus.

Images