Emergency and critical disease nursing training system and method based on vr

Abstract

The invention discloses an acute and critical disease nursing training system and method based on vr, and relates to the technical field of medical systems. A learning mode and an assessment mode are selected through a handle trigger; in the learning mode, the nursing operation process is dynamically displayed through vr glasses and a scene generation engine, and the scene generation engine constructs a disease evolution path based on a knowledge graph and associates user historical data to match personalized teaching cases; receiving a selection instruction of a user for an assessment item in the assessment mode, receiving an operation instruction of the user at the same time, recording time sequence data of an operation path and selecting a behavior logic chain; generating a score according to a dynamic scoring algorithm including an operation track accuracy parameter and a decision timeliness parameter; the user is personally on the scene to carry out nursing training and examination on acute and critical diseases, and trial and error in a highly-simulated critical scene are carried out; a targeted training scene is intelligently pushed according to the specialty and ability of the user; and the speed and quality of acute and critical disease nursing are balanced through multi-dimensional scoring.

Description

Technical Field

[0001] The present application relates to the field of medical system technology, and in particular to a VR-based critical care training system and method. Background Art

[0002] Emergency and critical care nursing training is a systematic education program for medical staff to improve their rescue capabilities in emergency medical scenarios. Its core goal is to cultivate professionals with rapid response, accurate assessment and efficient handling capabilities. Traditional emergency and critical care nursing training mostly relies on physical simulation equipment and clinical internship models. Currently, VR is introduced for nursing training. Although some venue restrictions have been solved through the construction of a virtual environment, its scene generation mostly uses a preset static case library as the core teaching resource, and the technical architecture mostly relies on fixed script disease scenarios, such as standard cardiopulmonary resuscitation procedures, resulting in a serious disconnect between the training content and the complexity and dynamics of real clinical practice. For example, in shock treatment training, traditional systems can only simulate changes in preset parameters such as blood pressure and heart rate, and it is difficult to generate a chain reaction of complications, such as acute kidney injury triggered by untimely rehydration, making it difficult for trainees to grasp the causal logic and dynamic decision-making ability of disease evolution.

[0003] At the same time, most of the current scoring systems use operation accuracy and step correctness as the basis for scoring, and lack the emphasis on the timeliness pressure of clinical decision-making, such as the "1-hour bundled treatment" time window for septic shock. Therefore, although the medical staff have a high pass rate in standardized case training, the trainees' handling errors caused by sudden complications and other situations in real emergency scenarios are still not low, exposing the contradiction between static training and dynamic clinical needs. Therefore, it is urgent to develop a VR critical care training system that integrates knowledge graph drive and has real-time dynamic response capabilities to break through the static and homogeneous bottlenecks of traditional training. Summary of the invention

[0004] This application aims to provide a VR-based critical care training system and method. The system breaks through the static and homogeneous bottlenecks of traditional training by integrating knowledge graph drive and real-time dynamic response. The specific technical solutions are as follows: In a first aspect of the present application, a VR-based critical care training system is provided, comprising: Operation perception module, triggering mode selection signal through handle trigger, including learning mode and assessment mode; Dynamic learning module, in learning mode, uses VR glasses to dynamically display nursing operation processes through a scene generation engine. The scene generation engine builds disease evolution paths based on knowledge graphs and associates user historical data to match personalized teaching cases; The intelligent assessment module receives the user's selection instructions for assessment items in the assessment mode, and also receives the user's operation instructions based on the system prompts, and records the time series data of the operation path and the logic chain of the selection behavior in real time; The multidimensional scoring module generates a total score and sub-item scores based on a dynamic scoring algorithm. The dynamic scoring algorithm includes an operation trajectory accuracy parameter and a decision timeliness parameter. The operation trajectory accuracy parameter is calculated based on the distance between the user's operation path and the standard path; the decision timeliness parameter uses the time series difference method to evaluate the duration of the operation node interval.

[0005] In one embodiment of the present application, the operation perception module further includes: A feature acquisition unit, which acquires the user's physiological data through a wearable sensor, wherein the physiological data includes heart rate variability, pupil diameter change rate, and body posture vector; The dynamic scoring algorithm in the multidimensional scoring module also includes a physiological stability parameter, calculated as follows: S=α||P||+β⋅HRV -1 , where ||P|| is the modulus of the somatosensory posture vector, HRV is the heart rate variability index, and α and β are normalization coefficients.

[0006] In one embodiment of the present application, the dynamic learning module and the intelligent assessment module further include a multimodal interaction submodule, specifically including: The voice instruction unit recognizes first aid terms and generates voice operation instructions. The dynamic learning module triggers corresponding voice prompts according to the different states of the patient. The intelligent assessment module prompts different assessment contents and corresponding operation options according to the user's historical data; The tactile feedback unit is integrated into the operating end of the VR handle and simulates the operating resistance of the medical device through the damper; The device parameter generation unit generates various physiological indicators of patients in real time through virtual reality rendering, and simulates changes in physical signs based on the patients' physiological indicators.

[0007] In one embodiment of the present application, the feature acquisition unit collects the user's physiological index data in real time through the wearable device, specifically including collecting the heart rate through the heart rate sensor, capturing the changes in pupil diameter and position through the infrared camera in the VR glasses, and collecting the hand tremor frequency and amplitude through the electromyography detection device; The dynamic learning module and the intelligent assessment module also include a scene difficulty dynamic binding unit, which generates a scene complexity adjustment coefficient according to the physiological indicator data. The higher the scene complexity adjustment coefficient, the more events that need to be operated in the scene, and the higher the event difficulty: When the user's heart rate exceeds a preset threshold range, the stress relief subunit is activated to reduce the scene complexity adjustment coefficient; When the pupil diameter change rate exceeds a preset threshold per unit time, the attention calibration subunit is triggered, the key operation area is highlighted in the VR field of view, the operation is completed under the highlighted prompt, and the scene complexity adjustment coefficient is reduced; The dynamic scoring algorithm in the multi-dimensional scoring module also includes an operation difficulty index, and the operation difficulty index is positively correlated with the scene complexity adjustment coefficient of each operation.

[0008] In one embodiment of the present application, it also includes a difficulty adaptation module, which organizes the scene complexity adjustment coefficient in each assessment of the user, gradually increases the initial difficulty of the dynamic learning mode and the intelligent assessment mode, and at the same time, when the scene complexity adjustment coefficient decreases continuously during the user operation, the initial difficulty of the dynamic learning mode and the intelligent assessment mode is reduced.

[0009] In one embodiment of the present application, the intelligent assessment module further includes a team collaboration training submodule, and the submodule includes: The role management unit is configured to create an emergency team operation scenario including doctors, nurses and pharmacists in the local area network. Differentiated permissions are configured for each role according to clinical operation specifications: the doctor role has the right to make diagnosis decisions and the approval authority for the activation of medical equipment; the nurse role executes nursing operation instructions and monitors the patient's vital signs parameters; the pharmacist role is responsible for drug dosage calculation and compatibility taboo review; Terminal synchronization unit, real-time synchronization of multi-terminal operation data, including when the nurse performs venipuncture, the virtual view of the doctor's terminal synchronously displays the 3D model of the puncture progress; the pharmacist's medication dispensing operation error will trigger the terminal warning interface of the whole team; The role replacement unit can replace the roles in the emergency team with virtual roles before the assessment; during the assessment, when it is detected that the local member's operation is interrupted, the virtual role replacement is started: the virtual role can be operated through the terminal access of the remote personnel, or through the system operation that generates a standardized operation sequence based on the emergency knowledge graph.

[0010] In one embodiment of the present application, the role replacement unit further includes replacing the patient, specifically: The virtual patient generation subunit randomly generates virtual patient models of different ages, genders and body types; The real person identification subunit operates with the real person as the patient. The image acquisition device collects and analyzes the key bone point data of the patient in real time and calculates the height and body shape characteristics, and generates age and gender prediction coefficients at the same time; The adaptive subunit is operated to adjust the nursing operation position according to the randomly generated virtual patient parameters or the identified real personnel parameters, and at the same time, the dynamic offset algorithm of the instrument positioning coordinates is executed.

[0011] In one embodiment of the present application, an emergency connection module is also included, and the module includes: The assessment docking unit synchronizes the scoring data in the user multi-dimensional scoring module with the hospital emergency dispatch platform database. Users who pass the assessment will be granted device use rights and participate in hospital emergency dispatch: The enhanced guidance unit establishes a real-time mapping relationship between the physical space coordinate system and the virtual coordinate system through an augmented reality space registration algorithm based on feature point matching. When the user participates in real first aid, a dynamic operation guidance layer is superimposed in the field of view of the AR glasses, displaying the patient's vital signs trend curve and recommended medication dosage threshold in real time. It also generates a pressure feedback light column at the patient's operation position. When the operation position is inaccurate or the operation method is incorrect, a red ripple warning is triggered accompanied by tactile feedback.

[0012] In one embodiment of the present application, the emergency connection module also includes a multimodal data fusion unit, which accesses the hospital PACS system to retrieve the patient's historical imaging data and marks the abnormal bone structure warning area on the AR interface; at the same time, it uploads the emergency operation video to the quality control platform in real time, and generates an unalterable emergency capability certification record based on blockchain technology.

[0013] In a second aspect of the present application, a VR-based critical care training method is provided, comprising: The trigger mode selection signal is selected by the handle trigger, including learning mode and assessment mode; In the learning mode, the nursing operation process is dynamically displayed through VR glasses through the scene generation engine. The scene generation engine constructs the disease evolution path based on the knowledge graph and associates the user's historical data to match personalized teaching cases; In the assessment mode, the user receives the selection instructions for the assessment items, and at the same time receives the user's operation instructions based on the system prompts, and records the time series data of the operation path and the logic chain of the selection behavior in real time; The total score and sub-item scores are generated according to the dynamic scoring algorithm. The dynamic scoring algorithm includes the operation trajectory accuracy parameter and the decision timeliness parameter. The operation trajectory accuracy parameter is calculated based on the distance between the user's operation path and the standard path; the decision timeliness parameter uses the time series difference method to evaluate the interval duration of the operation nodes.

[0014] This application has the following beneficial effects: 1. VR technology enables users to conduct immersive emergency and critical care training and assessment, allowing trainees to try and make mistakes multiple times in highly simulated critical scenarios (such as ventricular fibrillation rescue); at the same time, a dynamic case generation mechanism driven by knowledge graphs simulates the disease development and complications that may occur in patients in real scenarios, and can intelligently push targeted training scenarios based on the user's professional and ability shortcomings. For example, for users with a tracheal intubation pass rate of less than 80%, priority is given to pushing difficult airway treatment cases to achieve targeted reinforcement; and decision timeliness is introduced in the scoring, and the time series difference method is used to evaluate the interval length of the operation nodes, and the speed and quality of emergency and critical care are balanced through multi-dimensional scoring.

[0015] 2. Integrate heart rate variability and body posture data to break through the single-dimensional evaluation limitation of traditional VR training that only relies on operation accuracy. For example, when a trainee performs tracheal intubation, the system can simultaneously identify hand tremors (abnormal body posture) and heart rate surges (HRV reduction), accurately locate weak links in skills, add physiological stability parameters to the dynamic scoring algorithm, and supplement the scoring of psychological quality. For example, if a user is in an extremely nervous state (hand tremors, heart rate surges), although he or she completes the assessment content quickly and accurately, the instability caused by completing the assessment in a nervous state also needs to be reflected in the score difference.

[0016] 3. Dynamic simulation of instrument resistance is achieved through dampers, making virtual operation force feedback similar to real clinical scenarios. The stepped resistance design strengthens muscle memory training, especially improving the success rate of delicate operations such as deep vein puncture; the physiological parameter model based on the finite element algorithm can accurately reflect the pathological and physiological chain reactions. For example: when the compression depth of cardiopulmonary resuscitation is insufficient, the system will simultaneously present the disappearance of carotid artery pulsation and pupil dilation signs, enhancing the immersion of training.

[0017] 4. By integrating multi-dimensional physiological indicators such as heart rate, pupil changes and hand tremors, the system breaks through the limitation of traditional VR training that relies only on operation accuracy. For example, when the user performs cardiopulmonary resuscitation, the hand tremor amplitude exceeds the standard (>5mm) will trigger a real-time correction prompt, and the abnormal increase in heart rate (>120bpm) will reduce the complexity of the scene, forming a dual-track evaluation mechanism of "operation accuracy-psychological quality". The same operation will score higher in the more difficult scenes, and the difficulty will gradually increase as the user becomes more proficient in the operation. At the same time, when the scene complexity adjustment coefficient decreases due to continuous nervous hand tremors, the initial difficulty of the dynamic learning mode and the intelligent assessment mode is reduced; different from the dynamic difficulty in the above-mentioned assessment process, it is the reduced initial difficulty that helps users consolidate the operation of a lower level of difficulty; that is, the dynamic binding of the scene complexity adjustment coefficient and the knowledge graph node weight enables the system to intelligently push training content according to the user's ability. For example, for users with a low pass rate for venous puncture, high-weight node-related scenes (such as venous collapse simulation for shock patients) are matched first, and the initial difficulty is gradually increased based on historical data to shorten the skill enhancement cycle.

[0018] 5. The differentiated configuration of role permissions and the terminal data synchronization mechanism enable doctors, nurses, and pharmacists in the emergency team to form an efficient collaborative closed loop. For example, when a nurse performs a venipuncture, the doctor uses a three-dimensional model to guide the needle insertion angle in real time, and the pharmacist simultaneously reviews the medication plan, effectively shortening the emergency time and passing the group assessment in the assessment, making critical care closer to the real scene; at the same time, the virtual character can be controlled by a remote person to operate, or it can be directly operated by the system based on the emergency knowledge graph to generate a standardized operation sequence, avoiding the situation where different roles are not in place during the assessment or training, and effectively responding to abnormal situations such as the sudden departure of personnel. For example, when a nurse's operation is detected to be interrupted, the system can seamlessly switch to the AI ​​agent to execute the standardized nursing process to ensure that the emergency operation is not interrupted; further, by randomly replacing virtual patients, the virtual training scene covers all age groups and all pathological types. For example, the system automatically adjusts the attachment position of the defibrillator electrode pads according to the body shape of obese patients, simulating the actual clinical operation resistance gradient; and the system can be operated by real people as patients to make the operation scene more realistic. After the body shape and gender of the real person are identified by the image acquisition device, the position of the person being operated is synchronously adjusted.

[0019] 6. The enhanced guidance unit is based on physical-virtual coordinate system mapping, and establishes a spatial alignment model through feature point matching; the AR dynamic guidance layer includes: vital sign trend curve: integrating real-time data from the hospital HIS system, and using the LSTM model to predict changes in vital signs in the next 5 minutes; medication dose threshold warning: calculating the recommended dose range (such as epinephrine 0.01-0.03mg / kg) based on the patient's weight and liver and kidney function data; the pressure feedback system includes: tactile feedback module: piezoelectric actuator generates frequency-adjustable vibration (50-200Hz), and wrong operation triggers gradient enhanced feedback; visual warning layer: using a particle system to generate a red ripple diffusion effect, and activating a full-screen warning when the offset error is greater than 3mm. The patient's CT / MRI image data is retrieved through the DICOM protocol, and the U-Net algorithm is used to automatically annotate the abnormal bone area (such as fracture line, joint dislocation); at the same time, the abnormal structure is highlighted in translucent red (transparency is negatively correlated with the severity of the lesion), and mechanical simulation data (such as the maximum load-bearing threshold of the fracture site) is superimposed. The deep integration of the PACS system and AR annotation enables on-site emergency personnel to quickly identify hidden injuries (such as spinal compression fractures). The abnormal structure warning area is superimposed with biomechanical simulation data to effectively reduce the incidence of secondary injuries. At the same time, the operation video is analyzed, and a 3D convolutional neural network is used to extract the operation trajectory features (such as instrument angle, compression depth), and the similarity is matched with the standard operation library. Blockchain evidence is further stored: key frames of the first aid process (such as defibrillation timestamps, medication records) generate hash values ​​and upload them to the chain, and the evidence data is synchronized to the supervision platform to effectively protect patients and users. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0021] Figure 1 A schematic diagram of the electronic device structure of the hardware operating environment involved in an embodiment of the present application.

[0022] Figure 2 It is a functional module diagram of a VR-based critical care nursing training system provided in an embodiment of the present application.

[0023] Figure 3 This is a module construction diagram of a VR-based critical care nursing training system provided in an embodiment of the present application.

[0024] Figure 4 This is a step flow chart of a VR-based critical care nursing training method provided in an embodiment of the present application.

[0025] Symbols in the figure: 1001 - processor, 1002 - communication bus, 1003 - user interface, 1004 - network interface, 1005 - memory. DETAILED DESCRIPTION

[0026] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0027] The solution of the present application is further described below in conjunction with the accompanying drawings.

[0028] like Figure 1 As shown, the electronic device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) memory, or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0029] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0030] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module and a data storage module.

[0031] exist Figure 1In the electronic device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the electronic device of the present invention can be set in the electronic device, and the electronic device calls a VR-based critical care nursing training system stored in the data storage module in the memory 1005 through the processor 1001, and executes a VR-based critical care nursing training method provided in an embodiment of the present application.

[0032] Based on the aforementioned hardware operating environment and system architecture, in the first aspect of the present application, refer to Figure 2 and Figure 3 As shown, a VR-based critical care training system is provided, comprising: Operation perception module, triggering mode selection signal through handle trigger, including learning mode and assessment mode; It should be noted that when the handle trigger is in learning mode, the basic operation guidance is activated; when the assessment mode is selected, the timed evaluation is started; Dynamic learning module, in learning mode, uses VR glasses to dynamically display nursing operation processes through a scene generation engine. The scene generation engine builds disease evolution paths based on knowledge graphs and associates user historical data to match personalized teaching cases; It should be noted that the scenario generation engine builds a knowledge graph based on the Neo4j graph database, which includes: disease type nodes: critical illnesses such as acute respiratory failure and cardiogenic shock; operation relationship chain: time sequence logic of tracheal intubation - ventilator parameter setting - vital signs monitoring; Personalized case matching uses collaborative filtering algorithms, taking the user's historical operation trajectory as the input vector and outputting teaching content that matches the user's professional field; The intelligent assessment module receives the user's selection instructions for assessment items in the assessment mode, and also receives the user's operation instructions based on the system prompts, and records the time series data of the operation path and the logic chain of the selection behavior in real time; It should be noted that the behavior logic chain uses the decision tree model to mark deviation types such as "sequence error" and "missing steps"; The multidimensional scoring module generates a total score and sub-item scores based on a dynamic scoring algorithm. The dynamic scoring algorithm includes an operation trajectory accuracy parameter and a decision timeliness parameter. The operation trajectory accuracy parameter is calculated based on the distance between the user's operation path and the standard path; the decision timeliness parameter uses the time series difference method to evaluate the duration of the operation node interval.

[0033] In this implementation, VR technology enables users to conduct immersive emergency and critical care training and assessment, allowing trainees to try and make mistakes multiple times in highly simulated critical scenarios (such as ventricular fibrillation rescue); at the same time, a dynamic case generation mechanism driven by knowledge graphs simulates the disease development and complications that may occur in patients in real scenarios, and can intelligently push targeted training scenarios based on user expertise and ability shortcomings. For example, for users with a tracheal intubation pass rate of less than 80%, priority is given to pushing difficult airway treatment cases to achieve targeted reinforcement; and decision timeliness is introduced in the scoring, and the time series difference method is used to evaluate the interval length of the operation nodes, and the speed and quality of emergency and critical care are balanced through multi-dimensional scoring.

[0034] In one embodiment of the present application, the operation perception module further includes: A feature acquisition unit, which acquires the user's physiological data through a wearable sensor, wherein the physiological data includes heart rate variability, pupil diameter change rate, and body posture vector; The dynamic scoring algorithm in the multidimensional scoring module also includes a physiological stability parameter, calculated as follows: S=α||P||+β⋅HRV -1 , where ||P|| is the modulus of the somatosensory posture vector, HRV is the heart rate variability index, and α and β are normalization coefficients.

[0035] In this implementation, the heart rate variability and body posture data are integrated to break through the single-dimensional evaluation limitation of traditional VR training that only relies on operation accuracy. For example, when a trainee performs tracheal intubation, the system can simultaneously identify hand tremors (abnormal body posture) and heart rate surges (HRV reduction), accurately locate weak links in skills, add physiological stability parameters to the dynamic scoring algorithm, and supplement the scoring of psychological quality. For example, if a user is in an extremely nervous state (hand tremors, heart rate surges), although he or she completes the assessment content quickly and accurately, the instability caused by completing the assessment in a nervous state also needs to be reflected in the score difference.

[0036] In one embodiment of the present application, the dynamic learning module and the intelligent assessment module further include a multimodal interaction submodule, specifically including: The voice instruction unit recognizes first aid terms and generates voice operation instructions. The dynamic learning module triggers corresponding voice prompts according to the different states of the patient. The intelligent assessment module prompts different assessment contents and corresponding operation options according to the user's historical data; It should be noted that in the dynamic learning module, users are taught through voice operation guidance, and the dynamic learning module includes video teaching and training teaching; in the intelligent assessment module, assessment content and operation options are output to users through voice operation guidance; The tactile feedback unit is integrated into the operating end of the VR handle, and simulates the operating resistance of the medical device through the damper. It should be noted that the damper simulates the operating resistance gradient of the medical device through the deformation of the piezoelectric layer (such as chest compression resistance: 30-50N; venous puncture resistance: 5-8N), and adjusts the feedback intensity in real time according to the operating accuracy, including linear reduction of resistance under precise operation and step-by-step resistance enhancement under the offset threshold. The device parameter generation unit generates various physiological indicators of patients in real time through virtual reality rendering, and simulates changes in physical signs based on the patients' physiological indicators.

[0037] It should be noted that physiological indicators include core indicators such as heart rate, blood pressure, and respiratory rate. Finite element algorithms are used to simulate pathological changes (such as the hemodynamic model of hemorrhagic shock), and vital sign parameters are adjusted according to operational feedback (such as the heart rate waveform recovering sinus rhythm after correct chest compression; the blood pressure curve dropping sharply after incorrect medication).

[0038] In this embodiment, the dynamic simulation of the instrument resistance is achieved through the damper, so that the virtual operation force feedback is similar to the real clinical scene. The stepped resistance design strengthens muscle memory training, especially improves the success rate of delicate operations such as deep vein puncture; the physiological parameter model based on the finite element algorithm can accurately reflect the pathophysiological chain reaction. For example: when the cardiopulmonary resuscitation compression depth is insufficient, the system synchronously presents the disappearance of the carotid artery pulsation and the sign of pupil dilation, enhancing the immersion of training.

[0039] In one embodiment of the present application, the feature acquisition unit collects the user's physiological index data in real time through the wearable device, specifically including collecting the heart rate through the heart rate sensor, capturing the changes in pupil diameter and position through the infrared camera in the VR glasses, and collecting the hand tremor frequency and amplitude through the electromyography detection device; The dynamic learning module and the intelligent assessment module also include a scene difficulty dynamic binding unit, which generates a scene complexity adjustment coefficient according to the physiological indicator data. The higher the scene complexity adjustment coefficient, the more events that need to be operated in the scene, and the higher the event difficulty: When the user's heart rate exceeds a preset threshold range, the stress relief subunit is activated to reduce the scene complexity adjustment coefficient; When the pupil diameter change rate exceeds a preset threshold per unit time, the attention calibration subunit is triggered, the key operation area is highlighted in the VR field of view, the operation is completed under the highlighted prompt, and the scene complexity adjustment coefficient is reduced; The dynamic scoring algorithm in the multi-dimensional scoring module also includes an operation difficulty index, and the operation difficulty index is positively correlated with the scene complexity adjustment coefficient of each operation.

[0040] It should be noted that the difficulty of the assessment process is dynamic. When the user becomes nervous or inattentive, the assessment difficulty is reduced to gradually train the user's psychological quality. The dynamic scoring algorithm also includes an operation difficulty index, which is positively correlated with the scene complexity adjustment coefficient of each operation. That is to say, the final scores of the same sub-item operations in different scenarios are different. For example, the score of a scenario where only venous puncture is required will be lower than that of a venous puncture that also requires preparation for hemostasis and defibrillation. In other words, the score in a scenario with a high scene complexity adjustment coefficient will be higher than that in a scenario with a low scene complexity adjustment coefficient.

[0041] In one embodiment of the present application, it also includes a difficulty adaptation module, which organizes the scene complexity adjustment coefficient in each assessment of the user, gradually increases the initial difficulty of the dynamic learning mode and the intelligent assessment mode, and at the same time, when the scene complexity adjustment coefficient decreases continuously during the user operation, the initial difficulty of the dynamic learning mode and the intelligent assessment mode is reduced.

[0042] In this embodiment, by integrating multi-dimensional physiological indicators such as heart rate, pupil changes and hand tremor, the system breaks through the limitation of traditional VR training that only relies on operation accuracy. For example, when the user performs cardiopulmonary resuscitation, the hand tremor amplitude exceeds the standard (>5mm) will trigger a real-time correction prompt, and the abnormal increase in heart rate (>120bpm) will reduce the complexity of the scene, forming a "operation accuracy-psychological quality" dual-track evaluation mechanism. The same operation, the higher the score in the more difficult scene, the difficulty gradually increases as the user becomes more proficient in the operation, and when the scene complexity adjustment coefficient decreases due to continuous nervous hand shaking, the initial difficulty of the dynamic learning mode and the intelligent assessment mode is reduced; different from the dynamic difficulty in the above-mentioned assessment process, it is the reduced initial difficulty that helps users consolidate the operation of a lower level of difficulty; that is, the dynamic binding of the scene complexity adjustment coefficient and the knowledge graph node weight enables the system to intelligently push training content according to the user's ability. For example, for users with a low pass rate for venous puncture, high-weight node-related scenes (such as venous collapse simulation of shock patients) are matched first, and the initial difficulty is gradually increased based on historical data to shorten the skill enhancement cycle.

[0043] In one embodiment of the present application, the intelligent assessment module further includes a team collaboration training submodule, and the submodule includes: The role management unit is configured to create an emergency team operation scenario including doctors, nurses and pharmacists in the local area network. Differentiated permissions are configured for each role according to clinical operation specifications: the doctor role has the right to make diagnosis decisions and the approval authority for the activation of medical equipment; the nurse role executes nursing operation instructions and monitors the patient's vital signs parameters; the pharmacist role is responsible for drug dosage calculation and compatibility taboo review; Terminal synchronization unit, real-time synchronization of multi-terminal operation data, including when the nurse performs venipuncture, the virtual view of the doctor's terminal synchronously displays the 3D model of the puncture progress; the pharmacist's medication dispensing operation error will trigger the terminal warning interface of the whole team; The role replacement unit can replace the roles in the emergency team with virtual roles before the assessment; during the assessment, when it is detected that the local member's operation is interrupted, the virtual role replacement is started: the virtual role can be operated through the terminal access of the remote personnel, or through the system operation that generates a standardized operation sequence based on the emergency knowledge graph.

[0044] In one embodiment of the present application, the role replacement unit further includes replacing the patient, specifically: The virtual patient generation subunit randomly generates virtual patient models of different ages, genders and body types; The real person identification subunit operates with the real person as the patient. The image acquisition device collects and analyzes the key bone point data of the patient in real time and calculates the height and body shape characteristics, and generates age and gender prediction coefficients at the same time; The adaptive subunit is operated to adjust the nursing operation position according to the randomly generated virtual patient parameters or the identified real personnel parameters, and at the same time, the dynamic offset algorithm of the instrument positioning coordinates is executed.

[0045] In this embodiment, the differentiated configuration of role permissions and the terminal data synchronization mechanism enable the doctors, nurses, and pharmacists in the emergency team to form an efficient collaborative closed loop. For example, when the nurse performs intravenous puncture, the doctor guides the needle insertion angle in real time through the three-dimensional model, and the pharmacist simultaneously reviews the medication plan, effectively shortening the emergency time, passing the group assessment in the assessment, and making critical care closer to the real scene; at the same time, the virtual character can be controlled by remote personnel to operate, or it can be directly operated by the system based on the emergency knowledge graph to generate a standardized operation sequence, so as to avoid the situation where different roles are not in place during the assessment or training, and effectively respond to abnormal situations such as the sudden departure of personnel. For example, when the nurse's operation is interrupted, the system can seamlessly switch to the AI ​​agent to execute the standardized nursing process to ensure that the emergency operation is not interrupted; further, by randomly replacing virtual patients, the virtual training scene covers all age groups and all pathological types. For example, the system automatically adjusts the attachment position of the defibrillator electrode pads according to the body shape of obese patients, simulating the actual clinical operation resistance gradient; and the system can be operated by real people as patients to make the operation scene more realistic. After the body shape and gender of the real person are identified by the image acquisition device, the position of the person being operated is synchronously adjusted.

[0046] In one embodiment of the present application, an emergency connection module is also included, and the module includes: The assessment docking unit synchronizes the scoring data in the user multi-dimensional scoring module with the hospital emergency dispatch platform database. Users who pass the assessment will be granted device use rights and participate in hospital emergency dispatch: The enhanced guidance unit establishes a real-time mapping relationship between the physical space coordinate system and the virtual coordinate system through an augmented reality space registration algorithm based on feature point matching. When the user participates in real first aid, a dynamic operation guidance layer is superimposed in the field of view of the AR glasses, displaying the patient's vital signs trend curve and recommended medication dosage threshold in real time. It also generates a pressure feedback light column at the patient's operation position. When the operation position is inaccurate or the operation method is incorrect, a red ripple warning is triggered accompanied by tactile feedback.

[0047] It should be noted that the enhanced guidance unit is based on physical-virtual coordinate system mapping and establishes a spatial alignment model through feature point matching; the AR dynamic guidance layer includes the vital signs trend curve: integrating real-time data from the hospital HIS system, and using the LSTM model to predict changes in vital signs in the next 5 minutes; medication dosage threshold warning: calculating the recommended dosage range based on the patient's weight and liver and kidney function data (such as epinephrine 0.01-0.03 mg / kg); the pressure feedback system includes a tactile feedback module: the piezoelectric actuator generates frequency-adjustable vibration (50-200Hz), and incorrect operations trigger gradient enhanced feedback; the visual warning layer: a particle system is used to generate a red ripple diffusion effect, and a full-screen warning is activated when the offset error is greater than 3mm.

[0048] In one embodiment of the present application, the emergency connection module also includes a multimodal data fusion unit, which accesses the hospital PACS system to retrieve the patient's historical imaging data and marks the abnormal bone structure warning area on the AR interface; at the same time, it uploads the emergency operation video to the quality control platform in real time, and generates an unalterable emergency capability certification record based on blockchain technology.

[0049] It should be noted that the patient's CT / MRI image data is retrieved through the DICOM protocol, and the U-Net algorithm is used to automatically annotate the abnormal bone areas (such as fracture lines and joint dislocations); at the same time, the abnormal structure is highlighted in translucent red (transparency is negatively correlated with the severity of the lesion), and the mechanical simulation data (such as the maximum load-bearing threshold of the fracture site) is superimposed. The deep integration of the PACS system and AR annotation enables on-site emergency personnel to quickly identify hidden injuries (such as spinal compression fractures). The abnormal structure warning area is superimposed with biomechanical simulation data to effectively reduce the incidence of secondary injuries; at the same time, the operation video is analyzed, and the 3D convolutional neural network is used to extract the operation trajectory features (such as instrument angle, compression depth), and the similarity is matched with the standard operation library; further blockchain evidence is stored: the key frames of the emergency process (such as defibrillation timestamps, medication records) generate hash values ​​and upload them to the chain, and the evidence data is synchronized to the supervision platform to effectively protect patients and users.

[0050] In the second aspect of the present application, reference is made to Figure 4 As shown, a VR-based critical care training method is provided, including: The trigger mode selection signal is selected by the handle trigger, including learning mode and assessment mode; In the learning mode, the nursing operation process is dynamically displayed through VR glasses through the scene generation engine. The scene generation engine constructs the disease evolution path based on the knowledge graph and associates the user's historical data to match personalized teaching cases; In the assessment mode, the user receives the selection instructions for the assessment items, and at the same time receives the user's operation instructions based on the system prompts, and records the time series data of the operation path and the logic chain of the selection behavior in real time; The total score and sub-item scores are generated according to the dynamic scoring algorithm. The dynamic scoring algorithm includes the operation trajectory accuracy parameter and the decision timeliness parameter. The operation trajectory accuracy parameter is calculated based on the distance between the user's operation path and the standard path; the decision timeliness parameter uses the time series difference method to evaluate the interval duration of the operation nodes.

[0051] It should be noted that the specific implementation method of a VR-based critical care nursing training method in an embodiment of the present application refers to the specific implementation method of a VR-based critical care nursing training system proposed in the first aspect of the aforementioned embodiment of the present application, and will not be repeated here.

[0052] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that an article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such article or device. In the absence of more restrictions, the elements defined by the sentence "includes..." do not exclude the existence of other identical elements in the article or device including the elements.

[0053] The above is a detailed introduction to the VR-based critical care nursing training system provided. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the VR-based critical care nursing training system and its core idea of ​​the present application; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A VR-based critical care nursing training system, characterized in that: include: Operation perception module, triggering mode selection signal through handle trigger, including learning mode and assessment mode; Dynamic learning module, in learning mode, uses VR glasses to dynamically display nursing operation processes through a scene generation engine. The scene generation engine builds disease evolution paths based on knowledge graphs and associates user historical data to match personalized teaching cases; The intelligent assessment module receives the user's selection instructions for assessment items in the assessment mode, and also receives the user's operation instructions based on the system prompts, and records the time series data of the operation path and the logic chain of the selection behavior in real time; The multidimensional scoring module generates a total score and sub-item scores based on a dynamic scoring algorithm. The dynamic scoring algorithm includes an operation trajectory accuracy parameter and a decision timeliness parameter. The operation trajectory accuracy parameter is calculated based on the distance between the user's operation path and the standard path; the decision timeliness parameter uses the time series difference method to evaluate the duration of the operation node interval.

2. A VR-based critical care nursing training system according to claim 1, characterized in that: The operation perception module also includes: A feature acquisition unit, which acquires the user's physiological data through a wearable sensor, wherein the physiological data includes heart rate variability, pupil diameter change rate, and body posture vector; The dynamic scoring algorithm in the multidimensional scoring module also includes a physiological stability parameter, calculated as follows: S=α||P||+β⋅HRV -1 , where ||P|| is the modulus of the somatosensory posture vector, HRV is the heart rate variability index, and α and β are normalization coefficients.

3. A VR-based critical care nursing training system according to claim 2, characterized in that: The dynamic learning module and intelligent assessment module also include a multimodal interaction submodule, specifically including: The voice instruction unit recognizes first aid terms and generates voice operation instructions. The dynamic learning module triggers corresponding voice prompts according to the different states of the patient. The intelligent assessment module prompts different assessment contents and corresponding operation options according to the user's historical data; The tactile feedback unit is integrated into the operating end of the VR handle and simulates the operating resistance of the medical device through the damper; The device parameter generation unit generates various physiological indicators of patients in real time through virtual reality rendering, and simulates changes in physical signs based on the patients' physiological indicators.

4. A VR-based critical care nursing training system according to claim 2, characterized in that: The feature acquisition unit collects the user's physiological index data in real time through the wearable device, specifically including collecting the heart rate through the heart rate sensor, capturing the changes in pupil diameter and position through the infrared camera in the VR glasses, and collecting the hand tremor frequency and amplitude through the electromyography detection device; The dynamic learning module and the intelligent assessment module also include a scene difficulty dynamic binding unit, which generates a scene complexity adjustment coefficient according to the physiological indicator data. The higher the scene complexity adjustment coefficient, the more events that need to be operated in the scene, and the higher the event difficulty: When the user's heart rate exceeds a preset threshold range, the stress relief subunit is activated to reduce the scene complexity adjustment coefficient; When the pupil diameter change rate exceeds a preset threshold per unit time, the attention calibration subunit is triggered, the key operation area is highlighted in the VR field of view, the operation is completed under the highlighted prompt, and the scene complexity adjustment coefficient is reduced; The dynamic scoring algorithm in the multi-dimensional scoring module also includes an operation difficulty index, and the operation difficulty index is positively correlated with the scene complexity adjustment coefficient of each operation.

5. A VR-based critical care nursing training system according to claim 4, characterized in that: It also includes a difficulty adaptation module, which organizes the scene complexity adjustment coefficient in each user's assessment, gradually increases the initial difficulty of the dynamic learning mode and the intelligent assessment mode, and at the same time, when the scene complexity adjustment coefficient decreases continuously during the user's operation, reduces the initial difficulty of the dynamic learning mode and the intelligent assessment mode.

6. A VR-based critical care nursing training system according to any one of claims 1 to 5, characterized in that: The intelligent assessment module also includes a team collaboration training submodule, which includes: The role management unit is configured to create an emergency team operation scenario including doctors, nurses and pharmacists in the local area network. Differentiated permissions are configured for each role according to clinical operation specifications: the doctor role has the right to make diagnosis decisions and the approval authority for the activation of medical equipment; the nurse role executes nursing operation instructions and monitors the patient's vital signs parameters; the pharmacist role is responsible for drug dosage calculation and compatibility taboo review; Terminal synchronization unit, real-time synchronization of multi-terminal operation data, including when the nurse performs venipuncture, the virtual view of the doctor's terminal synchronously displays the 3D model of the puncture progress; the pharmacist's medication dispensing operation error will trigger the terminal warning interface of the whole team; The role replacement unit can replace the roles in the emergency team with virtual roles before the assessment; during the assessment, when it is detected that the local member's operation is interrupted, the virtual role replacement is started: the virtual role can be operated through the terminal access of the remote personnel, or through the system operation that generates a standardized operation sequence based on the emergency knowledge graph.

7. A VR-based critical care nursing training system according to claim 6, characterized in that: The role replacement unit also includes replacing the patient, specifically: The virtual patient generation subunit randomly generates virtual patient models of different ages, genders and body types; The real person identification subunit operates with the real person as the patient. The image acquisition device collects and analyzes the key bone point data of the patient in real time and calculates the height and body shape characteristics, and generates age and gender prediction coefficients at the same time; The adaptive subunit is operated to adjust the nursing operation position according to the randomly generated virtual patient parameters or the identified real personnel parameters, and at the same time, the dynamic offset algorithm of the instrument positioning coordinates is executed.

8. A VR-based critical care nursing training system according to claim 7, characterized in that: It also includes an emergency liaison module, the module comprising: The assessment docking unit synchronizes the scoring data in the user multi-dimensional scoring module with the hospital emergency dispatch platform database. Users who pass the assessment will be granted device use rights and participate in hospital emergency dispatch: The enhanced guidance unit establishes a real-time mapping relationship between the physical space coordinate system and the virtual coordinate system through an augmented reality space registration algorithm based on feature point matching. When the user participates in real first aid, a dynamic operation guidance layer is superimposed in the field of view of the AR glasses, displaying the patient's vital signs trend curve and recommended medication dosage threshold in real time. It also generates a pressure feedback light column at the patient's operation position. When the operation position is inaccurate or the operation method is incorrect, a red ripple warning is triggered accompanied by tactile feedback.

9. A VR-based critical care nursing training system according to claim 8, characterized in that: The emergency connection module also includes a multimodal data fusion unit, which accesses the hospital's PACS system to retrieve the patient's historical imaging data and marks the abnormal bone structure warning area on the AR interface; at the same time, it uploads the emergency operation video to the quality control platform in real time, and generates an unalterable emergency capability certification record based on blockchain technology.

10. A VR-based critical care nursing training method, characterized in that: include: The trigger mode selection signal is selected by the handle trigger, including learning mode and assessment mode; In the learning mode, the nursing operation process is dynamically displayed through VR glasses through the scene generation engine. The scene generation engine constructs the disease evolution path based on the knowledge graph and associates the user's historical data to match personalized teaching cases; In the assessment mode, the user receives the selection instructions for the assessment items, and at the same time receives the user's operation instructions based on the system prompts, and records the time series data of the operation path and the logic chain of the selection behavior in real time; The total score and sub-item scores are generated according to the dynamic scoring algorithm. The dynamic scoring algorithm includes the operation trajectory accuracy parameter and the decision timeliness parameter. The operation trajectory accuracy parameter is calculated based on the distance between the user's operation path and the standard path; the decision timeliness parameter uses the time series difference method to evaluate the interval duration of the operation nodes.

Images