Virtual reality-based disaster emergency rescue nursing skill training system

Abstract

The application discloses a disaster emergency rescue nursing skill training system based on virtual reality and relates to the technical field of virtual reality.The application constructs a high-simulation virtual disaster scene based on real data, generates diversified virtual wounded persons matched with the virtual disaster scene, superimposes secondary disaster risk diffusion effects and emergency prompt information in real time, and dynamically simulates physiological state changes such as blood pressure and respiration of the wounded persons caused by nursing operation differences in the rescue process, so that the problems of single scene and fixed injury in traditional training are solved, multi-dimensional operation characteristics are accurately compared and hierarchically recognized, instant multi-modal feedback of the operation of trainees is realized, the defects of lagging guidance and vagueness are overcome, personalized and adaptive training schemes are generated according to comprehensive evaluation results, and the progress is dynamically adjusted, so that scene simulation, operation practice, accurate feedback and personalized improvement are integrated.

Description

Technical Field

[0001] This invention relates to the field of virtual reality technology, and in particular to a disaster emergency rescue and nursing skills training system based on virtual reality. Background Technology

[0002] Traditional disaster emergency rescue and nursing skills training mainly relies on theoretical lectures, mannequin practice, and limited on-site drills, which has many limitations. For example, Chinese patent application CN114049808A discloses an emergency knowledge training system based on virtual reality. This patent application uses modules for rubble burial, mobile escape, and successful escape prompts in its emergency knowledge training components to virtually realize emergency knowledge. Combined with physical equipment, it immerses users in simulated disaster scenarios, thereby enhancing the fun of learning emergency knowledge and achieving efficient and intelligent emergency knowledge training.

[0003] However, while the aforementioned patents provide virtual scenario support for emergency training, the following problems still exist:

[0004] Existing technologies lack a virtual patient with dynamic physiological state and a precise operation interaction mechanism. They cannot simulate the physiological state changes of patients caused by differences in nursing operations, cannot provide feedback to trainees on whether the operation is standardized, and are difficult to improve the proficiency and accuracy of nursing skills. Furthermore, they do not quantify and monitor trainees' data in nursing operations, making it impossible to scientifically assess nursing skill levels and achieve individualized teaching to develop targeted nursing skills enhancement programs. Summary of the Invention

[0005] The purpose of this invention is to provide a disaster emergency rescue nursing skills training system based on virtual reality. Through highly realistic dynamic disaster scenario simulation, precise nursing operation interactive feedback, quantitative assessment and personalized training program adaptation, it effectively improves trainees' practical ability and standardization in disaster emergency nursing, thereby solving the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A virtual reality-based disaster emergency rescue nursing skills training system includes:

[0008] The disaster scenario reconstruction module is used to construct virtual disaster scenarios that are consistent with real disaster scenarios based on basic data from various disaster sites, and dynamically adjust scenario parameters to simulate rescue environments for different disaster types and levels.

[0009] The virtual scenario simulation module is used to generate virtual patients with different injury characteristics and overlay them into a virtual disaster scenario. At the same time, it receives nursing operation data from trainees in real time and simulates the physiological changes of the virtual patients during the rescue process.

[0010] The nursing operation interaction module is used to establish an interactive connection between trainees and the virtual disaster environment and virtual wounded patients, capture various emergency rescue nursing operations performed by trainees in the virtual reality environment, identify standardized operations, and provide feedback on the operation results to trainees.

[0011] The training monitoring and evaluation module is used to monitor various data of trainees in real time during the training process, including operational data, learning time data, and virtual wounded status change data. It evaluates the training effect and skill level of trainees according to preset evaluation indicators and generates evaluation results.

[0012] Furthermore, the process of constructing a virtual disaster scenario using the disaster scenario reconstruction module includes:

[0013] Collect basic data from various disaster sites and preprocess it to form a standardized basic dataset;

[0014] A disaster environment model library is constructed based on a standardized basic dataset, including a terrain basic model, a building debris basic model, and a disaster source basic model. Environmental physical rules are set for the disaster environment model library. Each basic model in the basic disaster scenario model library corresponds to a typical disaster type.

[0015] Based on the preset training objectives, the system calls up the matching terrain base model, building debris base model, and disaster source base model from the disaster environment model library, initializes the scene parameters, and adjusts the parameters of each scene according to the training difficulty requirements.

[0016] Based on environmental physical rules, the terrain base model, building debris base model and disaster source base model are dynamically rendered, and the rendered scene is verified in multiple dimensions to generate a virtual disaster scene.

[0017] Furthermore, the virtual disaster scenario also includes:

[0018] Construct interactive trigger points for disaster scenarios, set trigger conditions and feedback logic for each interactive trigger point, and trigger corresponding environmental feedback based on the trainees' operational behavior in the scenario or the changes in the status of virtual wounded;

[0019] By acquiring secondary disaster risk diffusion data and combining it with the status information of disaster scenario interaction trigger points, an emergency prompt information layer is overlaid in the virtual disaster scenario to display the corresponding emergency prompt information.

[0020] Furthermore, the specific implementation process of overlaying the emergency alert information layer for disaster scenarios includes:

[0021] Real-time monitoring of the spread of secondary disasters in virtual disaster scenarios, including spread speed, scope of impact and degree of harm, and determination of the risk level of secondary disasters;

[0022] The system calculates the duration of each level of risk within a set statistical period, determines the proportion of the current risk level's duration to the total duration of the set statistical period, records the frequency of risk level changes within the set statistical period, and analyzes the risk diffusion trend.

[0023] By combining the risk level assessment results with the risk diffusion trend analysis results, the actual impact of secondary disasters on the rescue environment is evaluated. At the same time, the real-time location data of trainees and the virtual casualty distribution data are linked to prioritize the assessment of the impact of secondary disasters on the trainees' current rescue operation site.

[0024] Based on the assessment results of the impact of secondary disasters on the trainees' current rescue operation site, the display mode of the emergency prompt information layer is controlled in a hierarchical manner.

[0025] Furthermore, the process by which the virtual scenario simulation module generates virtual wounded individuals with different injury characteristics includes:

[0026] Collect real patient case data from clinical disaster relief, extract information on injury type, injury location, injury severity and corresponding physiological parameters from real patient medical records, and classify and integrate them based on disaster type to establish an injury characteristic database;

[0027] Obtain information on the type of virtual disaster scenario generated by the disaster scenario reconstruction module, filter the injury combination matching in the injury feature database, and determine the number and type ratio of injury combinations of the virtual casualties to be generated by combining the injury distribution of the scenario parameters.

[0028] The appearance feature parameters corresponding to the selected injury combinations are mapped to the corresponding parts of the three-dimensional human body basic model to generate the appearance model of the virtual patient. The physiological state parameter range of the corresponding injury combination is extracted from the patient injury feature database, and the initial physiological state parameters of the virtual patient are randomly generated.

[0029] The appearance model of the virtual injured person is associated with the corresponding physiological state of the accident to generate virtual injured persons with different injury characteristics. The generated virtual injured persons with different injury characteristics are then superimposed on the space area that can be rescued and operated in the virtual disaster scene.

[0030] Furthermore, the nursing operation interaction module performs standardized operation recognition, specifically including:

[0031] Based on the clinical emergency rescue nursing operation standards, combined with the generated virtual patient injury characteristics, the operation type dimensions are divided, and corresponding standardized feature parameters are determined for each operation type.

[0032] Real-time data collection of nursing operation data performed by trainees on virtual wounded in virtual disaster scenarios;

[0033] Extract the operational features from nursing operation data, and compare the types of operational features with the corresponding standard feature parameters, including comparison of operation site, operation force, operation sequence, and operation duration;

[0034] If all comparison results are deemed acceptable, the operation is identified as standard. If any comparison result is deemed unacceptable, it is determined whether it is a critical item. If a critical item exists and the comparison difference exceeds the preset deviation threshold, the operation is identified as seriously non-standard. If it is a non-critical item and the comparison difference does not exceed the preset deviation threshold, the operation is identified as slightly non-standard.

[0035] Feedback information is generated based on the operation recognition results and transmitted to the trainee. It outputs slight vibrations while simultaneously providing positive feedback and deviation prompts via voice. Visual markers in the virtual scene help the trainee correct their operation.

[0036] Furthermore, the training monitoring and evaluation module assesses the training effectiveness and skill level of trainees through the following processes:

[0037] Based on the correlation between the timestamps of nursing operation recognition data and the data on changes in the physiological state of virtual wounded patients, learning duration data, and scene interaction data, an assessment dataset for a single training session of trainees is constructed.

[0038] Based on the assessment dataset, scores for each indicator are calculated in combination with preset assessment indicators, including operational accuracy, operational efficiency, nursing effect and emergency response speed. The calculation results of each indicator are matched with the corresponding skill level, including advanced, intermediate and basic.

[0039] Each indicator is assigned a value, and a comprehensive evaluation is conducted based on the assigned values ​​to generate a single training evaluation report for the trainee.

[0040] Furthermore, this includes developing training programs based on the trainees' skill levels and corresponding assessment results, as detailed in the following process:

[0041] Establish a training program template library, which includes basic training templates, advanced training templates, and intensive training templates. Each template contains corresponding training objectives, training content, training duration, and assessment standards.

[0042] The initial training template is matched to the trainee's skill level and adjusted based on the statistics of non-standard operation types in the evaluation report.

[0043] Based on the adjustment results, a training plan for trainees is generated, and the changes in the adjusted special training content, time allocation and assessment standards are marked, and synchronized to the training monitoring and evaluation module.

[0044] Furthermore, the training monitoring and evaluation module includes:

[0045] The operation data serialization submodule is used to group the trainee's operation data according to nursing needs, and record a complete nursing operation as a time series data containing different modal data, including spatial modal data, mechanical modal data, temporal modal data and trajectory modal data;

[0046] The attention network model submodule is used to build and train an attention network model that includes a feature extraction layer, a cross-modal fusion layer, and a skill evaluation output layer.

[0047] The feature extraction layer contains a dedicated feature extraction subnetwork for each modality of data. Each feature extraction subnetwork uses an intramodal attention mechanism to extract low-level features from the modal data in the corresponding time series data to obtain modal features.

[0048] The cross-modal fusion layer is used to concatenate modal features extracted from the same time series data to obtain feature sequences of the same modal data. The feature sequences corresponding to each same modal data are input into a fusion layer, and a cross-modal attention mechanism is introduced. The cross-modal weights are determined according to the contribution of the modal data to the evaluation. The fusion algorithm based on the cross-modal weights is used to fuse the cross-modal data to obtain the fused feature representation.

[0049] The skill evaluation output layer has a multi-task processing output channel. The feature representation is sent to the multi-task processing output channel for processing and then outputs the skill evaluation result. The skill evaluation result includes the overall skill score result, error pattern recognition result, and key segment localization result.

[0050] The feedback correction suggestion generation submodule is used to combine error pattern recognition results and key segment location results to generate actionable correction suggestions.

[0051] Furthermore, the cross-modal fusion layer specifically performs the following fusion process:

[0052] By performing dimensional projection processing, the modal feature dimensions of the feature sequences of each modal data are unified, and a multi-head attention mechanism is used to calculate cross-modal weights and implement cross-modal attention score calculation.

[0053] Cross-modal attention weighted computation is performed to obtain weighted features; a gating mechanism is used for refined fusion, and sparse constraints are applied to the cross-modal weights of the gating to promote modality selection; residual connections and layer normalization are implemented based on the feedforward network.

[0054] In cross-modal fusion, temporal convolution enhancement is introduced. By using grouped convolution to reduce the number of parameters, the attention mechanism is sparsified. After enhancement, the cross-modal fusion algorithm is used to process the enhanced feature representation, resulting in a cross-modal fusion feature representation with temporal awareness.

[0055] Compared with the prior art, the beneficial effects of the present invention are:

[0056] This invention proposes a virtual reality-based disaster emergency rescue nursing skills training system. It constructs highly realistic, dynamically evolving virtual disaster scenarios based on real data and generates diverse virtual patients with varying injuries. It overlays secondary disaster risk diffusion effects and emergency alerts in real time, dynamically simulating changes in the patient's physiological state, such as blood pressure and respiration, due to differences in nursing procedures during rescue. This solves the problems of single scenarios and fixed injury conditions in traditional training. It accurately captures trainees' nursing actions and achieves immediate, multimodal feedback through precise comparison and hierarchical recognition of multi-dimensional operational features. This overcomes the shortcomings of delayed and generalized guidance. Based on comprehensive evaluation results, it generates personalized and adaptive training programs and dynamically adjusts the pace. By integrating scenario simulation, operational practice, precise feedback, and personalized improvement, it significantly enhances the scientific rigor, relevance, and effectiveness of emergency nursing skills training. Attached Figure Description

[0057] Figure 1 This is a block diagram of the disaster emergency rescue nursing skills training system based on virtual reality according to the present invention. Detailed Implementation

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] To address the technical challenges of existing technologies lacking a dynamic physiological state-based virtual patient interaction mechanism for precise operation, which hinders the improvement of nursing skill proficiency and accuracy, and failing to quantify and monitor trainees' data during nursing operations to scientifically assess nursing skill levels, and to develop targeted nursing skill enhancement programs, please refer to [link to relevant documentation]. Figure 1 This embodiment provides the following technical solution:

[0060] A virtual reality-based disaster emergency rescue nursing skills training system includes:

[0061] The disaster scenario reconstruction module is used to construct virtual disaster scenarios that are consistent with real disaster scenarios based on basic data from various disaster sites, and dynamically adjust scenario parameters to simulate rescue environments for different disaster types and levels.

[0062] The disaster types include typical disaster scenarios such as earthquake collapse, fire explosion, flood siege, and chemical leak, while also covering complex scenarios with multiple disasters superimposed. The scenario parameters include key elements such as geographical environment, meteorological conditions, day and night, seasonal changes, distribution of injuries and the allocation of rescue resources. By combining multiple parameters, a highly realistic training environment is constructed to provide virtual environment support for emergency rescue and nursing skills training.

[0063] The virtual scenario simulation module is used to generate virtual patients with different injury characteristics and overlay them into a virtual disaster scenario. At the same time, it receives nursing operation data from trainees in real time and simulates the physiological changes of the virtual patients during the rescue process, providing diverse nursing operation objects for training.

[0064] Among them, the characteristics of the injury include: the type of injury such as head, trunk, limbs, etc.; the location of the injury such as abrasion, contusion, fracture, bleeding, etc.; and the severity of the injury, mild, moderate, and severe; physiological parameters include heart rate, blood pressure, blood oxygen saturation, and the state of consciousness such as conscious, drowsy, or comatose.

[0065] The nursing operation interaction module is used to establish an interactive connection link between trainees and the virtual disaster environment and virtual casualties. It captures various emergency rescue nursing operation actions performed by trainees in the virtual reality environment, performs standardized operation recognition, and provides feedback to trainees on operation results, including feedback on the standardization of operation actions, such as whether the operation site is accurate, whether the operation force is appropriate, and whether the operation sequence is correct, and feedback on operation effect, such as hemostasis effect, fixation stability, and whether the depth and frequency of cardiopulmonary resuscitation are up to standard.

[0066] The training monitoring and evaluation module is used to monitor various data of trainees in real time during the training process, including operational data, learning time data, and virtual wounded status change data. It evaluates the trainees' training effectiveness and skill level based on preset evaluation indicators and generates evaluation results, specifically including:

[0067] Based on the correlation between the timestamps of nursing operation recognition data and the data on changes in the physiological state of virtual wounded patients, learning duration data, and scene interaction data, an assessment dataset for a single training session of trainees is constructed.

[0068] Based on the assessment dataset, scores for each indicator are calculated in combination with preset assessment indicators, including operational accuracy, operational efficiency, nursing effect and emergency response speed. The calculation results of each indicator are matched with the corresponding skill level, including advanced, intermediate and basic.

[0069] Values ​​are assigned to each indicator level, and a comprehensive evaluation is conducted based on the assigned values ​​to generate a single training evaluation report for the trainees.

[0070] In this embodiment, the training plan is also developed based on the trainees' skill levels and corresponding assessment results, as follows:

[0071] Establish a training program template library, which includes basic training templates, advanced training templates, and intensive training templates. Each template contains corresponding training objectives, training content, training duration, and assessment standards.

[0072] The initial training template is matched according to the trainee's skill level. If the trainee's overall assessment is unqualified, the basic training template is matched; if the trainee's overall assessment is qualified, the advanced training template is matched; if the trainee's overall assessment is excellent, the reinforcement training template is matched. The initial training template is also adjusted in a personalized manner based on the statistics of non-standard operation types in the assessment report.

[0073] Based on the adjustment results, a personalized training plan is generated for each trainee, and the changes in the adjusted special training content, time allocation and assessment standards are marked. This plan is then synchronized to the training monitoring and evaluation module for targeted tracking of subsequent training effectiveness.

[0074] In this embodiment, the personalized adjustments include: if the trainee's compression depth deviation frequency is high, a chest compression-specific training module is added to the corresponding training template, the duration is increased by 1 hour, and compression depth adaptation training is set for different virtual patient body types; if the trainee's emergency response speed score is low, a sudden risk scenario simulation module is added to the training template, such as randomly triggering secondary disasters, to train the trainee to respond quickly.

[0075] In this embodiment, during the training process, if a trainee's operational accuracy in a certain training module reaches the assessment standard for that module twice consecutively, the trainee will proceed to the next training module ahead of schedule; if a trainee's operational accuracy in a certain training module is lower than 80% of the assessment standard twice consecutively, the training time for that module will be extended.

[0076] In this embodiment, a highly simulated and diverse disaster environment is constructed by covering multiple disaster types and dynamically combining multiple parameters. Trainees can be exposed to complex and ever-changing disaster scenarios in this module, significantly improving their adaptability to different types of disasters. This effectively solves the problems of insufficient training scenarios and difficulty in simulating real and complex disasters in traditional training. Furthermore, the virtual scenario simulation module enables the generation of virtual patients with various injuries, simulating their physiological state changes in real time. This allows for nursing practice for different injuries, making up for the shortcomings of single patient models and fixed physiological states in traditional training. It provides accurate feedback on the standardization and actual effect of trainees' operations. During the training process, trainees can promptly identify problems in their operations, facilitating rapid correction of errors. This successfully solves the problems of delayed and inaccurate operational feedback in traditional training. Combined with graded matching templates and personalized adjustment functions, it achieves a scientific and comprehensive assessment of trainees' skill levels and generates exclusive training plans. The training progress is dynamically adjusted based on the trainees' operational accuracy, ensuring both training efficiency and quality, and effectively avoiding ineffective training.

[0077] In this embodiment, the process of constructing a virtual disaster scenario by the disaster scenario reconstruction module includes:

[0078] Basic data from various disaster sites were collected and preprocessed. Density-based outlier detection algorithms were used to remove extreme outliers from topographic and building structure data. Time series interpolation was used to fill in missing values ​​in disaster visual feature data and disaster sound effect feature data to form a standardized basic dataset.

[0079] A disaster environment model library is constructed based on a standardized basic dataset, including a terrain basic model, a building debris basic model, and a disaster source basic model. Environmental physical rules are set for the disaster environment model library. Each basic model in the basic disaster scenario model library corresponds to a typical disaster type.

[0080] Based on the preset training objectives, the system calls up the matching terrain base model, building debris base model, and disaster source base model from the disaster environment model library, initializes the scene parameters, and adjusts the parameters of each scene according to the training difficulty requirements.

[0081] Based on environmental physical rules, the terrain base model, building debris base model and disaster source base model are dynamically rendered, and the rendered scene is verified in multiple dimensions to generate a virtual disaster scene with realistic visual and auditory effects.

[0082] In this embodiment, the basic data includes topographic data, building structure data, disaster visual feature data, and disaster sound effect feature data. The topographic data includes the elevation variation of the disaster area, terrain type, and obstacle distribution. The building structure data includes the degree of building collapse, component fracture morphology, and remaining supporting structure parameters. The disaster visual feature data includes flame color, spread speed, smoke concentration and diffusion direction, water depth and flow rate, and chemical leak color and diffusion range. The disaster sound effect feature data includes sounds of building collapse, flame combustion, water impact, and cries for help from the injured.

[0083] In this embodiment, the environmental physical rules include object collision detection rules, such as the force feedback logic when virtual objects collide, and the calculation method of displacement and deformation after collision; gravity rules, such as the acceleration parameters of falling objects in virtual scenes, and the difference coefficient of gravity affecting objects of different materials; and disaster diffusion rules, such as the correlation formula between flame spread speed and wind speed and combustible density in fire scenes, and the calculation model of toxic gas diffusion range and meteorological conditions in chemical leak scenes.

[0084] In this embodiment, the virtual disaster scenario further includes:

[0085] Construct interactive trigger points for disaster scenarios, including operable rescue equipment and changeable environmental elements. Set trigger conditions and feedback logic for each interactive trigger point, and trigger corresponding environmental feedback based on the trainees' operational behavior in the scenario or the changes in the status of virtual wounded.

[0086] By acquiring secondary disaster risk diffusion data and combining it with the status information of disaster scenario interaction trigger points, an emergency prompt information layer is overlaid in the virtual disaster scenario to display the corresponding emergency prompt information.

[0087] In this embodiment, environmental feedback includes secondary disaster events, such as aftershocks causing additional building debris to fall in an earthquake collapse scenario, or a sudden flashover expanding the flame range in a fire scenario; rescue resource replenishment events, such as the generation of virtual rescue supplies models like emergency medicine boxes and bandages at designated locations in the scenario after a trainee completes a single nursing operation; and temporary environmental change events, such as a sudden rise in water level in a flood siege scenario, or a temporary change in wind direction in a chemical leak scenario. The trigger probability and intensity of the environmental feedback are dynamically adjusted according to the training progress. For example, the trigger probability is less than 30% and the trigger intensity is mild in the early stages of training, while the trigger probability increases to over 60% and the trigger intensity is moderate to severe in the later stages of training, in order to enhance the challenge and realism of the scenario.

[0088] In this embodiment, the specific implementation process of overlaying the disaster scenario emergency alert information layer includes:

[0089] Real-time monitoring of the spread of secondary disasters in virtual disaster scenarios, including spread speed, scope of impact and degree of harm, and determination of the risk level of secondary disasters;

[0090] The system calculates the duration of each level of risk within a set statistical period, determines the proportion of the current risk level's duration to the total duration of the set statistical period, records the frequency of risk level changes within the set statistical period, and analyzes risk diffusion trends, such as continuous enhancement, stabilization, or gradual weakening.

[0091] By combining the risk level assessment results with the risk diffusion trend analysis results, the actual impact of secondary disasters on the rescue environment is evaluated. At the same time, the real-time location data of trainees and the virtual casualty distribution data are linked to prioritize the assessment of the impact of secondary disasters on the trainees' current rescue operation site.

[0092] Based on the assessment results of the impact of secondary disasters on the trainees' current rescue operation site, the display mode of the emergency prompt information layer is controlled in a hierarchical manner.

[0093] In this embodiment, the risk level of secondary disasters is determined as shown in the table below:

[0094]

[0095] In this embodiment, when the risk level is Level 1 and the current risk level consistently accounts for less than 30%, the actual impact is determined to be low; when the risk level is Level 2, or when the risk level is Level 1 and the current risk level consistently accounts for 30%-60%, the actual impact is determined to be medium; when the risk level is Level 3, or when the risk level is Level 2 and the current risk level consistently accounts for more than 60%, the actual impact is determined to be high.

[0096] In this embodiment, when the impact is determined to be low, a basic evacuation route is displayed only when trainees enter the risk edge area; when the impact is determined to be medium, a dynamic risk warning is continuously displayed, including the direction of secondary disaster spread and the remaining time for safe evacuation, while the current secondary disaster risk range is marked in the virtual disaster scenario, and the color block boundary is updated in real time as the risk spreads; when the impact is determined to be high, an emergency warning mode is triggered, displaying the evacuation plan with flashing red text, including the optimal transfer route and emergency response steps, and simultaneously activating the buzzer alarm and vibration prompt of the interactive device.

[0097] In this embodiment, the statistical period is dynamically adjusted according to the type of secondary disaster: 5 minutes for fire scenario, 10 minutes for flood scenario, and 8 minutes for chemical leak scenario.

[0098] In this embodiment, the process by which the virtual scenario simulation module generates virtual wounded individuals with different injury characteristics includes:

[0099] Collect real patient case data from clinical disaster relief, extract information on injury type, injury location, injury severity and corresponding physiological parameters from real patient medical records, and classify and integrate them based on disaster type to establish an injury characteristic database;

[0100] Obtain information on the type of virtual disaster scenario generated by the disaster scenario reconstruction module, filter the injury combination matching in the injury feature database, and determine the number and type ratio of injury combinations of the virtual casualties to be generated by combining the injury distribution of the scenario parameters.

[0101] The three-dimensional human body base model is called, and the appearance feature parameters corresponding to the selected injury combination are mapped to the corresponding parts of the three-dimensional human body base model to generate the appearance model of the virtual wounded. The physiological state parameter range of the corresponding injury combination is extracted from the wounded injury feature library, and the initial physiological state parameters of the virtual wounded are randomly generated.

[0102] The appearance model of the virtual injured person is associated with the corresponding physiological state of the accident to generate virtual injured persons with different injury characteristics. The generated virtual injured persons with different injury characteristics are then superimposed on the space area that can be rescued and operated in the virtual disaster scene.

[0103] In this embodiment, the nursing operation interaction module performs standardized operation recognition, specifically including:

[0104] Based on the clinical emergency rescue nursing operation standards, combined with the generated virtual patient injury characteristics, the operation type dimensions are divided, and corresponding standardized feature parameters are determined for each operation type.

[0105] Real-time data collection of nursing operation data of trainees performing nursing operations on virtual wounded in virtual disaster scenarios, including the limb joint angles corresponding to the operation nodes, the real-time coordinates of the contact points between the trainee's hand and the virtual wounded, the pressure values ​​fed back by the interactive device, the start time, end time and interval of each operation step, and the movement path of the trainee's hand on the surface of the virtual wounded.

[0106] Extract the operational features from nursing operation data, and compare the types of operational features with the corresponding standard feature parameters, including comparison of operation site, operation force, operation sequence, and operation duration;

[0107] If all comparison results are deemed acceptable, the operation is identified as standard. If any comparison result is deemed unacceptable, it is determined whether it is a critical item. If a critical item exists and the comparison difference exceeds the preset deviation threshold, the operation is identified as seriously non-standard. If it is a non-critical item and the comparison difference does not exceed the preset deviation threshold, the operation is identified as slightly non-standard.

[0108] Feedback information is generated based on the operation recognition results and transmitted to the trainee. It outputs slight vibrations while simultaneously providing positive feedback and deviation prompts via voice. Visual markers in the virtual scene help the trainee correct their operation.

[0109] In this embodiment, the standardized feature parameters include the coordinate range of the operation site, the threshold of the operation force, the logic of the operation sequence, and the operation duration requirement, etc.

[0110] In this embodiment, the virtual scenario simulation module constructs a database of injury characteristics based on real clinical cases, combines virtual disaster scenario types and parameter-based injury combinations, and generates objects by associating 3D models with physiological parameters. The generated virtual patients closely resemble real rescue injuries and can accurately match the scenario, making the nursing objects faced by trainees more realistic and diverse. This significantly enhances the immersion and training relevance of disaster rescue scenarios. By comparing and identifying key and non-key items of standardized feature parameters, the module accurately judges the degree of operational standardization and provides timely feedback through multiple channels to help trainees correct their mistakes. This helps trainees quickly master standardized operations, effectively improving their ability to standardize complex emergency nursing operations, enhancing teaching and training effectiveness, and achieving individualized instruction.

[0111] In this embodiment, the training monitoring and evaluation module includes:

[0112] The operation data serialization submodule is used to group the trainee's operation data according to nursing needs, recording a complete nursing operation as a time series data S = {(m1_t, m2_t,…mi_t…, mN_t)}, where t is the time point, and m1, m2, mi, and mN are different modal data, including spatial modal data, mechanical modal data, temporal modal data, and trajectory modal data. Spatial modal data can be skeletal joint data from VR controllers / VR gloves, such as hand joint positions, joint postures, and / or joint angles; mechanical modal data can be pressure / force data fed back by haptic devices; temporal modal data can be the start, end, and duration stamps of each operation step; trajectory modal data can be the motion trajectory of the operation tool (such as virtual gauze, pressing hand, etc.) in three-dimensional space.

[0113] The attention network model submodule is used to build and train an attention network model that includes a feature extraction layer, a cross-modal fusion layer, and a skill evaluation output layer.

[0114] The feature extraction layer contains a dedicated feature extraction subnetwork for each modality of data. Each feature extraction subnetwork uses an intramodal attention mechanism to extract low-level features from the modal data in the corresponding time series data to obtain modal features.

[0115] The cross-modal fusion layer is used to concatenate modal features extracted from the same time series data to obtain a feature sequence of the same modality. The feature sequences corresponding to each same modality data are input into a fusion layer, and a cross-modal attention mechanism is introduced. The cross-modal weights are determined according to the contribution of the modal data to the evaluation. For example, when judging the compression depth, the mechanical cross-modal weight is the highest; when judging the compression position, the spatial cross-modal weight is the highest; and when evaluating the overall rhythm, the temporal cross-modal weight is the highest. The fusion algorithm based on the cross-modal weights performs cross-modal data fusion to obtain the fused feature representation.

[0116] The skill evaluation output layer has a multi-task processing output channel. The feature representation is sent to the multi-task processing output channel for processing and then outputs the skill evaluation result. The skill evaluation result includes the overall skill score result, error pattern recognition result, and key segment localization result.

[0117] The feedback correction suggestion generation submodule is used to combine error pattern recognition results and key segment location results to generate actionable correction suggestions.

[0118] In this embodiment, a hierarchical attention network enables a deep understanding and fine-grained analysis of multimodal and temporal operational data. This allows for the focus on key operational aspects and cross-modal collaborative relationships, much like a human expert. This upgrades automated assessment from "result comparison" to "process diagnosis," accurately locating micro-defects in complex skills and providing expert-level, personalized corrective guidance, significantly improving the efficiency and effectiveness of skills training. Instead of simply providing a vague "non-standard" evaluation, it can generate specific and actionable corrective suggestions by combining error pattern recognition and key segment localization results. For example, feedback might include: "Your second and fifth presses did not reach the required depth of 5 cm (key segment highlighted). We suggest increasing the upper body tilt angle to utilize body weight. Also, your pressing frequency gradually increases in the later stages of the cycle (based on temporal modality analysis). Please maintain a stable rhythm of 100-120 presses per minute." Furthermore, it allows for side-by-side visual comparison of expert-performed key segments with the trainee's performance, providing a clear overview for the trainee.

[0119] In this embodiment, the fusion process performed by the cross-modal fusion layer is as follows:

[0120] By performing dimensional projection processing, the modal feature dimensions of the feature sequences of each modal data are unified, and a multi-head attention mechanism is used to calculate cross-modal weights and implement cross-modal attention score calculation.

[0121] Cross-modal attention weighted computation is performed to obtain weighted features; a gating mechanism is used for refined fusion, and sparse constraints are applied to the cross-modal weights of the gating to promote modality selection; residual connections and layer normalization are implemented based on the feedforward network.

[0122] In cross-modal fusion, temporal convolution enhancement is introduced. By using grouped convolution to reduce the number of parameters, the attention mechanism is sparsified. After enhancement, the following cross-modal fusion algorithm is used to obtain a feature representation of cross-modal fusion with temporal awareness.

[0123]

[0124] in, Let i be the output feature representation of the target mode i at time step t. The modal index, from 1 to , For the number of modes, To sum over all source modes, The indices for the attention heads are from 1 to H. To focus on the number of heads, To sum over all attention heads, For cross-modal weights, To determine the amount of information that target modality i should focus on from the h-th attention head of source modality j within time t, the information content is used as the attention weight. Let h be the feature representation of source mode j at time t in the h-th attention head. To project the output of the h-th attention head back from the head dimension to the model dimension.

[0125] In this embodiment, cross-modal fusion processing dynamically learns the importance of each modality through an attention mechanism, and then achieves refined fusion through a gating mechanism. This not only preserves the unique information of each modality, but also taps into the synergistic effect of cross-modal data. Gating fusion achieves smooth fusion of original features and temporally enhanced features. Cross-modal weights can automatically learn the optimal balance point. This provides a specific and selectable method for how cross-modal attention mechanisms can finely integrate multi-source information, which can fully tap into the complementary information in multimodal data and provide a high-quality unified feature representation for subsequent nursing skills assessment.

[0126] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A disaster emergency rescue nursing skills training system based on virtual reality, characterized in that, include: The disaster scenario reconstruction module is used to construct virtual disaster scenarios that are consistent with real disaster scenarios based on basic data from various disaster sites, and dynamically adjust scenario parameters to simulate rescue environments for different disaster types and levels. The virtual scenario simulation module is used to generate virtual patients with different injury characteristics and overlay them into a virtual disaster scenario. At the same time, it receives nursing operation data from trainees in real time and simulates the physiological changes of the virtual patients during the rescue process. The nursing operation interaction module is used to establish an interactive connection between trainees and the virtual disaster environment and virtual wounded patients, capture various emergency rescue nursing operations performed by trainees in the virtual reality environment, identify standardized operations, and provide feedback on the operation results to trainees. The training monitoring and evaluation module is used to monitor various data of trainees in real time during the training process, including operation data, learning time data and virtual wounded status change data. It evaluates the training effect and skill level of trainees according to preset evaluation indicators and generates evaluation results. The training monitoring and evaluation module includes: The operation data serialization submodule is used to group the trainee's operation data according to nursing needs, and record a complete nursing operation as a time series data containing different modal data, including spatial modal data, mechanical modal data, temporal modal data and trajectory modal data; The attention network model submodule is used to build and train an attention network model that includes a feature extraction layer, a cross-modal fusion layer, and a skill evaluation output layer. The feature extraction layer contains a dedicated feature extraction subnetwork for each modality of data. Each feature extraction subnetwork uses an intramodal attention mechanism to extract low-level features from the modal data in the corresponding time series data to obtain modal features. The cross-modal fusion layer is used to concatenate modal features extracted from the same time series data to obtain feature sequences of the same modal data. The feature sequences corresponding to each same modal data are input into a fusion layer, and a cross-modal attention mechanism is introduced. The cross-modal weights are determined according to the contribution of the modal data to the evaluation. The fusion algorithm based on the cross-modal weights is used to fuse the cross-modal data to obtain the fused feature representation. The skill evaluation output layer has a multi-task processing output channel. The feature representation is sent to the multi-task processing output channel for processing and then outputs the skill evaluation result. The skill evaluation result includes the overall skill score result, error pattern recognition result, and key segment localization result. The feedback correction suggestion generation submodule is used to combine error pattern recognition results and key segment location results to generate actionable correction suggestions; The cross-modal fusion layer specifically performs the following fusion process: By performing dimensional projection processing, the modal feature dimensions of the feature sequences of each modal data are unified, and a multi-head attention mechanism is used to calculate cross-modal weights and implement cross-modal attention score calculation. Cross-modal attention weighted computation is performed to obtain weighted features; a gating mechanism is used for refined fusion, and sparse constraints are applied to the cross-modal weights of the gating to promote modality selection; residual connections and layer normalization are implemented based on the feedforward network. In cross-modal fusion, temporal convolution enhancement is introduced. By using grouped convolution to reduce the number of parameters, the attention mechanism is sparsified. After enhancement, the cross-modal fusion algorithm is used to process the enhanced feature representation, resulting in a cross-modal fusion feature representation with temporal awareness.

2. The disaster emergency rescue nursing skills training system based on virtual reality as described in claim 1, characterized in that, The process of constructing a virtual disaster scenario using the disaster scenario reconstruction module includes: Collect basic data from various disaster sites and preprocess it to form a standardized basic dataset; A disaster environment model library is constructed based on a standardized basic dataset, including a terrain basic model, a building debris basic model, and a disaster source basic model. Environmental physical rules are set for the disaster environment model library. Each basic model in the basic disaster scenario model library corresponds to a typical disaster type. Based on the preset training objectives, the system calls up the matching terrain base model, building debris base model, and disaster source base model from the disaster environment model library, initializes the scene parameters, and adjusts the parameters of each scene according to the training difficulty requirements. Based on environmental physical rules, the terrain base model, building debris base model and disaster source base model are dynamically rendered, and the rendered scene is verified in multiple dimensions to generate a virtual disaster scene.

3. The disaster emergency rescue nursing skills training system based on virtual reality as described in claim 2, characterized in that, The process of constructing the virtual disaster scenario also includes: Construct interactive trigger points for disaster scenarios, set trigger conditions and feedback logic for each interactive trigger point, and trigger corresponding environmental feedback based on the trainees' operational behavior in the scenario or the changes in the status of virtual wounded; By acquiring secondary disaster risk diffusion data and combining it with the status information of disaster scenario interaction trigger points, an emergency prompt information layer is overlaid in the virtual disaster scenario to display the corresponding emergency prompt information.

4. The disaster emergency rescue nursing skills training system based on virtual reality as described in claim 3, characterized in that, The specific implementation process of overlaying the emergency alert information layer in disaster scenarios includes: Real-time monitoring of the spread of secondary disasters in virtual disaster scenarios, including spread speed, scope of impact and degree of harm, and determination of the risk level of secondary disasters; The system calculates the duration of each level of risk within a set statistical period, determines the proportion of the current risk level's duration to the total duration of the set statistical period, records the frequency of risk level changes within the set statistical period, and analyzes the risk diffusion trend. By combining the risk level assessment results with the risk diffusion trend analysis results, the actual impact of secondary disasters on the rescue environment is evaluated. At the same time, the real-time location data of trainees and the virtual casualty distribution data are linked to prioritize the assessment of the impact of secondary disasters on the trainees' current rescue operation site. Based on the assessment results of the impact of secondary disasters on the trainees' current rescue operation site, the display mode of the emergency prompt information layer is controlled in a hierarchical manner.

5. The disaster emergency rescue nursing skills training system based on virtual reality as described in claim 4, characterized in that, The process by which the virtual scenario simulation module generates virtual patients with different injury characteristics includes: Collect real patient case data from clinical disaster relief, extract information on injury type, injury location, injury severity and corresponding physiological parameters from real patient medical records, and classify and integrate them based on disaster type to establish an injury characteristic database; Obtain information on the type of virtual disaster scenario generated by the disaster scenario reconstruction module, filter the injury combination matching in the injury feature database, and determine the number and type ratio of injury combinations of the virtual casualties to be generated by combining the injury distribution of the scenario parameters. The appearance feature parameters corresponding to the selected injury combinations are mapped to the corresponding parts of the three-dimensional human body basic model to generate the appearance model of the virtual patient. The physiological state parameter range of the corresponding injury combination is extracted from the patient injury feature database, and the initial physiological state parameters of the virtual patient are randomly generated. The appearance model of the virtual injured person is associated with the corresponding physiological state of the accident to generate virtual injured persons with different injury characteristics. The generated virtual injured persons with different injury characteristics are then superimposed on the space area that can be rescued and operated in the virtual disaster scene.

6. The disaster emergency rescue nursing skills training system based on virtual reality as described in claim 5, characterized in that, The nursing operation interaction module performs standardized operation recognition, specifically including: Based on the clinical emergency rescue nursing operation standards, combined with the generated virtual patient injury characteristics, the operation type dimensions are divided, and corresponding standardized feature parameters are determined for each operation type. Real-time data collection of nursing operation data performed by trainees on virtual wounded in virtual disaster scenarios; Extract the operational features from nursing operation data, and compare the types of operational features with the corresponding standard feature parameters, including comparison of operation site, operation force, operation sequence, and operation duration. If all comparison results are deemed acceptable, the operation is identified as standard. If any comparison result is unacceptable, it is determined whether it is a critical item. If a critical item exists and the comparison difference exceeds the preset deviation threshold, the operation is identified as seriously non-standard. If it is a non-critical item and the comparison difference does not exceed the preset deviation threshold, the operation is identified as slightly non-standard. Feedback information is generated based on the operation recognition results and transmitted to the trainee. It outputs slight vibrations while simultaneously providing positive feedback and deviation prompts via voice. Visual markers in the virtual scene also assist the trainee in correcting the operation.

7. The disaster emergency rescue nursing skills training system based on virtual reality as described in claim 6, characterized in that, The training monitoring and evaluation module assesses the training effectiveness and skill level of trainees through the following processes: Based on the correlation between the timestamps of nursing operation recognition data and the data on changes in the physiological state of virtual wounded patients, learning duration data, and scene interaction data, an assessment dataset for a single training session of trainees is constructed. Based on the assessment dataset, scores for each indicator are calculated in combination with preset assessment indicators, including operational accuracy, operational efficiency, nursing effect and emergency response speed. The calculation results of each indicator are matched with the corresponding skill level, including advanced, intermediate and basic. Each indicator is assigned a value, and a comprehensive evaluation is conducted based on the assigned values ​​to generate a single training evaluation report for the trainee.

8. The disaster emergency rescue nursing skills training system based on virtual reality as described in claim 1, characterized in that, This also includes developing training programs based on trainees' skill levels and corresponding assessment results, specifically including the following process: Establish a training program template library, which includes basic training templates, advanced training templates, and intensive training templates. Each template contains corresponding training objectives, training content, training duration, and assessment standards. The initial training template is matched to the trainee's skill level and adjusted based on the statistics of non-standard operation types in the evaluation report. Based on the adjustment results, a training plan for trainees is generated, and the changes in the adjusted special training content, time allocation and assessment standards are marked, and synchronized to the training monitoring and evaluation module.

Images