Emergency drilling device based on multiple modes and evaluation system thereof

Abstract

The invention relates to a multi-modal-based emergency drilling device and an evaluation system thereof, and belongs to the technical field of emergency drilling. The multi-modal-based emergency drilling device comprises a model building module, a multi-modal feedback module, fire extinguishing equipment and a processor; establishing a scene model by scanning a target industrial scene, and dynamically configuring an accident virtual scene in the scene model; an interaction logic unit in the processor triggers vibration feedback, high temperature feedback and sound feedback when a fire occurs in the accident virtual scene; and the processor controls the fire extinguishing equipment to perform fire extinguishing operation to obtain operation data, maps the operation data to the accident virtual scene and dynamically adjusts a fire evolution process in the accident virtual scene to obtain a processing result. By restoring the multi-sensory experience of a real scene, the psychological adaptability of participants is improved, a closed loop of real operation and virtual feedback is realized through fire extinguishing equipment, and the defect that traditional VR drilling lacks interactive experience is overcome.

Description

Technical Field

[0001] The present invention relates to the technical field of emergency drills, and in particular to a multi-modal emergency drill device and an evaluation system thereof. Background Art

[0002] Emergency drills are a systematic process of training participants' emergency response capabilities through simulated emergency scenarios. As a crucial component of production safety and emergency management, emergency drills' core value lies in enhancing personnel's rapid response capabilities, teamwork, and psychological resilience in real-world disaster situations, while also verifying the feasibility of emergency plans. High-quality emergency drills can significantly reduce casualties and property losses in actual accidents, making them an indispensable training tool in industries such as industry, the military, and public safety.

[0003] Current emergency drills primarily rely on physical locations to create simulated scenarios, manually setting up simple disaster scenarios while participants operate real equipment. These systems utilize VR / AR technology to create three-dimensional virtual environments, providing visual and auditory feedback through head-mounted displays (HMDs) and supporting some gesture interaction. Some systems also incorporate limited motion capture technology to record operational traces.

[0004] However, traditional physical drills in existing technologies are difficult to reproduce complex disasters, while virtual drills lack multi-sensory stimulation and cannot simulate key physical effects such as thermal radiation and shock waves; operations and feedback in virtual systems are disconnected, especially when physical equipment is involved, a natural mapping relationship cannot be established; and existing systems mostly rely on coaches' subjective observation and scoring, and lack quantitative analysis capabilities for key dimensions such as operational standardization and stress physiological indicators, resulting in distorted evaluation of training effects. Summary of the Invention

[0005] In response to the shortcomings in the relevant technologies, the purpose of the present invention is to provide a multimodal emergency drill device and its evaluation system to solve the technical problems in the existing technology that traditional physical drills are difficult to reproduce complex disasters, and virtual drills lack multi-sensory stimulation and cannot simulate key physical effects such as thermal radiation and shock waves; operations and feedback in virtual systems are disconnected, especially when physical equipment is involved, a natural mapping relationship cannot be established; and existing systems mostly rely on coaches' subjective observation and scoring, and lack quantitative analysis capabilities for key dimensions such as operational standardization and stress physiological indicators, resulting in distorted training effect evaluation.

[0006] The present invention provides a multi-modal emergency drill device, comprising: A model building module is used to scan the target industrial scene to establish a scene model, and dynamically configure a virtual accident scene in the scene model, wherein the virtual accident scene includes the process of fire occurrence and smoke diffusion; A multimodal feedback module, equipped with a multi-region vibration device, a temperature response module and a directional sound field device, is used to generate vibration feedback, high temperature feedback and sound feedback; Fire extinguishing equipment, installed in the physical rehearsal area of ​​the target industrial scene and corresponding to the coordinates of the virtual fire extinguishing equipment in the virtual accident scene, for performing fire extinguishing operations in fire accidents; a processor, communicatively connected to the multimodal feedback module and the fire extinguishing equipment; The processor is further configured with an interactive logic unit for triggering vibration feedback, high temperature feedback, and sound feedback according to the multimodal feedback module when a fire occurs in the virtual accident scene; The processor controls the fire extinguishing equipment to perform fire extinguishing operations to obtain operation data, maps the operation data to the accident virtual scene, and dynamically adjusts the fire evolution process in the accident virtual scene to obtain a processing result.

[0007] The embodiment of the present invention scans the target industrial scene through a model building module to construct a scene model with real physical characteristics, and dynamically configures a virtual accident scene including the fire and smoke diffusion process in the scene model, which can accurately restore the physical characteristics of the industrial environment. At the same time, by dynamically loading disaster simulation content, the drill scene can maintain the spatial relationship of the real environment and flexibly simulate various accident scenarios, providing an accurate physical space benchmark and disaster evolution basis for subsequent multimodal interaction.

[0008] In some embodiments of the present invention, the multimodal feedback module specifically includes: a vibration generating device, configured with a vibration motor array, for generating graded vibration feedback data according to the location and intensity of the fire accident in the virtual accident scene, and performing vibration feedback at different frequencies and amplitudes; a temperature sensing generating device, configured with a thermoelectric module, for generating hot air feedback matching the virtual fire source temperature according to the virtual fire source temperature of the fire accident in the virtual accident scene; A sound generating device, comprising ultrasonic array speakers distributed throughout the target industrial scene, for locating explosion sounds using a VBAP algorithm based on the positions of the participants and the explosion point when a fire occurs; The visual generation device is equipped with a head-mounted display for displaying the occurrence and handling process of the fire accident in the virtual accident scene in real time.

[0009] The multimodal feedback module of an embodiment of the present invention is configured with a vibration generating device, a temperature generating device, a sound generating device and a visual generating device. When a fire is triggered in a virtual accident scene, the vibration motor array generates graded vibration feedback according to the location and intensity of the disaster. The thermoelectric module outputs hot air feedback that matches the temperature of the virtual fire source, and the ultrasonic speaker array realizes spatial sound field positioning through the VBAP algorithm. It can simultaneously stimulate the tactile, thermal and auditory senses of the participants, enhancing the immersion and realism of the drill.

[0010] In some embodiments of the present invention, the fire extinguishing equipment specifically includes: a pressure sensor, installed inside the handle of the fire extinguishing equipment, for measuring the grip force; The RFID tag is installed at the bottom of the fire extinguishing equipment and is used to store the unique identification code and calibration parameters of the fire extinguishing equipment.

[0011] The fire-fighting equipment of an embodiment of the present invention monitors the grip force in real time by installing a pressure sensor inside the handle, and an RFID tag is set at the bottom to store the device's unique identification and calibration parameters, so that the fire-fighting equipment can accurately identify and establish a mapping relationship with the corresponding equipment in the virtual accident scene. By capturing the actual operation force and converting it into fire-fighting parameters in the virtual scene, the interaction between real operation and virtual feedback is realized, providing realistic force feedback and operation feel for operation training.

[0012] In some embodiments of the present invention, the physical rehearsal area of ​​the target industrial scene is further equipped with valve equipment; The valve device corresponds to the virtual valve device coordinates of the virtual accident scene, which includes: A motion sensor, mounted on the end of the rotating shaft of the valve device, for detecting the rotation angle of the valve device in real time; An operation feedback unit is communicatively connected to the motion sensor and is used to map the opening of the valve device into a leakage rate.

[0013] The valve device of the embodiment of the present invention detects the rotation angle in real time by installing a motion sensor at the end of the rotating shaft, and maps the opening to the leakage rate through the operation feedback unit, so that the physical operation of the valve device by the participants can directly affect the fluid state in the virtual accident scene. By establishing a precise correspondence between the actual operation and the virtual parameters, the authenticity and instant feedback effect of the operation training are enhanced, so that the participants can intuitively feel the scene changes brought about by the operation.

[0014] In some embodiments of the present invention, the controller is further configured to: After obtaining the gripping force data according to the pressure sensor, the gripping force data is transmitted in real time to the virtual accident scene via wireless communication to control the fire extinguishing spray range of the virtual fire extinguishing equipment in the virtual accident scene; The controller is communicatively connected to the valve device, and is used to obtain rotation angle data based on the motion sensor, and then transmit the rotation angle data in real time to the accident virtual scene through wireless communication, so as to control the leakage rate corresponding to the opening of the virtual valve device in the accident virtual scene.

[0015] The controller of the embodiment of the present invention transmits the gripping force and valve device rotation angle data in real time through wireless communication, and maps it to the virtual accident scene, dynamically adjusts the fire extinguishing spray range and leakage rate, and establishes a closed-loop feedback between the actual operation and the virtual disaster evolution, so that the drill process can change in real time according to the operation. By instantly responding to the operational behavior of the participants, the pertinence and effectiveness of the training are improved, and the drill effect is closer to the actual situation.

[0016] In some embodiments of the present invention, the vibration generating device is further configured as follows: When a fire accident occurs in the virtual accident scene, fragment collision data is obtained through the physics engine, and impact force data is calculated based on the fragment collision data combined with the physics engine. Vibration feedback of different frequencies and amplitudes is performed through the vibration motor array, and the posture of the participants is tracked in real time according to the inertial measurement unit to dynamically adjust the vibration direction.

[0017] The vibration generating device of an embodiment of the present invention obtains fragment collision data and calculates the impact force through a physical engine, drives the vibration motor array to generate vibration feedback with directionality and strength, and at the same time combines the inertial measurement unit to track the posture of the performers in real time to dynamically adjust the vibration direction. It can accurately simulate the physical characteristics of explosion shock waves and fragment impacts, and enhance the physical reality of the disaster scene by providing tactile feedback corresponding to the spatial position and intensity of the virtual disaster, so that the performers can perceive the spatial distribution of the disaster through touch.

[0018] In some embodiments of the present invention, the temperature sensing device is further configured as follows: When a fire accident occurs in the virtual accident scene, the temperature is dynamically adjusted through the thermoelectric module and the PID control algorithm according to the heat source position and temperature change trend of the fire accident, and the temperature is dynamically correlated with the flame height through the thermoelectric module.

[0019] The temperature sensing generating device of the embodiment of the present invention dynamically adjusts the output temperature through a thermoelectric module and a PID control algorithm, so that it matches the temperature of the virtual fire source and the changes in flame height in real time, and can accurately restore the characteristics of thermal radiation attenuation with distance. By providing thermal feedback consistent with the virtual fire source in spatial position and temperature gradient, it achieves accurate simulation of high-temperature environments, significantly improving the participants' perception of thermal hazards, enabling them to judge the location and intensity of the fire source through temperature changes.

[0020] In some embodiments of the present invention, the sound generating device is further configured as follows: When a fire accident occurs in the virtual accident scene, the coordinates of the explosion point in the virtual accident scene are mapped to the physical position of the ultrasonic array speaker through the VBAP algorithm, and the sound pressure level is adjusted in real time according to the distance between the participants and the explosion point when the fire occurs.

[0021] The sound generating device of the embodiment of the present invention uses the VBAP algorithm to map the coordinates of the explosion point to the ultrasonic speaker array, and adjusts the sound pressure level in real time according to the distance. It can accurately reproduce the propagation characteristics of sound waves in three-dimensional space. By providing auditory feedback with precise direction and distance, the participants can accurately locate the disaster location through hearing, enhance the three-dimensionality and authenticity of environmental perception, and provide an important sensory basis for emergency decision-making.

[0022] In some embodiments of the present invention, the visual generation device is further configured to: When a fire accident occurs in the virtual accident scene, the fragment scattering trajectory obtained according to the fragment collision data is displayed through the head-mounted display, and the explosion light and shadow display when the fire occurs is triggered through the network protocol.

[0023] The visual generation device of the embodiment of the present invention presents the trajectory of flying debris and the light and shadow effects of explosions through a head-mounted display, and combined with the dynamic light and shadow changes triggered by the network protocol, it can fully display the visual characteristics of the disaster. By providing visual effects consistent with the results of physical calculations, it effectively reduces the brain's cognitive load on virtual scenes and improves the realism and effectiveness of the drill.

[0024] Some embodiments of the present invention further provide an evaluation system for a multimodal emergency drill device, comprising: The data acquisition module is used to collect the operation trajectory, physiological indicators and environmental parameters generated by the participants during the emergency drill device; A data processing module is used to calculate evaluation indicators based on the operation trajectory, physiological indicators and environmental parameters, and establish a multi-dimensional evaluation model through the AHP-entropy weight method; The evaluation and analysis module is used to analyze the evaluation indicators according to the multi-dimensional evaluation model, obtain analysis results, and dynamically adjust the scene parameters according to the analysis results.

[0025] The embodiment of the present invention acquires multidimensional data of operation trajectories, physiological indicators and environmental parameters through a data acquisition module, constructs a multidimensional evaluation model using the AHP-entropy weight method through a data processing module, and finally dynamically adjusts the scenario parameters through an evaluation and analysis module. This allows for comprehensive quantitative analysis of the drill effect, and achieves a scientific evaluation of emergency response capabilities by establishing a correlation model between operation behaviors, physiological responses and environmental changes. Dynamic adjustment based on the evaluation results enables the system to adapt to different training needs, continuously optimize the drill effect, and provide an objective basis for improving emergency response capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work. Figure 1 A schematic diagram of a multi-modal emergency drill device provided in an embodiment of the present invention Figure 1 ; Figure 2 A schematic structural diagram of a multimodal feedback module 2 provided in an embodiment of the present invention; Figure 3 A schematic diagram of a multi-modal emergency drill device provided in an embodiment of the present invention Figure 2 ; Figure 4 A schematic diagram of the functional architecture of a multi-modal emergency drill device provided in an embodiment of the present invention; Figure 5 A schematic diagram of the functional architecture interaction of a multimodal emergency drill device provided in an embodiment of the present invention; Figure 6 A schematic diagram of the structure of an evaluation system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is described and illustrated below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely used to explain this application and are not intended to limit this application. Based on the embodiments provided in this application, all other embodiments obtained by those of ordinary skill in the art without making any creative efforts are within the scope of protection of this application. It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. In the absence of conflict, the embodiments of the present invention and the features thereof may be combined with each other. The technical solution of the present invention is described in detail below in conjunction with specific embodiments and the accompanying drawings.

[0028] As attached Figure 1 As shown, the present invention provides a multi-modal emergency drill device, comprising: The model building module 1 is used to scan the target industrial scene to build a scene model, and dynamically configure a virtual accident scene in the scene model. The virtual accident scene includes the process of fire occurrence and smoke diffusion.

[0029] Optionally, the target industrial scene is photographed and scanned by a drone, a point cloud model is generated using the 3D modeling software RealityCapture3, and a scene model is created using 3dsMax software; the scene model includes chemical plant equipment, building structures, and piping systems.

[0030] In the target industrial scene, fixed structures such as buildings and equipment bases are built using low-polygon static models, and normal maps are used to enhance the details. In the target industrial scenario, interactive devices use high-precision dynamic models and reserve joint animation interfaces; The static model and the dynamic model are combined to generate a scene model.

[0031] Multimodal feedback module 2, equipped with a multi-region vibration device, a temperature response module and a directional sound field device, for generating vibration feedback, high temperature feedback and sound feedback; Fire extinguishing equipment 3, installed in the physical drill area of ​​the target industrial scene, and corresponding to the coordinates of the virtual fire extinguishing equipment in the virtual accident scene, for performing fire extinguishing operations in fire accidents; Processor 4, communicatively connected to the multimodal feedback module and the fire extinguishing equipment; The processor 4 is further configured with an interactive logic unit for triggering vibration feedback, high temperature feedback and sound feedback according to the multimodal feedback module when a fire occurs in the virtual accident scene; The processor controls the fire extinguishing equipment to perform fire extinguishing operations to obtain operation data, maps the operation data to the accident virtual scene, and dynamically adjusts the fire evolution process in the accident virtual scene to obtain a processing result.

[0032] Through the above-mentioned emergency drill device, the target industrial scene is scanned to construct a scene model with real physical characteristics, and a virtual accident scene including the fire and smoke diffusion process is dynamically configured in the scene model, which can accurately restore the physical characteristics of the industrial environment. At the same time, by dynamically loading disaster simulation content, the drill scene can maintain the spatial relationship of the real environment and flexibly simulate various accident scenarios, providing an accurate physical space benchmark and disaster evolution basis for subsequent multimodal interaction.

[0033] In some embodiments of the present invention, Figure 2 As shown, the multimodal feedback module 2 specifically includes: The vibration generating device 21 is configured with a vibration motor array, and is used to generate graded vibration feedback data according to the location and intensity of the fire accident in the virtual accident scene, and to provide vibration feedback at different frequencies and amplitudes; The temperature sensing generating device 22 is configured with a thermoelectric module and is used to generate hot air feedback that matches the virtual fire source temperature according to the virtual fire source temperature of the fire accident in the virtual accident scene; The sound generating device 23 includes ultrasonic array speakers distributed in the target industrial scene, and is used to locate the explosion sound using the VBAP algorithm based on the positions of the participants and the explosion point when the fire occurs; The visual generation device 24 is equipped with a head-mounted display for displaying the occurrence and handling process of the fire accident in the virtual accident scene in real time.

[0034] The multimodal feedback module 2 is configured with a vibration generating device 21, a temperature generating device 22, a sound generating device 23 and a visual generating device 24. When a fire is triggered in the virtual accident scene, the vibration motor array generates graded vibration feedback according to the location and intensity of the disaster. The thermoelectric module outputs hot air feedback that matches the temperature of the virtual fire source. The ultrasonic speaker array realizes spatial sound field positioning through the VBAP algorithm, which can simultaneously stimulate the tactile, thermal and auditory senses of the participants, enhancing the immersion and realism of the drill.

[0035] In some embodiments of the present invention, the vibration generating device 21 is further configured as follows: When a fire accident occurs in the virtual accident scene, the fragment collision data is obtained through the physics engine. The impact force data is calculated based on the fragment collision data combined with the physics engine. Vibration feedback of different frequencies and amplitudes is performed through the vibration motor array, and the posture of the participants is tracked in real time based on the inertial measurement unit to dynamically adjust the vibration direction.

[0036] Furthermore, a multi-region vibration transposition is constructed using a vibration motor array. This allows for independent control of the vibration frequency and amplitude of each motor. Alternatively, the multi-region vibration device can be a tactile vest. Twelve to sixteen motors are deployed in key areas of the vest, such as the chest, back, and shoulders, to cover the main sensory areas of the human body.

[0037] The motor vibration frequency ranges from 50Hz to 500Hz, and the motor amplitude ranges from 0.1G to 3G.

[0038] A low-frequency vibration module is integrated into the vibration generating device 21 to simulate the deep vibration of the explosion shock wave when a fire accident occurs; wherein the low-frequency vibration module can be a ButtKickerGamerPro module; the vibration frequency range of the low-frequency vibration module is 5Hz to 200Hz.

[0039] For example, when a fire accident occurs, vibrations with a frequency of 50Hz and an amplitude of 2G are triggered within 10 meters of the explosion center.

[0040] A 9-axis inertial measurement unit (IMU) is embedded in the tactile vest to track the performers' posture in real time and dynamically adjust the vibration direction. The 9-axis inertial measurement unit (IMU) includes an accelerometer, a gyroscope, and a magnetometer.

[0041] For example, when a fire accident occurs, if the explosion is located on the left side of the performer, the amplitude of the left motor of the tactile vest worn by the performer will be increased to 200%.

[0042] Furthermore, the physical properties of explosion fragments are defined through the physics engine PhysX, and the fragment scattering trajectory is calculated in real time; among them, the physical properties of explosion fragments include mass, velocity and collision elasticity.

[0043] Obtain fragment collision data, such as collision position, impact force, and other fragment collision data, through the PxSimulationEventCallback interface of the multi-platform physics simulation engine development tool PhysXSDK, and trigger tactile feedback instructions.

[0044] The motor amplitude is dynamically adjusted based on the debris collision data through the tactile feedback mapping algorithm. The calculation model of the tactile feedback mapping algorithm is:

[0045] in, is the motor amplitude; is the impact force in the detected debris collision data; is the maximum impact force, set to 1000N; The maximum amplitude of the motor is set to 3G.

[0046] For example, when a fire accident occurs and debris hits the wall, the physics engine PhysX calculates the trajectory of the debris flying, generates a collision event including the collision position and explosion force, and sends the collision event to the tactile controller integrated in the tactile vest through the UDP protocol.

[0047] The haptic controller analyzes the collision event, calculates the target motor ID, motor amplitude and duration, and sends the target motor ID, motor amplitude and duration as a PWM signal to drive the motor array.

[0048] The 9-axis inertial measurement unit (IMU) is used to track the posture of the performers in real time, and the motor activation priority is calculated based on the spatial information in the debris collision data; for example, the back motor is activated first in a rear collision.

[0049] The collision point coordinates are obtained based on the spatial position in the debris collision data. Based on the 9-axis inertial measurement unit (IMU), the collision point coordinates are converted into the local coordinate system of the performer and matched with the motor layout of the tactile vest.

[0050] For example, if the collision point coordinates are located 30° to the right front of the participant, the right chest and right shoulder motors of the tactile vest will be activated, and the amplitude will be distributed according to the angle weight.

[0051] In addition, when a fire accident occurs, the explosion shock wave generated by the explosion adopts short pulse vibration with a duration of 0.1s to 0.5s, and intermittent vibration is triggered by fragment impact every 0.2s to simulate continuous impact.

[0052] The ButtKicker module integrated in the tactile vest outputs low-frequency vibration to simulate the instantaneous impact of an explosion; the low-frequency vibration has a frequency of 50Hz, an amplitude of 2G, and lasts for 0.3s.

[0053] The vibration motor array triggers regional vibrations of the haptic vest based on the debris density distribution in the debris impact data; for example, the motor amplitude of the haptic vest is increased to 2.5G in areas with dense debris.

[0054] Furthermore, the vibration intensity is calculated based on the distance between the participants and the explosion center through a dynamic attenuation calculation model. The dynamic attenuation calculation model is:

[0055] in, is the vibration intensity; The distance between the participants and the center of the explosion.

[0056] Optionally, real-time data synchronization between the physics engine PhysX and the tactile vest is implemented using Bluetooth 5.0 or a customized wireless protocol; wherein the Bluetooth 5.0 delay is less than 20ms; the wireless protocol can be the Nordic nRF5340 chipset.

[0057] Through timestamp alignment technology, we ensure that the error between the physics engine PhysX calculation and the tactile feedback response is lower than the human perception threshold. Among them, the delay of the physics engine PhysX calculation is about 5ms; the tactile feedback response time is less than 20ms; and the human perception threshold is set to 50ms.

[0058] The vibration generating device 21 obtains the fragment collision data and calculates the impact force through the physical engine, drives the vibration motor array to generate vibration feedback with directionality and strength, and combines the inertial measurement unit to track the posture of the participants in real time to dynamically adjust the vibration direction. It can accurately simulate the physical characteristics of the explosion shock wave and fragment impact, and enhance the physical reality of the disaster scene by providing tactile feedback corresponding to the spatial position and intensity of the virtual disaster, so that the participants can perceive the spatial distribution of the disaster through touch.

[0059] In some embodiments of the present invention, the temperature sensing device 22 is further configured as follows: When a fire accident occurs in the virtual accident scene, it is used to dynamically adjust the temperature through the thermoelectric module and PID control algorithm according to the heat source location and temperature change trend of the fire accident, and dynamically correlate the temperature with the flame height through the thermoelectric module.

[0060] Optionally, the thermoelectric module can adopt a Peltier thermoelectric module. The semiconductor cooling element TEC1-1270 in the Peltier thermoelectric module can cover the temperature range of 40°C to 80°C, meeting the requirements of flame thermal radiation simulation; wherein, the voltage of the semiconductor cooling element TEC1-1270 is set to 12V, the current is set to 6A, and the maximum temperature difference is 67°C.

[0061] Deploy 4 to 6 groups of Peltier thermoelectric modules around the head and torso of the performers to form multi-zone temperature control; among them, two groups of Peltier thermoelectric modules can be deployed on the chest and back of the performers respectively.

[0062] The temperature sensing device 22 integrates a temperature sensor, which can use a high-precision digital sensor DS18B20 to monitor the ambient temperature in real time for closed-loop control; wherein, the accuracy of the high-precision digital sensor DS18B20 is ±0.5°C.

[0063] The temperature sensing device 22 is also equipped with a microcontroller, which can be an STM32F4 microcontroller. The STM32F4 microcontroller supports PWM output and multi-channel ADC acquisition, and is used to process temperature and drive the Peltier thermoelectric module.

[0064] The temperature sensing device 22 is also equipped with a driving circuit, which can adopt an H-bridge driving circuit. The H-bridge driving circuit controls the current direction and power of the Peltier thermoelectric module through the L298N chip or the DRV8833 chip to achieve two-way regulation of heating and cooling.

[0065] The temperature sensing device 22 is also equipped with an aluminum heat sink and a high-speed fan to ensure the heat dissipation efficiency of the cold end of the Peltier thermoelectric module and maintain the temperature difference at more than 50°C.

[0066] Furthermore, the temperature in the Peltier thermoelectric module is blown toward the performers through a high-speed fan to simulate the effect of flame radiation hot air, and the fan speed is automatically adjusted according to the ambient temperature.

[0067] Among them, the high-speed fan supports adjustment of the air outlet angle from 30° to 120° to match the direction of the virtual fire source; for example, when the flame is on the left side of the performer, the left nozzle of the high-speed fan is activated, and when the ambient temperature is greater than 30°C, the speed of the high-speed fan is increased to 5000RPM.

[0068] The temperature sensing device 22 is also configured with a virtual engine interface and a hardware controller. The virtual fire source temperature is defined in the virtual engine Unity or the virtual engine Unreal, and the virtual fire source temperature variable is transmitted to the hardware controller in real time through the virtual engine interface via the TCP protocol or the UDP protocol.

[0069] The hardware controller dynamically correlates the temperature in the Peltier thermoelectric module with the flame height through a mapping model. The mapping model is:

[0070]

[0071] in, is the temperature in the Peltier thermoelectric module; is the flame height; is the virtual flame temperature.

[0072] For example, when the virtual flame temperature variable When the temperature in the Peltier thermoelectric module is 300℃, The flame height is 185℃. is 4.5m.

[0073] The PID control algorithm dynamically adjusts the analog signal duty cycle according to the temperature difference between the target temperature and the temperature measured by the temperature sensor. The calculation model of the control algorithm is:

[0074]

[0075] in, is the duty cycle of the analog signal; is the temperature error value; is the proportional coefficient, which is 2.5; is the integral coefficient, and its value is 0.1; is the differential coefficient, which is 0.5; is the target temperature; The temperature measured by the temperature sensor. By adjusting the proportional coefficient, integral coefficient, and differential coefficient, the response time is less than 3s and the overshoot is less than 5%.

[0076] The position of the virtual flame is obtained through the 9-axis inertial measurement unit (IMU) or the performer's perspective data, and the Peltier thermoelectric modules in the corresponding area are activated preferentially based on the position of the virtual flame. For example, when the virtual flame is located on the right side of the performer, the power of the Peltier thermoelectric module on the right side is increased to 80%.

[0077] By predicting the trend of virtual flame temperature changes, the power of the Peltier thermoelectric module is adjusted in advance to compensate for hardware thermal inertia. For example, when the virtual flame temperature is predicted to rise exponentially, the power of the Peltier thermoelectric module is adjusted and the temperature lag is reduced through the Kalman filter algorithm.

[0078] By establishing a lookup table of temperature and high-voltage curves in the Peltier thermoelectric module, the problem of decreased efficiency in the high-temperature zone is solved; for example, when the temperature in the Peltier thermoelectric module is greater than 150°C, the power of the Peltier thermoelectric module is increased by 20%.

[0079] In addition, when the temperature in the Peltier thermoelectric module is greater than 200°C, or the skin contact temperature of the performer is greater than 45°C, an emergency power off is triggered; optionally, the circuit is cut off through a MOSFET relay.

[0080] The temperature sensing device 22 dynamically adjusts the output temperature through the thermoelectric module and PID control algorithm to match it in real time with the temperature and flame height changes of the virtual fire source. It can accurately restore the characteristics of thermal radiation attenuation with distance. By providing thermal feedback consistent with the spatial position and temperature gradient of the virtual fire source, it achieves accurate simulation of high-temperature environments, significantly improving the participants' perception of thermal hazards, enabling them to judge the location and intensity of the fire source through temperature changes.

[0081] In some embodiments of the present invention, the sound generating device 23 is further configured as follows: When a fire accident occurs in the virtual accident scene, the VBAP algorithm is used to map the coordinates of the explosion point in the virtual accident scene to the physical position of the ultrasonic array speaker, and the sound pressure level is adjusted in real time according to the distance between the participants and the explosion point when the fire occurs.

[0082] Furthermore, the ultrasonic array speaker can use the HyperSoundTPS-1 speaker, which can achieve ±15° narrow beam directional sound transmission through ultrasonic carrier modulation technology, so that the effective distance of sound source propagation reaches 0.5m to 8m, and the sound pressure level can reach 105dB. By allowing performers to wear headphones, 360° sound field positioning can be achieved.

[0083] The sound generating device 23 is further provided with a DSP audio processor. The DSP audio processor supports conversion between the Ambisonics format and the Binaural format and is used for real-time processing of spatialized audio data.

[0084] The VBAP algorithm is used to map the explosion point coordinates in the virtual accident scene to the ultrasonic array speaker channels, ensuring that the sound field positioning error is less than 5°.

[0085] For example, when the explosion occurs 45° to the right front of the performer, the sound pressure level of the right front speaker of the ultrasonic array speaker increases to 120%, and the delay parameter is set to 1.5ms.

[0086] The sound pressure level is calculated using a dynamic sound field attenuation algorithm based on the distance between the performers and the sound source. The calculation model of the dynamic sound field attenuation algorithm is:

[0087] in, is the actual sound pressure level at the performers’ positions; is the sound pressure level at the sound source position; is the distance between the performers and the sound source.

[0088] The sound generating device 23 uses the VBAP algorithm to map the coordinates of the explosion point to the ultrasonic speaker array, and adjusts the sound pressure level in real time according to the distance. It can accurately reproduce the propagation characteristics of sound waves in three-dimensional space. By providing auditory feedback with precise direction and distance, participants can accurately locate the disaster location through hearing, enhancing the three-dimensionality and authenticity of environmental perception, and providing an important sensory basis for emergency decision-making.

[0089] In some embodiments of the present invention, the visual generation device 24 is further configured to: When a fire accident occurs in the virtual accident scene, the debris scattering trajectory obtained based on the debris collision data is displayed through the head-mounted display, and the explosion light and shadow display when the fire occurs is triggered through the network protocol.

[0090] In combination with the sound generating device 23, according to the event trigger of the virtual engine Unity or the virtual engine Unreal, the sound and light instructions are synchronously sent through the Art-Net protocol, wherein the timestamp alignment error is less than 10ms.

[0091] Audio data is transmitted over the IP network based on the AES67 standard, with a data transmission delay of less than 2ms.

[0092] Furthermore, when an explosion is detected, the LED lights are controlled to strobe quickly through the DMX protocol to simulate the explosion flash effect, and OSC commands are sent at the same time to adjust the sound and image direction; The value of DMX channel 101 is set to 255 to control the LED brightness to 100%. The value of DMX channel 102 is set to 200 to control the strobe frequency of the light to 100Hz. By calling the audio system, the spatialization effect of the sound is adjusted according to the coordinates of the explosion point, so that the explosion sound comes from a specific direction.

[0093] The dynamic range compression algorithm limits the peak sound pressure to less than 85dB, simulating the sensitivity of the human ear to different frequencies to comply with OSHA hearing protection standards.

[0094] When a performer takes off their headphones or triggers the emergency stop button, the audio output is cut off within 0.1 seconds.

[0095] The visual generation device 24 presents the trajectory of flying debris and the light and shadow effects of the explosion through a head-mounted display, and combined with the dynamic light and shadow changes triggered by the network protocol, it can fully display the visual characteristics of the disaster. By providing visual effects consistent with the results of physical calculations, it effectively reduces the brain's cognitive load on virtual scenes and improves the realism and effectiveness of the drill.

[0096] In some embodiments of the present invention, fire and smoke diffusion are simulated in a virtual accident scene, flame particles are generated by the UnrealNiagara particle system, and the flame diffusion path is simulated in combination with the Navier-Stokes equation; smoke diffusion is rendered in real time through a GPU-accelerated voxelization algorithm, and the diffusion direction is dynamically adjusted based on wind direction parameters and wind speed parameters.

[0097] Combined with the temperature sensing device 22, the temperature in the Peltier thermoelectric module is mapped to the flame height. Every 100°C increase in temperature corresponds to an increase in the flame height of 1.5 meters. At the same time, hot air is output according to the Peltier thermoelectric module to simulate the thermal radiation effect.

[0098] The NVIDIA Flex engine is used to simulate the viscosity and diffusion path of leaked liquids, and the shader is used to dynamically generate ground wetness maps and real-time reflection effects; the leaked liquids include oils, chemicals, etc.

[0099] When the virtual flame ignites the leaked liquid, a secondary explosion event is triggered. The physics engine PhysX is used to calculate the trajectory of the fragments, and the sound generation transposition 23 is linked to simulate the explosion sound effect, and the vibration generation device 21 is combined to simulate the explosion impact vibration.

[0100] In some embodiments of the present invention, the fire extinguishing equipment 3 specifically includes: A pressure sensor, installed inside the handle of the fire extinguishing device 3, is used to measure the grip force; The RFID tag is installed at the bottom of the fire extinguishing device 3 and is used to store the unique identification code and calibration parameters of the fire extinguishing device 3.

[0101] Furthermore, the fire extinguishing equipment 3 also includes a microcontroller, which uses the microcontroller ESP32-WROOM-32, installed in the hidden place of the interlayer of the bottle body of the fire extinguishing equipment 3, uses a dual-core 240MHz, and integrates WiFi or Bluetooth to support parallel collection and wireless transmission of multi-sensor data.

[0102] Optionally, the pressure sensor uses a thin film pressure sensor FlexiForceA201. The thin film pressure sensor FlexiForceA201 has a measuring range of 0 pounds to 100 pounds and a thickness of 0.2 mm, which can fit the curvature of the handle of the fire extinguishing equipment 3.

[0103] The thin-film pressure sensor FlexiForce A201 is embedded in the handle of the fire extinguishing equipment 3 and fixed by a custom 3D-printed bracket, with its surface covered with a silicone protective layer.

[0104] The RFID tag adopts the UHF passive tag Impinj Monza R6, and the reading distance of the passive tag Impinj Monza R6 is 0m to 8m.

[0105] The rubber cover of the handle of the fire extinguishing equipment 3 was removed, and the thin film pressure sensor FlexiForceA201 was attached to the inner wall of the handle of the fire extinguishing equipment 3. It was connected to the microcontroller ESP32-WROOM-32 via an FPC flexible cable. The appearance of the handle of the fire extinguishing equipment 3 was reconstructed while maintaining the original ergonomic design.

[0106] A UHF passive tag Impinj Monza R6 is installed at the bottom of the fire extinguishing equipment 3 and is kept 2 mm away from the metal bottle of the fire extinguishing equipment 3 to prevent metal from shielding and interfering with the RFID signal while ensuring that the tag can be read stably.

[0107] The fire extinguishing equipment 3 is also equipped with a lithium-ion battery. The lithium-ion battery adopts ER34615, with a voltage of 3.6V and a current of 19Ah. Combined with a DC-DC step-down circuit, it ensures that the fire extinguishing equipment has a battery life of more than 1 year; among them, the DC-DC step-down circuit adopts ER34615.

[0108] The fire extinguishing device 3 monitors the grip force in real time by installing a pressure sensor inside the handle. An RFID tag is set at the bottom to store the device's unique identification and calibration parameters, so that the fire extinguishing device 3 can accurately identify and establish a mapping relationship with the virtual fire extinguishing device 3 in the virtual accident scene. By capturing the actual operating force and converting it into fire extinguishing parameters in the virtual scene, the interaction between real operation and virtual feedback is realized, providing realistic force feedback and operation feel for operation training.

[0109] In some embodiments of the present invention, Figure 3 As shown, the physical rehearsal area of ​​the target industrial scenario is also equipped with valve equipment 5; The valve device 5 corresponds to the virtual valve device coordinates of the accident virtual scene, which includes: A motion sensor is installed at the end of the rotating shaft of the valve device 5 and is used to detect the rotation angle of the valve device 5 in real time; The operation feedback unit is in communication with the motion sensor and is used to map the opening of the valve device 5 into a leakage rate.

[0110] Optionally, the motion sensor uses a six-axis motion sensor MPU-6050, which has a range of ±2000° / s and uses an I²C interface.

[0111] The six-axis motion sensor MPU-6050 is embedded in the machined groove at the end of the rotating shaft of the valve device 5 and encapsulated with epoxy resin for shockproofing. The six-axis motion sensor MPU-6050 is fixed by a snap to ensure that the rotating axis of the valve device 5 is aligned with the Z axis of the motion sensor.

[0112] The valve device 5 further includes a magnetic encoder, which uses the magnetic encoder AS5600, for replicating and calibrating the absolute rotation angle of the valve device 5; wherein the accuracy of the magnetic encoder AS5600 is ±0.5°.

[0113] The valve device 5 also includes a battery compartment, in which a button battery CR2032 is installed. By combining it with an LDO voltage regulator circuit TPS7A05, the battery life of the valve device 5 is guaranteed to be greater than 500 hours.

[0114] The valve device 5 is also integrated with an energy harvesting module, which uses LTC3588-1 to generate electricity by utilizing mechanical rotational kinetic energy.

[0115] The valve device 5 detects the rotation angle in real time by installing a motion sensor at the end of the shaft, and maps the opening to the leakage rate through the operation feedback unit, so that the physical operation of the valve device by the participants can directly affect the fluid state in the virtual accident scene. By establishing a precise correspondence between the actual operation and the virtual parameters, the authenticity and instant feedback effect of the operation training are enhanced, allowing the participants to intuitively feel the scene changes brought about by the operation.

[0116] In some embodiments of the present invention, the analog signals in the pressure sensor and the motion sensor are converted into 16-bit digital signals through the signal conditioning circuit ADS1115, and sampled 860 times per second to meet the dynamic pressure detection requirements.

[0117] The high-frequency noise of the motion sensor is eliminated by a low-pass filter. The low-pass filter allows signals no greater than 50Hz to pass through and can attenuate higher-frequency noise.

[0118] In some embodiments of the present invention, the controller 4 is further configured to: After obtaining the grip force data from the pressure sensor, the grip force data is transmitted to the accident virtual scene in real time via wireless communication to control the fire extinguishing spray range of the virtual fire extinguishing equipment 3 in the accident virtual scene; The controller 4 is in communication with the valve device 5 and is used to obtain the rotation angle data according to the motion sensor, and then transmit the rotation angle data to the accident virtual scene in real time through wireless communication to control the leakage rate corresponding to the opening of the virtual valve device in the accident virtual scene.

[0119] Furthermore, wireless communication can use Bluetooth 5.0 or ZigBee 3.0 for communication; among them, the transmission rate of Bluetooth 5.0 is 2Mbps and the delay is less than 10ms; the transmission distance of ZigBee 3.0 is 100m, and transmission is carried out through Mesh networking.

[0120] The grip force data and rotation angle data are collected by the microcontroller ESP32-WROOM-32 and transmitted via wireless communication. The wireless transmission protocol data packet structure includes: 2-byte header, which is the start flag of the data packet and is used to synchronize the receiving end; 4-byte device ID, which is a unique identifier of the device and is determined by the RFID tag; 2-byte pressure value. The pressure value is the reading of the pressure sensor. Its range is 0 to 4095, corresponding to the actual pressure of 0 pounds to 100 pounds.

[0121] 2-byte angle value, which is the reading of the motion sensor and ranges from -180° to +180°; 2-byte CRC check code, used to detect transmission errors.

[0122] The pressure value and the angle value are uploaded synchronously. The transmission frequency of wireless communication is 50Hz, and a frame of data packet is sent every 20ms. The data volume is 14 bytes per frame. To maintain the current communication rate, at least Stable channel bandwidth.

[0123] For example, the analog signals from the pressure sensor and motion sensor are converted into pressure values ​​and angle values ​​through the signal conditioning circuit ADS1115, and the converted pressure values ​​and angle values ​​are transmitted to the accident virtual scene through wireless communication.

[0124] Furthermore, the analog signal from the pressure sensor is processed using a pressure signal processing formula. The specific process is as follows: pressure data is collected 100 times per second, and the average of 10 consecutive sampling points is taken using a moving average filter to suppress high-frequency jitter noise. The linear fit R² of the pressure sensor is greater than 0.99, indicating excellent linearity.

[0125] The pressure signal processing formula is:

[0126] in, is the calibrated pressure value; is the original voltage value.

[0127] The process of processing the analog signal in the motion sensor is as follows: obtaining the rotation Euler angle of the valve device 5 through the quaternion solution of Madgwick filtering, the acquisition accuracy of the rotation Euler angle is ±1°, avoiding the integral drift of the motion sensor, and mapping the rotation angle of the valve device 5 to the opening of the valve device 5 through the rotation angle mapping algorithm.

[0128] The calculation model of the rotation angle mapping algorithm is:

[0129] in, is the opening of the valve device 5; is the yaw angle.

[0130] After the converted pressure value and angle value are transmitted to the accident virtual scene via wireless communication ZigBee3.0, the controller 4 controls the accident virtual scene to trigger the fire extinguishing spray range of the virtual fire extinguishing equipment and trigger the leakage rate corresponding to the opening of the virtual valve equipment.

[0131] Optionally, the wireless communication ZigBee 3.0 performs data transmission through a ZigBee communication module, and the ZigBee communication module includes a terminal node and a coordinator.

[0132] The chip model of the terminal node is TICC2530, which supports the ZigBee3.0 protocol; the frequency band of the terminal node is 2.4GHz, the transmission rate is 250kbps, and the power consumption is 1μA sleep current.

[0133] The terminal node is integrated into the fire extinguishing equipment 3 and the valve equipment 5, and is used to collect data from the pressure sensor and the motion sensor and upload the data to the coordinator.

[0134] The coordinator's chip model is TICC2652R, which supports multiple protocols; the coordinator's transmission power is +20dBm, and the RAM storage space is 128KB. The coordinator is connected to the controller 4 as a network hub.

[0135] The coordinator receives the data and sends it to the accident virtual scene through the USB interface or UART interface.

[0136] Through the data preprocessing module, the rotation Euler angle obtained by quaternion solution and the pressure sensor signal filtering obtained by moving average filtering are fused. The main control chip of the data preprocessing module is STM32F103, which uses Cortex-M3 core, 72MHz, and has a built-in 12-bit ADC.

[0137] The Unity virtual engine develops a C# plug-in using the ZigBee DLL library to receive coordinator data through a serial port or USB interface; the ZigBee DLL library can be TIZ-Stack SDK.

[0138] The Unreal virtual engine uses C++ to write the ZigBee data parsing thread and forwards it to the Unreal blueprint system through TCPSocket.

[0139] The controller 4 maps the pressure value received in the accident virtual scene to the fire extinguishing spray radius through the pressure mapping algorithm. The calculation model of the pressure mapping algorithm is:

[0140] in, is the pressure value, ranging from 0.5m to 20.5m; is the fire extinguishing spray radius, which is calculated to range from 0N to 1000N; The controller 4 maps the angle value received in the accident virtual scene to the leakage flow through the valve device mapping algorithm. The calculation model of the valve device mapping algorithm is:

[0141] in, is the angle value, ranging from 0° to 180°; is the leakage flow rate, and the calculated range is 0 m³ / h to 16.2 m³ / h.

[0142] In the Unreal virtual engine, through the Niagar special effects system and the Chaos physics engine, the angle value of the valve device 5 is used to drive the fluid leakage rate in the accident virtual scene in real time.

[0143] In addition, during the wireless communication process, a mesh network architecture is set up. When a node in the network fails or the signal is interrupted, the data packet will automatically select other available paths for transmission to ensure uninterrupted communication. The failure of any single node will not cause the entire network to be paralyzed, and the data can be forwarded through multiple hops of adjacent nodes.

[0144] During wireless communication, dynamic channel switching is also set up. The node monitors the received signal strength in real time and identifies channel interference. When the current channel RSSI is lower than the threshold, the network automatically switches to the channel with the least interference.

[0145] The controller 4 transmits the gripping force and the rotation angle data of the valve device 5 in real time through wireless communication, and maps it to the virtual accident scene, dynamically adjusting the fire extinguishing spray range and leakage rate, and establishing a closed-loop feedback between the actual operation and the virtual disaster evolution, so that the exercise process can change in real time according to the operation. By instantly responding to the operational behavior of the participants, the pertinence and effectiveness of the training are improved, and the exercise effect is closer to the actual situation.

[0146] like Figure 4-Figure 5 As shown, according to the model building module 1, multimodal feedback module 2, fire extinguishing equipment 3 and processor 4 in the above-mentioned multimodal emergency drill device, the emergency drill device is divided into an input layer, an output layer, a storage layer, a communication layer and an interaction layer according to the system functional architecture.

[0147] Among them, the input layer includes sensor modules and interactive devices.

[0148] The sensor module is further divided into motion tracking module, gesture recognition module, voice input module and eye tracking module.

[0149] The motion tracking module is implemented through motion sensors (gyroscopes, accelerometers), 9-axis inertial measurement units (IMUs), and optical tracking cameras; The gesture recognition module is implemented through a depth camera, an infrared camera, and data gloves; The voice input module is implemented through microphone array and voice recognition technology; The eye tracking module is implemented through an infrared camera, which is used to analyze the gaze point.

[0150] The interactive devices are further divided into handles, tactile feedback devices, foot pedals, etc. Among them, the tactile feedback device can be a force feedback glove.

[0151] The output layer includes visual output, auditory output, and tactile feedback.

[0152] Visual output is achieved through head-mounted displays or stereo glasses, based on optical solutions such as high-resolution OLED or LCD screens; Auditory output is achieved through stereo directional audio combined with ultrasonic array speakers, based on head-related transfer function HRTF technology and multi-channel stereo field; ‌Haptic feedback is achieved through tactile feedback devices such as motors, electrical stimulation tactiles, temperature simulation devices, and handles.

[0153] The storage layer includes local storage or cloud storage for storing 3D scene data, user profiles, and motion capture databases; The storage layer also includes a content management system based on VR or AR technology to implement multimodal simulation scenarios, virtual emergency drill scenario resource libraries, and dynamic content streaming loading.

[0154] The communication layer includes a real-time communication module, which achieves multi-user synchronization through the low-latency network protocol WebRTC, supports cloud rendering based on edge computing nodes and 5G / 6G transmission, and ensures data security through encrypted transmission and user privacy protection.

[0155] The interaction layer configures interaction rules and user status management.

[0156] Interaction rules include game mechanics and training procedures; User state management includes location, permissions, and personalization.

[0157] The process of the drill conducted by the participants according to the emergency drill device is as follows: (1) Scene loading According to the drill requirements, select the corresponding drill type in the system interface of the emergency drill device, such as chemical leakage, fire accident, etc. After selecting the drill type, the scene model and accident virtual scene will be automatically loaded.

[0158] Check the appearance of fire extinguishing equipment 3 and valve equipment 5, and check whether the pressure sensor, motion sensor, and RFID tag are functioning properly. Use the pre-set installation method to bind fire extinguishing equipment 3 and valve equipment 5 to the virtual fire extinguishing equipment and virtual valve equipment in the virtual accident scene to ensure that the operation data can be accurately transmitted to the virtual accident scene.

[0159] (2) Wearing equipment Wear the haptic vest flat on your body and adjust the shoulder straps and waist belt to ensure it fits snugly. Check that the vibration motors on the chest, back, and shoulders of the vest are securely fastened and connected properly.

[0160] Wear your headset and adjust its position to ensure a clear field of view and that it does not interfere with operation. Wear your headphones and adjust their position to ensure that the sound is clear and free of noise. Check the connection between the headset and headphones to ensure that the sound is playing accurately.

[0161] Verify that the Peltier modules deployed around the head and torso are positioned correctly and that the connecting cables are not tangled or damaged. If using a head-mounted device, adjust its position to ensure a clear field of view and without affecting operation.

[0162] (3) Drill begins After donning the device, confirm that all devices are properly connected and sensor data is displaying normally on the system interface of the accident simulation scene. Click the "Start Drill" button to perform the final initialization and load the resources and parameters required for the drill.

[0163] After entering the virtual accident scene, the participants should first adapt to the surrounding environment.

[0164] At this point, the accident scenario is presented through multimodal feedback: The accident virtual scene shows a high-precision modeled accident scene with burning flames and smoke; The directional sound field device emits explosion sounds, cries for help, etc. The temperature response module simulates flame heat radiation, allowing participants to feel the temperature rise; The haptic vest generates vibration feedback based on conditions such as fire and explosions; Based on multimodal information, the participants determined the location, scale and degree of danger of the accident and began to plan their route of action.

[0165] Follow the drill tasks prompted by the system, such as closing leaking valves, extinguishing fires, evacuating personnel, etc. During the operation, you must remain calm, pay attention to operating procedures, and communicate and collaborate with the team in a timely manner.

[0166] The participants pick up the fire extinguishing equipment 3 according to the fire situation. The built-in pressure sensor of the fire extinguishing equipment 3 collects the grip force data in real time and transmits it wirelessly to the virtual accident scene via Bluetooth or ZigBee.

[0167] According to the pressure value, the corresponding spray radius is presented in the virtual accident scene through the pressure mapping algorithm to simulate the fire extinguishing effect.

[0168] At the same time, the tactile vest provides different vibration feedback according to the fire extinguishing situation and explosion impact in the virtual accident scene. For example, the vibration is weakened when the fire is extinguished and the vibration is strengthened when the explosion impacts. The thermoelectric module adjusts the temperature feedback according to the degree of flame extinguishing, and the temperature gradually decreases; The sound generating device simulates the sound of flame extinguishing, etc.

[0169] When the actors operate the valve device 5, the motion sensor on the valve device 5 detects the rotation angle data in real time and transmits it to the virtual accident scene.

[0170] According to the rotation angle mapping algorithm, the leakage rate corresponding to the opening degree of the valve device 5 is simulated in the virtual accident scene, and the changes in the liquid leakage situation are displayed in the virtual accident scene.

[0171] The tactile vest generates vibration feedback based on the impact, explosion, etc. of leaking liquid; The thermoelectric module adjusts its temperature according to the temperature characteristics of the leaking liquid and changes in the surrounding environment; The sound generating device simulates the sound of liquid leakage, pipe rupture, etc.

[0172] During the drill, if an explosion occurs, the PhysX physics engine calculates the trajectory of the explosion fragments.

[0173] The haptic vest simulates the vibrations of explosion shock waves and debris impacts. The vibration motor amplitude is adjusted according to the haptic feedback mapping algorithm, and the motor at the corresponding position is activated based on the posture of the performer and the spatial location of the collision point. Thermoelectric modules simulate high-temperature radiation at the moment of explosion; The sound generating device emits an explosion sound, and the ultrasonic array speaker locates the sound according to the explosion position; The visual generation device displays explosion light and shadow effects and images of flying debris; Based on the above feedback, the participants avoided danger in a timely manner and adjusted their action strategies.

[0174] (4) End of the drill When the drill task is completed or the drill end condition is reached, the operation of the fire extinguishing equipment 3 and the valve equipment 5 is stopped, and the processing result is waited for on the spot.

[0175] The evaluation system will automatically collect and analyze the operation trajectories, physiological indicators and environmental parameters generated during the drill, and generate evaluation indicators, including operation trajectory deviation indicators, emergency response indicators, response time indicators, instruction accuracy indicators, and task completion indicators. Analyze based on the evaluation indicators, obtain the analysis results, carefully read the improvement suggestions, and understand the shortcomings in the drill.

[0176] After reviewing the analysis results, carefully remove the wearable device and place the tactile vest, headphones, and head-mounted device in the designated locations.

[0177] Return the fire extinguishing equipment 3 and valve equipment 5 to their original positions, tidy up the connecting lines, and keep the equipment tidy.

[0178] like Figure 6As shown, some embodiments of the present invention further provide an evaluation system for emergency drill devices based on multimodal means, comprising: The data acquisition module 6 is used to collect the operation trajectory, physiological indicators and environmental parameters generated by the participants during the emergency drill device; Data processing module 7, used to calculate evaluation indicators based on operation trajectory, physiological indicators and environmental parameters, and establish a multi-dimensional evaluation model through AHP-entropy weight method; The evaluation and analysis module 8 is used to analyze the evaluation indicators according to the multi-dimensional evaluation model, obtain analysis results, and dynamically adjust the scene parameters according to the analysis results.

[0179] Through the above-mentioned evaluation system, multidimensional data of operation trajectories, physiological indicators and environmental parameters are obtained, and a multidimensional evaluation model is constructed using the AHP-entropy weight method. Finally, the scenario parameters are dynamically adjusted to comprehensively quantify and analyze the drill effects. By establishing a correlation model between operation behavior, physiological reactions and environmental changes, a scientific assessment of emergency response capabilities is achieved. Dynamic adjustment based on the evaluation results enables the system to adapt to different training needs, continuously optimize the drill effects, and provide an objective basis for improving emergency response capabilities.

[0180] Optionally, the VR tracker ViveTracker is used to record the operating trajectory and action standardization of the participants to obtain motion capture data; wherein the action standardization can be the use angle of the fire extinguishing equipment 3.

[0181] The wearable device Empatica E4 is used to collect the heart rate and skin electrical response of the participants, obtain physiological monitoring data, and evaluate their decision-making ability under stress.

[0182] The response time of key nodes is recorded through event triggers of the Unity virtual engine or the Unreal virtual engine to obtain operation log data; wherein, the response time of key nodes can be an alarm delay time.

[0183] The voice software development kit AzureSpeechSDK is used to record the command accuracy and obtain voice interaction data.

[0184] The FLACS physics engine records the task completion status and obtains task completion data; for example, the success rate of leak plugging is recorded.

[0185] The operation trajectory deviation index is calculated based on the motion capture data. The calculation formula is:

[0186] in, It is the operation trajectory deviation index; is the total number of sampling times; For the The horizontal coordinate of the sub-sampled operation trajectory; For the The ordinate of the sub-sampled operation trajectory; is the standard path reference horizontal coordinate; It is the standard path reference vertical coordinate.

[0187] The emergency response index is calculated based on the physiological monitoring data. The calculation formula is:

[0188] in, It is an emergency response indicator; is the maximum heart rate of the participants; is the peak value of the performer's skin conductance; normalize the heart rate to 180 bpm and the skin conductance to 15 μS, and take the weighted sum to reflect the performer's comprehensive stress level.

[0189] The response time efficiency index is calculated based on the operation log data. The calculation formula is:

[0190] in, It is the response time indicator; is the actual response indicator; is the preset time threshold; when the response time exceeds the time threshold, the score of the response time efficiency indicator is 0 points.

[0191] The command accuracy index is calculated based on the voice interaction data. The calculation formula is:

[0192] in, is the instruction accuracy indicator; The number of instructions that match the plan keywords through NLP semantic analysis; is the total number of instructions.

[0193] The task completion index is calculated based on the task completion data. The calculation formula is:

[0194] in, It is the task completion indicator; To provide fire extinguishing coverage; is the total volume requirement; Time consuming task.

[0195] Preprocessing the calculated operation trajectory deviation index, emergency response index, response time index, instruction accuracy index, and task completion index; Based on Kalman filtering, data fusion algorithm is used for preprocessing. The specific preprocessing process is as follows: Resample the data uniformly to the same time axis for spatiotemporal alignment; Delete data that deviates from the mean by more than 3 times the standard deviation to avoid extreme values ​​interfering with the fusion results; Perform weighted fusion on the data according to preset weights.

[0196] A multi-dimensional evaluation model is constructed based on the AHP-entropy weight method according to the operation trajectory deviation index, emergency response index, response time index, instruction accuracy index, and task completion index, including: Construct a judgment matrix and set initial weights for indicators using expert scoring methods; The entropy weight method is used to automatically adjust the weight according to the discrete degree of the indicator data. The calculation model of the entropy weight method is:

[0197]

[0198] in, For the Information entropy of an indicator; For the Under the indicator, Normalized proportion of samples; is the normalization constant; For the Entropy weight of each indicator; is the total number of indicators; The initial weight and the revised weight are fused through the fusion calculation formula, and the fusion calculation formula is:

[0199] in, is the weight after fusion; is the initial weight; is the modified weight; is the empirical coefficient, and its value is 0.6.

[0200] Calculate the indicators using a comprehensive evaluation formula based on the multi-dimensional evaluation model to obtain indicator scores, and analyze the indicators based on the scores to obtain analysis results; The comprehensive evaluation calculation formula is:

[0201] in, Score the indicators; For the The weight of each indicator; For the Items of standardized indicators; is the number of serious errors; serious errors include failure to close the valve, selection of the wrong escape route, etc. The total number of preset key operation nodes.

[0202] Dynamically adjust scene parameters based on analysis results. For example, if a performer's heart rate exceeds 120 bpm or skin conductance exceeds 20 μS in their physiological monitoring data, the scene parameters are adjusted according to the adjustment algorithm. The calculation model of the adjustment algorithm is:

[0203] in, is the scene parameter; is the initial weight; is the current emergency coefficient, which ranges from 0 to 1.

[0204] It should be noted that the above is a reference method for a multi-modal emergency drill device and an evaluation system thereof, and the present invention is not limited thereto.

[0205] The embodiments of the present invention realize the restoration of the multi-sensory experience of the real scene through the multimodal feedback module, improve the psychological adaptability of the participants, realize the closed loop of real operation and virtual feedback through the sensors in the fire extinguishing equipment and valve equipment, solve the disadvantage of the lack of interactive experience in traditional VR drills, and quantify the drill effect based on the multi-dimensional evaluation model of evaluation indicators, providing a scientific basis for improving the training program, and solving the technical problems that traditional physical drills in the existing technology are difficult to reproduce complex disasters, and virtual drills lack multi-sensory stimulation and cannot simulate key physical effects such as thermal radiation and shock waves; operations and feedback in the virtual system are disconnected, especially when physical equipment is involved, a natural mapping relationship cannot be established; and the existing system mostly relies on the subjective observation and scoring of the coach, lacks the quantitative analysis capability of key dimensions such as operation standardization and stress physiological indicators, resulting in distorted training effect evaluation.

[0206] Finally, it should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other. The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the same. Although the present invention has been described in detail with reference to preferred embodiments, persons skilled in the art should understand that the specific implementation methods of the present invention may still be modified or some technical features may be replaced by equivalents without departing from the spirit of the technical solutions of the present invention, and all of these should fall within the scope of the technical solutions claimed for protection by the present invention.

Claims

1. A multi-modal emergency drill device, characterized in that: include: A model building module is used to scan the target industrial scene to establish a scene model, and dynamically configure a virtual accident scene in the scene model, wherein the virtual accident scene includes the process of fire occurrence and smoke diffusion; A multimodal feedback module, equipped with a multi-region vibration device, a temperature response module and a directional sound field device, is used to generate vibration feedback, high temperature feedback and sound feedback; Fire extinguishing equipment, installed in the physical rehearsal area of ​​the target industrial scene and corresponding to the coordinates of the virtual fire extinguishing equipment in the virtual accident scene, for performing fire extinguishing operations in fire accidents; a processor, communicatively connected to the multimodal feedback module and the fire extinguishing equipment; The processor is further configured with an interactive logic unit for triggering vibration feedback, high temperature feedback, and sound feedback according to the multimodal feedback module when a fire occurs in the virtual accident scene; The processor controls the fire extinguishing equipment to perform fire extinguishing operations to obtain operation data, maps the operation data to the accident virtual scene, and dynamically adjusts the fire evolution process in the accident virtual scene to obtain a processing result.

2. The multi-modal emergency drill device according to claim 1, characterized in that: The multimodal feedback module specifically includes: a vibration generating device, configured with a vibration motor array, for generating graded vibration feedback data according to the location and intensity of the fire accident in the virtual accident scene, and performing vibration feedback at different frequencies and amplitudes; a temperature sensing generating device, configured with a thermoelectric module, for generating hot air feedback matching the virtual fire source temperature according to the virtual fire source temperature of the fire accident in the virtual accident scene; A sound generating device, comprising ultrasonic array speakers distributed throughout the target industrial scene, for locating explosion sounds using a VBAP algorithm based on the positions of the participants and the explosion point when a fire occurs; The visual generation device is equipped with a head-mounted display for displaying the occurrence and handling process of the fire accident in the virtual accident scene in real time.

3. The multi-modal emergency drill device according to claim 1, characterized in that: The fire extinguishing equipment specifically includes: a pressure sensor, installed inside the handle of the fire extinguishing equipment, for measuring the grip force; The RFID tag is installed at the bottom of the fire extinguishing equipment and is used to store the unique identification code and calibration parameters of the fire extinguishing equipment.

4. The multi-modal emergency drill device according to claim 3, characterized in that: The physical rehearsal area of ​​the target industrial scenario is also equipped with valve equipment; The valve device corresponds to the virtual valve device coordinates of the virtual accident scene, which includes: A motion sensor, mounted on the end of the rotating shaft of the valve device, for detecting the rotation angle of the valve device in real time; An operation feedback unit is communicatively connected to the motion sensor and is used to map the opening of the valve device into a leakage rate.

5. The multi-modal emergency drill device according to claim 4, characterized in that: The controller is further configured to: After obtaining the gripping force data according to the pressure sensor, the gripping force data is transmitted in real time to the virtual accident scene via wireless communication to control the fire extinguishing spray range of the virtual fire extinguishing equipment in the virtual accident scene; The controller is communicatively connected to the valve device, and is used to obtain rotation angle data based on the motion sensor, and then transmit the rotation angle data in real time to the accident virtual scene through wireless communication, so as to control the leakage rate corresponding to the opening of the virtual valve device in the accident virtual scene.

6. The multi-modal emergency drill device according to claim 2, characterized in that: The vibration generating device is further configured as: When a fire accident occurs in the virtual accident scene, fragment collision data is obtained through the physics engine, and impact force data is calculated based on the fragment collision data combined with the physics engine. Vibration feedback of different frequencies and amplitudes is performed through the vibration motor array, and the posture of the participants is tracked in real time according to the inertial measurement unit to dynamically adjust the vibration direction.

7. The multi-modal emergency drill device according to claim 2, characterized in that: The temperature sensing device is further configured as follows: When a fire accident occurs in the virtual accident scene, the temperature is dynamically adjusted through the thermoelectric module and the PID control algorithm according to the heat source position and temperature change trend of the fire accident, and the temperature is dynamically correlated with the flame height through the thermoelectric module.

8. The multi-modal emergency drill device according to claim 2, characterized in that: The sound generating device is further configured as follows: When a fire accident occurs in the virtual accident scene, the coordinates of the explosion point in the virtual accident scene are mapped to the physical position of the ultrasonic array speaker through the VBAP algorithm, and the sound pressure level is adjusted in real time according to the distance between the participants and the explosion point when the fire occurs.

9. The multi-modal emergency drill device according to claim 2, characterized in that: The visual generation device is further configured to: When a fire accident occurs in the virtual accident scene, the fragment scattering trajectory obtained according to the fragment collision data is displayed through the head-mounted display, and the explosion light and shadow display when the fire occurs is triggered through the network protocol.

10. An evaluation system for emergency drill devices based on multi-modality, characterized in that: include: The data acquisition module is used to collect the operation trajectory, physiological indicators and environmental parameters generated by the participants during the emergency drill device; A data processing module is used to calculate evaluation indicators based on the operation trajectory, physiological indicators and environmental parameters, and establish a multi-dimensional evaluation model through the AHP-entropy weight method; The evaluation and analysis module is used to analyze the evaluation indicators according to the multi-dimensional evaluation model, obtain analysis results, and dynamically adjust the scene parameters according to the analysis results.

Images