Mixed Reality Emergency Response Training System

Abstract

This invention provides a mixed reality emergency response training system, comprising a head-mounted device, a system server, and physical training objects. The head-mounted device is electrically connected to the system server and includes a display module, an image capture module, and a processing module. When a placement guidance screen is projected, the system captures the placement image within the physical placement frame. After verifying the placement of the physical training object using positioning anchor points, the display module projects various simulated environment screens. At this time, the image capture module captures time frame images within the physical placement frame. The processing module uses a gesture recognition model and a trajectory analysis model to detect the time frame images and obtain the gesture actions and corresponding trajectory information. Then, it uses preset data to compare the gesture actions and performs logical calculations on the corresponding trajectory information to obtain an effective action value. Based on this, a training result is accumulated and output.

Claims

1. A mixed reality emergency response training system, comprising a head-mounted device and a system server, wherein the head-mounted device is electrically connected to the system server and comprises a display module, an image capture module, and a processing module, characterized in that: the emergency response training system includes a physical training object; the processing module comprises a gesture recognition model and a trajectory analysis model; the display module comprises at least one placement guidance screen and a plurality of simulated environment screens; the placement guidance screen comprises a physical placement frame with a positioning anchor point, for placing the physical training object in the physical placement frame according to the positioning anchor point; when the display module projects the placement guidance screen, the image capture module captures at least one placement image within the physical placement frame, and the processing module uses the positioning anchor point to check the placement image. When the placement of the training object is confirmed to be correctly determined by the placement image, the display module is driven to project various simulated environment images. At this time, the image capture module captures multiple time frame images within the placement frame of the object. The processing module uses the gesture recognition model and the trajectory analysis model to detect these time frame images and determine at least one gesture and at least one corresponding trajectory information. Then, a preset data is used to compare the gesture and perform logical operations on the corresponding trajectory information to obtain a valid action value. Based on this, a training result output is accumulated and generated. The training result includes the percentage of correct steps completed, the percentage of accurate movements, suggested improvements, and a result score. The system server includes a business module with at least one shopping screen for displaying and interactively selling the physical training object. When the business module receives and determines that the result score is greater than a passing value, it outputs a discount token to the processing module for display and selection on the display module to access the corresponding shopping screen. The discount token is designed as a jigsaw puzzle or a key, and the shopping screens are multiple. When the jigsaw puzzle is selected, the display module displays the corresponding shopping screen, which has a missing element corresponding to the jigsaw puzzle or key. When the discount token and the missing element are correctly matched, the shopping screen displays at least one item with a corresponding discounted price for purchase at a discount.

2. The emergency response training system as described in claim 1, wherein, The training object is either a fire extinguisher substitute or a CPR (Cardiopulmonary Resuscitation) dummy. The fire extinguisher substitute has the overall shape of a fire extinguisher, a movable nozzle, a switch handle, and a safety pin.

3. The emergency response training system as described in claim 1, wherein, The processing module uses the positioning anchor point to check the placement image. When it confirms that the physical training object is correctly placed in the physical placement frame, the processing module outputs a placement completion signal and drives the display module to continue to project the corresponding simulated environment screens. Conversely, when the processing module detects that the physical training object is incorrectly placed, it outputs a placement error signal to warn and guide the placement of the physical training object.

4. The emergency response training system as described in claim 1, wherein, These simulated environment scenes are arranged in a sequential and interconnected manner to form a training theme, which is fire extinguisher fire extinguishing training or CPR (Cardiopulmonary Resuscitation) training. The placement guidance screen is connected between these simulated environment scenes in accordance with the sequence of events of the training theme.

5. The emergency response training system as described in claim 1, wherein, When the head-mounted device is first activated, the processing module requests the setting of a user ID and a unit name of one of the trainees through the display module for integration into the training results; the system server is equipped with an analysis module that receives multiple training results and organizes and collects them according to the unit name.

6. The emergency response training system as described in claim 5, wherein, This analysis module parses the training results corresponding to a given unit name and generates a unit training results report to monitor the overall training progress and status of the unit.

7. The emergency response training system as described in claim 5 includes at least one depth camera electrically connected to at least one of the head-mounted device and the system server, for mounting around the physical training object to assist the processing module in further accurately detecting and obtaining the hand gesture and the corresponding trajectory information.

Images