MR-based medical rescue training evaluation system and method
By using an aviation medical rescue training platform based on MR technology and a multimodal data fusion evaluation method, the problem of quantitative evaluation of the effectiveness of aviation medical rescue training was solved, and the training effect was optimized and improved in real time.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GENERAL HOSPITAL OF PLA
- Filing Date
- 2022-12-14
- Publication Date
- 2026-06-19
AI Technical Summary
Currently, there is a lack of quantitative evaluation methods for aviation medical rescue training, making it impossible to evaluate the training effectiveness in real time and optimize it, which affects the rescue results.
An aviation medical rescue training platform based on MR technology is adopted. Training data is acquired through six-degree-of-freedom flight state simulation, human simulation model and multimodal sensors. A convolutional neural network is constructed to perform multimodal feature fusion evaluation and generate a training effect report.
It enables quantitative and systematic evaluation of aviation rescue personnel training, improves the effectiveness of rescue training, and allows for timely improvement of training programs.
Smart Images
Figure CN116151655B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of aviation medical rescue, and in particular to an MR-based aviation medical rescue training effectiveness evaluation system and method. Background Technology
[0002] In the event of critical illnesses, emergencies, or unexpected disasters, timely organization of rescue forces to provide immediate and effective on-site rescue and necessary medical treatment to individuals or groups can save lives and alleviate injuries and suffering. Air medical rescue refers to the use of helicopters or fixed-wing aircraft to transport or deliver the wounded, medical personnel, and emergency equipment. Due to its rapid response, high mobility, wide operational range, and fewer geographical limitations, air medical rescue is on average 3-5 times faster than ground-based rescue, effectively reducing accident mortality by approximately 40%. It is considered the "most efficient rescue method," capable of quickly opening new lifelines in hard-to-reach disaster sites thanks to its mobility and speed.
[0003] The operational environment of aviation medical emergency rescue differs from that of ordinary ground rescue. Helicopter rescue, in particular, inevitably involves vibration, noise, changes in flight attitude, high levels of coordination, and the intersection of flight and medical risks. Therefore, emergencies are more frequent and complex than in ordinary ground rescue, placing special demands on flight crews, medical personnel, and support staff in terms of psychological resilience, environmental adaptability, equipment usage, and professional skills. Thus, conducting professional medical personnel training, developing new ground training equipment, and improving the training system are of great significance for ensuring the quality of medical care and patient safety throughout the entire aviation medical emergency rescue process, improving rescue efficiency, and reducing disability rates.
[0004] In existing technologies, there is an evaluation method for VR flight simulator training (patent number: 201910587040.3). This method decomposes all operational actions into basic action elements and each training subject into several evaluation nodes. It uses machine learning to obtain the node evaluation network of the evaluation nodes, and then uses deep learning to obtain the comprehensive evaluation result of the entire training subject. However, this method only records flight attitude data by establishing a subject pattern library and does not evaluate the training effect of aviation medical rescue trainees. There is a method for evaluating the effectiveness of specialist nurse training (patent number: 201811324898.2). This method includes an information management module, an assessment processing module, a data statistics module, a storage module, and a display / input module. The display / input module is a touch screen. The assessment processing module has a data transmission module. The assessment processing module generates assessment forms, which are then used for assessment via the display / input module or a mobile PC and imported into the data statistics and storage modules. The storage module has a USB interface. This system is used to conduct multi-dimensional dynamic evaluation of the effectiveness of specialist nurse training from the reaction layer, learning layer, transfer layer, and result layer in two time periods. The evaluation results are objective and realistic, making up for the design deficiencies of previous evaluation methods. However, this method only evaluates the effectiveness of basic nurse skills training and does not cover the evaluation of the effectiveness of medical personnel training in an aviation medical rescue environment. There is also a method for evaluating exercise training based on a virtual environment (patent number: 201810062199.9). This method includes the following steps:
[0005] Step 1: Construct a virtual teaching platform; Step 2: Calculate motion trajectory; Step 3: Evaluate exercise training. The virtual teaching platform is designed using Unity3D, based on skeletal points acquired by Kinect V2. It maintains a 30-frame-per-second frame rate to capture standard vectors and spinal point coordinates for standard movements, and calculates the height of the person performing the standard movements. The offset coordinates of the spinal points are determined by comparing the heights of the trainees with those of the person performing the standard movements. Then, the spatial motion curve of the standard movements is calculated based on the skeletal length of the trainees, thereby recognizing and scoring continuous movements. However, this method belongs to the field of motion-sensing interaction and does not involve aviation medical rescue training. There is a method for a training and evaluation platform suitable for aviation emergency rescue (patent number: 202210902368.1). This method integrates complex rescue scenario information and rescue equipment functional information through the design of a task decision design unit, a training decision interaction unit, and a data processing unit, enabling more intuitive management of training tasks and flight equipment released by the platform. By combining real complex rescue scenario data with real aircraft operating costs, trainees can design and set up rescue systems, deploy equipment, and schedule equipment resources within the entire rescue system. However, this method only familiarizes trainees with basic rescue procedures and does not involve aviation medical rescue training or effectiveness evaluation. There is a nursing teaching and training method based on virtual reality technology (patent number: 202210572420.1). This method includes a virtual reality module worn on the trainee's head and a tactile practice module providing realistic tactile sensations. The tactile practice module is equipped with an external extension module simulating implantable tubing. The tactile practice module includes a skeletal unit and, from the inside out, human-like muscle simulation units, subcutaneous simulation units, and epidermal simulation units. As the pressure of the surface pressure sensing unit, subcutaneous pressure sensing unit, and muscle pressure sensing unit increases, the trainee experiences pain, thus improving the trainee's accuracy in remembering incorrect movements and areas of error. This allows the trainee to form an associative memory between the operation location and pain, specifically improving the understanding of nurses without clinical experience and making their nursing actions more standardized. However, this method is only a static ground-based medical nursing teaching and training method and does not involve research on aviation medical rescue training.
[0006] In summary, at present, there is a lack of research on aviation medical rescue training, no evaluation system for aviation medical rescue training has been established, and there are no methods and means to quantitatively assess the effectiveness of aviation medical rescue training. It is impossible to evaluate and verify the effectiveness of aviation medical rescue training in real time, and there is no timely optimization of training programs based on training results, which affects the actual rescue effectiveness. Summary of the Invention
[0007] Therefore, the technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide an evaluation system and method for the effectiveness of aviation medical rescue training based on MR, which can realize quantitative and systematic evaluation of aviation rescue personnel training and improve the effectiveness of rescue training.
[0008] To address the aforementioned technical problems, this invention provides an MR-based aviation medical rescue training effectiveness evaluation system, comprising an aviation medical rescue training platform, an evaluation acquisition module, an evaluation module, and an evaluation report module.
[0009] The aviation medical rescue training platform uses MR technology to simulate flight visuals and rescue environments, and trainees perform medical rescues within the platform.
[0010] The evaluation acquisition module acquires the rescue data of the trainees in real time and transmits it to the evaluation module. The evaluation module extracts multimodal features from the rescue data for evaluation.
[0011] The evaluation report module associates the evaluation results of the evaluation module with the trainee information to generate an evaluation report.
[0012] In one embodiment of the present invention, the aviation medical rescue training platform includes a six-degree-of-freedom flight state simulation platform, a helicopter simulation cabin, a six-degree-of-freedom platform ground cage, and a support platform.
[0013] The six-degree-of-freedom flight state simulation platform is used to simulate six flight attitudes: pitch, roll, yaw, vertical takeoff and landing, longitudinal displacement, and lateral displacement.
[0014] The six-degree-of-freedom flight state simulation platform, helicopter simulation cabin, six-degree-of-freedom platform cage and support, flight visual simulation, and rescue environment simulation together realize the simulation of the aviation environment.
[0015] In one embodiment of the present invention, the evaluation acquisition module includes a miniature low-power sensor, a human model, a camera device, and a character recognition system, wherein the human model is equipped with multiple different types of sensors;
[0016] Trainees perform medical rescue on the mannequin model. The miniature low-power sensor acquires the trainee's physiological parameters, the camera device acquires facial images and rescue action images of the trainee, multiple different types of sensors acquire rescue action data of the trainee, and the text recognition system acquires data on the quality of the medical documents written by the trainee.
[0017] In one embodiment of the present invention, the simulated human model has a built-in pressure sensor, flow sensor, and Hall sensor.
[0018] The pressure sensor acquires data on the compression position and pressure applied by trainees during CPR training; the flow sensor acquires data on the artificial respiration operation performed by trainees; and the Hall sensor acquires data on the endotracheal intubation operation performed by trainees.
[0019] In one embodiment of the present invention, the evaluation module includes an attention tension level evaluation module, a treatment operation standardization evaluation module, and a medical equipment operation ability evaluation module.
[0020] The attention and tension level evaluation module evaluates the trainees' attention and tension levels during medical rescue based on their physiological parameters and facial images.
[0021] The evaluation module for the standardization of rescue operations evaluates the standardization of rescue operations performed by trainees based on their rescue action data and the quality of their written medical documents.
[0022] The medical equipment operation capability evaluation module obtains data on the trainees' equipment operation standardization and equipment data interpretation ability based on the rescue action images, and evaluates the trainees' medical equipment operation capability when carrying out medical rescue.
[0023] In one embodiment of the present invention, multimodal features are extracted from the rescue data for evaluation, specifically as follows:
[0024] Construct a convolutional neural network that includes a feature extractor, an attention mechanism, and an output layer.
[0025] The rescue data is divided into three modalities: image, physiological signal and numerical data. The feature extractor is used to extract features from the image, physiological signal and numerical data and reduce the dimensionality to obtain the image feature matrix P, the physiological signal feature matrix S and the numerical feature matrix N.
[0026] An attention mechanism is used to cascade and fuse the image feature matrix P, the physiological signal feature matrix S, and the numerical feature matrix N to obtain the fused feature matrix F.
[0027] The fused feature matrix F is input to the output layer to obtain the evaluation result.
[0028] In one embodiment of the present invention, the step of using an attention mechanism to cascade and fuse the image feature matrix P, the physiological signal feature matrix S, and the numerical feature matrix N to obtain a fused feature matrix F specifically involves:
[0029] The image feature matrix P, the physiological signal feature matrix S, and the numerical feature matrix N are respectively used as the Key in the attention mechanism, and the Query is used as the output vector after recognition based on the fused feature matrix F.
[0030] The similarity between the Query and the Key is calculated, and the weighted coefficients are summed to obtain the image attention coefficient w1, the physiological signal attention coefficient w2, and the numerical attention coefficient w3.
[0031] The final fusion feature matrix F is obtained as: F = [w1×P, w2×S, w3×N].
[0032] In one embodiment of the present invention, the convolutional neural network includes an input layer, three convolutional layers, three max-pooling layers, one fully connected layer, and one softmax classification layer.
[0033] The rescue data is passed through the input layer and then through three convolutional layers and three pooling layers to obtain three modalities of data. The three modalities of data are then processed by the fully connected layer to extract the feature matrix. The feature matrix is then processed by the Softmax classification layer to obtain the evaluation result.
[0034] In one embodiment of the present invention, the evaluation report includes basic information of trainees, a training evaluation form, and a training optimization method.
[0035] This invention also provides a method for evaluating the effectiveness of aviation medical rescue training based on MR, comprising: constructing an aviation medical rescue training platform that includes flight visual simulation and rescue environment simulation using MR technology, and having trainees perform medical rescue within the aviation medical rescue training platform;
[0036] Real-time acquisition of rescue data from trainees, and extraction of multimodal features from the rescue data for evaluation;
[0037] The evaluation results are linked to the trainees' information to generate an evaluation report.
[0038] The technical solution of the present invention has the following advantages compared with the prior art:
[0039] This invention constructs an aviation medical rescue training platform using MR technology and conducts aviation medical rescue training within the platform. By using multimodal data perception fusion to evaluate the rescue data of trainees acquired in real time, it achieves a quantitative and systematic assessment of aviation rescue personnel training, which helps to improve the effectiveness of personnel rescue training in a timely manner and enhance the overall effectiveness of rescue training. Attached Figure Description
[0040] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein:
[0041] Figure 1 This is a schematic diagram of the structure of the present invention.
[0042] Figure 2 This is a schematic diagram of the training effectiveness evaluation method in this invention.
[0043] Figure 3 This is a schematic diagram of a six-degree-of-freedom training platform in an embodiment of the present invention.
[0044] Figure 4 This is a schematic diagram of a simulated human model in an embodiment of the present invention. Detailed Implementation
[0045] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.
[0046] Mixed Reality (MR) technology is a further development of virtual reality technology. MR enhances the realism of the user experience by simulating virtual scene information and establishing an interactive feedback loop between the real and virtual worlds. Therefore, this invention integrates MR technology into aviation medical rescue training, such as... Figure 1 As shown, a MR-based aviation medical rescue training effectiveness evaluation system is disclosed, including an aviation medical rescue training platform, an evaluation acquisition module, an evaluation module, and an evaluation report module. The aviation medical rescue training platform uses MR technology to simulate flight visuals and rescue environments, allowing trainees to perform medical rescues within the platform. The evaluation acquisition module acquires the trainees' rescue data in real time and transmits it to the evaluation module, which extracts multimodal features from the rescue data for evaluation. The evaluation report module correlates the evaluation results from the evaluation module with the trainees' information to generate an evaluation report. Based on the evaluation report, a conclusion can be reached in real time regarding whether the trainees' current operations are standardized, facilitating targeted optimization of the trainees' rescue operations.
[0047] An aviation rescue training platform is the foundation and basic condition for conducting aviation rescue training. This aviation medical rescue training platform includes a six-degree-of-freedom flight state simulation platform, a helicopter simulation cabin, a six-degree-of-freedom platform cage, and a support platform. For example... Figure 3As shown, the six-degree-of-freedom flight state simulation platform is used to simulate six flight attitudes: pitch, roll, yaw, vertical takeoff and landing, longitudinal displacement, and lateral displacement. The helicopter simulation cabin, based on the overall design requirements of the AC313A helicopter's aviation medical emergency rescue simulation training cabin, uses a 1:1 simulation cabin section of the AC313A real aircraft, excluding the tail section and propeller section. The fuselage structure includes the forward fuselage, mid-fuselage, transition section, engine room, and cabin door. The six-degree-of-freedom flight state simulation platform, helicopter simulation cabin, six-degree-of-freedom platform cage and support, flight visual simulation, and rescue environment simulation together realize the simulation of the aviation environment.
[0048] like Figure 2 As shown, the evaluation acquisition module includes a miniature low-power sensor, such as... Figure 4 The diagram shows a mannequin model, a camera device, and a text recognition system. The mannequin model is equipped with multiple different types of sensors. Trainees perform medical rescue on the mannequin model. The miniature low-power sensors acquire physiological parameters such as the trainee's electrocardiogram and respiration. The camera device acquires facial images and rescue action images of the trainee. The multiple different types of sensors acquire rescue action data of the trainee. The text recognition system acquires data on the quality of the medical documents written by the trainee. In this embodiment, the text recognition system is an OCR recognition system, which can achieve quality control of aviation medical documents. The medical documents are patient case records written by trainees during aviation rescue. These case records mainly consist of checkmarks and include descriptions of key information about the patient. The quality of the medical documents refers to the quality of the case records written by trainees during aviation rescue, which is an evaluation dimension of this invention when evaluating training effectiveness. The quality data of the medical documents is obtained through two methods: OCR recognition and scoring. OCR targets the text portion of the medical documents, using algorithms such as natural language processing models to identify the trainee's writing quality. Scoring targets the checkmark selection portion of the medical documents, judging the trainee's writing quality by scoring the checkmark selections.
[0049] In this embodiment, the mannequin model is equipped with a pressure sensor, a flow sensor, and a Hall sensor. The pressure sensor acquires data on the compression position and pressure applied by the trainee during CPR training. The flow sensor acquires data on the trainee's artificial respiration operation. The Hall sensor acquires data on the trainee's endotracheal intubation operation.
[0050] This study constructs a multimodal dataset integrating facial images, electrocardiograms, respiration, chest compression positions, compression pressure, artificial respiration, endotracheal intubation, medical documentation quality, equipment operation standardization, and equipment data interpretation capabilities of aviation rescue trainees. A comprehensive and systematic evaluation is conducted across three dimensions: attention span, treatment procedure standardization, and medical equipment operation skills. A decision-level-based evaluation method for rescue team members is developed through multimodal data processing and feature extraction.
[0051] The evaluation module includes an attention and tension level evaluation module, a rescue operation standardization evaluation module, and a medical equipment operation ability evaluation module. The attention and tension level evaluation module assesses the trainee's attention and tension levels during medical rescue based on the trainee's physiological parameters and facial images. The rescue operation standardization evaluation module assesses the trainee's rescue operation standardization based on the trainee's rescue action data (i.e., data acquired by pressure sensors, flow sensors, and Hall effect sensors) and the quality of the medical documentation written by the trainee. The medical equipment operation ability evaluation module assesses the trainee's medical equipment operation ability based on data obtained from the rescue action images, including equipment operation standardization and equipment data interpretation skills. Equipment operation standardization is determined by judging whether the trainee's rescue actions in the rescue action images captured by the camera are standardized. Data interpretation ability is also determined by judging whether the rescue actions of the trainees are standardized in the rescue action images captured by the camera. For example, if the blood pressure and heart rate of the injured or sick person are found to be fluctuating greatly through the monitor, and the trainees have corresponding emergency treatment measures to stabilize the vital signs of the injured or sick person, then the trainees' data interpretation ability meets the requirements.
[0052] In this embodiment, multimodal features are extracted from the rescue data for evaluation, specifically as follows:
[0053] S1: Construct a convolutional neural network including a feature extractor, an attention mechanism, and an output layer. The convolutional neural network includes an input layer, three convolutional layers, three max pooling layers, one fully connected layer, and one softmax classification layer.
[0054] S2: The rescue data is divided into three modalities: image (i.e., data collected by camera devices), physiological signal (i.e., data collected by miniature low-power sensors), and numerical (i.e., data collected by multiple different types of sensors and text recognition systems). The feature extractor extracts features from the image, physiological signal, and numerical data, retaining 7 main features for each, and reducing the dimensionality to obtain the image feature matrix P, the physiological signal feature matrix S, and the numerical feature matrix N. After passing through the input layer, the rescue data undergoes 3 convolutional layers and 3 pooling layers to obtain the data for the three modalities. The data for the three modalities are then processed by the fully connected layer to extract the feature matrix.
[0055] S3: Using an attention mechanism, the image feature matrix P, the physiological signal feature matrix S, and the numerical feature matrix N are concatenated and fused to obtain the fused feature matrix F.
[0056] S3-1: The image feature matrix P, the physiological signal feature matrix S, and the numerical feature matrix N are respectively used as the Key in the attention mechanism for input, and the Query is used as the output vector after recognition based on the fused feature matrix F.
[0057] S3-2: Calculate the similarity between the Query and the Key, and calculate the image attention coefficient w1, physiological signal attention coefficient w2 and numerical attention coefficient w3 by weighted summation of the weight coefficients;
[0058] S3-3: The final fusion feature matrix F is obtained as: F = [w1×P, w2×S, w3×N].
[0059] S4: Input the fused feature matrix F into the output layer to obtain the evaluation result. The feature matrix is then processed through a Softmax classification layer to obtain the evaluation result. In this embodiment, the ReLU function is used as the activation function. To improve the network's generalization performance, local response normalization is added to the convolutional outputs of the first and second convolutional layers after applying ReLU.
[0060] To facilitate aviation trainees' timely understanding of their skills training progress and improvement methods, the constructed evaluation report includes basic information about the trainees, a training evaluation form (mainly including evaluations of helicopter hemorrhagic shock, cardiac arrest, and acute respiratory distress syndrome treatment procedures), and training optimization methods. Based on the evaluation results of the evaluation modules, optimization plans are proposed to improve the trainees' skills.
[0061] This invention also discloses a method for evaluating the effectiveness of aviation medical rescue training based on MR, comprising the following steps:
[0062] An aviation medical rescue training platform, which includes flight visual simulation and rescue environment simulation, is constructed using MR technology. Trainees enter the flight cabin after wearing wearable devices such as ECG devices.
[0063] Trainees perform rescue procedures in accordance with training requirements on the aviation medical rescue training platform; wearable devices such as ECG, cameras (motion capture), and sensors on silicone mannequins collect data in real time.
[0064] The multimodal features extracted from the rescue data are evaluated using a convolutional neural network. The evaluation results are then correlated with the trainees' information to generate an evaluation report, which is then printed out.
[0065] The technical solution of the present invention has the following advantages compared with the prior art:
[0066] 1. This invention constructs an aviation medical rescue training platform using MR technology and conducts aviation medical rescue training on the platform. It evaluates the rescue data of trainees acquired in real time through multimodal data perception fusion, and constructs an aviation medical rescue training effectiveness evaluation method based on decision-level fusion. This enables quantitative and systematic evaluation of aviation rescue personnel training, which helps to improve the effectiveness of personnel rescue training in a timely manner and enhance the overall rescue training effect.
[0067] 2. This invention uses pressure sensors combined with data analysis to evaluate and optimize training. It quantitatively and systematically assesses the training effectiveness from three dimensions: the trainees' level of attention and tension, the standardization of treatment procedures, and the ability to operate medical equipment. The evaluation is comprehensive and scientific.
[0068] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0069] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0070] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0071] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0072] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.
Claims
1. A training effectiveness evaluation system for aviation medical rescue based on MR, characterized in that: This includes an aviation medical rescue training platform, an evaluation acquisition module, an evaluation module, and an evaluation report module. The aviation medical rescue training platform uses MR technology to simulate flight visuals and rescue environments, and trainees perform medical rescues within the platform. The evaluation acquisition module acquires the rescue data of the trainees in real time and transmits it to the evaluation module. The evaluation module extracts multimodal features from the rescue data for evaluation. The evaluation report module associates the evaluation results of the evaluation module with the trainee information to generate an evaluation report; The evaluation acquisition module includes a miniature low-power sensor, a mannequin model, a camera device, and a text recognition system. The mannequin model is equipped with multiple different types of sensors. Trainees perform medical rescue on the mannequin model. The miniature low-power sensor acquires the trainee's physiological parameters, the camera device acquires facial images and rescue action images of the trainee, the multiple different types of sensors acquire rescue action data of the trainee, and the text recognition system acquires data on the quality of the medical documents written by the trainee. The mannequin model has built-in pressure sensors, flow sensors, and Hall sensors. The pressure sensors acquire data on the compression position and pressure during cardiopulmonary resuscitation training, the flow sensors acquire data during artificial respiration, and the Hall sensors acquire data during endotracheal intubation. The evaluation module includes an attention and tension level evaluation module, a rescue operation standardization evaluation module, and a medical equipment operation ability evaluation module. The attention and tension level evaluation module evaluates the trainees' attention and tension levels during medical rescue based on their physiological parameters and facial images. The rescue operation standardization evaluation module evaluates the trainees' rescue operation standardization based on their rescue action data and the quality of their written medical documents. The medical equipment operation ability evaluation module evaluates the trainees' medical equipment operation ability during medical rescue based on the rescue action images, which provides data on their equipment operation standardization and equipment data interpretation skills. The evaluation process involves extracting multimodal features from the rescue data. Specifically, this includes constructing a convolutional neural network that includes a feature extractor, an attention mechanism, and an output layer. The rescue data is divided into three modalities: image, physiological signal, and numerical data. The feature extractor is used to extract features from the image, physiological signal, and numerical data and then dimensionality-reduced to obtain the image feature matrix P, the physiological signal feature matrix S, and the numerical feature matrix N. An attention mechanism is used to cascade and fuse the image feature matrix P, the physiological signal feature matrix S, and the numerical feature matrix N to obtain a fused feature matrix. The fused feature matrix The input and output layers obtain the evaluation results; The attention mechanism is used to cascade and fuse the image feature matrix P, the physiological signal feature matrix S, and the numerical feature matrix N to obtain a fused feature matrix. Specifically, the image feature matrix P, the physiological signal feature matrix S, and the numerical feature matrix N are respectively used as components in the attention mechanism. Enter the input, As based on the fusion feature matrix The output vector after recognition; calculate the... and The similarity is used to calculate the image attention coefficient w1, physiological signal attention coefficient w2, and numerical attention coefficient w3 by weighted summation of the weight coefficients; The final fusion feature matrix is obtained. for: .
2. The MR-based aviation medical rescue training effectiveness evaluation system according to claim 1, characterized in that: The aviation medical rescue training platform includes a six-degree-of-freedom flight state simulation platform, a helicopter simulation cabin, a six-degree-of-freedom platform cage, and a support platform. The six-degree-of-freedom flight state simulation platform is used to simulate six flight attitudes: pitch, roll, yaw, vertical takeoff and landing, longitudinal displacement, and lateral displacement. The six-degree-of-freedom flight state simulation platform, helicopter simulation cabin, six-degree-of-freedom platform cage and support, flight visual simulation, and rescue environment simulation together realize the simulation of the aviation environment.
3. The MR-based aviation medical rescue training effectiveness evaluation system according to claim 1, characterized in that: The convolutional neural network includes an input layer, three convolutional layers, three max-pooling layers, one fully connected layer, and one softmax classification layer. The rescue data is passed through the input layer and then through three convolutional layers and three pooling layers to obtain three modalities of data. The three modalities of data are then processed by the fully connected layer to extract the feature matrix. The feature matrix is then processed by the Softmax classification layer to obtain the evaluation result.
4. The MR-based aviation medical rescue training effectiveness evaluation system according to any one of claims 1-3, characterized in that: The evaluation report includes basic information about the trainees, a training evaluation form, and training optimization methods.
5. A method for evaluating the effectiveness of aviation medical rescue training based on MR, characterized in that, include: An aviation medical rescue training platform, which includes flight visual simulation and rescue environment simulation, is constructed using MR technology. Trainees then conduct medical rescue operations within this platform. Real-time acquisition of rescue data from trainees, and extraction of multimodal features from the rescue data for evaluation; The evaluation results are linked to the trainees' information to generate an evaluation report; The evaluation acquisition module includes a miniature low-power sensor, a mannequin model, a camera device, and a text recognition system. The mannequin model is equipped with multiple different types of sensors. Trainees perform medical rescue on the mannequin model. The miniature low-power sensor acquires the trainee's physiological parameters, the camera device acquires facial images and rescue action images of the trainee, the multiple different types of sensors acquire rescue action data of the trainee, and the text recognition system acquires data on the quality of the medical documents written by the trainee. The mannequin model has built-in pressure sensors, flow sensors, and Hall sensors. The pressure sensors acquire data on the compression position and pressure during cardiopulmonary resuscitation training, the flow sensors acquire data during artificial respiration, and the Hall sensors acquire data during endotracheal intubation. The evaluation module includes an attention and tension level evaluation module, a rescue operation standardization evaluation module, and a medical equipment operation ability evaluation module. The attention and tension level evaluation module evaluates the trainees' attention and tension levels during medical rescue based on their physiological parameters and facial images. The rescue operation standardization evaluation module evaluates the trainees' rescue operation standardization based on their rescue action data and the quality of their written medical documents. The medical equipment operation ability evaluation module evaluates the trainees' medical equipment operation ability during medical rescue based on the rescue action images, which provides data on their equipment operation standardization and equipment data interpretation skills. The evaluation process involves extracting multimodal features from the rescue data. Specifically, this includes constructing a convolutional neural network that includes a feature extractor, an attention mechanism, and an output layer. The rescue data is divided into three modalities: image, physiological signal, and numerical data. The feature extractor is used to extract features from the image, physiological signal, and numerical data and then dimensionality-reduced to obtain the image feature matrix P, the physiological signal feature matrix S, and the numerical feature matrix N. An attention mechanism is used to cascade and fuse the image feature matrix P, the physiological signal feature matrix S, and the numerical feature matrix N to obtain a fused feature matrix. The fused feature matrix The input and output layers obtain the evaluation results; The attention mechanism is used to cascade and fuse the image feature matrix P, the physiological signal feature matrix S, and the numerical feature matrix N to obtain a fused feature matrix. Specifically, the image feature matrix P, the physiological signal feature matrix S, and the numerical feature matrix N are used as the Keys in the attention mechanism. Enter the input, As based on the fusion feature matrix The output vector after recognition; calculate the... and The similarity is used to calculate the image attention coefficient w1, physiological signal attention coefficient w2, and numerical attention coefficient w3 by weighted summation of the weight coefficients; The final fusion feature matrix is obtained. for: .