Emergency rescue training system in bronchoscope operation
Through the bronchoscopic intraoperative emergency rescue training system, brain-computer interface and virtual simulation equipment are used to evaluate the operator's psychological state and operational proficiency, which solves the problem of lack of real scene experience and multidisciplinary collaboration in existing technologies, and achieves the effect of immersive training and multidisciplinary collaboration.
Patent Information
- Application Number
- CN202510879757.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-27
- Publication Date
- 2025-09-23
AI Technical Summary
In the existing technology, the training of emergency situations during pediatric electronic bronchoscopy mainly relies on theoretical teaching and animal experiments, lacks real-life scene experience, makes it difficult to achieve collaborative training of multidisciplinary teams, and cannot meet the clinical requirements for multidisciplinary collaboration.
A bronchoscopic intraoperative emergency rescue training system is provided, which includes a brain-computer interface, virtual simulation equipment, patient models and cameras. By collecting EEG signals, it evaluates the operator's mental state and operational proficiency, and evaluates operational accuracy, reaction speed and teamwork ability, providing an immersive training experience.
It has improved the ability of medical staff to deal with emergencies during surgery, promoted the collaboration of multidisciplinary teams, and improved the effectiveness and efficiency of training.
Smart Images
Figure CN120690073A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of virtual simulation, and in particular to a bronchoscopic emergency rescue training system. Background Art
[0002] Pediatric electronic bronchoscopy is increasingly used in the diagnosis and treatment of pediatric respiratory diseases. However, the procedure is challenging and can lead to a variety of emergencies, including anesthetic allergy, bleeding, laryngospasm, hypoxemia, asphyxia, arrhythmia, cardiac arrest, and pneumothorax. Improper handling can seriously threaten the child's life and health.
[0003] Currently, training for emergency response during pediatric electronic bronchoscopy primarily relies on a combination of traditional theoretical instruction and animal experiments. This lack of real-world experience makes it difficult for medical staff to effectively integrate theoretical knowledge with practical application, resulting in limited training effectiveness. Furthermore, traditional training methods struggle to achieve collaborative training within multidisciplinary teams and fail to meet the clinical requirements for multidisciplinary collaboration.
[0004] Therefore, how to provide immersive training scenarios to improve team collaboration capabilities is a technical problem that needs to be solved urgently. Summary of the Invention
[0005] In view of the above problems, the present invention provides a bronchoscopic intraoperative emergency rescue training system that overcomes the above problems or at least partially solves the above problems.
[0006] The present invention provides a bronchoscopic emergency rescue training system, comprising:
[0007] A brain-computer interface, worn on the operator's head, is used to collect the operator's EEG signals;
[0008] A virtual simulation device for presenting to the operator a virtual scenario of one or more emergency situations occurring in a virtual patient during bronchoscopy;
[0009] Patient model, used for operator operation;
[0010] a camera, configured to capture images of an emergency rescue operation performed by an operator, wherein the emergency rescue operation is an operation performed by the operator in response to a virtual scene of the one or more sudden emergency situations;
[0011] The processing module is connected to the brain-computer interface, virtual simulation equipment and camera, and is used to evaluate the operator's psychological state and operational proficiency based on EEG signals; based on the operator's psychological state and operational proficiency, it updates the virtual scene and evaluates the operator's operational accuracy, reaction speed and teamwork ability.
[0012] Preferably, the operator is any one or more of the following:
[0013] Doctors, nurses and anesthesiologists.
[0014] Preferably, the emergency situation is any one or more of the following:
[0015] Allergy to anesthetic drugs, bleeding, laryngospasm, hypoxemia, asphyxia, arrhythmia, cardiac arrest, and pneumothorax.
[0016] Preferably, the virtual simulation device is any one of the following:
[0017] AR devices and XR devices.
[0018] Preferably, the virtual simulation device is used to present a virtual scene of a sudden emergency situation of a virtual surgical instrument, a virtual vital sign detection device, and a virtual patient to the operator.
[0019] Preferably, the virtual simulation device is used to:
[0020] Presenting a virtual scene of inserting a bronchoscope into the trachea and a virtual scene of an emergency situation of a virtual patient to a doctor-type operator;
[0021] Presenting a virtual scene of virtual surgical instruments to a nurse-type operator, wherein the virtual surgical instruments include: a bronchoscope, foreign body forceps, a balloon, a basket, a laser, and a cryoinstrument;
[0022] Presenting a virtual scene of a virtual patient's anesthesia state to an anesthesiologist-type operator;
[0023] A doctor-type operator, a nurse-type operator, and an anesthesiologist-type operator are presented with virtual emergency equipment including a suction machine, central oxygen supply, and an emergency cart.
[0024] Preferably, the processing module is specifically configured to:
[0025] Extract EEG feature signals based on EEG signals;
[0026] Based on the EEG characteristic signal and the psychological state recognition model, identifying the operator's psychological state, including: emotional state, attention state, and stress state;
[0027] The operator's proficiency is determined based on EEG signals and the operator's response time to emergency rescue operations.
[0028] Preferably, the processing module is specifically configured to:
[0029] When the operator's mental state and operating proficiency are not good, the current virtual scene is updated to a relaxing virtual scene.
[0030] Preferably, the processing module is specifically configured to:
[0031] Based on the image of the emergency rescue operation and the operator's voice information, determining whether the emergency rescue operation performed by the operator is correct to determine the accuracy of the operation;
[0032] determining, based on the EEG signal, a response time of the operator to an emergency rescue operation;
[0033] determining a reaction speed based on the response time;
[0034] Determine team collaboration capabilities based on the connection time between operations of different operators.
[0035] Preferably, the processing module is further configured to:
[0036] Provide a virtual competition environment, including setting competition rules and scoring criteria, and present them through virtual simulation equipment.
[0037] One or more technical solutions in the embodiments of the present invention have at least the following technical effects or advantages:
[0038] The present invention provides a bronchoscopic intraoperative emergency rescue training system, comprising: a brain-computer interface, worn on the operator's head, for collecting the operator's electroencephalogram (EEG) signals; a virtual simulation device, for presenting to the operator a virtual scene of one or more emergency situations occurring in a virtual patient during bronchoscopy; a patient model, for providing the operator with an operation; a camera, for collecting images of emergency rescue operations performed by the operator, where the emergency rescue operations are operations performed by the operator for one or more virtual scenes of emergency situations; a processing module, connected to the brain-computer interface, the virtual simulation device and the camera, for evaluating the operator's psychological state and operational proficiency based on EEG signals; updating the virtual scene based on the operator's psychological state and operational proficiency; and evaluating the operator's operational accuracy, reaction speed and teamwork ability based on the images and EEG signals of the emergency rescue operations. By providing medical workers with an immersive training experience, the ability to respond to emergencies during surgery is improved, and the collaborative cooperation of multidisciplinary teams is promoted. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present invention. Throughout the drawings, the same reference figures denote the same components. In the drawings:
[0040] Figure 1 A schematic diagram of the structure of a bronchoscopic emergency rescue training system according to an embodiment of the present invention is shown;
[0041] Figure 2 Schematic diagram of an EEG interface according to an embodiment of the present invention is shown;
[0042] Figure 3 A schematic diagram of the rescue process for microscopic bleeding according to an embodiment of the present invention is shown;
[0043] Figure 4 A schematic diagram of the rescue process for laryngeal spasm according to an embodiment of the present invention is shown. DETAILED DESCRIPTION
[0044] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.
[0045] Example 1:
[0046] The embodiment of the present invention provides a bronchoscopic emergency rescue training system, such as Figure 1 Shown, including:
[0047] The brain-computer interface 101 is worn on the operator's head and is used to collect the operator's brain electrical signals;
[0048] The virtual simulation device 102 is used to present to the operator a virtual scene of one or more emergency situations occurring in a virtual patient during bronchoscopy;
[0049] Patient model 103, used for operation by the operator;
[0050] Camera 104 is used to capture images of emergency rescue operations performed by an operator, where the emergency rescue operations are performed by the operator in response to a virtual scene of one or more sudden emergency situations;
[0051] The processing module 105 is connected to the brain-computer interface 101, the virtual simulation device 102 and the camera 103, and is used to evaluate the operator's psychological state and operation proficiency based on the EEG signal; based on the operator's psychological state and operation proficiency, update the virtual scene, and evaluate the operator's operation accuracy, reaction speed, and teamwork ability.
[0052] In a specific embodiment, the system can be used by multiple medical workers, including nurses, doctors, and anesthesiologists. For a surgery, a team is needed to complete it, and each operator needs to cooperate with each other to make the surgery go smoothly.
[0053] For this team member, each operator needs to wear a brain-computer interface 101 and use a virtual simulation device 102 to present a virtual scene. Virtual simulation device 102 is an AR device or an XR device. In the case of an AR device, each operator needs to wear an AR device, specifically glasses. Different virtual scenes can be presented to different operators.
[0054] For example, a nurse can see a virtual patient and the required virtual surgical instruments. They can also see the patient's intraoperative emergencies and the situation under the bronchoscopy, specifically, the bronchoscope, foreign body forceps, balloon, basket, laser, and cryostat. The nurse can pass the necessary virtual surgical instruments to the doctor and put them back in place. The doctor needs to perform a bronchoscopic operation on the virtual patient, so the doctor is presented with a virtual scene of inserting the bronchoscope into the trachea, as well as one or more virtual emergency scenarios that may occur with the virtual patient.
[0055] When the virtual simulation device is an XR device, the same virtual screen is presented to different operators through projection.
[0056] The anesthesiologist needs to manage the anesthesia status of the virtual patient, such as adjusting the anesthesia duration, etc. Therefore, a virtual scene of the virtual patient's anesthesia status needs to be presented to the anesthesiologist-type operator.
[0057] In addition, virtual emergency equipment including a suction machine, central oxygen supply, and an emergency vehicle is presented to the doctor-type operator, the nurse-type operator, and the anesthesiologist-type operator.
[0058] At the same time, vital signs detection equipment can also be presented to nurse-type, doctor-type and anesthesiologist-type operators so that doctors, nurses and anesthesiologists can all see the vital signs monitoring results of virtual patients, such as electrocardiogram monitoring equipment, blood pressure monitoring equipment, respiratory monitoring equipment, etc.
[0059] The brain-computer interface 101 is worn on the heads of different types of operators to collect EEG signals of different operators, such as Figure 2 shown.
[0060] Describe different virtual simulation devices:
[0061] AR devices stand for augmented reality, while XR devices stand for extended reality. AR devices overlay the real world with virtual information. The virtual information typically appears in the form of images, text, and video, enhancing perception and understanding of the real world. XR devices can blend and switch between the real and virtual worlds to varying degrees depending on needs. They can add virtual elements to real scenes like AR, provide a completely virtual experience like VR, and even enable real-time interaction and fusion of virtual and real elements, presenting more complex and diverse effects.
[0062] In order to enable the operator to see the patient model and various virtual instruments and equipment at the same time, AR equipment is used.
[0063] The emergency situation presented by the virtual simulation device 102 is specifically any one or more of the following:
[0064] Allergy to anesthetic drugs, bleeding, laryngospasm, hypoxemia, asphyxia, arrhythmia, cardiac arrest, and pneumothorax.
[0065] These sudden emergency situations cause some physical injuries to the virtual patients, and if they are not rescued in time, the situation will worsen further.
[0066] When presenting an allergy to anesthetic drugs, it is reflected through a rash on the virtual patient's skin or difficulty breathing.
[0067] When presenting bleeding, the bleeding situation is reflected by the virtual patient bleeding in different parts of the airway.
[0068] When laryngeal spasm occurs, it is reflected by the virtual patient coughing, cyanosis of the lips and face, distended neck veins, abnormal breath sounds, increased heart rate, and decreased blood oxygen saturation.
[0069] When hypoxemia occurs, it is monitored by the virtual patient's blood oxygen saturation monitoring device.
[0070] When suffocation occurs, the virtual patient will experience cyanosis of the lips and face, distended neck veins, disappearance of breath sounds, a rapid rise in heart rate followed by a sudden drop, and a sharp drop in blood oxygen saturation.
[0071] When arrhythmia occurs, the virtual patient is monitored by an ECG monitoring device.
[0072] When cardiac arrest occurs, the virtual patient will experience pulse loss, respiratory arrest, decreased blood pressure and blood oxygen saturation.
[0073] When pneumothorax is present, it is manifested by symptoms such as chest bulging, percussion tympany, tracheal deviation, subcutaneous emphysema and dyspnea in the virtual patient.
[0074] Any one or more of the above-mentioned emergency situations are presented through the virtual simulation device 102 so that the operator can effectively identify them, and the identification results are transmitted to the processing module 105 in a selected manner.
[0075] Next, after correct identification, the operator can perform corresponding emergency rescue operations based on these sudden emergency situations.
[0076] The camera 104 is used to collect images of the emergency rescue operation performed by the operator and then transmit them to the processing module 105 .
[0077] The processing module 105 can evaluate the operator's psychological state and operational proficiency based on EEG signals, update the virtual scene based on the operator's psychological state and operational proficiency, and evaluate the operator's operational accuracy, reaction speed, and teamwork ability.
[0078] The evaluation of the operator's mental state and operational proficiency is specifically achieved through the following methods:
[0079] Extract EEG feature signals based on EEG signals;
[0080] Based on EEG characteristic signals and psychological state recognition model, identify the operator's psychological state, including emotional state, attention state and stress state;
[0081] The operator's proficiency is determined based on EEG signals and the operator's response time to emergency rescue operations.
[0082] Specifically, historical EEG signals and corresponding historical psychological states of historical patients are collected. Before training, the corresponding historical EEG feature signals need to be extracted from the historical EEG signals. This can be achieved by using EEG time-domain feature extraction techniques, including short-time Fourier transform and wavelet transform. Feature extraction saves hardware and software resources, reduces computation time, and reduces complexity. The historical EEG feature signals and historical psychological states are then input into a neural network model (CNN) for training, thereby obtaining the psychological state recognition model.
[0083] Next, the currently collected EEG signal of the operator is first subjected to EEG feature signal extraction, and then the EEG feature signal is input into the mental state recognition model, thereby recognizing the mental state of the operator.
[0084] To analyze the operator's proficiency, the operator's response time to emergency rescue operations is first determined. This can be determined by capturing images of the operator performing emergency rescue operations through a camera. The time from the generation of the EEG signal to the operator's response to the emergency rescue operation is then used to derive the corresponding reaction speed, thereby determining the operator's proficiency. For skilled operators, this reaction speed is closer to the standard reaction speed.
[0085] The emergency rescue operations performed by the operator can be the following:
[0086] Doctors perform surgical operations, such as bronchoscopy, or deal with sudden emergency situations; nurses organize and deliver virtual surgical instruments, use emergency drugs and equipment, and cooperate with doctors in emergency treatment, including but not limited to medication, airway opening, cardiopulmonary resuscitation, etc.; anesthesiologists monitor the patient's anesthetic status.
[0087] When the operator's psychological state and operation proficiency are obtained, they are compared with their respective corresponding standard values. If both are lower than their respective corresponding standard values, it is determined that the operator's psychological state and operation proficiency are poor. At this time, the processing module 104 updates the current virtual scene to a relaxed virtual scene. By updating the virtual scene, the operator's psychological state is improved to improve the operation accuracy.
[0088] For example, the environment in the operating room is relatively strict. By updating it to a ward environment, the operator's psychological state can be alleviated, thereby promoting the completion of the operation.
[0089] When evaluating the operator's accuracy, reaction speed, and teamwork ability, the evaluation is carried out in the following manner:
[0090] Based on the emergency rescue operation image and the operator's voice information, determine whether the emergency rescue operation performed by the operator is correct to determine the accuracy of the operation;
[0091] Determine the operator's response time to emergency rescue operations based on EEG signals;
[0092] Determine the reaction speed based on the response time;
[0093] Determine team collaboration capabilities based on the connection time between operations of different operators.
[0094] Specifically, different emergency situations require different emergency rescue operations. For example, in the case of cardiac arrest, cardiopulmonary resuscitation is immediately performed; in the case of anesthetic allergy, anti-allergic drugs can be injected, and the operator's voice information can be collected to determine the drug dosage; in the case of bleeding, drug treatment can be used under bronchoscopy, and interventional procedures can be used to stop bleeding when necessary. By analyzing the images of emergency rescue operations, it is possible to determine whether the emergency rescue operations performed by the operator are correct based on key points such as key instruments and the site of action.
[0095] Specific to different emergency situations, such as Figure 3 、 Figure 4 The following are the corresponding emergency rescue operations. Figure 3 The rescue process for microscopic bleeding; Figure 4 The rescue process for laryngospasm;
[0096] In a specific embodiment, the system collects the operator's operation images and voice information, and thus judges the operator's operation accuracy according to the above-mentioned rescue process.
[0097] To assess reaction speed, we can extract the ERP waveform based on the EEG signal and then build a linear regression model between the ERP waveform component latency and the emergency response time. The ERP waveform component latency is formed after the emergency occurs and is related to the response time. Finally, based on this linear regression model, the response time is obtained. This reaction time is a portion of the response time.
[0098] Teamwork capability is determined by the connection time between different operators' operations. This connection time can be determined by the time between the triggering event and the response time. For example, if the triggering event is a doctor requesting a bronchoscope, the corresponding response event is the nurse handing the bronchoscope to the doctor. Based on the timing between the two events, the connection time is obtained, and thus the teamwork capability is determined based on the connection time between multiple events.
[0099] This allows the operator's accuracy, reaction speed, and teamwork ability to be evaluated and an evaluation report to be generated, providing a more specific conclusion on the operator's performance.
[0100] In a specific embodiment, the operator can be one or more of the following types. For example, only the doctor type operator is trained, and the other two types of operators are obtained by virtualization. Of course, if all three types of operators participate in the training, there is no need to simulate any one type of operator.
[0101] In an optional embodiment, the processing module 105 is further configured to provide a virtual competition environment, including setting competition rules and scoring criteria, and presenting them through the virtual simulation device 102 .
[0102] By setting up a virtual competition environment, the operators' learning enthusiasm and competitive awareness can be stimulated. Operators can learn from each other and improve together through competition.
[0103] Moreover, through multi-modal training, the flexibility and efficiency of training can be improved.
[0104] One or more technical solutions in the embodiments of the present invention have at least the following technical effects or advantages:
[0105] The present invention provides a bronchoscopic intraoperative emergency rescue training system, comprising: a brain-computer interface, worn on the operator's head, for collecting the operator's electroencephalogram (EEG) signals; a virtual simulation device, for presenting to the operator a virtual scene of one or more emergency situations occurring in a virtual patient during bronchoscopy; a patient model, for providing the operator with an operation; a camera, for collecting images of emergency rescue operations performed by the operator, where the emergency rescue operations are operations performed by the operator for one or more virtual scenes of emergency situations; a processing module, connected to the brain-computer interface, the virtual simulation device and the camera, for evaluating the operator's psychological state and operational proficiency based on EEG signals; updating the virtual scene based on the operator's psychological state and operational proficiency; and evaluating the operator's operational accuracy, reaction speed and teamwork ability based on the images and EEG signals of the emergency rescue operations. By providing medical workers with an immersive training experience, the ability to respond to emergencies during surgery is improved, and the collaborative cooperation of multidisciplinary teams is promoted.
[0106] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.
[0107] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.
Claims
1. A bronchoscopic emergency rescue training system, characterized by: include: A brain-computer interface, worn on the operator's head, is used to collect the operator's EEG signals; A virtual simulation device for presenting to the operator a virtual scenario of one or more emergency situations occurring in a virtual patient during bronchoscopy; Patient model, used for operator operation; a camera, configured to capture images of an emergency rescue operation performed by an operator, wherein the emergency rescue operation is an operation performed by the operator in response to a virtual scene of the one or more sudden emergency situations; The processing module is connected to the brain-computer interface, virtual simulation equipment and camera, and is used to evaluate the operator's psychological state and operational proficiency based on EEG signals; based on the operator's psychological state and operational proficiency, it updates the virtual scene and evaluates the operator's operational accuracy, reaction speed and teamwork ability.
2. The method according to claim 1, wherein The operator is any one or more of the following: Doctors, nurses and anesthesiologists.
3. The method according to claim 1, wherein The emergency situation is specifically any one or more of the following: Allergy to anesthetic drugs, bleeding, laryngospasm, hypoxemia, asphyxia, arrhythmia, cardiac arrest, and pneumothorax.
4. The method according to claim 1, wherein Virtual simulation device, specifically any of the following: AR devices and XR devices.
5. The method according to claim 1, wherein The virtual simulation device is used to present a virtual scene of a sudden emergency situation of a virtual surgical instrument, a virtual vital sign detection device, and a virtual patient to the operator.
6. The method according to claim 2, wherein The virtual simulation device is used to: Presenting a virtual scene of inserting a bronchoscope into the trachea and a virtual scene of an emergency situation of a virtual patient to a doctor-type operator; Presenting a virtual scene of virtual surgical instruments to a nurse-type operator, wherein the virtual surgical instruments include: a bronchoscope, foreign body forceps, a balloon, a basket, a laser, and a cryoinstrument; Presenting a virtual scene of a virtual patient's anesthesia state to an anesthesiologist-type operator; A doctor-type operator, a nurse-type operator, and an anesthesiologist-type operator are all presented with virtual emergency equipment including a suction machine, central oxygen supply, and an emergency cart.
7. The method according to claim 1, wherein Processing module, specifically used for: Extract EEG feature signals based on EEG signals; Based on the EEG characteristic signal and the psychological state recognition model, identifying the operator's psychological state, including: emotional state, attention state, and stress state; The operator's proficiency is determined based on EEG signals and the operator's response time to emergency rescue operations.
8. The method according to claim 1, wherein Processing module, specifically used for: When the operator's mental state and operating proficiency are not good, the current virtual scene is updated to a relaxing virtual scene.
9. The method according to claim 1, wherein Processing module, specifically used for: Based on the image of the emergency rescue operation and the operator's voice information, determining whether the emergency rescue operation performed by the operator is correct to determine the accuracy of the operation; determining, based on the EEG signal, a response time of the operator to an emergency rescue operation; determining a reaction speed based on the response time; Determine team collaboration capabilities based on the connection time between operations of different operators.
10. The method according to claim 1, wherein The processing module is also used to: Provide a virtual competition environment, including setting competition rules and scoring criteria, and present them through virtual simulation equipment.