Airway management virtual reality training

Abstract

Airway management virtual reality (VR) training systems and methods are provided that use a relatively simple and passive physical patient model to train a user to perform airway management procedures. The patient model can include: a respiratory tract, a moveable head, and an openable jaw that are anatomically realistic; and electromagnetic sensors to measure the motion of the various parts of the physical model and to measure the motion of tools used to treat the model. Parameters of the medical procedure being performed are sensed, tracked, and displayed to provide a continuous, detailed, and coherent VR representation of the training situation in a realistic scenario, with a hierarchy of sensors leveraged to focus the VR representation on key features of the training and to produce interactivity with the VR patient and assessment of the trainee's performance.

Description

Background of the Invention Technical Field

[0002] This invention relates to the field of virtual reality systems for medical training, and more particularly to airway management training systems.

[0003] background

[0004] Existing airway management training systems utilize mechanically complex patient models, which include multiple motors and associated actuators, as well as complex model structural features, to make the patient model as realistic as possible. Invention Overview

[0006] The following is a simplified overview to provide a preliminary understanding of the invention. This overview does not necessarily identify key elements, nor does it limit the scope of the invention, but is merely intended to introduce the following description.

[0007] One aspect of the present invention provides an airway management training system comprising: a physical, generally passive patient model, having an airway, a head movable from one side to the other, and an openable jaw; a plurality of electromagnetic sensors configured to measure the relative position of the head and jaw and the relative position of at least one airway management tool relative to the airway; and a virtual reality (VR) system configured to provide a VR representation to a user or other trainee of: a scene, at least one patient corresponding to the physical patient model, at least one airway management tool, and the trainee’s hand manipulating or operating at least one airway management tool.

[0008] One aspect of the present invention provides an airway management training method, comprising: training airway management in a virtual reality (VR) environment; enhancing the VR environment using a physical patient model and at least one airway management tool, wherein the physical patient model has an airway, a head movable from one side to the other, and an openable jaw, and the at least one airway management tool has at least one pressure sensor; measuring the relative position of the head and jaw and the relative position of the at least one airway management tool relative to the airway using a plurality of electromagnetic sensors associated with the passive patient model; and displaying a scene, a patient corresponding to the physical patient model, at least one airway management tool, and the trainee's hand to a trainee in the VR environment.

[0009] These, additional and / or other aspects and / or advantages of the invention are set forth in the following detailed description; can be inferred from the detailed description; and / or can be learned by practice of the invention. Brief description of the attached diagram

[0011] To better understand embodiments of the invention and to show how they can be implemented, reference will now be made to the accompanying drawings by way of example only, with similar reference numerals throughout the drawings indicating corresponding elements or portions.

[0012] In the attached diagram:

[0013] Figure 1A and Figure 1B This is a high-level schematic block diagram of an airway management training system according to some embodiments of the present invention.

[0014] Figures 2-4 This is a high-level schematic diagram of airway management training system components according to some embodiments of the present invention.

[0015] Figure 5A and Figure 5B Examples of VR representations of scenes, patients, tools, and trainees' hands are provided according to some embodiments of the present invention.

[0016] Figure 6 This is a high-level flowchart illustrating an airway management training method according to some embodiments of the present invention.

[0017] Figure 7 This is a high-level block diagram of an exemplary computing device that can be used in embodiments of the present invention.

[0018] Detailed description of the invention

[0019] In the following description, various aspects of the invention are depicted. For purposes of explanation, specific configurations and details are set forth to provide a thorough understanding of the invention. However, it will also be apparent to those skilled in the art that the invention can be practiced without the specific details set forth herein. Furthermore, well-known features may be omitted or simplified so as not to obscure the invention. With reference to the specific drawings, it is to be emphasized that the details shown are by way of example and are merely for the purpose of illustrative discussion of the invention, and are presented to provide the most useful and readily understood description of what is believed to be the principles and concepts of the invention. In this respect, no attempt is made to show the structural details of the invention in more detail than necessary for a basic understanding of the invention; the description, taken in conjunction with the drawings, enables those skilled in the art to understand how several forms of the invention can be embodied in practice.

[0020] Before explaining at least one embodiment of the invention in detail, it should be understood that the invention, in its application, is not limited to the structural and arrangement details of the components set forth in the following description or shown in the drawings. The invention is applicable to other embodiments that can be practiced or implemented in various ways, as well as combinations of the disclosed embodiments. Furthermore, it should be understood that the wording and terminology used herein are for descriptive purposes and should not be considered limiting.

[0021] Unless otherwise specifically stated, it will be apparent from the following discussion that terms used throughout the specification, such as “processing,” “computing,” “calculating,” “determining,” “enhancing,” “deriving,” etc., refer to the actions and / or processes of a computer or computing system or similar electronic computing device that manipulate and / or convert data represented as physical quantities (such as electronic quantities) in the registers and / or memory of the computing system into other data represented as similar physical quantities in the memory, registers, or other information storage, transmission, or display devices of the computing system.

[0022] Embodiments of the present invention provide an efficient and economical method and mechanism for virtual reality training in airway management, thereby improving the technical field of medical simulators. Specifically, an airway management virtual reality training system is provided that uses a relatively simple and passive physical patient model to train a user (e.g., a trainee) to perform airway management procedures. The patient model includes a modeled airway, a head movable from side to side relative to the torso, and an openable jaw, all anatomically realistic. The physical patient model includes electromagnetic sensors that provide system data on the movement of various parts of the model and on the movement of tools entering, within, and exiting the model.

[0023] Users or trainees can manipulate airway management tools, such as laryngoscopes, manual resuscitators, and other tools used to perform airway management procedures, while pressure sensors on the tools provide input regarding the mechanical interactions between the airway management tools and various parts of a physical patient model. These tools can be real tools, tool models, or multi-purpose components simulated as specific tools in virtual reality (VR). Both the models and tools have trackers that provide position and orientation data to the system. Furthermore, the system incorporates multiple VR sensors, such as cameras, to monitor the airway management procedures being performed by the trainee.

[0024] The system can track and sense many parameters of an ongoing medical procedure and use them to provide trainees with a continuous, detailed, and coherent virtual reality representation, displaying training scenarios in a real-world setting. It can utilize and modify the hierarchy of sensors to focus the representation on key features of the training and generate interactive VR-simulated patient models and assessments of trainee performance.

[0025] Although the patient models in some embodiments are simple and passive compared to existing training models, the VR system compensates for this simplicity by providing a realistic representation and display of the procedure being performed to the user (trainee). The system can be configured to have visual representations that suggest at least some of the tactile inputs provided by more complex existing models. Specifically, the data collected by the system is organized in a hierarchical and situation-related manner, thus providing a continuous and realistic VR representation of the performed procedure, for example, regarding the position and orientation of the airway management tool relative to the modeled airway, the pressure applied by the airway management tool to the modeled airway, and the trainee's hand manipulating the tool.

[0026] To this end, the disclosed system dynamically identifies treatment situations performed by the trainee and dynamically manages the situation-related hierarchy among sensors to maintain the continuity and coherence of representation and display relative to the identified treatment situation. For example, when a trainee manipulates or operates a manual resuscitator, the system can represent or display the trainee's hand hidden beneath it, based on data from pressure sensors on the resuscitator (supplementing visual data collected by a camera). In another example, multi-purpose (e.g., general-purpose) tools can be used by the trainee to perform assistive actions and are represented and displayed only within the VR scene as corresponding specific real-world tools (e.g., scalpels, forceps, catheters such as ET (endotracheal) tubes, suction devices, stethoscopes, carbon dioxide detectors, pulse oximeters, etc.)—further simplifying the system. The VR system can simulate various operational scenarios and multiple assistants, which can be used to monitor the trainee's interactions. For example, cameras can be used to sense the trainee's position and posture and correlate them with instructions given by the trainee to a real or virtual assistant. Furthermore, even outside the camera's sensing range, the trainee's hand can be represented by the VR system—by adjusting the VR representation based on the recognition, for example, showing the trainee's hand extended to receive a (simulated) tool. Embodiments of the disclosed system are shown in a non-limiting manner in the accompanying drawings below.

[0027] One advantage of this system is the simplicity of its physical model, which makes it cheaper to purchase and use, and more durable for outdoor training (e.g., for military medics). Another advantage is the provision of reliable feedback across a wide range of procedures and user actions, allowing trainees to practice different airway management procedures in many environmental situations.

[0028] Figure 1A and Figure 1B This is a high-level schematic block diagram of an airway management training system 100 according to some embodiments of the present invention. Figures 2-4This is a high-level schematic diagram of the components of an airway management training system 100 according to some embodiments of the present invention. The airway management training system 100 includes a virtual reality (VR) system 150 that provides trainees with a simulated scenario 162 and instructions related to the airway management procedures applied by the trainee on a passive physical patient model 110 using a variety of real, modeled and / or simulated airway management tools 120.

[0029] like Figure 1A As illustrated, the VR system 150 can be configured to dynamically identify treatment situations and respond to situation-related hierarchies from multiple sensors in the training system 100 to continuously provide a realistic representation of the airway management procedures applied by trainees using the system 150.

[0030] The trainee can use a VR headset 102, to which a VR system 150 provides a VR representation 140. The VR headset 102 may include an eye tracker (not shown) that provides data on the trainee's eye movements to the VR system 150. One or more trackers 104 can be attached to the VR headset 102 to track the trainee's head. The trainee may further use one or two gloves 103 (see...). Figure 1A and Figure 1B The glove 103 may be equipped with an additional tracker 104 (e.g., on the trainee's hand and / or on the trainee's fingers) to track the position of the trainee's hand and fingers. Suitable gloves used may include Manus VR gloves or Noitom Hi5 VR gloves, or other suitable gloves. In some embodiments, an optical sensor 152, such as a camera, may also be attached to the trainee's head (or headgear 102), hand (or glove 103), or other body part to provide close-up images of the therapeutic procedure performed by the trainee. In some embodiments, one or more gloves 103 may be configured to measure forces applied by the trainee during manipulation of the physical model 110 (e.g., such as...). Figure 2 (Illustrated schematically) and transmits the measurement results to VR system 150, which can use the measurement results as additional sensor data. In some embodiments, one or more gloves 103 may be configured to provide tactile feedback to the trainee, thereby applying forces to the trainee's hand in addition to the forces experienced due to manipulating the physical model 110, for example, to enhance the tactile simulation, simulate additional structural features (e.g., structural features in the airway of a VR patient), etc.

[0031] like Figure 1BAs illustrated, the VR system 150 can be configured to represent the patient model 110 as a VR patient 141 in a VR environment (140), the tools used or commonly used as corresponding VR tools 125, the trainee's hands (and possibly other body parts of the trainee) as VR 146, and the pressure 144 applied by the tools (the pressure VR representation can be visual, as an instruction, using tactile cues, or otherwise), the surrounding scene 162, the virtual or real assistant 164, medical devices (not shown), etc. Figure 2 The physical training setup is illustrated schematically. Figure 3 The diagram schematically illustrates some details of the physical structure of patient model 110, and Figure 4 Some tools 120 are shown schematically.

[0032] According to some embodiments of the present invention, Figure 5A and Figure 5B Scene 162, Patient 141, Tool Display 125B, 125A (tool displays for manual resuscitator 120B and laryngoscope 120A respectively, with laryngoscope 120A displayed on screen) are provided. Figure 5B (See further explanation below) and an example of VR representation 140 of the trainee's hand 146. Figure 5A and Figure 5B Each of these includes an example of the actual VR representation 140 and line drawings indicating the parts of the representation as listed above. The continuous and coherent matching of the VR representation 140 with the trainee's movements on the patient model 110 can produce realistic training, effective learning, and reliable assessment of the trainee's abilities.

[0033] The airway management training system 100 includes a passive physical patient model 110 (e.g., a mannequin or a portion thereof) having a modeled airway 111, a head 112 connected to the trunk 115 and movable from one side to the other, and an openable jaw 113, such as... Figure 2 and Figure 3 As shown. Arrow 112A schematically indicates the direction of movement of the head 112, and arrow 113A schematically indicates the direction of movement of the jaw 113. Note that the VR system 150 compensates for the simplicity of the patient model 110 (compared to mechanically complex prior art patient models that include multiple motors and associated actuators, more directions of movement, and more complex model features), replacing at least some of the tactile information in prior art systems with visual information. Advantageously, the simplicity of the passive physical patient model 110 allows it to be durable and deployable in the field, for example, for training military medics and / or civilian medical personnel under realistic conditions.

[0034] The patient model 110 also includes one or more electromagnetic sensors 114 configured to measure the relative position of the head 112 and jaw 113, and the relative position of the airway management tool 120 relative to the modeled airway 111. For example, electromagnetic sensors 114A, 114B (in...) Figure 3 As schematically shown, electromagnetic sensors 114A and 114B (located inside the head 112) can be configured to measure jaw and head movements, respectively. Figure 2 The schematic diagram shows data 153 provided from an electromagnetic sensor. Figure 3 Also shown is a mechanical device 112B (connecting the head 112 to the torso 115 and supporting the rotational movement of the head 112), which is configured to allow the head 112 to move from one side to the other. Figure 2 (Motion 112A shown in the figure). The modeled airway 111 is not explicitly shown, but it is modeled to provide realistic interaction with the application tool 120 corresponding to the airway anatomy. In some embodiments, the patient model 110 may also include pressure sensors 117 located at specific locations important during airway management to complement the pressure sensors 122 on the tool 120. The pressure sensors 122 and optional flexion sensors 123 on the tool 120, as well as the optical sensors 152 of the VR system 150 (and / or the tracker 104 on the head-mounted device 102), provide data 151 about the tool 120, which the VR system 150 uses to identify treatment situations 156 performed by the trainee and dynamically manage situation-related hierarchies 158 of multiple sensors in the system 100.

[0035] Airway management tools 120 may include, for example, a laryngoscope 120A, a manual resuscitator 120B (e.g., Resuscitator equipment or other resuscitators), multi-purpose tool 120C, which can be represented or displayed in VR as any of a range of tools (e.g., scalpel, forceps, catheters (e.g., ET (endotracheal) tube), suction device, stethoscope, carbon dioxide detector, pulse oximeter, etc.), such as Figure 4 As shown. The airway management tool 120 may include one or more pressure sensors 122 and possibly trackers 124, which provide the system 100 with feedback on the physical interaction between the tool 120 and the patient model 110, as well as on the position and orientation 142 of the tool 120. This feedback can be used to assess the trainee’s performance and / or enhance or modify the VR representation 140 of the procedure, generating, for example, responses from the simulated patient.

[0036] In various embodiments, the laryngoscope 120A can be real or modeled, and pressure sensors 122 along the blades of the laryngoscope 120A provide feedback on the forces exerted by the laryngoscope 120A on a modeled airway 111 (e.g., teeth, jaw, or internal airway region) when the trainee manipulates or operates the laryngoscope 120A. This feedback can be translated into VR indications, such as simulated patient injury or response, and / or assessments of the quality of the trainee's application of appropriate airway management procedures.

[0037] In various embodiments, the manual resuscitator 120B may be real or modeled, having pressure sensors 122 and / or flexure sensors 123 along at least a portion of its circumference and / or on its mouthpiece 122A. When a trainee operates the manual resuscitator 120B, the pressure sensors 122, 122A can be used to provide feedback on the force exerted by the manual resuscitator 120B on the modeled airway 111 (e.g., teeth or jaw 113). This feedback can be translated into VR indications, such as simulated patient injury or response, and / or an assessment of the quality of the trainee's application of appropriate airway management procedures. The degree of airtightness between the mouthpiece of the manual resuscitator 120B and the patient model 110 can also be measured and indicated in the VR representation 140 and / or indicated by modifying the simulated patient response in the VR representation 140 (e.g., insufficient airtightness may result in insufficient or no chest movement when operating the manual resuscitator 120B).

[0038] Furthermore, data from pressure sensor 122 can be used to enhance or modify the trainee's VR representation. For example, data from pressure sensor 122 indicating that the trainee's hand is located below manual resuscitator 120B and is not visible to the optical sensor 152 of VR system 150 can be used to accurately or approximately represent and display hand 146 (although at least one hand is at least partially hidden by manual resuscitator 120B) to enhance the continuity and realism of the VR. For example, VR system 150 can be configured to represent the trainee's hand 146 to correspond to detected pressure and / or deflection applied to the circumference of manual resuscitator 120B.

[0039] In various embodiments, the multipurpose tool 120C with a general-purpose design can be used to adjustably represent or display any of a variety of assistive tools, such as scalpels, forceps, catheters (e.g., ET (endotracheal) catheters), aspirators, stethoscopes, carbon dioxide detectors, pulse oximeters, etc. For example, upon request from the trainee (e.g., from a real or simulated assistant 164) and / or based on dynamically identified treatment conditions 156, the VR system 150 can be configured to virtually provide and display the tool 120C, such as any of scalpels, forceps, ET catheters, aspirators, stethoscopes, carbon dioxide detectors, pulse oximeters, etc.

[0040] According to some embodiments of the present invention, Figure 5A and Figure 5B An example of a VR representation 140 is provided, showing scene 162, patient 141, tool 125, and trainee's hand 146. Virtual Reality (VR) system 150 can be configured to direct the trainee (wearing VR headset 102, e.g., with attached tracker 104) into a VR environment. Figure 2 (Illustrated schematically) A scenario 162 is provided, which includes at least a patient 141 corresponding to a physical patient model 110, and a representation 140 of a medical procedure performed by a trainee on the passive physical patient model 110 using an airway management tool 120. A VR headset 102 may include a head-mounted device that provides the VR representation 140 to the trainee, and may include, for example, a display and processor associated with and communicating with the VR system 150 via a wired or wireless connection to a computing device 154 disclosed below. The VR headset 102 may include a stereoscopic head-mounted display that provides sound, and may also include head motion and / or eye-tracking sensors, as well as potentially associated and linked controllers. Through the VR system 150 and the VR headset 102, the trainee can receive training in airway management procedures within the virtual and controlled environment of the VR representation 140.

[0041] VR representation 140 may include tool representation 125 of tool 120, and indications of at least the location and orientation 142 of airway management tool 120 relative to a modeled airway 111 (including tool representation 125) (see...). Figure 5A and Figure 5B (Example in the text), representation of the pressure 144 applied by airway management tool 120 to the modeled airway 111 (in...) Figure 1A and Figure 1B (Illustratively shown in the image), and a representation of the hand of a trainee 146 manipulating an airway management tool 120. The position and orientation 142 of the tool 120 can be visually represented, such as... Figure 5A and Figure 5B As shown in the provided non-limiting examples, pressure 144 may be measured, for example, by optical sensor 152 and / or tracker 124. Pressure 144 may be represented, for example, by an indicator and / or by the resistance of tool movement.

[0042] VR system 150 may include optical sensor 152, which is configured to (e.g., use) Figure 2The tracker 116 shown at least tracks the position of the passive patient model 110, the airway management tool 120 (e.g., deriving its position and orientation), and the trainee's hands. The VR system 150 can also be configured to dynamically identify treatment situations 156 performed by the trainee and dynamically manage situation-related hierarchies 158 among sensors 152, 114, and 122, maintaining the representation 140 continuous and coherent relative to the identified treatment situation. The VR system 150 is configured to receive data from sensors 114, 122, and 123 and trackers 104, 116, and 124 via wired and / or wireless connections.

[0043] It should be noted that the continuity of the VR representation includes the continuous motion of the represented element, without any jumps or jerks that do not correspond to real motion. The situation-related sensor hierarchy 158 involves the relative reliability of various sensors and can be used to exclude potential discontinuous representations of elements that some sensors may imply due to factors such as their limited field of view relative to higher-level sensors that provide more reliable data, their relative distance to the corresponding element, or their less relevant sensing patterns.

[0044] It is further noted that the coherence of the VR representation includes the consistent position of the represented elements, without any disappearances or large-scale changes that do not correspond to actual movement. The situation-related sensor hierarchy 158 relates to the relative reliability of various sensors and can be used to exclude the potential presence or disappearance of elements in the VR representation of elements that some sensors may imply due to factors such as their limited field of view relative to higher-level sensors that provide more reliable data, their relative distance to the corresponding element, or their less relevant sensing patterns. For example, in the case where a hand or part thereof disappears from the field of view of the optical sensor 152 because it is hidden under the manual resuscitator 120B, data from the flexure sensor 123 above it can be used to provide a VR representation of the hand to keep it present in the VR representation 140 in an appropriate manner. Another example involves instructions given by a trainee that may be accompanied by hand movements beyond the sensing range. In this case, the representation of the hand can be supplemented by tracker data or estimated position.

[0045] Specific, non-limiting examples of situation-related sensor levels 158 are shown in Table 2 below. As a general rule, for each or some identified treatment situations 156, the VR system 150 may have rules for determining which sensors and trackers in the system 100 are more reliable and less reliable relative to the geometry of the treatment situation (e.g., potentially hidden elements or elements that may extend beyond the sensing range) and / or relative to the sensing modality (e.g., in some cases, pressure data may be more reliable than optical data). Situation-related sensor levels 158 can be determined for each or some identified treatment situations 156 based on these rules.

[0046] VR system 150 can be configured to use any of a variety of VR modeling procedures, such as using polygonal meshes and adding surface features (see, for example, see...). Figure 3 Examples of polygonal meshes and Figure 5A and Figure 5A (Example of adding surface features) to generate a patient representation 141 from the patient model 110. Note that the physical patient model 110 is used as a real-world reference for the medical procedure applied by the trainee and corresponds to an internal data model in the VR system 150 used to construct the VR representation 140 of the patient 141, which is displayed to the trainee via the VR headset 102. The VR system 150 (and / or the airway management training system 100) is configured to further enhance the VR representation 140 with: a visual representation of the airway management tool 120 (based on the position and orientation of the airway management tool 120 relative to the modeled airway 111), a representation and / or indication of the pressure 144 applied by the airway management tool 120 to the modeled airway 111, and a visual representation of the trainee's hand 146 manipulating the airway management tool.

[0047] Table 1 provides several non-restricted examples of sensors and data in the airway management training system 100. Trackers typically have 6 degrees of freedom (DoF) and may include available trackers with corresponding performance.

[0048] Table 1: Examples of sensors and data in the airway management training system.

[0049]

[0050] Table 2 provides several non-limiting examples of treatment status, monitored actions, sensors used, and corresponding VR representations and feedback.

[0051] Table 2: Examples of treatment status, monitored actions, sensors used, and corresponding VR representations and feedback.

[0052]

[0053]

[0054] In the following sections, specific, non-limiting examples related to the cases described in Table 2 are provided for the identified treatment case 156, corresponding to the relevant indication and / or its response, the virtual patient simulation features, the expected trainee response, and the sensor hierarchy used to assess the actual trainee response.

[0055] For example, during treatment preparation and / or various treatment phases, the VR representation 140 may include simulated patient movements, sounds, head movements, various breathing patterns manifested in the head and chest areas, and related to medical conditions, chest movements, substances (such as fluid in or out of the patient's airway), etc. The VR representation 140 may include typical patient behaviors according to different patient states, such as alertness, consciousness, partial or complete unconsciousness, asphyxia, sedation, various breathing patterns, etc.

[0056] VR representation 140 can be modified based on the trainee's actions or inactions, such as specific instructions, diagnostic measures, procedures applied to the patient model, measurements and instructions taken, and the use of tools in diagnosis and treatment. Specifically, incorrect or incomplete application of tools to the patient model 110 may modify VR representation 140 in a manner reflecting inappropriate application. For example, incorrect use of manual resuscitator 120B may result in the virtual patient not responding as expected (e.g., not breathing as expected in terms of chest and head movements and related sounds), requiring correction by the trainee. Note that virtual patient responses can be represented in VR representation 140 and / or relevant medical data and indicators.

[0057] Examples of modifications to the sensor hierarchy 158 and the VR representation of the passive patient model 110 regarding the dynamically identified treatment situation 156 include, for example, the following related to the sensors listed in Table 1. System 100 can be configured to modify the resolution of the sensor hierarchy 158 and / or the VR representation 140 based on specific elements related to the identified treatment situation 156. For example, when a trainee performs fine motor actions, the finger tracker 104 can receive higher priority than other sensors, and the resolution of the VR representation 140 in the corresponding area can be increased. In another example, when a trainee provides instructions and receives tools, the resolution of the VR representation 140 can be reduced, and sensor priorities can be assigned to large-scale scene tracking. Additional simulations of a virtual assistant can be added to the VR representation 140. When a trainee applies manual procedures on the modeled airway 111, head 112, and jaw 113, the corresponding electromagnetic sensors 114 can receive priorities to influence the VR representation 140 of the applied procedure. When a trainee uses tool 120, when the procedure is external and the hand and fingers are visible, hand and / or finger tracker 104 can receive priority, pressure sensor 122 can receive priority regarding internal applications of tool 120 (e.g., insertion of an ET catheter or laryngoscope blade), and when the application is external and the hand is not easily tracked, such as when the hand is under a resuscitator, other sensors (e.g., flexion sensor 122 on manual resuscitator 120B) can receive priority. Alternatively or additionally, gesture recognition can be used to enhance the trainee's application of specific procedures.

[0058] The airway management training system 100 and the virtual reality system 150 may include a computing device 154 or components thereof, such as a processor (see below for example). Figure 7 It is configured to execute the published procedures and system responses to trainees, continuously adjust the VR representation 140, and manage the evaluation of trainees' actions.

[0059] Figure 6 This is a high-level flowchart illustrating an airway management training method 200 according to some embodiments of the present invention. The method stages can be performed with respect to the aforementioned airway management training system 100, which may optionally be configured to implement method 200. Method 200 may be implemented at least partially by at least one computer processor. Some embodiments include a computer program product comprising a computer-readable storage medium having a computer-readable program contained therein, and configured to perform relevant stages of method 200 (e.g., see below). Figure 7 Method 200 may include the following stages, regardless of their order.

[0060] Airway management training method 200 may include training airway management in a virtual reality environment (phase 205); using a passive physical patient model represented in the virtual reality environment and at least one airway management tool (phase 210), wherein the physical patient model has a modeled airway, a head movable from one side to the other, and an openable jaw, and at least one airway management tool has at least one pressure sensor; measuring the relative position of the head and jaw and the relative position of at least one airway management tool relative to the modeled airway (phase 220); and providing a trainee with a VR scene in the virtual reality environment, the scene including at least one VR patient corresponding to the physical patient model (using multiple electromagnetic sensors associated with the physical patient model), and a VR representation of a medical procedure performed by the trainee on the patient model, the VR representation including at least one airway management tool and the trainee's hand (phase 230). VR representation may include at least showing the position and orientation of at least one airway management tool relative to the airway and the hands of a trainee operating at least one airway management tool (stage 240), and visually and / or using tactile cues to indicate the pressure applied to the modeled airway by at least one airway management tool (stage 241).

[0061] The airway management training method 200 may further include at least tracking the position of a physical patient model, tracking the position and orientation of at least one airway management tool, and tracking the trainee's hand (stage 222).

[0062] The airway management training method 200 may further include dynamically identifying treatment situations performed by the trainee (stage 224) and dynamically managing situation-related hierarchies between sensors, which maintains the continuity and coherence of the VR representation relative to the identified treatment situation (stage 226).

[0063] In some embodiments, at least one airway management tool includes a laryngoscope, at least one pressure sensor on its blade, and method 200 further includes providing the trainee with VR feedback (phase 242) on pressure measurements applied by the laryngoscope blade to a modeled airway.

[0064] In some embodiments, the at least one airway management tool includes a manual resuscitator, the at least one pressure sensor is located on its interface tube, at least one pressure and / or flexure sensor is located on at least a portion of its circumference, and method 200 further includes providing the trainee with measurements of the pressure applied by the trainee to the manual resuscitator (and / or the trainee's flexure of the manual resuscitator) and feedback on the degree of airtightness between the manual resuscitator and the mouth of the passive patient model (stage 244).

[0065] Airway management training method 200 may further include displaying or representing the trainee’s hand in correspondence with detected pressure and / or deflection applied to the circumference of the manual resuscitator (stage 246).

[0066] The airway management training method 200 may further include the adjustable display or representation of a multi-purpose tool, as at least one airway management tool in the VR representation (stage 248), such as at least one of a scalpel, forceps, catheter, ET (endotracheal) catheter, aspirator, stethoscope, carbon dioxide detector, and pulse oximeter.

[0067] Figure 7 This is a high-level block diagram of an exemplary computing device 154 that can be used in embodiments of the present invention. The computing device 154 may include a controller or processor 173, which may be or may include, for example, one or more central processing unit processors (CPUs), one or more graphics processing units (GPUs or general-purpose GPUs - GPGPUs), a chip or any suitable computing or computing device, an operating system 171, a memory 172, a storage device 175, an input device 176, and an output device 177. The airway management training system 100 and the virtual reality system 150 may be or may include, for example, Figure 7 The computer system shown.

[0068] Operating system 171 may be or may include any code segment designed and / or configured to perform tasks related to coordinating, scheduling, arbitrating, supervising, controlling, or otherwise managing the operation of computing device 154, such as the execution of a scheduler. Memory 172 may be or may include, for example, random access memory (RAM), read-only memory (ROM), dynamic RAM (DRAM), synchronous DRAM (SD-RAM), dual data rate (DDR) memory chips, flash memory, volatile memory, non-volatile memory, cache memory, buffers, short-term memory cells, long-term memory cells, or other suitable memory cells or storage device cells. Memory 172 may be or may include multiple possibly different memory cells. Memory 172 may store, for example, instructions for executing methods (e.g., code 174), and / or data such as user responses, interrupts, etc.

[0069] Executable code 174 can be any executable code, such as an application, program, process, task, or script. Executable code 174 can be executed by controller 173 under the control of operating system 171. For example, according to embodiments of the invention, when executed, executable code 174 can cause the generation or compilation of computer code, or application execution such as VR execution or inference. Executable code 174 can be code generated by the methods described herein. For the various modules and functions described herein, one or more computing devices 154 or components of computing device 154 can be used. Devices including components similar to or different from those included in computing device 154 can be used and can be connected to a network and used as a system. One or more processors 173 can be configured to implement embodiments of the invention by, for example, executing software or code.

[0070] Storage device 175 may be or may include, for example, a hard disk drive, floppy disk drive, optical disc (CD) drive, CD-R drive, Universal Serial Bus (USB) device, or other suitable removable and / or fixed storage unit. Data such as instructions, code, VR model data, parameters, etc., may be stored in storage device 175 and may be loaded from storage device 175 into memory 172, where it may be processed by controller 173. In some embodiments, in Figure 7 Some of the components shown can be omitted.

[0071] Input device 176 may be, for example, a mouse, keyboard, touchscreen, or touchpad, or any suitable input device. It will be appreciated that, as shown in block 176, any suitable number of input devices may be operatively connected to computing device 154. Output device 177 may include one or more displays, speakers, and / or any other suitable output devices. It will be appreciated that, as shown in block 177, any suitable number of output devices may be operatively connected to computing device 154. Any suitable input / output (I / O) device may be connected to computing device 154; for example, a wired or wireless network interface card (NIC), modem, printer or fax machine, universal serial bus (USB) device, or external hard drive may be included in input device 176 and / or output device 177.

[0072] Embodiments of the present invention may include one or more articles (e.g., memory 172 or storage device 175) that encode, include, or store instructions (e.g., computer-executable instructions), such as a computer or processor non-transitory readable medium or a computer or processor non-transitory storage medium, such as a memory, disk drive, or USB flash drive, which, when executed by a processor or controller, perform the methods disclosed herein.

[0073] The foregoing description of various aspects of the invention has been based on flowchart illustrations and / or partial illustrations of methods, apparatus (systems), and computer program products according to embodiments of the invention. It is to be understood that each part of the flowchart illustrations and / or partial illustrations, as well as combinations of multiple parts in the flowchart illustrations and / or partial illustrations, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus for manufacturing machines, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create instructions for implementing the functions / actions specified in or in portions of the flowchart illustrations and / or partial illustrations.

[0074] These computer program instructions may also be stored in a computer-readable medium that can direct a computer, other programmable data processing apparatus or other device to operate in a particular manner, causing the instructions stored in the computer-readable medium to produce an article of manufacture, including instructions that implement the functions / actions specified in the flowcharts and / or partial diagrams or portions thereof.

[0075] Computer program instructions may also be loaded onto a computer, other programmable data processing apparatus or other equipment to cause a series of operational steps to be performed on the computer, other programmable apparatus or other equipment, thereby producing a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide for implementing a flowchart and / or a partial illustration of a process or a process for which a function / action is specified in a portion thereof.

[0076] The aforementioned flowcharts and diagrams illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each part in the flowchart or partial diagram may represent a module, segment, or portion of code, which includes one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions mentioned in this part may not appear in the order shown in the diagram. For example, two parts shown consecutively may actually be executed substantially simultaneously, or these parts may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each part in the partial diagrams and / or flowchart illustrations, and combinations of multiple parts in the partial diagrams and / or flowchart illustrations, may be implemented by a system based on dedicated hardware, or a combination of dedicated hardware and computer instructions, that performs the specified function or action.

[0077] In the above description, embodiments are examples or implementations of the invention. Various appearances of "an embodiment," "an embodiment," "some embodiments," or "a number of embodiments" do not necessarily refer to the same embodiment. Although various features of the invention may be described in the context of a single embodiment, these features may also be provided individually or in any suitable combination. Conversely, although the invention may be described herein in the context of a single embodiment for clarity, the invention may also be implemented in a single embodiment. Some embodiments of the invention may include features different from those of the embodiments disclosed above, and some embodiments may incorporate elements from other embodiments disclosed above. The disclosure of elements of the invention in the context of a particular embodiment should not be construed as limiting its use to a single particular embodiment. Furthermore, it should be understood that the invention may be carried out or practiced in different ways, and the invention may be implemented in some embodiments other than those outlined in the above description.

[0078] This invention is not limited to these illustrations or corresponding descriptions. For example, the process does not need to move through every shown box or state, or move in the exact same order as illustrated and described. The technical and scientific terms used herein have the meanings as commonly understood by one of ordinary skill in the art to which this invention pertains, unless otherwise defined. While the invention has been described with respect to a limited number of embodiments, these should not be construed as limiting the scope of the invention, but rather as examples of some preferred embodiments. Other possible variations, modifications, and applications also fall within the scope of the invention. Accordingly, the scope of the invention should not be limited by what has been described to date, but rather by the appended claims and their legal equivalents.

Claims

1. An airway management training system, comprising: A physical patient model with an airway, a head that can be moved from one side to the other, and an openable jaw. Multiple electromagnetic sensors are configured to measure the relative position of the head and the jaw, and the relative position of at least one airway management tool relative to the airway. as well as A virtual reality (VR) system configured to provide a trainee with a VR representation of a scene, a VR representation of at least one patient corresponding to the physical patient model, a VR representation of the at least one airway management tool, and a VR representation of the trainee's hand operating the at least one airway management tool. Wherein, VR represents the relative position, at least in part, based on measurements taken by the electromagnetic sensor.

2. The airway management training system according to claim 1, wherein: The virtual reality system includes optical sensors configured to track at least the position of the physical patient model, the position and orientation of the at least one airway management tool, and the trainee's hand. The virtual reality system is configured to dynamically identify treatment situations performed by the trainee and dynamically manage situation-related hierarchies between the optical sensors, the hierarchies maintaining the continuity and coherence of the VR representation of the scene relative to the identified treatment situations.

3. The airway management training system according to claim 1, wherein, The VR representation of the scene associates the visual representation of the at least one airway management tool with the airway.

4. The airway management training system according to claim 1, wherein, The VR representation of the scene indicates the pressure applied to the airway by the at least one airway management tool.

5. The airway management training system according to claim 1, further comprising the at least one airway management tool, wherein the at least one airway management tool has at least one pressure sensor.

6. The airway management training system according to claim 5, wherein, The at least one airway management tool includes a laryngoscope, and the at least one pressure sensor is located on the blade of the laryngoscope.

7. The airway management training system according to claim 1, wherein, The at least one airway management tool includes a manual resuscitator having at least one pressure sensor and / or at least one flexure sensor on at least a portion of its circumference.

8. The airway management training system according to claim 7, wherein, The virtual reality system is configured to represent the trainee's hand in accordance with the detected pressure or deflection applied to the circumference of the manual resuscitator.

9. The airway management training system according to any one of claims 5-8, wherein, The at least one airway management tool includes a multi-purpose tool, which is visually represented as at least one of a scalpel, forceps, catheter, ET (endotracheal) catheter, aspirator, stethoscope, carbon dioxide detector, and pulse oximeter.

10. An airway management training method, comprising: Training airway management in a virtual reality (VR) environment; The VR environment is enhanced using a physical patient model and at least one airway management tool, wherein the physical patient model has an airway, a head that can be moved from one side to the other, and an openable jaw. Using multiple electromagnetic sensors associated with the passive patient model, the relative positions of the head and jaw, and the relative positions of the at least one airway management tool with respect to the airway, are measured; and In the VR environment, a VR representation of the scene, a VR representation of the patient corresponding to the physical patient model, a VR representation of the at least one airway management tool, and a VR representation of the trainee's hand are displayed to the trainee. The VR represents, at least in part, a relative position as measured.

11. The airway management training method of claim 10, further comprising at least tracking the position of the physical patient model, the at least one airway management tool, and the trainee's hand.

12. The airway management training method of claim 10, further comprising dynamically identifying treatment situations performed by the trainee, and dynamically managing situation-related hierarchies between the sensors, the hierarchies maintaining the continuity and coherence of the VR representation relative to the identified treatment situation.

13. The airway management training method according to claim 10, wherein, The at least one airway management tool includes a laryngoscope and at least one pressure sensor on the blade of the laryngoscope, and the method further includes providing the trainee with VR feedback on a measurement of the pressure applied to the airway by the blade of the laryngoscope.

14. The airway management training method according to claim 10, wherein, The at least one airway management tool includes a manual resuscitator having at least one pressure sensor and / or at least one flexure sensor on at least a portion of its circumference, and the method further includes providing the trainee with measurements of the pressure applied by the trainee to the manual resuscitator and VR feedback on the degree of airtightness between the manual resuscitator and the mouth of the passive patient model.

15. The airway management training method of claim 14, further comprising representing the trainee’s hand as corresponding to a detected pressure or deflection applied to the circumference of the manual resuscitator.

16. The airway management training method according to any one of claims 10-15, further comprising visually representing the multi-purpose tool as the at least one airway management tool in an adjustable manner, as at least one of a scalpel, forceps, catheter, ET (endotracheal) catheter, aspirator, stethoscope, carbon dioxide detector, and pulse oximeter.

Images