Medical device imaging system and method

Abstract

A time-synchronized video imaging system (10) includes an environmental camera (32) that acquires environmental images of a patient while a video laryngoscope (12) acquires airway images of the patient. A monitor (30) receives the airway images from the video laryngoscope and the environmental images from the environmental camera. The monitor time-synchronizes the airway images and the environmental images to one another to generate a combined view video file of time-synchronized airway and environmental images to allow viewing of macro or environmental effects of intubation of the patient.

Description

Background Technology

[0001] This disclosure relates generally to medical devices, and more specifically to medical device imaging systems and methods.

[0002] This section is intended to introduce the reader to various aspects of the technology that may be related to the aspects of this disclosure described below and / or claimed. It is believed that this discussion will help provide the reader with background information to facilitate a better understanding of the various aspects of this disclosure. Therefore, it should be understood that these statements should be read in this context and not as an endorsement of prior art.

[0003] During patient treatment, catheters or other medical devices can be used to control the flow of air, food, fluids, or other substances into the patient's body. For example, an endotracheal tube can be used, for instance, during mechanical ventilation to control the flow of air or other gases through the patient's trachea and into the lungs. Such endotracheal tubes can include endotracheal tubes, tracheostomy tubes, or transtracheal tubes. A laryngoscope is typically used during intubation (the insertion of an endotracheal tube into the patient's trachea).

[0004] Video laryngoscopy is a form of indirect laryngoscopy in which a medical professional (such as a doctor, therapist, nurse, or other practitioner) views video images of the patient's larynx on a display screen. Video laryngoscopy may include an integrated display that is in the laryngoscope operator's line of sight, allowing the patient's airway to be viewed in real time on the display screen to facilitate navigation within the airway and insertion of an endotracheal tube. Summary of the Invention

[0005] Some implementations are outlined below. These implementations are not intended to limit the scope of this disclosure. In fact, this disclosure may cover a variety of forms that may be similar to or different from the implementations set forth below.

[0006] In one embodiment, a video imaging system is provided, including a monitor with a display screen. The video laryngoscope of the video imaging system includes a camera for acquiring images of the patient's airway. An environmental camera of the video imaging system acquires environmental images of the patient's environment. The monitor of the video imaging system operates to: receive airway images from the video laryngoscope and environmental images from the environmental camera; synchronize the airway images and environmental images with each other; and combine the synchronized airway images and environmental images into a combined video file.

[0007] In one embodiment, a medical device system includes a video laryngoscope for acquiring images of a patient's airway and an environmental camera for acquiring images of the environment. The system's monitor operates to: receive airway images from the video laryngoscope; receive environmental images from the environmental camera; determine a first time difference between the monitor's clock and the video laryngoscope's clock, and a second time difference between the monitor's clock and the environmental camera's clock; shift the airway images relative to the environmental images based on the first and second differences to synchronize the airway images and the environmental images with each other in time; and generate a video file including the time-synchronized airway images and environmental images.

[0008] In one embodiment, a method includes the following steps: receiving airway images and laryngoscope clock information from a video laryngoscope at a monitor; acquiring environmental images from an environmental camera on the monitor facing the room, wherein the environmental images and airway images are acquired simultaneously; determining a relative time difference between the laryngoscope clock information and the monitor clock information; synchronizing the airway images and environmental images in time based on the relative time difference; and generating a composite view video file having a combined view of the time-synchronized airway images and environmental images. Attached Figure Description

[0009] The advantages of the disclosed technology will become apparent from the following detailed description and with reference to the accompanying drawings, in which:

[0010] Figure 1 This is a schematic diagram of a patient environment including a video imaging system according to an embodiment of this disclosure;

[0011] Figure 2 This is a schematic diagram of certain components of a video imaging system according to an embodiment of this disclosure;

[0012] Figure 3 This is a schematic diagram of the communication path of a video imaging system according to an embodiment of this disclosure;

[0013] Figure 4 This is a flowchart of a method for using a video imaging system in accordance with an embodiment of this disclosure;

[0014] Figure 5 This is an example display showing a combined view of airway images and environmental images according to an embodiment of this disclosure;

[0015] Figure 6 This is an example display showing a combined view of airway images and environmental images according to an embodiment of this disclosure;

[0016] Figure 7 This is a flowchart of a method for using a video imaging system in accordance with an embodiment of this disclosure;

[0017] Figure 8 This is an example display showing a combined view of airway images and environmental images according to an embodiment of this disclosure;

[0018] Figure 9 This is an example display showing a combined view of airway images and environmental images according to an embodiment of this disclosure;

[0019] Figure 10 This is an example display showing a combined view of airway images and environmental images according to an embodiment of this disclosure; and

[0020] Figure 11 This is a block diagram of a video imaging system according to an embodiment of the present disclosure. Detailed Implementation

[0021] Healthcare professionals can use a laryngoscope to view a patient's oral cavity to facilitate the insertion of an endotracheal tube (e.g., an endotracheal tube, tracheostomy tube, or transtracheal tube) through the patient's mouth or nose and into the patient's trachea as part of an intubation procedure. A video laryngoscope includes a camera on a portion of the laryngoscope that is inserted into the patient's oral cavity to obtain images of the oral cavity (e.g., still images and / or moving images, such as video). These images can then be displayed during the intubation procedure to allow healthcare professionals to visualize the oral cavity and facilitate manipulation and insertion of the endotracheal tube. The video laryngoscope may include an integrated display screen located in the laryngoscope operator's line of sight but may not be highly visible to other clinicians in the room. During the operation of the video laryngoscope, the acquired images can be communicated, for example, by real-time streaming (e.g., via a wireless connection), to a separate display screen showing the video laryngoscope images to allow other clinicians to view them. The separate display screen can be one or more screens located in the procedure room or a separate room (such as a transfer room, nursing station, or other area). In this way, the patient's airway status is visible to other clinicians who can view these images as part of their training, providing relevant information or advice to the laryngoscopy operator and / or preparing for subsequent procedures based on the progress of intubation. In one example, the surgical team in the transit area can be moved into position and prepare the initial steps of the surgical procedure based on video laryngoscopy images indicating near completion of intubation. In another example, the laryngoscopy operator can be informed of potential intubation difficulties based on the visible airway anatomy.

[0022] During intubation, images from a video laryngoscopy are obtained from inside the patient (such as inside the patient's mouth or upper airway), and therefore these images provide only a limited perspective of the intervention being performed on the patient or other events occurring in the room. In video laryngoscopy images obtained from inside the patient's mouth or airway, events or interventions outside the patient are not necessarily apparent. Such external or macroscopic events or backgrounds can be patient movement (such as coughing, instinctive reactions, or other movements), administration of sedation or other medications to the patient, restraint or positioning of the patient, concurrent medical events in the patient (such as emergency intervention, local preparation, surgical incision and / or incision closure, a single stage of surgery, or other events), the laryngoscope operator's technique, the type of airway tube or other instrument used, communication and interaction between clinicians in the patient setting, the presence and location (including arrival or departure) of members of the medical team, the state or layout of the room, changes in the room environment, and many other external or macroscopic events or backgrounds. In one example, an emergency intubation occurs while concurrent stabilization measures are being performed on the patient, which pushes the patient and displaces the airway, potentially prolonging the total intubation time or increasing the risk of esophageal intubation. Providing patients with information about their external environment during intubation procedures can provide training benefits to clinicians, better inform other members of the medical team, and enhance the individual patient’s medical record.

[0023] This document provides a technique for aligning images acquired by a video laryngoscope during an intubation procedure with environmental images acquired by a camera in the room or other devices, illustrating the situational events. This technique synchronizes images from devices that are not time-synchronized with each other (e.g., different devices with corresponding clocks (having different times)). In some embodiments, one or all devices in the system are not set to the correct local time. In one embodiment, the system includes a video laryngoscope acquiring images of the patient's airway and an environmental camera simultaneously acquiring environmental images from the procedure room. The airway and environmental images are transmitted to a processor that combines the images into a dual-image, time-synchronized view. In one embodiment, the processor synchronizes the images with each other. For example, synchronization may include determining the time difference between the system clock on the processor and the local clock on each of the video laryngoscope and the environmental camera. The system uses these time differences to synchronize the environmental images with the airway images so that they both show what is happening simultaneously. The processor presents a combined view (e.g., a video feed or video file) showing the two video images playing together, so that the environmental background in the room can be seen along with the progress of intubation within the patient's airway. This combined view can be used in real time during intubation or retrospectively after intubation to assess intubation success and improve clinical interventions and patient records. The disclosed technique addresses the device-specific challenges associated with evaluating images acquired from distributed medical devices that are not time-synchronized with each other. Furthermore, the techniques used to align these acquired images do not synchronize the device clocks with each other (e.g., by updating or changing different clocks to the same time). Instead, the images acquired from these devices are synchronized together based on differences in associated clock information from the respective devices. Therefore, this technique involves the time synchronization of images without requiring the synchronization of clocks on various distributed cameras, observation instruments, and devices.

[0024] While embodiments of this disclosure are discussed in the context of video laryngoscopes, it should be understood that this technology can be used in conjunction with other medical devices (such as endoscopes or other types of internal observation instruments) that acquire images of the patient's interior to synchronize or time-align images of the external environment with internal anatomical images. The disclosed technology addresses the technical computational and communication challenges associated with medical devices that do not operate on the same global / synchronized clock. For example, peripheral medical devices such as video laryngoscopes may be manufactured in a first time zone and then distributed globally to time zones worldwide. Therefore, the internal clock of the medical device does not need to be synchronized with the local time, and some medical devices may not be programmed to allow updates to their internal clocks. Furthermore, to reduce battery size, improve portability, or defend against network security vulnerabilities, some medical devices may have limited communication capabilities and may not be able to receive external synchronization commands. Therefore, multiple devices in a system may have conflicting clock information. This technology addresses such situations to facilitate time synchronization of images acquired by different devices.

[0025] Figure 1 This is a schematic diagram of a patient environment including a video imaging system 10, which operates to synchronize airway images and environmental images of the patient environment and generate a composite video file with a combined view of the synchronized images. The patient environment can be any room where intubation is performed, such as a medical suite, operating room or other procedural room in a hospital or other care facility, a patient recovery room, an emergency intubation site, or other environment. The video imaging system 10 may include a video laryngoscope 12 for airway visualization of the patient during operation. The video imaging system 10 may be used additionally or alternatively with other patient visualization instruments that acquire patient images (e.g., images of the patient's interior).

[0026] The laryngoscope operator 13 holds the laryngoscope body 14, which is connected to a display section 16 having a display screen 18. Acquired airway images 20 are displayed on the display screen 18. The video laryngoscope 12 can be used as part of an intubation procedure to advance an endotracheal tube into the patient's airway to secure the airway for mechanical ventilation. Thus, the operator 13 of the video laryngoscope 12 performs intubation and directly manipulates the endotracheal tube within the patient's airway, while other clinicians in the patient setting assist the laryngoscope operator, monitor the patient's condition, prepare or adjust medical equipment in the patient setting, and / or wait until the airway is secured for further procedures or interventions.

[0027] These airway images 20 can also be stored in memory (on the video laryngoscope 12 or a separate memory device) and linked to the patient's medical records. The acquired airway images 20 can reveal lesions, challenging airway anatomy, movement of the advanced endotracheal tube and / or endoscope, improper esophageal intubation, and other factors that may affect patient care. These images can be viewed for patient diagnosis or care, or to train operators in intubation techniques.

[0028] The airway image 20 acquired by the video laryngoscope 12 is visible on the laryngoscope display screen 18, located in the operator's line of sight. Furthermore, the acquired airway image 20 is transmitted to a monitor 30, such as a patient vital signs monitor, laptop computer, tablet computer, display screen, or other computing device. The monitor 30 also receives images from one or more environmental cameras 32 that acquire environmental images 33. Environmental images 33 are images acquired outside the patient and include a contextual view of the patient and / or the environment. Environmental images may include views of the protocol room, medical clinicians (such as laryngoscope operator 13 or assistant clinicians), and other devices or medical equipment in the environment. In one embodiment, environmental images 33 may include the video laryngoscope 12 used to capture information related to the intubation technique of the laryngoscope operator 13, including operator arm position, endotracheal tube angle, operator pushing movements associated with advancing the endotracheal tube, and / or the operator's use of available inputs to the video laryngoscope 12. The monitor 30 receives airway images 20 and environmental images 33, synchronizes them (so that they show simultaneous events), and generates a combined view 34 of the synchronized images and / or a combined view video file. The combined view 34 may be displayed on the monitor 30 and / or stored as a video file, for example, together with the patient's recording.

[0029] In the illustrated embodiment, monitor 30 is shown as a freestanding and portable display located on a stand 36 beside the patient's bed. However, monitor 30 may be a computer, laptop computer, tablet computer (e.g., a tablet computer connected to a portable stand), multi-parameter patient monitor, multi-functional medical device or instrument, networked computer outside the room, mobile device, cloud computing device communicating with a graphical user interface on an operator's local device, or other device capable of receiving and synchronizing airway images 20 and environmental images 33. Monitor 30 may be located in the patient environment or may be a remote device. If monitor 30 is located in the room, in one embodiment, one or more environmental cameras of environmental cameras 32 may also be mounted on / integrated on or in the housing of monitor 30.

[0030] The combined view 34 can be displayed on a screen 38 integrated with the monitor, streamed in real time to one or more separate screens 35 visible to other clinicians in the environment, and / or stored in the memory of the monitor 30 for later viewing. Furthermore, the monitor 30 may or may not include the integrated display 38 or the environmental camera 32.

[0031] Figure 2An embodiment is shown in which a camera 32 is included on a portable upright wheeled stand 36 to capture environmental images 33. In this configuration, the camera 32 can be moved around the room, for example by being propelled by wheels, such that the view from the environmental camera 32 captures the patient's view or other desired view during intubation or other medical procedures.

[0032] In an illustrated embodiment, camera 32 may be an integrated camera of a mobile device or tablet computer 40, which includes a camera assembly having a rear-facing camera 32a and / or a front-facing camera 32b as shown. In one embodiment, system 10 allows selection of one or both of the rear-facing camera 32a or the front-facing camera 32b as the ambient camera 32. The ambient camera 32 may be additionally or alternatively implemented as a portable camera, such as a removable webcam or a front-facing camera on a laptop computer. The ambient camera 32 may be a fixed-position camera mounted in a room (e.g., mounted on a fixture in a room, in a corner, on a ceiling, or on a wall surface), or it may be mounted on a portable stand 36, such as a wheeled stand, which includes an adjustable base to allow the ambient camera 32 to be moved and repositioned to a desired location in the environment. In one embodiment, system 10 may detect available cameras 32 in the environment and allow activation of the selected camera and transmission of acquired images to monitor 30. For example, system 10 may allow the ambient camera 32 to be integrated in a plug-and-play manner.

[0033] The tablet computer 40 may provide at least one environmental camera 32, for example, as an integrated camera of system 10, and display functionality via an integrated display 38. For example, the display 38 may display a live view of environmental image 33 to facilitate the desired positioning of camera 32. The live view of environmental image 33 is updated as the portable stand 36 is repositioned within the environment. The operator can use the displayed view of environmental image 33 as a guide to adjust the stand 36 for the desired field of view. As shown, the display 38 may additionally or alternatively display a combined view 34 generated by the monitor 30.

[0034] Furthermore, in some embodiments, the tablet computer 40 may perform one or more functions of the monitor 30. In one example, an operator may attach a mobile device or portable tablet computer 40 to a receiving slot on a portable stand 36. The portable tablet computer 40 may operate as the monitor 30, for example, via operation of a software application (e.g., an "app") or operation of an application-specific integrated circuit dedicated to the portable tablet computer 40. In one embodiment, one or more functions of the monitor 30 are implemented by a separate device, and the portable tablet computer 40 serves as a camera 32 and optionally as a display, transmitting captured environmental images 33 to the monitor 30.

[0035] In one embodiment, the bracket 36 may serve as a component housing the monitor 30. In this case, the bracket 36 may receive environmental and airway images, synchronize them, combine them into a composite video file, and display the composite video file 34 on the display screen 38. Thus, in one embodiment, the system 10 allows the environmental camera 32 and the display screen 38 of the tablet computer 40 to be removably coupled to the bracket 36, and the tablet computer 40 may operate as a modular or portable device component of the system 10. In one embodiment, the individual monitor 30 generates a composite view video 34 comprising time-aligned airway and environmental images, and the composite view video 34 may be transmitted to the display screen 38.

[0036] In the illustrated implementation, camera 32 (e.g., rear-facing camera 32a and / or front-facing camera 32b) can be oriented to face the desired field of view toward the patient environment. Portable stand 36 can be maneuvered around the room using wheels, and portable stand 36 may also include other adjustment features, such as height adjustment or tilt adjustment to tilt the housing 42 of tablet computer 40 and / or camera 32. The portable stand allows for real-time adjustments if the field of view is obstructed when clinicians move into and out of the patient procedure site. Alternatively or additionally, environmental images 33 can be streamed to a separate display screen 35 (see [link to relevant documentation]). Figure 1 ).

[0037] Positioning camera 32 on portable stand 36 allows camera 32 to be moved to obtain the desired view at different steps or stages of the clinical procedure. For example, intubation may involve fewer clinicians relative to the subsequent surgical procedure, and the desired view during intubation may only show the patient's torso and head area. However, the desired view during surgical procedure may show the entire patient and the clinicians positioned around the patient. After intubation, the clinician may assume their desired position, and portable stand 36 and associated camera 32 can be manipulated to capture the desired view of the patient's environment during the surgical procedure without interfering with the clinician's movement. As discussed herein, portable environment camera 32 may be one of a group of cameras 32 capturing environmental images 33.

[0038] In one example, the portable stand 36 and the portable tablet computer 40 remain with the patient, and the portable stand 36 can move with the patient if the patient moves to a different procedure room. Therefore, the system 10 acquires environmental images 33 at various locations via the environmental camera 32 of the portable tablet computer 40.

[0039] The monitor 30 of system 10 receives airway images 20 acquired by the camera 50 of video laryngoscope 12 to synchronize with environmental images 33. The airway images 20 are communicated during operation of the video laryngoscope 12, for example, in response to an actuation of a power button 62 that enables a medical professional to turn the video laryngoscope 12 on and off, or after interaction with an input device 64 (such as a touch sensor or proximity sensor, e.g., a capacitive sensor, proximity sensor, etc.) that enables a medical professional operating the video laryngoscope 12 to provide input or commands.

[0040] In the illustrated embodiment, the video laryngoscope 12 may include an identifiable marker 66 that can be detected by the environmental camera 32 (e.g., based on image or feature recognition) and positioned on the body 14, display portion 16, and / or blade 68 of the video laryngoscope 12 to facilitate capturing an image of the patient's airway inlet in the environmental image 33. The identifiable marker may include a barcode, QR code, reference, printed symbol, or text, or other camera-recognizable markers positioned on the outer surface of the video laryngoscope 12.

[0041] In some embodiments, multiple environmental cameras 32 may be present in an environment such as a patient setting, and the monitor 30 may allow the user to select one or more environmental cameras 32 to provide environmental images used to generate a combined view video 34 as provided herein. While the illustrated combined view 34 includes airway images 20 and environmental images 33 from a single camera source, the environmental images 33 may include two, three, or four video streams all placed together. The multi-source environmental images 33 may be simultaneously displayed in a portion of a display dedicated to the environmental images 33, such as in a gallery or panel view. In another embodiment, video feeds from different environmental cameras 32 in the environment may be displayed in an alternating manner, switching between different cameras based on user selection or quality metrics. For example, for one part of a procedure, a first environmental camera 32 may provide the desired view, while in a later part of the procedure, a second environmental camera 32 may provide the desired view. The operator can view the available video feeds from the environmental cameras 32 in the user interface of the monitor 30 to select the desired camera view. The combined view 34 may be generated based on user selection. In another embodiment, camera feed can be selected based on the identification of identifiable landmarks 66 in the acquired environmental image 33. If the identifiable landmarks 66 are no longer distinguishable, the system 10 can switch to a different environmental camera 32 until the identifiable landmarks 66 are visible in the environmental image 33.

[0042] Figure 3This is a schematic diagram of the data communication path in the video imaging system 10 used to facilitate the synchronization of data from different devices, which may include acquired images and tagged events. In use, system 10 enables the transmission of acquired airway images from video laryngoscope 12 to one or more individual (e.g., remote) devices or systems, such as monitors or flat panel displays 38 or individual displays 35, monitor 30 (e.g., remote monitors), computing systems, and / or hospital data storage systems. For example, video laryngoscope 12 acquires airway images, which may be locally stored (e.g., stored in a storage device housed in body 14) for later access, viewing, or transmission, or may be streamed in real time to monitor 30 or other external devices. In some embodiments, video laryngoscope 12 communicates with monitor 30 and / or transmits data (e.g., wirelessly streaming images substantially in real time) to monitor 30. In some implementations, the video laryngoscope 12 automatically transmits data from its storage device to the monitor 30, and / or transmits the data to one or more other remote devices or systems at certain times (e.g., when the video laryngoscope 12 and / or the monitor 30 are turned on or off, or when user input is received).

[0043] The video laryngoscope 12 and the environmental camera 32 can communicate with the monitor 30 and / or other remote devices or systems via any of a variety of technologies. For example, the video laryngoscope 12, the environmental camera 32, and the monitor 30 may include communication devices (e.g., wireless transceivers) that establish wireless communication with each other using any suitable protocol or be operatively coupled to such communication devices.

[0044] The video laryngoscope 12 includes an internal laryngoscope clock 102 that generates laryngoscope clock information 100, such as timestamp information encoded using acquired image files. The laryngoscope clock information 100 is generated by the internal clock 102 on the video laryngoscope 12, which may not be set to the correct local time and / or may not match the internal clocks of other devices in the system. For example, the laryngoscope clock 102 may be set to a different time zone, may differ from the local time by several seconds or minutes, may be incorrect for other reasons, or may be the correct local time but still not match other devices in the system whose clocks are not matched to the correct local time.

[0045] In one implementation, the video laryngoscope sends laryngoscope clock information 100, such as timestamp information, to the monitor 30. This could be a single time at the beginning or end of the video file, a time in the middle of the video file, the start time plus the duration of the video, a duration stamp throughout the video (e.g., at each image frame), or other time information that associates the acquired airway image 20 with the laryngoscope clock 102.

[0046] The laryngoscope clock information 100, along with any associated data, is transmitted to the monitor 30 via the communication paths disclosed herein. In one example, a user may initiate the recording or transmission of airway images using a start or calibration user input 105 (e.g., an actuated thumb-operable control), which includes the laryngoscope clock information 100 associated with the time of the user input. During intubation, the laryngoscope operator may also provide patient event markers associated with patient events such as coughing or patient movement, which are relevant to the analysis of the intubation protocol, via user input 105. Patient event markers may include timestamp information generated by the laryngoscope clock 102.

[0047] During intubation, the environmental camera 32 also records video images. The environmental camera 32 also includes a local camera clock 106, which may match or may not match the actual or absolute local time according to standard local clock information 114. The environmental camera 32 sends environmental camera clock information 104 to the monitor 30. The monitor 30 then uses the local clock information from the contributing device—laryngoscopy clock information 100 and environmental camera clock information 104—to align the airway video and environmental video to synchronized time, so that events occurring in the room and with the patient occur simultaneously in each video.

[0048] In embodiments where the environmental camera 32 is integrated with the monitor 30, the camera clock information 104 is the same as the monitor clock information 110, and the monitor 110 does not necessarily receive separate camera clock information 104 from the environmental camera 32. That is, the system 10 includes at least one or both of the video laryngoscope clock 102 and the environmental camera clock 106 or the monitor clock 112. It should be understood that the disclosed synchronization can be relative synchronization with the monitor 30, wherein the monitor clock 112 is a control time, regardless of whether the monitor clock 112 corresponds to the real local clock information 114. In other embodiments, synchronization is set to the real local clock information 114 received by the monitor 30, which in turn sets the monitor clock 112.

[0049] In one embodiment, the monitor receives an airway video image 20 including laryngoscope clock information 104 and an environmental video image 33 including camera clock information 104. The monitor 30 can automatically query the video laryngoscope 12 for the current time based on the laryngoscope clock 102, and the video laryngoscope 12 sends the current time to the monitor 102. Alternatively, the video laryngoscope can send the current time to initiate synchronization without being queried by the monitor. In response, the monitor 30 determines a first time difference or a first delta time between the monitor clock time and the video laryngoscope clock time. When the environmental camera 32 has a separate clock 106, the monitor 30 queries the environmental camera 32 for its current time. In response to receiving the current time from the camera clock 106, the monitor 30 determines a second time difference or a second delta time between the monitor time and the environmental clock time. Therefore, the monitor 30 is able to correct the two video image streams to the monitor's time and synchronize them.

[0050] Figure 4 The use of the laryngoscope system 10 according to the embodiments of this disclosure and with reference to Figures 1 to 3 A flowchart of the process of method 150, which features the characteristics discussed herein, is provided. Method 150 disclosed herein includes various steps represented by boxes. It should be noted that at least some steps of method 150 can be executed as an automated procedure by a system such as system 10. Although the flowchart shows the steps in a specific order, it should be understood that the steps can be performed in any suitable order, and some steps can be performed simultaneously where appropriate. Additionally, steps can be added to method 150 or omitted from the method. Furthermore, certain steps or portions of method 150 can be performed by separate devices. For example, a portion of method 150 can be performed by video laryngoscope 12, while other portions of method 150 can be performed by the processor of environmental camera 32 or monitoring instrument 30. Furthermore, within the scope of applying the steps of method 150 disclosed herein to images, it should be understood that images can be image data, processed images, or image files, and can be still images or video images.

[0051] Method 150 begins with the monitor receiving the current video laryngoscope time at the monitor (box 152). In some embodiments, where the ambient camera 32 does not share the monitor clock 112, method 150 includes receiving the current camera clock time at the monitor (box 154). The current laryngoscope clock time reflects the time sent at the video laryngoscope 12 in response to an inquiry from the monitor or automatically sent from the video laryngoscope 12 as part of initiating communication with the monitor 30. The monitor 30 may compare the current laryngoscope clock time with the current monitor clock time to identify a first difference (box 156), such as an increment time, reflecting the corresponding clock difference between the monitor 30 and the video laryngoscope 12 at a given time point, and compare the current camera clock time with the current monitor clock time to identify a second difference (box 156), such as a second increment time, reflecting the corresponding clock difference between the monitor 30 and the ambient camera 32 at a given time point. As disclosed herein, the ambient camera 32 may be integrated with or hardwired to the monitor 30 and therefore may share the monitor clock 112. In this case, there will be no difference between the camera clock time and the monitor clock time, and certain image time synchronization steps can be omitted in method 150.

[0052] The monitor 30 adjusts the time associated with the received airway image 20 from the video laryngoscope 12 based on a first identified difference (box 160). Adjusting the time may include shifting the timestamp information associated with the received airway image 20 to synchronize with the monitor's clock time. The monitor 30 may also adjust the time associated with the received environmental image 33 from the environmental camera 32 based on a second identified difference (box 162), such that the time is shifted to synchronize with the monitor's clock time. In operation, the monitor 30 generates a combined or composite view video of the time-aligned airway and environmental images (box 166). The monitor 30 provides a relative reference for the two image streams to be adjusted or shifted. If the frame rates of the different cameras are different, the frames can be interpolated to bring them together into the composite video file. The monitor uses the monitor / video laryngoscope, and if present, the relative difference between the monitor / environmental camera 32, to align the two image streams together. Once aligned, the image streams can be run forward together.

[0053] The combined view video may include consecutive combined views 34, where each combined view is a frame of airway image 20 displayed together with a frame of environmental image 33, the frame of which was acquired at the same absolute time point or the same time point according to monitoring clock 112. The combined view video may be displayed automatically as it is being generated, or it may be generated and stored in memory for later playback. Furthermore, the combined view video may be generated in real time or based on airway images 20 and environmental images previously acquired from a simultaneous time window.

[0054] Figure 5 As shown in the example frame of the combined view 34 provided herein, it illustrates an airway image 20 arranged side-by-side with an environmental image 33. It should be understood that other arrangements, such as vertical, overlapping, or picture-in-picture arrangements, are also conceivable. Figure 6 As shown. Back Figure 5 In the depicted example, frames of the combined view 34 are annotated with patient event markers 170, which may be marked at specific times or within time windows. For example, patient event marker 170 may be the result of user input (e.g., thumb-operable control) indicating patient movement, represented by dashed lines indicating movement relative to previous frames. Patient movement may be apparent in the airway image due to displacement 171 of the endotracheal tube 172 within the airway, while patient movement 174 may also be seen in the macroscopic view of the environmental image 33. Viewing the video file showing the combined view 34 may reveal the cause of the patient movement, such as coughing, intervention by the laryngoscope operator 13 or other clinicians, or insufficient sedation. The combined view 34 may also include a timer indicator set to the start of the procedure.

[0055] Figure 6 An alternative embodiment of the combined view 34 provided herein is shown, which illustrates an environmental image 33 embedded within a larger view of an airway image 20. It should be understood that other arrangements, such as vertical, overlapping, or picture-in-picture arrangements, are also conceivable. Figure 6 As shown. Back Figure 5 In the depicted example, frames of the combined view 34 are annotated with patient event markers 170, which may be marked at specific times or within time windows. For example, patient event marker 170 may be the result of user input (e.g., thumb-operable control) indicating patient movement, represented by dashed lines indicating movement relative to previous frames. Patient movement may be apparent in the airway image due to displacement of the advanced endotracheal tube 172 within the airway, and may also be seen in the macroscopic view of the environment image 33. Viewing the video file showing the combined view 34 may reveal the cause of the patient movement, such as coughing, intervention by the laryngoscope operator 13 or other clinicians, or insufficient sedation. The combined view 34 may also include a timer indicator set to the start of the procedure.

[0056] Figure 7 Use according to the implementation scheme of this disclosure Figure 1A flowchart of the method 200 of the laryngoscope system 10 is provided herein. The method 200 disclosed herein includes various steps indicated by boxes. It should be noted that at least some steps of method 200 can be performed as an automated procedure by a system such as system 10. Although the flowchart shows the steps in a specific order, it should be understood that the steps can be performed in any suitable order, and some steps can be performed simultaneously where appropriate. Additionally, steps can be added to or omitted from method 200. Furthermore, certain steps or portions of method 200 can be performed by separate devices. For example, a portion of method 200 can be performed by the video laryngoscope 12, while other portions of method 200 can be performed by the environmental camera 32, physiological sensors, or the processor of the monitor 30. Furthermore, within the scope of applying the steps of the method 200 disclosed herein to images, it should be understood that images can be image data, processed images, or image files, and can be still images or video images. Furthermore, within the scope of applying the steps of the method 200 disclosed herein to physiological monitoring data, it should be understood that the monitoring data can be raw data or processed data.

[0057] Method 200 uses a video laryngoscope 12 to acquire airway images (e.g., airway image 20). Figure 1 (Box 202) and transmits the airway image to the monitor 30 (Box 204) at the start. Simultaneously with acquiring the airway image, the environmental camera 32 acquires an environmental image showing the patient's environment (e.g., environmental image 33, ...). Figure 1 (Box 206). Physiological monitoring data is acquired by one or more medical sensors (Box 210) and transmitted to monitor 30 (Box 212). This data may include the patient's physiological parameters, such as vital signs, pulse rate, oxygen saturation, respiratory rate, temperature, blood pressure, and other parameters. Monitor 30 synchronizes the airway image 20 as provided herein with the environmental image 33 (Box 214) to generate a combined view video 34 (Box 216). The combined view video 34 is annotated with events extracted from the physiological monitoring data (Box 218). Monitor 30 may synchronize the physiological monitoring data to the synchronized airway image 20 and environmental image 33 such that physiological monitoring events are annotated on the combined view 34 at the time of occurrence. Based on the difference between the clock of the physiological monitoring device transmitting sensor data and the clock of monitor 30, the synchronization of physiological monitoring data may occur in a manner similar to that disclosed with respect to video laryngoscope 12 and / or environmental camera 32.

[0058] In one example, the annotations are based on monitoring data outside of a preset tolerance. Therefore, the monitor 30 can identify time periods associated with deviations in physiological parameters and annotate the composite view 34 to indicate that the deviation is occurring simultaneously with a specific portion of the composite view 34.

[0059] In one implementation, the system may also include an endoscope 175 (see Figure 8 Endoscope 175 can be a separate device that sends images to monitor 30, or endoscope 175 can be coupled to video laryngoscope 12 and send images to video laryngoscope, which then sends them to monitor. When endoscope 175 is coupled to video laryngoscope, the airway image 20 acquired by video laryngoscope and sent to monitor can be a combination of laryngoscope camera image 120 and endoscope image 122 together in a single video file, in which case endoscope image 122 is aligned with video laryngoscope clock information. Alternatively, endoscope image 122 can be sent separately to monitor 30 along with endoscope clock information from endoscope 175. Combined view 34 can be generated using one or both of laryngoscope camera image 120 and endoscope image 122.

[0060] Figure 8 This is an example illustration of an implementation using endoscopic airway images 20 from an endoscope coupled to a video laryngoscope 12, as shown in combined view 34. Endoscope 175 can be advanced into the patient's airway to capture endoscopic airway images 20, which are then provided to the coupled video laryngoscope 12. The video laryngoscope 12 may be able to display on display 18 images from a laryngoscope camera 50, endoscopic images from an endoscope camera, or both, and may alternate between display modes at the video laryngoscope 12 based on user input. Therefore, the video laryngoscope 12 can provide two sets of airway images 20 to the monitor 30. The first set of images includes laryngoscope camera images 120 and the second set includes endoscopic images 122. Figure 3 Each set of images is associated with clock information used to synchronize the images to a combined video file as disclosed herein (such as both being associated with laryngoscope clock information, or laryngoscope image 120 being associated with laryngoscope clock information and endoscope image 122 being associated with endoscope clock information).

[0061] The combined view 34 can be generated based on images from the laryngoscope camera 50 and / or the endoscope image 122. While the display 18 of the video laryngoscope 12 can switch back and forth between alternating displays of the laryngoscope camera image 120 and the endoscope image 122, the video laryngoscope 12 can display one or both of the corresponding images. Furthermore, in the combined video file, any image can be overlaid on top of the others. For example, Figure 9This is an example display of the combined view 34, where the endoscope image 122 is overlaid on top of both the laryngoscope camera image 120 and the ambient image 33. System 10 allows the user to switch the overlaid images back and forth between the endoscope image 122 and the laryngoscope camera image 120, such as by swiping one image to another. In one embodiment, the display panel or outline 176 of the endoscope image 122 may be displayed in a circular or different shape relative to other displayed images in a manner consistent with the display mode of the video laryngoscope 12, to create continuity with the display style of the video laryngoscope 12.

[0062] Figure 10 This is an alternative combined view 34 showing a multi-panel arrangement in which the laryngoscope camera image 120 is arranged in the first panel, the endoscope image 122 is arranged in the second panel, and the environmental image 33 is arranged in the third panel of the combined view 34. (See also: Regarding...) Figure 9 As discussed, the endoscopic image 122 can be displayed using display settings to maintain a specific shape (e.g., circular). Furthermore, the background color 180 or outline surrounding the panel (such as the panel displaying the endoscopic image 122) can be different relative to other panels to facilitate differentiation between images. In one embodiment, the relative positions and sizes of these panels can be adjusted via settings of the video imaging system 10. Additionally, one or more panels can be eliminated. In one example, if the endoscope is not connected to the video laryngoscope 12 such that the monitor 30 does not receive the endoscopic image 122, the combined view 34 automatically ignores any dedicated panels for the endoscopic image 122. The connection of the endoscope to the video laryngoscope 12 and the reception of the endoscopic image 122 by the monitor 30 trigger activation of the appropriate display panel on the combined view 34.

[0063] Figure 11 This is a block diagram of an embodiment of the video imaging system 10. As shown, system 10 includes a video laryngoscope 12 and a monitor 30. Furthermore, system 10 includes an environmental camera assembly 250, which includes an environmental camera 32. In some embodiments, the video laryngoscope 12 is coupled to an endoscope 175 having an integrated endoscope camera 254.

[0064] The video laryngoscope 12 and monitor 30 may include various components that enable the system 10 to perform the techniques disclosed herein. For example, the video laryngoscope 12 may include a display screen 18, a camera 50, coupled sensors 260 and input (e.g., touch sensors) 64, and a controller 262 (e.g., an electronic controller), one or more processors 264, hardware memory 266, a power supply (e.g., a battery) 268, input / output (I / O) ports 270, communication devices 272, and a system for generating laryngoscope clock information 100. Figure 3 ) Laryngoscope clock 102.

[0065] The monitoring device 30 may include a display screen 54, a controller 280 (e.g., an electronic controller), one or more processors 282, hardware memory 284, a power supply (e.g., a battery or an input from an external power source) 286, I / O ports 288, and a communication device 290. The power supplies 268, 286 may be rechargeable and / or replaceable batteries. The communication devices 272, 290, and other communication devices of the system 10 (such as communication device 292 of the separate display 22 or communication device 294 of the environmental camera assembly 250) may be wireless transceivers configured to establish wireless communication with each other. By way of example, the communication devices may be configured to communicate using the IEEE 802.15.4 standard and may communicate using, for example, ZigBee, WirelessHART, or MiWi protocols. Additionally or alternatively, the communication devices may be configured to communicate using one or more standards, such as the Bluetooth standard or the IEEE 802.11 standard. As described above, in some embodiments, a communication device 268 may be housed in an adapter 70, which is configured to connect to the monitor 30 to facilitate wireless communication between the video laryngoscope 12 and the monitor 30. The monitor 30 also includes a monitor clock information generation device 110. Figure 3 The monitor clock 112.

[0066] In some embodiments, the video laryngoscope 12 and monitor 30 include a circuitry configured to process signals, such as, for example, signals generated by camera 50, signals generated by ambient camera 32, and / or control signals provided via an input to monitor 30 or an input 64 on video laryngoscope 12. In an illustrated embodiment, processors 264, 282 may be used to execute software. For example, processor 264 of video laryngoscope 12 may be configured to receive signals from camera 50 and endoscope 175 when coupled and to execute software to generate images and / or perform any of the various processes according to this disclosure (e.g., displaying images, storing images, transmitting images, etc.). Processor 282 of monitor 30 may be programmed to generate a combined view 34 as provided herein and to synchronize airway image 20 and ambient image 33. Furthermore, processors 264, 282 may include multiple microprocessors, one or more “general purpose” microprocessors, one or more special purpose microprocessors, and / or one or more application-specific integrated circuits (ASICs) or some combination thereof. For example, processors 264 and 282 may include one or more Reduced Instruction Set Computing (RISC) processors. It should be understood that the individual processing steps may be executed by any of the processors 264 and 282, or may be distributed among the processors 264 and 282 in any suitable manner.

[0067] Hardware memories 266 and 284 may include: volatile memory, such as random access memory (RAM); and / or non-volatile memory, such as read-only memory (ROM). It should be understood that hardware memories 266 and 284 may include flash memory, hard disk drive or any other suitable optical storage medium, magnetic storage medium or solid-state storage medium, other hardware memories or combinations thereof. Memories 266 and 284 may store a variety of information and may be used for a variety of purposes. For example, memories 266 and 284 may store processor-executable instructions (e.g., firmware or software) for execution by processors 264 and 282, such as instructions for processing signals generated by camera 50 to generate images, providing images on display screen 18, and / or transmitting images to monitor 30. Hardware memories 266 and 284 may store data (e.g., previously acquired images, time data, etc.), instructions (e.g., software or firmware for generating images, storing images, transmitting images, etc.), and any other suitable data.

[0068] System 10 may include features for automatically adjusting the position or focus of the environmental camera 32. The environmental camera assembly may include a position / focus controller that executes commands to adjust the field of view of the environmental camera. For example, monitor 30 or controller 300 may use object recognition technology to process the environmental image 32 to identify identifiable markers 66 positioned on the video laryngoscope. Figure 2 And adjust the camera position so that the identifiable marker 66 is kept within the field of view (such as keeping it centered).

[0069] The ambient camera assembly 250 also includes a microphone 302 for capturing ambient sounds, which may include clinician instructions, patient voices, and medical device alarms, and includes microphone-captured data synchronized with the ambient image 33, such that the combined view 34 includes audio. As discussed herein, the ambient camera 32 may be hardwired to the monitor 30 to receive clock information from the monitor clock 112. However, the ambient camera assembly may include an integrated clock 106 that generates clock information associated with the acquired ambient image 33.

[0070] Because patients requiring mechanical ventilation may have complex and overlapping medical conditions, composite view video files generated by a video imaging system facilitate the viewing or analysis of intubation procedures with additional environmental context. Successful intubation to quickly and effectively secure the patient's airway may be related to the skill and training of the intubator and assistive clinicians, the clinician's familiarity with the available equipment, the internal airway anatomy, and the patient's clinical condition. The disclosed technique allows for more accurate viewing of intubation procedures by providing environmental context in a user-friendly display configuration that takes into account device-specific differences in internal clock information. Accurate viewing leads to a more accurate characterization of contributing factors to difficult or failed intubations, which in turn improves patient care. The disclosed technique addresses the problem of clock discrepancies between coupled devices to facilitate the synchronization of images acquired by asynchronous devices.

[0071] While this disclosure allows for various modifications and alternatives, the accompanying drawings have illustrated specific embodiments by way of example and have been described in detail herein. However, it should be understood that the embodiments provided herein are not intended to be limited to the specific forms disclosed. Rather, various embodiments may encompass all modifications, equivalents, and alternatives falling within the spirit and scope of this disclosure as defined by the appended claims. Furthermore, it should be understood that certain elements of the disclosed embodiments may be combined or interchanged with each other.

Claims

1. A video imaging system, the video imaging system comprising: The monitoring device includes a display screen; A video laryngoscope, the video laryngoscope including a camera that acquires images of a patient's airway; and An environmental camera, which acquires environmental images of the patient's environment; The monitoring device operates as follows: Receive the airway image from the video laryngoscope and the environmental image from the environmental camera; Receive the video laryngoscope clock time from the video laryngoscope; Determine the first difference between the video laryngoscope clock time and the monitor clock time; Based on the first difference, the airway image and the environmental image are synchronized; as well as Synchronized airway images and environmental images are combined into a composite video file.

2. The video imaging system of claim 1, wherein the combined video file comprises a side-by-side view of synchronized airway images and environmental images.

3. The video imaging system of claim 1, wherein the monitor operates to use the first difference to synchronize the airway image to the monitor clock time.

4. The video imaging system of claim 1, wherein the monitor operates to receive an ambient camera clock time from the ambient camera and to determine a second difference between the ambient camera clock time and the monitor clock time.

5. The video imaging system of claim 4, wherein the monitor operates to use the second difference to synchronize the environmental image to the monitor clock time.

6. The video imaging system of claim 1, wherein the environmental camera is housed in or on the monitoring instrument.

7. The video imaging system according to claim 1, wherein the monitoring device includes a portable tablet computer.

8. The video imaging system of claim 7, wherein the environmental camera is coupled to the portable tablet computer on the side opposite to the display screen.

9. The video imaging system of claim 1, wherein one or both of the monitoring device or the environmental camera are connected to a portable bracket.

10. The video imaging system of claim 1, wherein the monitor operates as follows: Identify the video laryngoscope in the environmental image; and Adjust the field of view of the environmental camera to keep the video laryngoscope within the field of view.

11. A medical device system, the medical device system comprising: Video laryngoscope, which acquires images of the patient's airway; An environmental camera, which acquires environmental images; The monitoring device operates as follows: Receive the airway image from the video laryngoscope; Receive the environmental image from the environmental camera; Determine the first time difference between the monitor clock and the video laryngoscope clock, and the second time difference between the monitor clock and the environmental camera clock; The airway image is shifted relative to the environmental image based on the first time difference and the second time difference, so that the airway image and the environmental image are time-synchronized with each other. as well as Generates a video file that includes time-synchronized airway images and environmental images.

12. The medical device system of claim 11, wherein the environmental camera is mounted on a portable stand.

13. The medical device system of claim 11, wherein the monitor receives the airway images wirelessly and in real time.

14. A method for video imaging, the method comprising: The monitor receives airway images and laryngoscope clock information from the video laryngoscope. The monitoring instrument acquires environmental images from an environmental camera facing the room, wherein the environmental images and the airway images are acquired simultaneously. Determine the relative time difference between the laryngoscope clock information and the monitor clock information; The airway image and the environmental image are synchronized in time based on the relative time difference; as well as Generate a composite video file with a combined view of time-synchronized airway and environmental images.

15. The method of claim 14, wherein the composite video file comprises a picture-in-picture view or a side-by-side view.

16. The method of claim 14, wherein the environmental image is timestamped using monitor clock information.

17. The method of claim 14, further comprising: The monitor receives endoscopic images and endoscopic clock information from the endoscope. Determine the second relative time difference between the endoscope clock information and the monitor clock information; as well as The endoscopic image is synchronized with the environmental image based on the second relative time difference. The combined view in the composite video file also includes the time-synchronized endoscopic image.

18. The method according to claim 14, further comprising: Receive patient vital signs data from the patient monitor; Identify patient events and their associated times in the patient's vital signs data; as well as Information indicating the patient's event will be overlaid onto the composite video file at the associated time.

19. The method of claim 14, further comprising displaying the composite video file on the display screen of the monitor.

Images