Adaptive multi-image display for video laryngoscope

Abstract

Systems and methods for adaptive multi-image display are described. A video laryngoscope may display multiple images concurrently on a display. Display of multiple images concurrently may be based on a desirability to display the images. The displayed images may have an assigned hierarchy (e.g., images may be assigned as priority or subordinate / secondary). Determination of the image hierarchy may change and / or be continuous. Relative sizing of the concurrently displayed images may be adaptive based on the changing, determined hierarchy of the images. Images with priority may be sized larger relative to subordinate images.

Description

A0013433W001ADAPTIVE MULTI-IMAGE DISPLAY FOR VIDEO LARYNGOSCOPECROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims the benefit of U.S. Provisional Patent Application Serial No. 63 / 761,019, filed February 20, 2025, the entire content of which is incorporated herein by reference.INTRODUCTION

[0002] Laryngoscopes are commonly used during intubation of a patient who may require breathing assistance. An intubation is a medical procedure in which a medical professional (e.g., a doctor, therapist, nurse, clinician, or other practitioner) inserts a breathing tube (e.g., an endotracheal tube) into the mouth of the patient, past the larynx, and into the trachea of the patient. The breathing tube may then be connected to a ventilator or other device for supplying breathing gases to a patient. A laryngoscope may assist medical professionals during intubation by helping manipulate portions of the patient’s anatomy (e.g., the tongue or the epiglottis) and obtain a view of the larynx. For additional visualization, and to facilitate navigation and insertion of tracheal tubes within the airway, some laryngoscopes may be configured with a video camera. A laryngoscope that includes a video camera may be referred to as a video laryngoscope (VL), utilizing video laryngoscopy to acquire and display a real-time video feed (e.g., sometimes referred to as an indirect view) of the patient’s airway.

[0003] It is with respect to this general technical environment that aspects of the present technology disclosed herein have been contemplated. Furthermore, although a general environment is discussed, it should be understood that the examples described herein should not be limited to the general environment identified herein.SUMMARY

[0004] Certain embodiments commensurate in scope with the originally claimed subject matter are summarized below. These embodiments are not intended to limit the scope of the disclosure. Indeed, the present disclosure may encompass a variety of forms that may be similar to or different from the embodiments set forth below.

[0005] Among other things, aspects of the present disclosure include systems and methods for adapting a multi-image display of a video laryngoscope. In an aspect, a method performedA0013433W001by a video laryngoscope is disclosed. When operating multiple visualization devices, such as a video laryngoscope and an endoscope (or other airway visualization tool), a medical professional may benefit from a single display that consolidates display of multiple real-time images from multiple visualization sources. For example, a display or graphical user interface of a video laryngoscope may display real-time images acquired from a camera of the VL and real-time images acquired from a camera of a connected endoscope. Display of the real-time images may be adaptive such that the real-time images that have priority may be displayed in a larger size relative to real-time images without priority (e.g., secondary images). Priority of images may be determined by the video laryngoscope and may be based on analysis of the realtime images, order that visualization devices or tools are connected, hierarchy of visualization tools, etc. Additionally or alternatively, priority of images may be based on interactions received at a user interface of the video laryngoscope.

[0006] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Additional aspects, features, and / or advantages of examples will be set forth in part in the description which follows and, in part, will be apparent from the description, or may be learned by practice of the disclosure.BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The following drawing figures, which form a part of this application, are illustrative of aspects of systems and methods described below and are not meant to limit the scope of the disclosure in any manner, which scope shall be based on the claims.

[0008] FIG. 1A is a schematic of an example patient environment including a video laryngoscope and an endoscope.

[0009] FIG. IB is a schematic of the video laryngoscope of FIG. 1.

[0010] FIG. 2A depicts another example combination of a laryngoscope and an endoscope.

[0011] FIG. 2B depicts a bottom perspective view of an example detachable cartridge.

[0012] FIGS. 3A-3B are block diagrams of components of example video laryngoscope and endoscope systems.A0013433W001

[0013] FIG. 4 shows example display screens of a video laryngoscope during adaptive image display.

[0014] FIG. 5 shows additional example display screens of a video laryngoscope during adaptive image display.

[0015] FIG. 6 shows an example method for adaptive image display with a video laryngoscope.

[0016] FIG. 7 shows another example method for adaptive image display with a video laryngoscope.

[0017] While examples of the disclosure are amenable to various modifications and alternative forms, specific aspects have been shown by way of example in the drawings and are described in detail below. The intention is not to limit the scope of the disclosure to the particular aspects described. On the contrary, the disclosure is intended to cover all modifications, equivalents, and alternatives falling within the scope of the disclosure and the appended claims.DETAILED DESCRIPTION

[0018] As discussed briefly above, laryngoscopes are commonly used during intubation of a patient who may require breathing assistance. An intubation is a medical procedure in which a medical professional (e.g., a doctor, therapist, nurse, clinician, or other practitioner) inserts a breathing tube (e.g., an endotracheal tube) into the mouth of the patient, past the larynx, and into the trachea of the patient. The breathing tube may then be connected to a ventilator or other device for supplying breathing gases to a patient. A laryngoscope may assist medical professionals during intubation by helping manipulate portions of the patient’s anatomy (e.g., the tongue or the epiglottis) and obtain a view of the larynx. For additional visualization, and to facilitate navigation and insertion of tracheal tubes within the airway, some laryngoscopes may be configured with a video camera. A laryngoscope that includes a video camera may be referred to as a video laryngoscope (VL), utilizing video laryngoscopy to acquire and display a real-time video feed (e.g., sometimes referred to as an indirect view) of the patient’s airway.

[0019] An endoscope is a long, narrow, flexible instrument or tube with a video camera that may be used to assist in placing a tracheal tube into a patient’s trachea during an intubation.A0013433W001With some patients, an endoscope may provide supplemental visualization of the patient’s airway to a medical professional to aid with intubations in difficult airway environments (e.g., inability to position the head or neck of the patient, such as due to injury; airway obstruction; atypical anatomy of the patient; other health considerations; or a combination of these or other factors). Some endoscopes may have a tip, or distal end, which is steerable. For example, the distal end may be actively controllable to bend, turn, rotate, or otherwise move the distal end in a desired direction, such as to navigate through or towards anatomy of the patient. An endoscope may perform the same or similar functions as an introducer and may alternatively be referred to as an introducer in some examples.

[0020] When a medical professional is using a combination of visualization techniques (e.g., video laryngoscopy and endoscopy), navigating or viewing multiple displays may be difficult. Additionally, viewing and / or navigating multiple real-time images while manipulating multiple devices (e.g., placing the VL, placing and / or steering the endoscope, placing the endotracheal tube, etc.) is also challenging.

[0021] When operating multiple visualization devices, such as a video laryngoscope and an endoscope (or other airway visualization tool), a medical professional may benefit from a single display that consolidates display of multiple real-time images from multiple visualization sources. For example, a display or graphical user interface of a video laryngoscope may display real-time images acquired from a camera of the VL and real-time images acquired from a camera of a connected endoscope. Display of the real-time images may be adaptive such that the real-time images that have priority may be displayed in a larger size relative to real-time images without priority (e.g., secondary images). Priority of images may be determined by the video laryngoscope and may be based on analysis of the real-time images, order that visualization devices or tools are connected, hierarchy of visualization tools, etc. Additionally or alternatively, priority of images may be based on interactions received at a user interface of the video laryngoscope.

[0022] FIG. 1 A shows an example patient environment 100 including a video laryngoscope 102 and an intubation tool 150 (e.g., an endoscope, an introducer, a tracheal tube or bougie, and / or an endotracheal tube, etc.). The patient environment 100 may be any room where an airway procedure (e.g., intubation) is being performed, such as a medical suite in a hospital or other care setting, an operating or other procedure room, patient recovery room, an emergency intubation setting, or other environments. As described herein, the video laryngoscope 102 mayA0013433W001be used for airway visualization of a patient 101 and / or visualization of an intubation tool 150 (e.g., endoscope 150) positioned inside the airway 140 of the patient 101. The endoscope 150 may be positioned in the airway 140 of the patient 101 concurrently with a portion of a blade of the video laryngoscope 102. The endoscope 150 includes a proximal end 154 and a distal end 152. Aspects of the video laryngoscope 102 are further shown in FIGS. IB, 2A, 2B, and 3 and aspects of the endoscope 150 are further shown in FIG. 2A and FIGS. 3A-B.

[0023] A medical professional 130 may hold a video laryngoscope 102 in a first hand 132 (e.g., a left hand 132 of the medical professional 130) and an endoscope 150 in a second hand 134 (e.g., a right hand 134 of the medical professional 130). As further described herein, the video laryngoscope 102 may be positioned in the airway 140 of the patient 101 to manipulate and / or visualize the patient’ s airway 140, such as with an arm 114 or blade 118 (shown in FIG.2). Visualization of the airway 140 of the patient 101 may include viewing patient’s anatomy (e.g., larynx, trachea, esophagus, vocal cords) with a camera 116 of the video laryngoscope 102. The medical professional 130 may move the endoscope 150 proximally (e.g., retract the endoscope 150) or distally (e.g., advance the endoscope 150), while watching the resulting images from the camera 116 of the video laryngoscope 102 on the display 108 of the video laryngoscope 102.

[0024] Video laryngoscope images (video images and / or still-shot images) may be acquired by the video laryngoscope 102 during operation by a medical professional. The acquired laryngoscope images may be displayed at an integrated display screen (e.g., display 108) of the video laryngoscope 102 in the line of sight of the operator (e.g., medical professional 130). The acquired laryngoscope images may be stored in a memory of the video laryngoscope 102. In some examples, the stored images may be reviewed or replayed on the display screen of the video laryngoscope 102 itself. In other examples, stored acquired images may be transferred to, or communicated to, a remote device / system (e.g., a computer). The stored images may be reviewed, edited, or otherwise interacted with at the remote device.

[0025] The video laryngoscope 102 may be configured to connect with or communicate with one or more tools or remote devices or systems via any of a variety of techniques. For example, the endoscope 150 may be configured to connect with the video laryngoscope 102 via a wired or wireless connection. The video laryngoscope 102 may include a tool port 126 configured to electrically couple with an input / output port of the endoscope 150. The tool port 126 may be anywhere on the body of the video laryngoscope 102 that is not intended to beA0013433W001inserted into a body of a patient (e.g., on the display portion 106, the handle portion 110, but not the arm). Images (video images and / or still-shot images) acquired by a camera of the endoscope 150 may be received by the video laryngoscope 102 for display at an integrated display screen (e.g., display 108) of the video laryngoscope 102. The laryngoscope images and the endoscope images may be displayed individually or concurrently at the display 108.

[0026] As an alternative to a direct electrical connection between the video laryngoscope 102 and the endoscope 150, communication between the video laryngoscope 102 and a tool or remote device / system may be facilitated by one or more communication devices (e.g., wireless transceivers or hubs, which may be a wireless adapter, dongle, bridge device) that are configured to establish wireless communication with one another using any suitable protocol and may be movable between multiple tools or remote devices / systems.

[0027] FIG. IB shows a schematic of a video laryngoscope 102. As shown, the video laryngoscope 102 has a body 104 (e.g., reusable body). The body 104 includes a display portion 106 having a display 108 that is configured to display images and / or other data, a handle portion 110 having a handle 112 that is configured to be gripped by the medical professional during the laryngoscopy procedure, and an elongate portion or arm 114 that supports a camera 116 and light source (e.g., light-emitting diodes (LEDs)) that is configured to obtain images, which may be still-shot images and / or moving images (e.g., a video feed). The camera 116 and light source may be incorporated on the distal end of the arm 114. The light source may be provided as part of the camera 116 or separate from the camera 116 on the blade 118 or arm 114.

[0028] In examples, the display portion 106 and the handle portion 110 may not be distinct portions, such that the display 108 is integrated into the handle portion 110. In the illustrated embodiment, an activating cover, such as a removable laryngoscope blade 118 (e.g., activating blade, disposable cover, sleeve, or blade), is positioned about the arm 114 of the body 104 of the video laryngoscope 102. Together, the arm 114 of the body 104 and the blade 118 form an insertable assembly that is configured to be inserted into the patient's oral cavity. It should be appreciated that the display portion 106, the handle portion 110, and / or the arm 114 that form the body 104 of the video laryngoscope 102 may be fixed to one another or integrally formed with one another (e.g., not intended to be separated by the medical professional during routine use) or may be removably coupled to one another (e.g., intended to be separated by the medical professional during routine use) to facilitate storage, use, inspection, maintenance, repair,A0013433W001cleaning, replacement, or interchangeable parts (e.g., use of different arms or extensions with one handle portion 110), for example.

[0029] The handle 112, arm 114, and / or display portion 106 may include one or more sensors 122 capable of monitoring functions (e.g., different, additional, and / or advanced monitoring functions). The sensors 122 may include a torque sensor, force sensor, strain gauge, accelerometer, gyroscope, magnet, magnetometer, proximity sensor, reed switch, Hall effect sensor, infrared sensor, wireless connectivity sensor, etc. disposed within or coupled to any suitable location of the video laryngoscope 102. The sensors 122 may detect interaction of the video laryngoscope 102 with other objects, such as a blade 118, physiological structures of the patient (e.g., teeth, tissue, muscle, etc.), or proximity of an airway tool (e.g., endoscope 150). For example, a sensor such as a magnet, pressure sensor, proximity sensor, etc. may enable the video laryngoscope 102 to determine when a blade is securely coupled (e.g., as distinguished between a secure coupling and a partial decoupling) with the arm 114 of the video laryngoscope 102.

[0030] The video laryngoscope 102 may also include a power button 120 that enables a medical professional to power the video laryngoscope 102 off and on. In examples, the video laryngoscope 102 may be powered by a power source (e.g., battery) that is coupled to the video laryngoscope 102. The power source may be removably coupled to the video laryngoscope 102. In an instance where the power source is removably couplable to the video laryngoscope 102, the power button 120 may be positioned on the removable power source, instead of on the video laryngoscope 102 itself. The power button 120 may also be used as an input device to access settings of the video laryngoscope 102, including a mode of operation (e.g., routine operation, settings, etc.). Additionally, the video laryngoscope 102 may include an input button or digital input device, such as a touch or proximity sensor 124 (e.g., capacitive sensor, proximity sensor, or the like) that is configured to detect a touch or object (e.g., a finger or stylus). The touch sensor 124 may enable the medical professional operating the video laryngoscope 102 to efficiently provide inputs or commands, such as inputs that cause the camera 116 to obtain or store an image on a memory of the video laryngoscope 102 and / or any other inputs relating to function of the video laryngoscope 102.

[0031] The video laryngoscope 102 may also include a tool port 126 configured to electrically couple with an input / output port of the endoscope 150. An alternative to an integrated tool port 126 is further described with respect to FIGS. 2A-2B. The tool port 126A0013433W001may be anywhere on the body of the video laryngoscope 102 that is not intended to be inserted into a body of a patient (e.g., on the display portion 106, the handle portion 110, but not the arm). Images (video images and / or still-shot images) acquired by a camera of the endoscope 150 may be received by the video laryngoscope 102 for display at an integrated display screen (e.g., display 108) of the video laryngoscope 102. The laryngoscope images and the endoscope images may be displayed individually or concurrently at the display 108.

[0032] As an alternative to the examples integrated tool port 126 shown in FIGS. 1 A-1B, FIG. 2A shows another example of coupling mechanism for a video laryngoscope 202 and an endoscope 206 (e.g., endoscope 206). FIG. 2B depicts a bottom perspective view of the example detachable cartridge 204 of FIG. 2 A. In the example system 200 shown in FIGS. 2A-2B, a video laryngoscope 202 (VL 202) is couplable to a steerable endoscope 206 via cartridge 204 (e.g., via endoscope port 207 that receives the endoscope proximal end 214), which, when coupled to the VL 202, comes into contact with, and engages, a drive housing 220. The endoscope 206 may have similar features as those described herein, such as a proximal end 214, a distal end 216, a steerable tip 218, and / or an accessory interface or camera 219. The endoscope proximal end 214 may include additional features that interact with and engage counter-facing features of the cartridge 204 (further described below) to aid in alignment between mechanical and / or electrical elements of the endoscope proximal end 214 and corresponding mechanical and / or electrical elements of the cartridge 204. The drive housing 220 includes elements that impart rotational force to elements of the cartridge 204 for control of (steering of) the endoscope steerable tip 218 by the VL 202. The VL 202 also includes a handle 208, blade 210, and display 212, which may be similar to, or the same as, corresponding elements described in FIGS. 1 A-1B. The drive housing 220 may be positioned on a rear surface 203 (opposite a front surface 201 which houses the display 212). A surface (e.g., the top surface 221) of the drive housing 220 includes an electrical interface 223C and VL output drives (e.g., hubs 224 A, 224B connected to motors within the drive housing 220 or elsewhere in the VL 202) that transmit rotational force from the drive housing 220 to the cartridge 204.

[0033] In the example shown in FIGS. 2A-2B, each of the drums 222 A-B in the introducer 206 (e.g., endoscope 206) may control bidirectional movement of the endoscope steerable tip 218 (distal tip 218) at the distal end 216 of the introducer 206 in a plane, and a single motor of the VL 202 may provide independent bidirectional control of the endoscope steerable tip 218 within that plane. Corresponding hubs 224A-B include features that engage and impart rotationA0013433W001to corresponding features of the cartridge mechanical receivers 209A-B when the cartridge 204 is coupled to the VL 202. The input mechanical receivers 209A-B in the cartridge 204 may include features capable of receiving rotational force from the hubs 224A-B.

[0034] As shown, the VL 202 also includes an electrical interface 223C that includes a set of conductive elements for providing power to the cartridge 204 and endoscope 206 and for allowing communication signals to be transmitted / received to / from the VL 202. The cartridge bottom surface 205 (FIG. 2B) includes a corresponding cartridge- VL electrical interface 223D that mates with the electrical interface 223C. The cartridge 204 also includes a cartridgeendoscope electrical interface for making electrical connection with the endoscope electrical interface 223 A. Additional elements and features of a detachable cartridge 204 for coupling an introducer to a video laryngoscope are described in U.S. Patent Application 18 / 421,476, filed November 25, 2024, which is hereby incorporated by reference in its entirety.

[0035] FIG. 3 A is a block diagram of components of the video laryngoscope 102 and an endoscope device / system 151. As shown, the video laryngoscope 102 and / or the endoscope device 151 may include various components that enable the video laryngoscope 102 to carry out the techniques disclosed herein. For example, the video laryngoscope 102 may include the display 108, the camera 116, a light source (e.g., which may integrated into the camera or separate from the camera), sensor(s) 122, and input (e.g., touch sensor and / or proximity sensor) 124, as well as a controller 160 (e.g., electronic controller), one or more processors 162, a hardware memory 164, a power source 166 (e.g., battery, which may be removably couplable with the video laryngoscope 102), input / output (VO) ports 168, a communication device 170, and a timer 172. In examples, the input / output (I / O) ports 168 may include the tool port 126 (e.g., endoscope input port 126), which may be configured to electrically couple an endoscope (e.g., endoscope system 151 or endoscope 150) with the video laryngoscope 102. In other examples, the timer 172 may track relative time (e.g., a start time, an end time, a frequency of image frame sampling), which may be referenced to acquire still-shot input images for analysis and / or to determine an operational timeout for powering down the video laryngoscope 102.

[0036] The endoscope device 151 may include a controller 174 (e.g., electronic controller), one or more processors 176, a hardware memory 178, I / O ports 182, and a communication device 184. As described herein, the endoscope system 151 or endoscope 150 may send or transmit images (e.g., video images and / or still-shot images, etc.) to the video laryngoscope 102 via I / O ports 182 and / or communication devices 170, 184. The I / O ports 182 of theA0013433W001endoscope system 151 may include an output port at a proximal end 154 to electrically couple the endoscope system 151 with the I / O ports 182 of the video laryngoscope 102. For example, the proximal end 154 of the endoscope 150 may include an electrical interface. The electrical interface may include electrical conductive elements (e.g., electrical contacts, conductive pads, receptacles, pins, balls, ports) for receiving power and / or transmitting / receiving signals to / from the video laryngoscope 102. The electrical interface may provide a power source (e.g., power source 180) for operating components of the endoscope system 151 and / or a data path. For example, the endoscope system 151 may transmit / send data via the electrical interface, such as sensor data, camera images (video and / or still-shot), information relating to the steering system 158, etc. For example, video data captured by the camera 159 of the endoscope 150 may be transmitted to the video laryngoscope 102 via the electrical interface of the endoscope system 151 (e.g., an I / O port 182) interfacing with the I / O ports 168 of the video laryngoscope 102 (e.g., such as tool port 126).

[0037] Additionally or alternatively, the communication device 170 of the video laryngoscope 102 may communicatively couple with the communication device 184 of the endoscope device 151 to allow communication between the video laryngoscope 102 and the endoscope device / system 151 (e.g., sending or receiving signal between the devices). The communication devices 170, 184 may enable wired or wireless communication. Wireless communication may include transceivers, adaptors, and / or wireless hubs that are configured to establish and / or facilitate wireless communication with one another. By way of example, the communication devices 170, 184 may be configured to communicate using the IEEE 802.15.4 standard, and may communicate, for example, using ZigBee, WirelessHART, or MiWi protocols. Additionally or alternatively, the communication devices 170, 184 may be configured to communicate using the Bluetooth standard or one or more of the IEEE 802.11 standards.

[0038] The video laryngoscope 102 may be capable of detecting the presence / absence of the endoscope system 151 or endoscope 150. When detected that the endoscope 150 is connected, the video laryngoscope 102 may energize power and / or signal connections to the endoscope 150 and / or may acquire or receive camera images from the camera 159 of the endoscope 150. The endoscope 150 may be connected or disconnected from the video laryngoscope 102 at any time during operation, any number of times.A0013433W001

[0039] As further described herein, the endoscope system 151 may further include sensors 156, a steering system 158, and the endoscope camera 159. The steering system 158 may include a steerable tip at a distal end 152 of the endoscope 150 that is controllable by a drive system. The drive system may include drum(s) and / or pull wires connected to the drums that are configured to move the steerable tip along two or more planes (e.g., via tension on the pull wires to bend the distal end 152 of the endoscope 150). The endoscope system 151 may receive steering instructions from the video laryngoscope 102, which may be determined from user input received at a user interface of the video laryngoscope 102 or determined based on image analysis (e.g., analysis of images acquired from the camera 116 of the video laryngoscope 102 and / or the camera 159 of the endoscope 150).

[0040] While the endoscope system 151 is connected to the video laryngoscope 102, the video laryngoscope 102 receives data (such as video images) from the endoscope system 151 / endoscope 150 through connection mechanism(s) (e.g., electrical interface and / or communication devices). The received data may be displayed at the display 108 of the video laryngoscope 102. The display 108 may be capable of displaying images from multiple cameras concurrently or simultaneously, such as images from the camera 116 of the video laryngoscope 102 and the camera 159 of the endoscope 150, such as by split screen, image overlay, picturein-picture, or other display methods. In examples, the display 108 may be touch-sensitive (e.g., a capacitive touch-sensitive display, such as a graphical user interface) that allows user input to be received through the display 108. Additionally, a user interface of the video laryngoscope (e.g., button, joystick, key, switch, graphical user interface at display 108, etc.) may be configured to receive steering inputs from a user of the video laryngoscope 102 to steer the endoscope 150 from interactions with the video laryngoscope 102.

[0041] The processors 162, 176 may include multiple microprocessors, one or more “general-purpose” microprocessors, one or more special-purpose microprocessors, and / or one or more application specific integrated circuits (ASICS), or some combination thereof. For example, the processors 162, 176 may include one or more reduced instruction set (RISC) processors. It should be appreciated that the various processing steps may be carried out by either processor 162, 176 or may be distributed between the processors 162, 176 in any suitable manner.

[0042] The hardware memory 164, 178 may include a volatile memory, such as random access memory (RAM), and / or a nonvolatile memory, such as read-only memory (ROM). ItA0013433W001should be appreciated that the hardware memory 164, 178 may include flash memory, a hard drive, or any other suitable optical, magnetic, or solid-state storage medium, other hardware memory, or a combination thereof. The memory 164, 178 may store a variety of information and may be used for various purposes. For example, the memory 164, 178 may store processorexecutable instructions (e.g., firmware or software) for the processors 162, 176 to execute, such as instructions for processing signals generated by the camera 116 to generate the image, provide the image on the display 108, etc. The hardware memory 164, 178 may store data (e.g., acquired images, tags, or labels, etc.), instructions (e.g., software or firmware for generating images, storing the images, analyzing the images, etc.), and any other suitable data.

[0043] FIG. 3B depicts another example of a video laryngoscope system. The system of FIG. 3B is similar to the system of FIG. 3 A with the exception that the video laryngoscope 102 directly controls and operates the endoscope 150. For example, the endoscope 150 may be removably connectable from the video laryngoscope 102, such as in the system depicted in FIGS. 2A-2B. In such examples, steering controls are generated by the controller 160 of the video laryngoscope 102 and transmitted to the steering system 158 of the endoscope 150. Similarly, the signals from the endoscope camera 159 and the endoscope sensor(s) 156 are transmitted back to the video laryngoscope 102. The processing and analysis of the images captured by both the endoscope camera 159 and the VL camera may then both be analyzed and processed by the controller 160 (e.g., processor 162) of the video laryngoscope 102.

[0044] FIG. 4 and FIG. 5 show example display screens of a video laryngoscope 102 (e.g., display 108 of video laryngoscope 102) during adaptive image display. As described herein, the laryngoscope camera 116 of the video laryngoscope 102 may acquire images in line of sight of the laryngoscope camera 116 (e.g., images of a medical room before and after an intubation, a patient’s upper airway anatomy during intubation, etc.). Additionally, the video laryngoscope 102 may acquire or receive images from the endoscope camera 159 of a connected endoscope 150. The laryngoscope images from the laryngoscope camera 116 of the video laryngoscope 102 and / or the endoscope images from the endoscope camera 159 of the endoscope 150 may be displayed on the display 108 of the video laryngoscope 102.

[0045] Referencing the example displays 400 of FIG. 4, when the video laryngoscope 102 is initially powered on (e.g., via a single push of a power button with a battery coupled to the video laryngoscope 102), a start-up display 402 may populate the display screen. The start-up display 402 may include VL images 404 (e.g., a camera feed acquired by a camera of the videoA0013433W001laryngoscope 102). In addition to the VL images, the start-up display 402 may display startup icons, such as a settings menu icon 406 and / or a battery life indicator. Although the startup icons are shown overlapping a bottom portion of the VL images 404 in the start-up display 402 of FIG. 4, any position of these startup icons, overlaid or not (e.g., in a banner outside of the display images), may be implemented, such as in an upper portion, in any corner, along a side of the display 108, etc.

[0046] The start-up display 402 may be displayed temporarily, until the video laryngoscope 102 detects functional use of the video laryngoscope and / or a connected tool and / or a user interaction with the video laryngoscope or tool. Functional use may be detected based on movement of the video laryngoscope 102 (e.g., as based on sensor data, image analysis, etc.), movement of the connected tool (e.g., endoscope), detection of airway anatomy in the VL images 404 or endoscope images 414, detection of a blade 410 coupled to the video laryngoscope 102, detection of an endoscope 150 coupled to the video laryngoscope 102, input received at a user interface, etc.

[0047] Based on inputs and / or data received at the video laryngoscope 102 during the temporary initialization period of the start-up display 402, the displays 400 may advance in a variety of ways. For example, after the start-up display 402, a display 400 of the video laryngoscope 102 may advance to a VL-specific display 408, an endoscope-specific display 412, or a multi-image adaptive display 416.

[0048] The VL-specific display 408 displays VL images 404 (e.g., images acquired from a camera of the video laryngoscope) without displaying other images that are acquired or received. The VL images 404 may fill the display 108 or substantially fill the display 108 of the video laryngoscope 102. The VL images 404 may be displayed without displaying another image (e.g., without displaying endoscope images) when the video laryngoscope 102 determines that displaying the VL images 404 is desirable while also determining that displaying other images is not desirable or unavailable. For example, the video laryngoscope 102 may determine that display of the VL images 404 is desirable based on sensor data or analysis of the VL images 404. For instance, when sensor data from a sensor (e.g., magnet, pressure sensor, proximity sensor, etc.) indicates that a blade is securely coupled with the arm of the video laryngoscope 102, display of VL images 404 may be desirable.A0013433W001

[0049] In another instance, display of VL images 404 may be desirable based on image analysis. For example, VL images 404 may be desirable for display when analysis of the VL images 404 is associated with detection of a blade 410 in the VL images (e.g., determining, based on image analysis of the VL images 404 that a blade is coupled to an arm of the video laryngoscope 102). In another example, VL images 404 may be desirable for display when analysis of the VL images 404 indicates that a user of the video laryngoscope 102 is employing functionality of the video laryngoscope 102, such as detecting airway anatomy (e.g., recognition of the larynx, vocal cords, etc. in the VL images) in the VL images 404, detecting movement(s) associated with intubation (which may additionally or alternatively use analysis of sensor data from the video laryngoscope 102), detecting airway tools (e.g., intubation tools, surgical tools, scoping tools, etc.), or other image analysis indicating use of the arm of the video laryngoscope 102. Image analysis (e.g., to detect a blade is fully coupled, patient anatomy, movement, a tool in the airway) may be performed using a trained model such as a neural network or types of artificial intelligence or machine learning models. Examples of detecting a blade based via image analysis are provided in U.S. Application No. 18 / 765,969, filed July 8, 2024, which is hereby incorporated by reference in its entirety. Examples of detecting patient anatomy and / or airway tools based on image analysis are provided in U.S. Application No.18 / 600,251, filed March 8, 2024, which is hereby incorporated by reference in its entirety.

[0050] As mentioned above, the VL-specific display 408 may display only the VL images 404, without displaying other images. The video laryngoscope 102 may determine not to display other images (other than the VL images 404) when display of other images may not be available or desirable to a user of the video laryngoscope 102. For example, no other images may be displayed at VL-specific display 408 when no other images are acquired or received (e.g., no endoscope images are received, such as while the endoscope is disconnected from the video laryngoscope), no airway anatomy is detected or recognized in the additional images (e.g., endoscope images are blurry or obscured, the endoscope images indicate that the endoscope is not placed in an airway or cavity of a patient, etc.), non-functional movement(s) or lack of movement(s) detected in the additional images (e.g., the additional camera is not being functionally controlled, such as the tool is freely hanging / dangling, the tool is not advancing in the airway or the camera images of the tool are not changing, etc.), or any other indication that a user of the video laryngoscope does not desire viewing images in addition to the VL images 404.A0013433W001

[0051] The endoscope-specific display 412, displays endoscope images 414 (e.g., images acquired from a camera of the endoscope) without displaying other images that are acquired or received. The endoscope images 414 may fill the display 108 or substantially fill the display 108 of the video laryngoscope 102. The endoscope images 414 may be displayed without displaying another image (e.g., without displaying VL images 404) when the video laryngoscope 102 determines that displaying the endoscope images 414 is desirable while also determining that displaying other images is not desirable. For example, the video laryngoscope 102 may determine that display of the endoscope images 414 is desirable based on an endoscope being coupled to the video laryngoscope 102, sensor data of the endoscope and / or video laryngoscope 102, analysis of the endoscope images 414, and / or analysis of the VL images 404. For instance, when sensor data from a sensor (e.g., magnet, pressure sensor, proximity sensor, etc.) of the endoscope indicates that the endoscope is operational. In another instance, when the video laryngoscope 102 determines that an endoscope is coupled (e.g., via a wired or wireless connection), display of endoscope images 414 may be desirable.

[0052] In another instance, display of endoscope images 414 may be desirable based on image analysis. For example, endoscope images 414 may be desirable for display when analysis of the endoscope images 414 indicates that a user of the video laryngoscope 102 is employing functionality of the endoscope, such as by detecting airway anatomy (e.g., recognition of the larynx, vocal cords, etc. in the endoscope images 414), detecting movement(s) associated with an endoscopic procedure (which may additionally or alternatively use analysis of sensor data from the endoscope), detecting airway tools (e.g., intubation tools such as an arm or blade of a video laryngoscope, surgical tools, scoping tools, etc.), or other image analysis indicating use of the endoscope. In another example, VL images 404 may be analyzed to determine if display of endoscope images 414 is desirable, such as detecting a portion of the endoscope in the VL images 404 (which may be used for analysis while not being displayed). As described above, image analysis (e.g., to detect a blade, patient anatomy, movement, a tool in the airway, etc.) may be performed using a trained model such as a neural network or types of artificial intelligence or machine learning models.

[0053] As described above, the endoscope-specific display 412 may display only the endoscope images 414, without displaying other images (such as VL images 404). The video laryngoscope 102 may determine not to display other images (other than the endoscope images 414) when display of other images may not be available or desirable to a user of the videoA0013433W001laryngoscope 102. For example, no other images may be displayed for the endoscope-specific display 412 when no blade is detected (e.g., no blade is determined to be coupled to the arm of the video laryngoscope 102, which may be determined via sensor analysis and / or image analysis), no airway anatomy is detected or recognized in the additional images (e.g., VL images 404 are blurry or obscured, the VL images 404 indicate that the arm of the video laryngoscope 102 is not placed in an airway or cavity of a patient, etc.), non-functional movement(s) or lack of movement(s) are detected in the additional images (e.g., the camera of the video laryngoscope 102 is not being functionally controlled, such as the video laryngoscope 102 is not advancing in the airway or the VL images 404 are not changing and / or are improperly placed, etc.), or any other indication that a user of the video laryngoscope 102 does not desire viewing images in addition to the endoscope images 414.

[0054] The multi-image adaptive display 416 displays multiple images on a display 108 of the video laryngoscope 102 (e.g., two or more images, at least two images, etc.). For example, the multi-image adaptive display 416 may show any number of images desirable for display to a user of the video laryngoscope 102. In an instance, the multi-image adaptive display 416 may show two images simultaneously / concurrently (e.g., the VL images 404 and the endoscope images 414). For instance, if the video laryngoscope 102 determines that display of the VL images 404 and the endoscope images 414 are both desirable, then both the VL images 404 and the endoscope images 414 may be displayed simultaneously / concurrently. Determining desirability of display of the VL images 404 and the endoscope images 414 is further described herein. Further discussion of multi-image adaptive display 416 is described in FIG. 5.

[0055] The displays 400 may change based on determinations made by the video laryngoscope (e.g., the image desirability circumstances described herein). In an example, a video laryngoscope 102 is powered on and displays start-up display 402. When VL images 404 are desirable for display and endoscope images 414 are not desirable (e.g., a blade is detected and an endoscope is not connected to the video laryngoscope 102, such as during an intubation), the VL-specific display 408 may be shown (e.g., flow from start-up display 402 to VL-specific display 408). In another example, when endoscope images 414 are desirable for display and VL images 404 are not desirable (e.g., an endoscope is connected to the video laryngoscope 102 and a blade coupling is not detected, such as during an awake endoscopic intubation), the endoscope-specific display 412 may be shown (e.g., flow from start-up display 402 to endoscope-specific display 412). In a further example, when both VL images 404 andA0013433W001endoscope images 414 are desirable for display (e.g., the video laryngoscope 102 detects a coupled blade and a connected endoscope), the multi-image adaptive display 416 may be shown (e.g., flow from start-up display 402 to multi-image adaptive display 416).

[0056] The displays 400 may automatically change between the VL-specific display 408, endoscope-specific display 412, and multi-image adaptive display 416 during operation of the video laryngoscope 102, based on determinations associated with desirability and / or availability of images for display. For example, VL-specific display 408 may change to the multi-image adaptive display 416 when endoscope images 414 are desirable for display (in addition to the VL images 404). Alternatively, VL-specific display 408 may change to the endoscope-specific display 412 when endoscope images 414 are desirable for display and VL images 404 are no longer determined to be desirable. In another example, endoscope-specific display 412 may change to the multi-image adaptive display 416 when VL images 404 are desirable for display (in addition to the endoscope images 414). Alternatively, endoscopespecific display 412 may change to the VL-specific display 408 when VL images 404 are desirable for display and endoscope images 414 are no longer determined to be desirable or available. In a further example, multi -image adaptive display 416 may change to VL-specific display 408 when endoscope images 414 are no longer desirable or available for display (while maintaining desirability of display of VL images 404). Alternatively, multi-image adaptive display 416 may change to endoscope-specific display 412 when VL images 404 are no longer desirable for display (while maintaining desirability of display of endoscope images 414).

[0057] FIG. 5 shows additional example display screens 500 of a video laryngoscope during adaptive image display. In particular, the display screens 500 of FIG. 5 show examples of how a multi -image adaptive display (e.g., multi-image adaptive display 416) may change or adapt based on image prioritization. As priority changes during adaptive image display, the display screens 500 may change between endoscope priority display 502, neutral display 508, and VL priority display 510.

[0058] Based on which image is determined to have priority on a multi-image display, relative sizes of the images may change. A display size of an image may be defined as an area of the display the image occupies, a percentage of the total area of the display screen the image occupies, a quantity of pixels the image fills or overlaps, a height and / or width of the image, or any other measurement for size of an image on display screen of a fixed physical size. In this way, a size of an image may be compared with a second image regardless of shape of theA0013433W001displayed images (e.g., a first image is rectangular and occupies a first percentage of the display screen, a second image is ovular and occupies a second percentage of the display screen, etc.). In an example, while a first image has priority over a second image (e.g., the second image is secondary or non-prioritized), the first image may be displayed in a larger size relative to the second image. Additionally or alternatively, image priority may determine an overlay order or stacking order, change a relatively translucency, change color or appearance, change aspect ratio, change outlining shape, change other visual indicators or emphasis, etc. Although FIG. 5 shows VL images 504 in a rectangular shape and endoscope images 514 in a circular shape, any shape for display of an image is appreciated, such as oval, circle, square, rectangle, polygon, triangle, etc. a portion of the image may not be displayed such that only a portion of the shape of the image is visible on the display 108, depending on the size of the image (e.g., as shown in endoscope priority display 502, edge portions of the circular shape of the endoscope images 514 are not visible on display 108, showing the endoscope images 514 as a circle with cropped edge(s)).

[0059] An image is assigned priority by the video laryngoscope 102 based on data received at the video laryngoscope 102. For example, priority may be determined based on an order that tools are connected / coupled to the video laryngoscope 102, a predetermined hierarchy of image types, image analysis, preselected operator settings, user input, other sensor data, etc. For instance, an image may have priority when the image is associated with the most recent or least recent tool connected / coupled to the video laryngoscope. For example, when an endoscope is the most recent tool connected to the video laryngoscope, the endoscope images may be prioritized. Alternatively, when the endoscope is the first tool connected to the video laryngoscope, the endoscope images may be given priority and maintain priority even after additional tools are coupled / connected. In another instance, certain image types may have a predetermined hierarchy of prioritization. For example, endoscope images may have a preset priority over VL images when both endoscope images and VL images are both desirable for display. Alternatively, VL images may have a predetermined priority over endoscope images when both endoscope images and VL images are both desirable for display. In another instance, priority may be based on image analysis. For example, analysis of the VL images and / or endoscope images, or analysis of sensor data, may determine which images are more desirable for a user of the video laryngoscope, such as recognition of functional movement, tools in the image, patient anatomy in the image, etc. In a further instance, priority may be preselected by an operator of the video laryngoscope, such as in a settings menu.A0013433W001

[0060] Priority may also be determined based on user interaction with a user interface of the video laryngoscope (e.g., user interaction with a GUI at display 108, button(s), joystick, movement, other haptic interface, etc.). For example, a user interaction with a GUI of the video laryngoscope may change the priority of images displayed. User interactions with a GUI at the display 108, to switch priority of images, may include a selection of an image, a double tap anywhere, a swipe up (e.g., to prioritize an image positioned at a lower half of the display 108), a swipe down (e.g., to prioritize an image positioned at an upper half of the display 108), a press and hold, a flick, or any other touch input.

[0061] Positioning of the images on the display 108 of the video laryngoscope 102 may be associated with functional considerations. Considering a two-image display (e.g., concurrent display of VL images 504 and endoscope images 514), a first image (e.g., VL image 504) may be positioned in an upper right quadrant of the display 108 and a second image (e.g., endoscope image 514) may be positioned in a lower left quadrant of the display 108. The images may partially overlap or be non-overlapping. In situations where the images partially overlap, VL images 504 may be positioned on the bottom (e.g., the VL image 504 may be overlapped by other images on top of the VL image 504, such as endoscope image 514). Overlapping portions of a top image may be translucent to allow for a user of the video laryngoscope 102 to see the overlapped portion of the bottom image. For example, the overlapping portion of the top image may have a translucency of 10%, 15%, 20%, 30%, etc. Images may be positioned such that overlapping or obscuring of a portion of the VL images 504 occurs in a lower half of the VL images 504 or a lower left quadrant of the VL images 504. The top half of the VL images 504 may be unobscured on the display 108, regardless of size of the images displayed, to allow unobstructed view of patient anatomy appearing in the top half of the VL images 504.

[0062] Endoscope images 514 may be positioned in a bottom left quadrant of the display 108 of the video laryngoscope 102. This position is adjacent an operator’s thumb, which may be used to control or steer an endoscope coupled to the video laryngoscope (e.g., the endoscope having a camera acquiring the endoscope image 514 being displayed at display 108). When the endoscope image 514 is positioned in the lower left quadrant, adj acent an operator’ s thumb, any touch inputs relating to steering of the endoscope may overlap with the display real estate showing the endoscope image 514 and reduce operator confusion associated with where to provide touch inputs relating to steering the endoscope. The endoscope image 514 may be movable with an associated user input (e.g., flick; tap, hold, and swipe; hold; multiple taps) to a region of the displayA0013433W001108 outside of the lower left quadrant. After the endoscope image 514 is moved, the endoscope image 514 may return to its lower left quadrant position after a set period of time, with another user input, when image priority changes, when tools are connected or disconnected from the video laryngoscope, or any other input or detection associated with a change in the display screens 500. Alternatively, the endoscope image 514 may not be movable about the display 108. This may be desirable to reduce user confusion and / or frustration relating to accidental movement of the endoscope image 514 during other touch inputs, such as endoscope steering inputs, image priority inputs, menu selection, etc.

[0063] Endoscope priority display 502 may be shown when endoscope images 514 are assigned priority. In endoscope priority display 502, the endoscope images 514 are larger than the VL images 504. The endoscope images 514 are enlarged such that a maximum width of the image is at least the width of the display 108. In an example, the maximum width of the endoscope images 514 during priority may be wider than the display 108 such that, when the endoscope images 514 are centered horizontally, the edges of the endoscope images 514 are not visible. The maximum width of priority images may be wider than the display 108 when the image shape is not the same shape and / or aspect ratio as the display 108. The secondary images (e.g., the VL images 504) may be shrunk to a smaller size, towards an upper right comer of the display 108, to maintain a maximum overlap of the endoscope images 514 and maintain a minimum unobscured portion of the VL images 504.

[0064] VL priority display 510 may be shown when endoscope images are assigned priority. In VL priority display 510, the VL images 504 are larger than the endoscope images 514. The VL images 504 are enlarged such that a maximum width of the image is at least the width of the display 108. In an example, the shape and aspect ratio of the VL images 504 may be the same as the display 108 to allow for a maximum width of the VL images 504 during priority to be the same as the width of the display 108. The secondary images (e.g., the endoscope images 514) may be shrunk to a smaller size, towards a lower left corner of the display 108, to maintain a maximum overlap of the endoscope images 514 and maintain a minimum unobscured portion of the VL images 504.

[0065] Neutral display 508 may be shown when no priority is assigned to an image. In neutral display 508, the VL images 504 and the endoscope images 514 are substantially the same size. The neutral width of each of the images may be optimized to maintain a maximumA0013433W001overlap of the endoscope images 514 and maintain a minimum unobscured portion of the VL images 504.

[0066] Although the examples shown in FIGS. 4 and 5 depict multi-image display with two images (e.g., VL images 404, 504 and endoscope images 414, 514), prioritization with any number of images is appreciated (e.g., two, three, or more images displayed concurrently). In examples where three or more images are prioritized, the image with priority may be the largest relative to the other images. Secondary, tertiary, etc. images may each be smaller, respectively, or may all be a substantially same, smaller size than the prioritized image. Additionally or alternatively, images of higher priority may be stacked or overlaid (with or without translucency) overtop of images with lower priority.

[0067] FIG. 6 and FIG. 7 show example methods according to the disclosed technology. The example methods include operations that may be implemented or performed by the systems and devices disclosed herein. For example, the video laryngoscope 102 and / or endoscope system 151 (or endoscope 150, 206) depicted in at least FIGS. 1-3 may perform the operations described in the methods. In addition, instructions for performing the operations of the methods disclosed herein may be stored in a memory of the video laryngoscope and / or endoscope (e.g., system memories 164, 178 described in FIGS. 3A-B).

[0068] Referencing FIG. 6, method 600 is an example method for adaptive image display with a video laryngoscope. At operation 602, the video laryngoscope is powered on. The video laryngoscope may be powered on from a power button positioned on the handle of the video laryngoscope. At operation 604, VL images are acquired from a VL camera of the video laryngoscope. The VL camera of the video laryngoscope may be coupled to an arm configured to receive a blade.

[0069] At operation 606, VL images and a settings indicator are displayed at the video laryngoscope. This VL images and settings indicator may be similar to those described in startup display 402 in FIG. 4. For example, after powering on the video laryngoscope and before determining if the video laryngoscope is being functionally used (e.g., detection of any tools connected / coupled to the video laryngoscope, such as a blade coupled to the arm, an endoscope connected to the video laryngoscope, etc.; movement of the video laryngoscope; detection of airway anatomy in an acquired image; user input received at a user interface; etc.), the startup display may be temporarily displayed.A0013433W001

[0070] At determination 608, the video laryngoscope determines if selection of a settings menu indicator is received. If the video laryngoscope determines that the settings menu indicator is selected, the method 600 flows “YES” to operation 610. At operation 610, a settings menu is displayed. The settings menu may include settings associated with assigning priority of images, such as preset priorities, user interactions enabled at a user interface to change priority, positioning and / or movability of images in an adaptive multi-image mode, or any other settings associated with display of multiple images at the display of the video laryngoscope and / or assigning priority. If, alternatively, no selection of the settings menu indicator is received, the method 600 flows “NO” to determination 612 and a settings menu is not displayed.

[0071] At determination 612, the video laryngoscope determines if an endoscope is connected. Determination of an endoscope being connected to the video laryngoscope may be based on receiving communication with an endoscope (e.g., via a wired or wireless connection). Communications with the endoscope may include receiving endoscope sensor data, receiving endoscope images, detecting an electrical interface connection with the endoscope, etc.

[0072] If the video laryngoscope detects that an endoscope is not connected, the method 600 flows “NO” to determination 614. At determination 614, the video laryngoscope determines if a blade is coupled to the arm of the video laryngoscope. A blade coupling (e.g., a complete blade coupling, without any partial decoupling such as a blade partially or insecurely positioned along the arm) may be determined based on sensor data and / or image analysis of the VL images (e.g., detecting a portion of a blade in the VL images). Alternatively, the video laryngoscope may determine if the arm is being functionally used. For example, determination of use of the arm of the video laryngoscope may include detecting airway anatomy in the VL images (e.g., recognition of the larynx, vocal cords, etc. in the VL images), detecting movement(s) associated with intubation (which may additionally or alternatively use analysis of sensor data from the video laryngoscope 102), detecting airway tools in the VL images (e.g., intubation tools, surgical tools, scoping tools, etc.), or other image analysis or sensor data analysis indicating use of the arm of the video laryngoscope 102.

[0073] If the video laryngoscope determines that a blade is not coupled (or partially decoupled) at determination 614 (or that the arm of the video laryngoscope is not being functionally used), the method 600 flows “NO” to determination 626, which is furtherA0013433W001described below. If the method 600 flows “NO” from determination 626, then flow proceeds back to determination 608 to determine if a selection of a settings menu indicator is received. In this way, operations / determinations 608-614 may repeat as required or desired to continue to temporarily display the startup display (e.g., display VL images and a settings menu indicator, if the method 600 flows “NO” from determination 608) or the settings menu (e.g., if the method 600 flows “YES” from determination 608). This temporary startup display or settings menu display may continue to be displayed until either an endoscope (or other tool with a camera) is connected and / or a blade is coupled to the video laryngoscope (or other functional use of the arm of the video laryngoscope is determined). Stated another way, the startup display or settings menu may be closed / stopped when a tool is connected to the video laryngoscope or functionality of the arm of the video laryngoscope is detected (e.g., blade coupling is detected).

[0074] If, alternatively, the video laryngoscope determines that a blade is coupled at determination 614 (or that the arm of the video laryngoscope is operational), the method 600 flows “YES” to operation 616. At operation 616, the VL images are displayed. The VL images may be displayed with a reduced set of indicators (e.g., a settings menu indicator is hidden / no longer displayed). The display of the VL images at operation 616 may be similar to VL-specific display 408 described with respect to FIG. 4. For example, the VL images may be displayed without displaying any other images (e.g., no endoscope images are displayed, and no priority is assigned because only one set of images is being displayed).

[0075] If, alternatively, the video laryngoscope does detect an endoscope is connected at determination 612, the method 600 flows “YES” to operation 618. At operation 618, endoscope images are received from a camera of the connected endoscope. The endoscope images may be communicated to the video laryngoscope over a wired or wireless connection. The camera of the endoscope may be positioned at a proximal end of the endoscope (e.g., at or adjacent a tip of the endoscope).

[0076] At determination 620, the video laryngoscope determines if a blade is coupled to the arm of the video laryngoscope (or if the arm of the video laryngoscope is being employed). Aspects of determination 620 may be similar to that of determination 614. If the video laryngoscope determines that a blade is coupled at determination 620 (or that the arm of the video laryngoscope is being functionally used), the method 600 flows “YES” to operation 622. At operation 622, the VL images and the endoscope images are concurrently displayed. TheA0013433W001concurrent display of the VL images and the endoscope images at operation 622 may be similar to multi-image adaptive display 416 described with respect to FIG. 4 and / or the adaptive image display screens 500 of FIG. 5.

[0077] If, alternatively, the video laryngoscope determines that a blade is not coupled at determination 620, the method 600 flows “NO” to operation 624. At operation 624, the endoscope images are displayed. The display of the endoscope images at operation 624 may be similar to endoscope-specific display 412 described with respect to FIG. 4. For example, the endoscope images may be displayed without displaying any other images (e.g., no VL images are displayed, and no priority is assigned because only one set of images is being displayed).

[0078] From operations 616, 622, and 624 (and from determination 614, described above), the method 600 may flow to determination 626. At determination 626, it is determined if the video laryngoscope is powering off. The video laryngoscope may determine to enter a power off state based on user input (e.g., selection of the power button), passing of a period of time (e.g., a timeout period), a remaining battery life, etc. If the video laryngoscope is powered off at determination 626, the method 600 flows “YES” to operation 628. At operation 628, the video laryngoscope is powered off.

[0079] If, alternatively, the video laryngoscope is not determined to be powered off at determination 626, the method may flow back to determination 612. Method operations and determinations 612-626 may repeat as required or desired. For example, concurrent display of the VL images and endoscope images may continue while the endoscope is connected and a blade is coupled (e.g., “YES” at determinations 612 and 620). In another example, display of only one image (e.g., operations 616 and 624) may display if the determinations of an endoscope connection and blade coupling are maintained / unchanged (e.g., same method flow at determinations 612, 614, and / or 620). In a further example, the display may change between concurrent display of multiple images (e.g., multi-image adaptive display at operation 622) and a single image (e.g., VL-specific display or endoscope-specific display at operations 616 and 624, respectively), based on endoscope connection and blade coupling determinations 612, 614, and / or 620. This change of display is further described with respect to FIG. 4.

[0080] FIG. 7 shows another example method for adaptive image display with a video laryngoscope. At operation 702, the video laryngoscope determines that a blade is coupled (or, as described herein, an arm of the video laryngoscope is being functionally employed) and anA0013433W001endoscope (or other tool with a camera) is connected to the video laryngoscope. These determinations may be similar to determinations 612, 614, and 620 described in FIG. 6.

[0081] At determination 704, image priority is determined. The video laryngoscope may determine priority based on an order that tools are connected / coupled to the video laryngoscope (e.g., most recent tool connected or least recent / first tool connected), a predetermined hierarchy of image types (e.g., VL images, connected tool images, remotely received images, etc.), image analysis (e.g., to determine which images a user may currently find more desirable, such as based on functional movement, positioning in the airway, tools detected, etc.), preselected operator settings (e.g., as may be provided in a settings menu), user input, other sensor data, etc. In some circumstances, such as if priority is not determinable or the video laryngoscope is instructed to not assign a hierarchy to images, image priority may not be assigned. Display of images may change based on image hierarchy. In the example method 700 provided in FIG. 7 (e.g., an adaptive display with two types of images), different stages of display may be shown based on image hierarchy. For instance, a first stage associated with priority of VL images displays VL images with a large size and endoscope images with relatively smaller size, a second stage associated with no priority of images displays VL images and endoscope images of substantially the same size, and a third stage associated with priority of endoscope images displays endoscope images with a large size and VL images with a relatively smaller size. Stated another way, images may have a set of sizes depending on their assigned hierarchy, such as a small size for secondary / subordinate images, a medium size for neutral / balanced-priority images, and a large size for priority images. Although three stages / sizes of display for image hierarchy are described, any number of stages / sizes is appreciated that allows for relative sizing of images based on assigned hierarchy. Additional discussion of determining image priority is provided at least with respect to FIG. 5.

[0082] If the video laryngoscope determines that an image has priority, the method 700 flows “YES” to operation 706. At operation 706, VL images and endoscope images are concurrently displayed that are relatively sized based on priority. For example, an image with priority may be displayed larger than an image without priority (e.g. a secondary image). For instance, when VL images have priority, the VL images are displayed larger than the endoscope images. Alternatively, when endoscope images have priority, the endoscope images are displayed larger than the VL images. Examples of displaying images with an assigned hierarchy (e.g., one of the concurrently displayed images is assigned priority) are furtherA0013433W001described in FIG. 5 regarding at least the discussion of endoscope priority display 502 and VL priority display 510.

[0083] If, alternatively, the video laryngoscope determines that there is no priority of images, the method 700 flows “NO” to operation 708. At operation 708, the VL images and the endoscope images are concurrently displayed and sized without priority. When no image is assigned priority, the concurrently displayed images may be sized similarly (e.g., the images may be substantially the same size). For instance, in the absence of an image hierarchy (e.g., none of the images are assigned priority), the VL images and the endoscope images may be displayed as substantially the same size on the display. Examples of displaying images without an assigned hierarchy (e.g., no priority is assigned) are further described in FIG. 5 regarding at least the discussion of neutral display 508.

[0084] At operation 710, an input is received at the video laryngoscope associated with priority. For example, a user interaction with a GUI of the video laryngoscope may change the priority of images displayed, such as a selection of an image, a double tap anywhere, a swipe, a press and hold, a flick, a pinch, interaction with a virtual joystick, or any other touch input. In another example, user interaction with any other user interface may change priority of the images displayed, such as input at buttons, shaking, joystick movement, or other movements or interaction with the video laryngoscope.

[0085] In some examples, the input associated with priority may be non-manual input. For instance, the input may be based on the images captured by the video laryngoscope and / or the images captured by the endoscope. For instance, if the VL images include interior patient anatomy, indicating that the VL camera is inserted into the body, the VL images may be given a higher priority (and thus displayed larger). In contrast, if the endoscope images include interior patient anatomy, indicating that the endoscope camera is inserted into the body, the endoscope images may be given a higher priority (and thus displayed larger). If both the VL images and the endoscope images include patient anatomy, the VL images and the endoscope images may be given equal priority.

[0001] To detect whether the images include patient anatomy, the images may be analyzed by the video laryngoscope, such as through the use of artificial intelligence (Al) and / or machine learning (ML) models. The images acquired by the cameras include multiple characteristics that may be analyzed by the technology disclosed herein. The characteristics may be used toA0013433W001determine if the camera is positioned inside of a body cavity or outside of a body cavity. Such characteristics may include image color (e.g., redness), detectable straight lines, detectable patient airway anatomy (e.g., vocal cords), fluctuations in color / brightness, etc. For example, image analysis of the images may determine that the camera is positioned outside of a body cavity of a patient when redness is below a threshold value, straight lines are detected (because straight lines rarely occur within patient anatomy), no airway anatomy is detected, and / or fluctuations in color / brightness are detected. Alternatively, in another example, image analysis of the images may determine that the camera is positioned inside a body cavity of a patient when redness is above a threshold value, straight lines are not detected, airway anatomy is detected, and / or fluctuations in color / brightness are not detected.

[0002] The object detection or identification may be accomplished through the use computer vision and / or AI / ML models that process the frames of the video feeds. In one example, a trained convolutional neural network (CNN) may be used to process the image frames of the video feed. A CNN is a class of deep neural networks that are effective at analyzing visual imagery. CNNs are composed of layers that include convolutional layers, pooling layers, and fully connected layers. The convolutional layers apply various filters to the input to create feature maps, which highlight specific features in the image. In the initial layers, simple features like edges and colors may be detected. As the data progresses through the network, more complex features like textures and patterns are identified. Some methods generate potential bounding boxes in the image where objects might be located. For each bounding box, the CNN predicts the probability of each object class (e.g., particular patient anatomy). The final output includes the class labels (e.g., patient anatomy) and bounding box coordinates for each detected object in the image or video frame.

[0086] Operations 702-710 may repeat as required or desired. For example, image hierarchy may be continuously determined to dynamically adapt / change the display. As image hierarchy changes, the relative sizing of concurrently displayed images in that hierarchy changes accordingly.

[0087] As should be appreciated from the foregoing, In an aspect, the technology relates to a method, performed by a video laryngoscope. The method includes acquiring first images using a camera of the video laryngoscope; detecting that a blade is coupled to the video laryngoscope; detecting that an endoscope is connected to the video laryngoscope; receiving second images using a camera of the endoscope; and in response to detecting the blade andA0013433W001detecting that the endoscope is connected to the video laryngoscope, concurrently displaying the first images and the second images at a display screen of the video laryngoscope.

[0088] In an example, the method further includes, before detecting that the endoscope is connected, displaying the first images acquired by the camera of the video laryngoscope and a settings indicator. In a further example, method further includes receiving a selection of the settings indicator; and in response to receiving the selection of the settings indicator, displaying a settings menu. In a still further example, the method further includes, in response to detecting the blade or detecting that the endoscope is connected, automatically hiding the settings menu and displaying at least one of the first images or the second images. In another example, the method further includes detecting a disconnect of the endoscope from the video laryngoscope; and in response to determining the disconnect of the endoscope, stopping display of the second images while continuing display of the first images. In still another example, the method further includes detecting a disconnect of the blade from the video laryngoscope; and in response to determining the disconnect of the blade, stopping display of the first images while continuing display of the second images. In yet another example, the second images at least partially overlap the first images. In still another example, the second images are at least partially translucent.

[0089] In another aspect, the technology relates to a video laryngoscope system that includes an endoscope having an endoscope camera that acquires endoscope images when the endoscope is powered on; and a video laryngoscope couplable to the endoscope. The video laryngoscope includes an integrated display; an arm; a laryngoscope camera positioned on the arm that acquires laryngoscope images; and a processor that operates to: detect that the endoscope is coupled to the video laryngoscope; in response to detecting that the endoscope is coupled to the video laryngoscope, display the endoscope images at the integrated display of the video laryngoscope; detect that a blade is coupled to the arm of the video laryngoscope; and in response to detecting that the blade is coupled to the arm, display the laryngoscope images at the integrated display of the video laryngoscope, wherein the endoscope images and the laryngoscope images are displayed concurrently.

[0090] In an example, the endoscope is connected to the video laryngoscope via a physical electrical connection. In another example, determining that the blade is coupled to the arm of the video laryngoscope is based at least in part on detecting at least a portion of the blade in the laryngoscope images. In a further example, the processor further operates to: based on theA0013433W001laryngoscope images, determine that the blade is at least a partially decoupled from the arm of the video laryngoscope; and in response to determining that the blade is at least partially decoupled, stop displaying the laryngoscope images. In yet another example, the processor further operates to: receive a touch input at the integrated display; and in response to the touch input, change a display size of the endoscope images relative to the laryngoscope images. In still another example, the endoscope images are displayed on a larger percentage of the integrated display than the laryngoscope images.

[0091] In another aspect, the technology relates to a method, performed by a video laryngoscope, for displaying two types of images concurrently. The method includes acquiring laryngoscope images from a laryngoscope camera of the video laryngoscope; receiving endoscope images from an endoscope camera of an endoscope electrically connected to the video laryngoscope; displaying the laryngoscope images and the endoscope images, wherein the endoscope images at least partially overlap a lower left quadrant of the laryngoscope images and wherein the endoscope images are relatively smaller than the laryngoscope images; receiving a first user input; in response to the first user input, changing display of the laryngoscope images and the endoscope images wherein the endoscope images are relatively similar in size compared to the laryngoscope images; receiving a second user input; and in response to the second user input, changing display of the laryngoscope images and the endoscope images wherein the endoscope images are relatively larger in size compared to the laryngoscope images.

[0092] In an example, wherein the first user input and the second user input are touch inputs. In a further example, the touch inputs are associated with changing an assigned image hierarchy of the laryngoscope images and the endoscope images. In another example, the laryngoscope images and the endoscope images are displayed as two different shapes. In still another example, the laryngoscope images are displayed in a substantially rectangular shape and wherein the endoscope images are displayed in a substantially ovular shape. In still another example, the endoscope images are at least partially translucent.

[0093] The techniques introduced above may be implemented for a variety of medical devices or devices where direct and indirect views are possible. A person of skill in the art will understand that the technology described in the context of a video laryngoscope for human patients could be adapted for use with other systems such as laryngoscopes for non-human patients or medical video imaging systems.A0013433W001

[0094] Those skilled in the art will recognize that the methods and systems of the present disclosure may be implemented in many manners and as such are not to be limited by the foregoing aspects and examples. In other words, functional elements being performed by a single component or multiple components, in various combinations of hardware and software or firmware, and individual functions, can be distributed among software applications at either the client or server level or both. In this regard, any number of the features of the different aspects described herein may be combined into single or multiple aspects, and alternate aspects having fewer than or more than all of the features herein described are possible.

[0095] Functionality may also be, in whole or in part, distributed among multiple components, in manners now known or to become known. Thus, a myriad of software / hardware / firmware combinations are possible in achieving the functions, features, interfaces, and preferences described herein. Moreover, the scope of the present disclosure covers manners for carrying out the described features and functions and interfaces, and those variations and modifications that may be made to the hardware or software firmware components described herein as would be understood by those skilled in the art now and hereafter. In addition, some aspects of the present disclosure are described above with reference to block diagrams and / or operational illustrations of systems and methods according to aspects of this disclosure. The functions, operations, and / or acts noted in the blocks may occur out of the order that is shown in any respective flowchart. For example, two blocks shown in succession may in fact be executed or performed substantially concurrently or in reverse order, depending on the functionality and implementation involved.

[0096] Further, as used herein and in the claims, the phrase “at least one of element A, element B, or element C” is intended to convey any of: element A, element B, element C, elements A and B, elements A and C, elements B and C, and elements A, B, and C. In addition, one having skill in the art will understand the degree to which terms such as “about” or “substantially” convey in light of the measurement techniques utilized herein. To the extent such terms may not be clearly defined or understood by one having skill in the art, the term “about” shall mean plus or minus ten percent.

[0097] Numerous other changes may be made which will readily suggest themselves to those skilled in the art and which are encompassed in the spirit of the disclosure and as defined in the appended claims. While various aspects have been described for purposes of this disclosure, various changes and modifications may be made which are well within the scopeA0013433W001of the disclosure. Numerous other changes may be made which will readily suggest themselves to those skilled in the art and which are encompassed in the spirit of the disclosure and as defined in the claims.

[0098] The following examples are illustrative of the techniques described herein.

[0099] Example 1. A method, performed by a video laryngoscope, comprising: acquiring first images using a camera of the video laryngoscope; detecting that a blade is coupled to the video laryngoscope; detecting that an endoscope is connected to the video laryngoscope; receiving second images using a camera of the endoscope; and in response to detecting the blade and detecting that the endoscope is connected to the video laryngoscope, concurrently displaying the first images and the second images at a display screen of the video laryngoscope.

[0100] Example 2. The method of Example 1, the method further comprising: before detecting that the endoscope is connected, displaying the first images acquired by the camera of the video laryngoscope and a settings indicator.

[0101] Example s. The method of Example 2, the method further comprising: receiving a selection of the settings indicator; and in response to receiving the selection of the settings indicator, displaying a settings menu.

[0102] Example 4. The method of Example 3, the method further comprising: in response to detecting the blade or detecting that the endoscope is connected, automatically hiding the settings menu and displaying at least one of the first images or the second images.

[0103] Example 5. The method of Example 1, the method further comprising: detecting a disconnect of the endoscope from the video laryngoscope; and in response to determining the disconnect of the endoscope, stopping display of the second images while continuing display of the first images.

[0104] Example 6. The method of Example 1, the method further comprising: detecting a disconnect of the blade from the video laryngoscope; and in response to determining the disconnect of the blade, stopping display of the first images while continuing display of the second images.A0013433W001

[0105] Example 7. The method of Example 1, wherein the second images at least partially overlap the first images.

[0106] Example 8. The method of Example 7, wherein the second images are at least partially translucent.

[0107] Example 9. A video laryngoscope system, comprising: an endoscope having an endoscope camera that acquires endoscope images when the endoscope is powered on; and a video laryngoscope couplable to the endoscope, the video laryngoscope comprising: an integrated display; an arm; a laryngoscope camera positioned on the arm that acquires laryngoscope images; and a processor that operates to: detect that the endoscope is coupled to the video laryngoscope; in response to detecting that the endoscope is coupled to the video laryngoscope, display the endoscope images at the integrated display of the video laryngoscope; detect that a blade is coupled to the arm of the video laryngoscope; and in response to detecting that the blade is coupled to the arm, display the laryngoscope images at the integrated display of the video laryngoscope, wherein the endoscope images and the laryngoscope images are displayed concurrently.

[0108] Example 10. The system of Example 9, wherein the endoscope is connected to the video laryngoscope via a physical electrical connection.

[0109] Example 11. The system of Example 9, wherein determining that the blade is coupled to the arm of the video laryngoscope is based at least in part on detecting at least a portion of the blade in the laryngoscope images.

[0110] Example 12. The system of Example 11, wherein the processor further operates to: based on the laryngoscope images, determine that the blade is at least a partially decoupled from the arm of the video laryngoscope; and in response to determining that the blade is at least partially decoupled, stop displaying the laryngoscope images.

[0111] Example 13. The system of Example 9, wherein the processor further operates to: receive a touch input at the integrated display; and in response to the touch input, change a display size of the endoscope images relative to the laryngoscope images.

[0112] Example 14. The system of Example 13, wherein the endoscope images are displayed on a larger percentage of the integrated display than the laryngoscope images.A0013433W001

[0113] Example 15. A method, performed by a video laryngoscope, for displaying two types of images concurrently, the method comprising: acquiring laryngoscope images from a laryngoscope camera of the video laryngoscope; receiving endoscope images from an endoscope camera of an endoscope electrically connected to the video laryngoscope; displaying the laryngoscope images and the endoscope images, wherein the endoscope images at least partially overlap a lower left quadrant of the laryngoscope images and wherein the endoscope images are relatively smaller than the laryngoscope images; receiving a first user input; in response to the first user input, changing display of the laryngoscope images and the endoscope images wherein the endoscope images are relatively similar in size compared to the laryngoscope images; receiving a second user input; and in response to the second user input, changing display of the laryngoscope images and the endoscope images wherein the endoscope images are relatively larger in size compared to the laryngoscope images.

[0114] Example 16. The method of Example 15, wherein the first user input and the second user input are touch inputs.

[0115] Example 17. The method of Example 16, wherein the touch inputs are associated with changing an assigned image hierarchy of the laryngoscope images and the endoscope images.

[0116] Example 18. The method of Example 15, wherein the laryngoscope images and the endoscope images are displayed as two different shapes.

[0117] Example 19. The method of Example 18, wherein the laryngoscope images are displayed in a substantially rectangular shape and wherein the endoscope images are displayed in a substantially ovular shape.

[0118] Example 20. The method of Example 19, wherein the endoscope images are at least partially translucent.

Claims

A0013433W001CLAIMSWhat is claimed is:

1. A method (600), performed by a video laryngoscope, comprising:acquiring (604) first images using a camera of the video laryngoscope;detecting (612) that a blade is coupled to the video laryngoscope;detecting (614) that an endoscope is connected to the video laryngoscope; receiving (618) second images using a camera of the endoscope; andin response to detecting the blade and detecting that the endoscope is connected to the video laryngoscope, concurrently displaying (622) the first images and the second images at a display screen of the video laryngoscope.

2. The method of claim 1, the method further comprising:before detecting that the endoscope is connected, displaying the first images acquired by the camera of the video laryngoscope and a settings indicator.

3. The method of claim 2, the method further comprising:receiving a selection of the settings indicator; andin response to receiving the selection of the settings indicator, displaying a settings menu.

4. The method of claim 3, the method further comprising:in response to detecting the blade or detecting that the endoscope is connected, automatically hiding the settings menu and displaying at least one of the first images or the second images.

5. The method of any one of claims 1-4, the method further comprising:detecting a disconnect of the endoscope from the video laryngoscope; andin response to determining the disconnect of the endoscope, stopping display of the second images while continuing display of the first images.A0013433W0016. The method of any one of claims 1-4, the method further comprising:detecting a disconnect of the blade from the video laryngoscope; andin response to determining the disconnect of the blade, stopping display of the first images while continuing display of the second images.

7. The method of any one of claims 1-6, wherein the second images at least partially overlap the first images.

8. The method of claim 7, wherein the second images are at least partially translucent.

9. A video laryngoscope system, comprising:an endoscope (150) having an endoscope camera that acquires endoscope images when the endoscope is powered on; anda video laryngoscope (102) couplable to the endoscope, the video laryngoscope comprising:an integrated display (108);an arm (114);a laryngoscope camera (116) positioned on the arm that acquires laryngoscope images; anda processor (162) that operates to:detect that the endoscope is coupled to the video laryngoscope;in response to detecting that the endoscope is coupled to the video laryngoscope, display the endoscope images at the integrated display of the video laryngoscope;detect that a blade is coupled to the arm of the video laryngoscope; and in response to detecting that the blade is coupled to the arm, display the laryngoscope images at the integrated display of the video laryngoscope, wherein the endoscope images and the laryngoscope images are displayed concurrently.

10. The system of claim 9, wherein the endoscope is connected to the video laryngoscope via a physical electrical connection.A0013433W00111. The system of any one of claims 9- 10, wherein determining that the blade is coupled to the arm of the video laryngoscope is based at least in part on detecting at least a portion of the blade in the laryngoscope images.

12. The system of claim 11, wherein the processor further operates to:based on the laryngoscope images, determine that the blade is at least a partially decoupled from the arm of the video laryngoscope; andin response to determining that the blade is at least partially decoupled, stop displaying the laryngoscope images.

13. The system of claim 9, wherein the processor further operates to:receive a touch input at the integrated display; andin response to the touch input, change a display size of the endoscope images relative to the laryngoscope images.

14. The system of claim 13, wherein the endoscope images are displayed on a larger percentage of the integrated display than the laryngoscope images.

15. The system of claim of any one of claims 9-14, wherein the laryngoscope images and the endoscope images are displayed as two different shapes.

Images