Systems and methods for automated intubation

Abstract

11. A system, method and apparatus for automatically performing endotracheal intubation on a patient, comprising: inserting a blade into the patient's upper airway and retracting an anatomical structure; inserting a bend and a tube disposed on the bend into the patient's airway; collecting airway data using at least one imaging sensor disposed on the bend; communicating the collected airway data to a processing circuit; predicting a target path for inserting the tube and generating a control signal using the processing circuit, wherein the target path is predicted based on at least one anatomical structure recognized by the processing circuit using the collected airway data; displaying target paths via a user interface to display at least one target path to an operator and also to allow the operator to select a target path; and communicating the control signal generated by the processing circuit to at least one actuation unit to actuate the three-dimensional movement of the tube.

Description

[Technical field]

[0001] The present invention relates to automated systems and methods for inserting invasive medical instruments inside a patient, and more particularly, to automated systems and methods for inserting invasive medical instruments into a body cavity of a patient using image-based guidance.

[0002] This section describes the technical field in detail and explains the problems faced in the art, and therefore statements in this section should not be construed as prior art.

[0003] Efficient implantation of medical devices within a patient's body is one of the greatest needs felt by the medical community today. One reason for the rise in need is the vast range of applications that invasive medical devices offer, ranging from the insertion of a pacemaker into the chest to keep the heart beating at the proper rate, to the insertion of a urinary catheter. Another reason is the numerous complexities and complexities that medical personnel, physicians, and anesthesiologists encounter during implantation procedures, which necessitates a sharp shift towards the prevention of morbidity and mortality.

[0004] One such application of invasive device implantation is endotracheal intubation, which is performed to keep a patient's airway open and support breathing. Endotracheal intubation (or ETI) is performed to visualize the glottic opening by using a laryngoscope and then insert a tube through it. The physician can see the glottis directly through his or her eyes after manipulating the anatomy of the upper airway with the laryngoscope and forming a "straight line of sight". Clear visualization of the glottic opening with the laryngoscope depends on several factors such as facial structure, Mallampati classification, dental condition, and joint stiffness. Thus, endotracheal intubation is a process that requires a lot of skill and training. Even with proper training, it can be difficult to visualize the glottic opening and insert a tube.

[0005] It is estimated that approximately 81% of endotracheal intubations are performed by non-physicians and 19% by physicians during first aid procedures. The unpredictable environment during first aid procedures adds to the complexity of successful intubation. The first attempt failure rate during endotracheal intubation is estimated to be as high as 41%. This delay in intubating the patient has serious consequences. Hypoxia can cause permanent brain damage within four minutes and death within ten minutes.

[0006] Alternative methods of intubation using videolarynoscopes offer a much better view because they include a camera at the tip of the scope and therefore do not require a "straight line of sight." The camera projects an image onto a monitor, which allows the endotracheal tube to be inserted manually by the physician by looking at the monitor. This also requires a lot of manual dexterity and visual spatial awareness, which are also difficult skills to master. The first-attempt failure rate with videolarynoscopes can also be higher.

[0007] If the patient cannot be intubated, several alternative methods are tried, including supraglottic ventilators, specialized airway devices such as King's tube or Combitube, mask ventilation, and in some cases even emergency cricothyrotomy, which means making an incision in the neck and trachea and inserting a tube through that opening. As expected, these procedures are not as effective as simple endotracheal intubation and can be much more invasive for the patient, with long-term sequelae.

[0008] Most of the current guided intubation systems and methods have limitations that result in challenges such as higher delays and failure rates during intubation. Therefore, there is a clear need to design systems and methods that can not only assist in rapid and successful intubation, but also operate completely autonomously and with minimal operator (or user) intervention. Operator and user may be used interchangeably.

[0009] Patients severely affected by severe respiratory infections such as the COVID-19 virus progress to respiratory distress and require intubation and mechanical ventilation. Healthcare personnel are in close proximity to infected patients and have direct contact with the saliva of such patients, and therefore are at risk of contracting the disease while following the standard of care for such patients. Moreover, the transfer of disease to healthcare personnel is directly related, among other things, to the duration and extent of contact with the patient, making ETI a high-risk procedure for the transfer of infection.

[0010] The present invention aims, inter alia, to overcome the drawbacks of the prior art, such as those mentioned above. Summary of the Invention

[0011] Use of terms such as "one embodiment," "at least one embodiment," "embodiment," "an example," "example," "for example," and the like indicates that the embodiment or example may include a particular feature, structure, characteristic, property, element, or limitation, but not all embodiments or examples necessarily include the particular feature, structure, characteristic, property, element, or limitation. Moreover, repeated use of the phrase "in an embodiment" does not necessarily refer to the same embodiment.

[0012] In an embodiment of the invention, an automated system inserts an invasive medical device into a body cavity of a patient. The automated system includes a processing circuit that receives data from at least one data source to recognize structures associated with the body cavity of the patient and predicts a target path for inserting the invasive medical device into the patient. The processing circuit further generates and communicates control signals to at least one actuation unit based on the target path to actuate three-dimensional movement of the invasive medical device.

[0013] The processing circuitry, together with the data received from the data source, can utilize the machine learning model to recognize structures associated with the patient's body cavity, thereby predicting a target path and actuating the three-dimensional movement of the invasive medical device by generating and communicating control signals to the actuation unit. The target path is a path along which the device will guide the invasive medical device once movement has begun. The generation of the machine learning model includes receiving or collecting training data in the form of a predefined data set for training at least one neural network. The form of the neural network can be an edge-implemented deep neural network-based object detector, as is known in the art. Other forms of machine learning other than neural networks are alternatives, as is known to those skilled in the art. The predefined data set can be, but is not limited to, images and videos.

[0014] The data source may be an imaging sensor. These sensors may include, but are not limited to, a camera, an infrared camera, an acoustic sensor, a microwave sensor, a light detector, or other sensors known to those skilled in the art may be employed to achieve the same purpose. The data received from the imaging sensor may be displayed on a user interface to provide the operator with a view of the patient's body cavity. Furthermore, the object path and recognized structures may be overlaid on the data received from the imaging sensor on the user interface for effective visual guidance to the operator.

[0015] In an exemplary embodiment of the invention, the automated intubation system predicts a target path for inserting a tube and generates a control signal for at least one actuation unit. The target path is predicted based on at least one anatomical structure recognized with data received from at least one imaging sensor. An overlay of the target path and / or the recognized anatomical structure is also displayed on the user interface on the data received by the user interface from the imaging sensor for effective visual guidance during intubation. The target path displayed on the user interface is also adjustable and / or overriden by the operator if the operator is not satisfied with the insertion target path. The operator may then select a proposed or adjusted target path for the system to follow during the intubation process.

[0016] Additionally, the object path overlay may be visualized on a user interface, in the form of augmented reality, and / or in any other form that provides effective visual guidance to the operator.

[0017] In a preferred embodiment, the automatic intubation system comprises a body, a bending section, a flexible section connecting the body to the bending section, a housing unit with at least one imaging sensor disposed on the bending section, an intubation tube disposed on the flexible section and the bending section, a circuit, a user interface, a disposable and / or reusable sleeve having a blade at one end for retracting the anatomical structure, and at least one actuation unit for actuating the three-dimensional movement of the tube. The length of the bending unit is variable and may be located only at the tip of the flexible section or may completely cover the flexible section. In other embodiments, the bending section may be located within any part of the flexible section, determined by several factors including, but not limited to, the relevant application and the anatomical structure that needs to be navigated. Preferably, the disposable and / or reusable sleeve is removably coupled to the body. The imaging sensor is preferably a camera, but sensors such as infrared, photodetectors, or other feasible means known to those skilled in the art may be employed to achieve the same purpose.

[0018] In a preferred embodiment of the present invention, the circuitry, the user interface and the operating unit are part of the main body. The circuitry further comprises a processing circuitry, a power circuitry and a communication circuitry.

[0019] In an alternative embodiment of the invention, the circuitry and user interface are located in at least one separate box, separate from the main body.

[0020] The processing circuit is utilized for both predicting a target path for inserting the tube based on at least one recognized anatomical structure and generating a control signal. The processing circuit is also utilized for recognizing the anatomical structure using data received from the imaging sensor and at least one pre-trained machine learning model. The actuation unit receives the control signal from the processing circuit and actuates the three-dimensional movement of the tube. The actuation unit specifically actuates the bending movement of the tube in the X and Y planes using a connection with the flexure. The actuation unit also has a sliding mechanism that actuates the sliding movement of the tube in the Z plane by moving the flexure and its associated actuation unit on a rail track. Alternatively, the sliding mechanism actuates the sliding movement of the tube in the Z plane by direct contact or abutment with the tube without displacing the flexure and its associated actuation unit. Those skilled in the art will also recognize that other three-dimensional coordinate schemes such as radial, polar, cylindrical, and spherical can be used in alternative forms of the x, y, and z coordinates described herein.

[0021] In another embodiment of the invention, a processing circuit is used only to predict the object path and generate control signals, and recognition of anatomical structures using imaging sensor data and machine learning models is performed by a separate and independent processing circuit.

[0022] The machine learning model is a portion of computer vision software developed by training one or more neural networks against a labeled dataset of images, the labeled dataset of images being constructed by converting collected intubation procedure footage into image files and labeling anatomical structures on the image files. In an alternative embodiment, generating the machine learning model includes receiving or collecting training data in the form of a predefined dataset for training at least one neural network. The predefined dataset may be, but is not limited to, images, audio, and video recorded and collected during the procedure.

[0023] In another embodiment of the invention, the control signals received by the actuation unit to actuate the three-dimensional movement of the tube are generated manually by an up / down button pair located on the exterior surface of the body or a touch button located on the user interface, so that the system provides a manual mode of actuation if required by the operator. The up / down button pair and the touch button may be used by the operator to override the automatic actuation of the tube if the operator is not satisfied with the target path.

[0024] In another aspect of the present invention, a method for automatically inserting an invasive medical device into a body cavity of a patient is provided, comprising inserting a bend and an invasive medical device disposed on the bend into the body cavity of the patient. The method includes collecting airway data using an imaging sensor disposed on the bend, and communicating the collected airway data to a processing circuit to predict a target path of insertion of the invasive medical device and generate a control signal. The control signal is then communicated to at least one actuation unit to actuate a three-dimensional movement of the invasive medical device. The target path is preferably predicted by the processing circuit based on recognition of at least one structure associated with the body cavity using data communicated from the imaging sensor.

[0025] Further, prediction of the target path of insertion and recognition of structures associated with the body cavity may be performed by the processing circuitry by utilizing a machine learning model together with data communicated from the imaging sensor. The generation of the machine learning model includes receiving or collecting training data in the form of a predetermined data set to train at least one neural network. The predetermined data set may be, but is not limited to, images and videos. It is foreseeable that the devices disclosed in this patent may be utilized in different body cavities other than the airway described herein, or perform different tasks within any of these body cavities.

[0026] In an exemplary embodiment of the invention, a method is provided for automatically intubating a patient by inserting a bend and a tube disposed on the bend into the patient's airway. The method further includes collecting airway data with an imaging sensor disposed on the bend and communicating the collected airway data to a processing circuit to predict a target path of insertion of the tube and generate control signals for actuating three-dimensional movement of the tube. The target path is preferably predicted by the processing circuit based on recognition of at least one anatomical structure using data communicated from the imaging sensor. The processing circuit utilizes a machine learning model and data communicated from the imaging sensor to recognize the anatomical structure and predict a target path of insertion of the tube.

[0027] The method may include displaying the airway data on a user interface to highlight a view of the airway to the operator, and further includes overlaying, on the user interface, the path of interest and recognized anatomical structures onto the data communicated from the imaging sensor for effective visual guidance to the operator.

[0028] There are advantages to having a semi-automated invasive device insertion system compared to a fully automated system. Commercialization of such a system requires regulatory approval from government agencies such as the FDA, and the pathway of a semi-automated system may be simpler and less complicated. Furthermore, having a fully automated system could potentially create layers of legal liability to which companies may be vulnerable. Furthermore, even if the technology is good, it is good for a trained professional to oversee the procedure and manually override it if necessary to ensure accurate intubation. When comparing a semi-automated system to a fully automated system, there may be fewer technical barriers to developing and producing a deployable system. Finally, having an inherent validation and control mechanism and layers of usability that achieve a precise pathway prevents injury and is safer for the patient.

[0029] In alternative embodiments, complementary sensors may be integrated into the device that can provide real-time information regarding the patient's relevant clinical parameters, such as vital signs including, but not limited to, pulse and heart rate, respiratory rate, oxygen saturation levels, temperature, blood pressure; and other laboratory results, including but not limited to, blood gas levels, glucose levels, and other results that would be known to a person trained in the state of the art.

[0030] In other embodiments, an operator may connect to the instrument remotely via the Internet and operate the instrument using a similar user interface.

[0031] Other embodiments and preferred features of the invention, together with corresponding advantages, will become apparent from the following description and claims. [Brief description of the drawings]

[0032] Various aspects as well as embodiments of the present invention may be better understood by reference to the following detailed description. For a better understanding of the present invention, the detailed description should be read in conjunction with the drawings.

[0033] [Figure 1] 1 illustrates an exemplary architecture of an automated system for inserting an invasive medical device inside a patient according to the present invention.

[0034] [Diagram 2] 1 illustrates an exemplary embodiment of an automated intubation system according to the present invention.

[0035] [Diagram 3] 1 shows the body, disposable sleeve, and tube assembly of an automatic intubation system according to the present invention.

[0036] [Figure 4] 1 illustrates an alternative embodiment of an automatic intubation system according to the present invention.

[0037] [Diagram 5] 3 shows the configuration of a bending portion according to the present invention.

[0038] [Figure 6] 1 illustrates an exemplary architecture of an automated intubation system according to the present invention.

[0039] [Figure 7] 1 shows a flow diagram for generating a machine learning model according to the present invention.

[0040] [Figure 8] 1 illustrates the use of an exemplary automated intubation method according to the present invention.

[0041] [Figure 9] 1 illustrates the use of a user interface according to the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0042] The present disclosure is best understood with reference to the detailed figures and descriptions set forth herein. Various embodiments are described with reference to the figures. However, those skilled in the art will readily understand that the detailed description provided herein with respect to the figures is for illustrative purposes only, since the method and system may be expanded beyond the embodiments described. For example, the teachings presented and the needs of a specific application may result in multiple alternatives and suitable approaches for implementing the functionality of any details described herein. Thus, any approach may be expanded beyond the specific implementation selection in the following embodiments.

[0043] The method of the present invention may be implemented by performing or executing selected steps or tasks manually, automatically, or a combination thereof. The term "method" refers to ways, means, techniques, and procedures for accomplishing a given task, including, but not limited to, ways, means, techniques, and procedures that are either known to those skilled in the art to which the invention pertains or that can be readily developed by those skilled in the art from known ways, means, techniques, and procedures. The descriptions, examples, methods, and materials presented in the claims and specification should not be construed as limiting, but are presented by way of example only. Those skilled in the art will envision numerous other possible variations within the scope of the technology described herein.

[0044] A person reading the description of the exemplary embodiment of the best mode of the invention (hereinafter referred to as the "exemplary embodiment") will consider the exemplary embodiment to be the best mode for carrying out the invention at the time of filing the patent application according to the inventor's belief. The exemplary embodiment should not be construed as limiting the invention to one embodiment, since a person skilled in the art may recognize substantially equivalent structure or substantially equivalent function that accomplishes the same result in the same or dissimilar manner.

[0045] A description of a species (or a specific item) refers to the genus (class of items) to which the species belongs and related species within this genus. Similarly, a description of a genus refers to the species known in the art. Moreover, as technology advances, many additional alternatives for implementing embodiments of the invention may emerge. Such advances will be incorporated within their respective genuses and recognized as functionally equivalent or structurally equivalent to the embodiments shown or described.

[0046] Unless expressly stated otherwise, conjunctions (such as "or," "and," "including," or "comprising") are to be construed in an inclusive sense and not in an exclusive sense.

[0047] As will be appreciated by those skilled in the art, various structures and devices are illustrated in block diagrams in order to avoid obscuring the present invention. In the following description, it should be noted that, unless otherwise stated, similarly named and functioning entities are implemented in a similar manner.

[0048] The foregoing explanations and definitions are provided for purposes of clarification, not limitation. Words and phrases are to be accorded their ordinary and plain meaning unless otherwise indicated.

[0049] The present invention can be better understood with reference to the figures. Figure 1 is a diagram of an exemplary architecture of an automated system 100 for inserting an invasive medical device into a body cavity of a patient. The system comprises a flexure 101, an imaging sensor 102, an invasive medical device 103, at least one actuation unit 104, a user interface 105, and a circuit 106. The circuit further comprises a processing circuit 106a for generating control signals based on inputs from the at least one imaging sensor and a machine learning model, a communication circuit 106b for providing data / signal communication between different components of the system, and a power circuit 106c. The actuation unit includes a slide mechanism 107 for providing the invasive medical device with movement in the Z-plane.

[0050] The processing circuit 106a may be a single processor, a logic circuit, a dedicated controller performing all functions, or a combination of processing support units depending on the functional requirements of the system. In an exemplary embodiment, the processing circuit has two independent processing support units 106aa and 106ab. The processing support unit 106aa is a computer vision software that performs at least one function (106aa1, 106aa2...106aaN) to automate the intubation process using machine learning techniques and data received from the imaging sensor 102. The functions include structure recognition around and within the patient's body cavity, and object path prediction for inserting the invasive medical device 103 into the patient. Alternatively, the processing circuit 106aa predicts the object path based on input from the imaging sensor, samples of past data received remotely from multiple device operating units, or machine learning models. The system further stores the object path to maintain a record of device operation for regulatory purposes in a memory (not shown in the system). The device record may be shared with remote devices for monitoring and control purposes. Images from one or more imaging sensors and further information such as status and decision points may be stored or shared with a remote server to further improve the machine learning model or for other purposes such as regulatory or training purposes. This information may be stored locally on the device or on a remote storage such as a server or on the cloud. The processing support unit 106ab generates a control signal based on the target path predicted by the processing support unit 106aa. To provide the invasive medical device with a three-dimensional movement, the control signal generated by the processing support unit 106ab is then communicated from the processing circuit to the actuation unit 104 via the communication circuit 106b on the basis of which the actuation unit actuates at least one of the bending portion 101 and the sliding mechanism 107. The processing support unit 106ab may be an integrated part of the actuation unit 104, and the control signal may be received by the actuation unit 104 via a wireless or wired communication circuit. The processing circuit 106aa may be remotely connected to the actuation unit 104 through a network or wireless medium to transmit the control signal.The communication circuitry may be an integral part of the operational unit. Each of the above mentioned functions may be combined with other functions, with each and every one of the above mentioned functions being combined with other functions in a single functional unit.

[0051] The communication circuitry 106b may be distributed in the complete system and function as an element of bidirectional data / signal transfer. The communication circuitry may be wired or wireless. The power circuitry 106c distributes power to all units of the system. The power circuitry may include a rechargeable battery or a direct regulated power source.

[0052] The actuation unit 104 may be a rotary motor, a linear motor, and / or a combination of both rotary and linear motors. In an exemplary embodiment, multiple actuation units (A1, A2...An) actuate the flexure 101 and the sliding mechanism 107 independently to provide three-dimensional movement. Alternatively, the flexure 101 and the sliding mechanism 107 may be actuated in an integrated manner with each other using a single actuation unit. The system can track the movement of the invasive medical device, compare it to a target path, calculate the deviation, and calibrate the movement. The calibration may be done automatically or by manual intervention. The actual movement data may be transmitted to a remote device for monitoring purposes.

[0053] The user interface 105 is in bidirectional communication with the processing circuitry 106a. The user interface is preferably a display device that displays data received from the imaging sensor 102 and an overlay of recognized structures and / or object paths from the processing circuitry on the data received from the imaging sensor to assist the operator with effective visual guidance. Alternatively, the user interface may be any device that may enable the operator to interact with an automated system, such as audio input / output, gesture input, an augmented reality enabled system, and / or a projection device. The user interface may be a head-up or head-mounted display that supports a virtual reality form of interaction. The user interface 105 may be used to select a proposed object path or to override the proposed path and select a modified object path formed by the operator modifying the proposed object path.

[0054] 2 is a diagram of an exemplary embodiment of an automated intubation system 200, which comprises a body 201, a flexible portion 202 connecting the body to a flexure 203, and a housing unit 204 attached to the flexure. The housing unit further supports at least one imaging sensor 205, at least one guide light 206, and at least one outflow channel 207. Preferably, the imaging sensor is a wide CMOS camera and the guide light is an LED light that is automatically turned on when the system is powered on. Alternatively, independent control switches for the guide light and the imaging sensor may be provided.

[0055] The body further comprises at least one actuation unit 208 that converts control signals received from the processing circuit into three dimensional movements for advancing the tube in the patient's body cavity. The actuation unit 208 may be a rotary motor, a linear motor, and / or a combination of both rotary and linear motors. Optionally, the exterior surface of the body 201 has at least one button or knob 209 for manual control of actuation, a light source 210 for indicating the power status of the automated system 200, a switch 211 for powering the automated system on or off, at least one port 212 for suction, and a tube release switch or lever 213 for decoupling the tube from the body.

[0056] In one embodiment, the actuation unit 208 further comprises a sliding mechanism 214. The sliding mechanism may be either an integral part of the actuation unit or a separate unit connected to the actuation unit. The sliding mechanism may be a movable base plate connected to the actuation unit via a rack and pinion mechanism (not shown), where the pinion is connected to the actuation unit for rotational motion and the rack is connected to the movable base plate to convert the rotational motion into vertical motion and / or displacement. A person skilled in the art will know other ways or mechanisms to connect the actuation unit to the movable base plate and achieve the same sliding mechanism. The main purpose of the sliding mechanism is to provide Z-plane movement to the tube. The use of the sliding mechanism actuation unit 208 is not required by this disclosure, and a number of electromechanical systems may be used to provide movement in the Z-plane to the inserted medical instrument, as disclosed below.

[0057] Alternatively, two independent actuation units may be used to actuate the flexure 203 and the sliding mechanism 214. A processing circuit (shown in FIG. 1) may send control signals for X and Y plane movement to an actuation unit that controls the movement of the flexure, and send control signals for Z plane movement to an actuation unit associated with the sliding mechanism.

[0058] Alternatively, many different arrangements of actuation units can be made for three dimensional movement of the tube, which will be readily apparent to one skilled in the art. These may include the use of rotary, geared, coiled, or screw based actuation units, and floating actuation units. Due care must be taken to allow for precise X and Y movement, and the amount of movement required in the Z plane.

[0059] A user interface 215 is also attached to the body 201 to display data received from the imaging sensor 205. Preferably, the user interface is a display device attached to the body. Alternatively, the user interface is a touch-sensitive display device having at least one button to trigger an actuation, a button to release the tube, and a power button (not shown). The user interface may be any device that may enable an operator's interaction with the automated system, such as audio input, audio output, or gesture input. In another embodiment, the user interface may have an intelligent agent that provides the necessary operator feedback.

[0060] The body 201 also contains circuitry 216, which further includes processing circuitry, communication circuitry, and power circuitry.

[0061] The flexure 203 is connected to the actuation unit 208. Preferably, the flexure 203 is connected to the actuation unit 208 via at least one cord (not shown in FIG. 2). The cord is connected to the actuation unit, passes through the flexible portion, reaches and connects to the flexure, and actuates the bending movement and / or movement of the flexure. Alternatively, the cord may be replaced by any feasible mechanical link, such as threads, wires, cables, and chains. A person skilled in the art will know other ways or means to connect the actuation unit to the flexure and provide the flexure 203 with a two-dimensional movement in the X and Y planes.

[0062] 3 is a diagram of the assembly of the body 201 with the tube 301 and sleeve 302 of the automatic intubation system 200. The tube may be disposed longitudinally over the flexible portion 202 and the bend 203. Alternatively, the tube may be disposed partially over the flexible portion and partially over the bend. Generally, the flexible portion extends through the tube and provides a view of the respiratory passageway via an imaging sensor supported by the housing unit 204. The tube is an endotracheal tube or any custom tube, which may include, but is not limited to, oral, nasal, cuffed, uncuffed, preform reinforced, double lumen tracheal tube.

[0063] The sleeve 302 may be mechanically connected to the body 201 and detachably connect the blade 303 to the body, preferably via a slip fit connection. Other feasible mechanical connections known to those skilled in the art may also be employed to achieve the same purpose. The detachable blade 303 at one end of the sleeve 302 is provided for retracting anatomical structures during the intubation procedure. The sleeve may be formed of disposable and / or reusable materials.

[0064] The blade 303 is designed to improve the effectiveness of the blade to provide better visibility during the intubation process and may be shaped similarly to the blades of conventional video laryngoscopic instruments. The blade also has an integrated pathway for guiding the tube in the early stages of intubation. The pathway may be an open tunnel through which the tube can pass, or may be formed in the blade with recesses, rails, grooves, or combinations thereof.

[0065] When the tube 301 is disposed on the flexible portion and the bend, it may contact the sliding mechanism 214. When the tube contacts the sliding mechanism, the tube can be displaced in the Z-plane along the flexible portion 202 and / or the bend 203 when the actuation unit 208 actuates the sliding mechanism.

[0066] Alternatively, the sliding mechanism 208 displaces the flexion portion 203 and the associated actuation unit in the Z-plane to insert and retract the flexion portion into the patient's trachea. The actuation unit associated with the flexion portion is specifically disposed on a rail guide (not shown) of the sliding mechanism, so that the actuation unit associated with the sliding mechanism can displace it accordingly.

[0067] The tube 301 is connected to the actuation unit 208 via its location on at least one of the flexible portion 202 and the bend 203. The actuation unit actuates the bend and thus the bending movement of the tube in the X and Y planes. In simple terms, the bend acts as a guide for the tube to navigate its direction within the patient's airway.

[0068] FIG. 4 is a diagram of an alternative embodiment of an automatic intubation system 400, again comprising a body 401, a flexible portion 402 connecting the body to a flex portion 403, and a housing unit 404 attached to the flex or flexible portion. The housing unit may support at least one imaging sensor 405, at least one guide light 406, and at least one outflow channel 407. The outflow channel 407 may be used to provide a channel if additional instruments need to be inserted, such as biopsy, suction, irrigation, etc. The outflow channel 407 may be used to provide a channel if additional instruments need to be inserted, such as biopsy, suction, irrigation, etc. The body further comprises at least one actuation unit 408, which may be a rotary motor, a linear motor, and / or a combination of both rotary and linear motors. Other types of motors will be readily apparent to one skilled in the art. The exterior of the body 401 may have some or all of the following: at least one button or knob 409 for manual control of actuation, a light source 410 for indicating the power status of the automated system, a switch 411 for powering the automated system on or off, at least one port 412 for suction, and a tube release switch or lever 413 for decoupling the tube from the body and bend when the tube reaches a desired position or location. The actuation unit 408 may further have a slide mechanism 414.

[0069] The system further comprises a user interface 415 and a circuit 416 arranged as separate units 417 outside the main body. The separate units are connected to the main body via a cable 418. Alternatively, the user interface 415, the circuit 416 and the system are connected by a wireless connection (not shown). The wireless connection may be established by Bluetooth, Wifi, Zigbee, telecommunication, NFC or any other communication mode available at the time of implementation of the system. The wireless communication also allows the device to be remotely controlled, together with data transfer. The remotely connected processing circuit may control multiple operating units at different times in multiple devices and provide centralized control to the hospital management and compliance department. The communication between the different units of the system may be secured by implementing technologies such as SSL.

[0070] FIG. 5 is a diagram of an exemplary embodiment of the configuration of the flexure 203 of FIG. 2 having multiple independent vertebrae 501 stacked on top of each other and connected by rivets 502. The vertebrae are connected in such an arrangement to allow partial and / or complete independent rotational movement around the rivet point of each vertebra. The rotational movement of each vertebra allows the flexure of the flexure. The vertebrae are connected to each other via a cord 503, one end of which is connected to an actuation unit (not shown in FIG. 5) and another end is connected to the vertebra at the distal end of the flexure. The vertebrae further have at least one eye loop 504 arranged on the inside. The cord from the actuation unit passes through the eye loop to a connection point at the distal end of the vertebra. Alternatively, a mesh or a combination of the above-mentioned configurations and meshes, or other feasible arrangements known to those skilled in the art, may be employed to achieve the same purpose.

[0071] 6 is a diagram of an exemplary architecture of an automatic intubation system 200, which comprises a flexure 203, an imaging sensor 205, a tube 301, at least one actuation unit 208, a user interface 215, and a circuit 216. The circuit further comprises a processing circuit 216a, which generates control signals based on input from the at least one imaging sensor, a communication circuit 216b, which provides data / signal communication between different components of the system, and a power circuit 216c. The actuation unit includes a sliding mechanism 213, which provides movement in the Z-plane for the tube.

[0072] The processing circuit 216a may be a single processor, a logic circuit, a dedicated controller performing all functions, or a combination of processing support units depending on the functional requirements of the system. In an exemplary embodiment, the processing circuit has two independent processing support units 216aa and 216ab. The processing support unit 216a is a computer vision software that utilizes machine learning techniques and data received from the imaging sensor 205 to perform at least one function (216aa1, 216aa2...216aaN). The functions include recognition of anatomical structures and prediction of a target path for the tube 301 to be inserted based on the recognition of at least one anatomical structure. The processing support unit and / or the processing circuit interact with the imaging sensor 205 to receive data during the intubation procedure and perform the functions described above.

[0073] In one embodiment, recognizing anatomical structures using imaging sensor data and machine learning techniques includes detecting respiratory structures such as the opening of the trachea, the glottis, the vocal cords, and / or the bifurcation between the esophagus and the trachea. In addition to or in lieu of detecting respiratory structures, other anatomical parts of the human body may also be detected and / or recognized.

[0074] Alternatively, the processing circuit 216aa predicts the target path based on input from the imaging sensor, samples of past data received remotely from multiple device actuation units, and machine learning models. The system further stores the target path to maintain a record of device operation for regulatory purposes in a memory (not shown in the system). The device record may be shared with remote devices for monitoring and control purposes. The processing support unit 216ab generates a control signal based on the target path predicted by the processing support unit 216aa. To provide three-dimensional movement to the invasive medical device, the control signal generated by the processing support unit 216ab is then communicated from the processing circuit to the actuation unit 208 via the communication circuit 216b on the basis of which the actuation unit actuates at least one of the flexure 203 and the sliding mechanism 214. The processing support unit 216ab may be an integrated part of the actuation unit 208, and the control signal is received by the actuation unit via a wireless or wired communication circuit. In one scenario, the processing circuitry 216aa may be remotely connected to the actuation unit 208 through the Internet or a wireless medium to transmit control signals. The communication circuitry may be an integrated part of the actuation unit.

[0075] The user interface 215 is in bidirectional communication with the processing circuit 106a. The user interface is preferably a display device that displays the data received from the imaging sensor 205 and an overlay of the recognized anatomical structures or object paths received from the processing circuit to assist the operator. Furthermore, the object path overlay may be visualized on the user interface in the form of augmented reality and / or in any other form that provides an effective visual guidance to the operator.

[0076] The user interface 215 may be a touch-sensitive display device that allows the operator to adjust the target path displayed thereon. The target path displayed on the user interface may be overridden by the operator if the operator is not satisfied with the target path of intubation. Additionally, it may have touch buttons associated with functions performed by buttons located on the exterior surface of the body, such as a button to trigger manual actuation, a tube release button, and / or a system power off button. Alternatively, the user interface may be any device that can enable interaction between the operator and the automated system, such as audio input, audio output, or gesture input, or any other control scheme that may be enabled by an intelligent agent.

[0077] FIG. 7 is an exemplary flow diagram for generating a machine learning model, comprising step 701 collecting a number of intubation procedure videos from an existing video laryngoscope, and step 702 separating the collected intubation procedure videos based on a predicted level of difficulty of the intubation procedure. The level of difficulty may be predicted in the form of either a traditional Mallampati classification or a custom intubation difficulty scale using an automatic fusion of computer vision models and known machine learning algorithms. The calculated or predicted difficulty score may be embedded in the metadata of the videos for easy searching and separation of the videos based on the calculated score. These videos may be supplemented with videos obtained from other sources, including the equipment described herein. There is no limitation on the video sources used for the training videos disclosed herein.

[0078] In step 703, the separated images are cropped to exclude portions of the images that contain occluded and / or unclear views of anatomical structures relevant to the intubation procedure. This step clears avoidable noise in the video data before moving on to the process of extensive training of the machine learning model.

[0079] In step 704, the cropped video file is converted to an image file, which is then labeled with anatomical structures in step 705 to build a dataset of labeled images. In step 706, this labeled dataset of images serves as a training dataset for training one or more neural networks to generate a machine learning model. The generated machine learning model is employed within or as part of the processing support unit 216aa (i.e., computer vision software) executed by the processing circuit 216a of FIG. 6 to recognize at least one anatomical structure during the intubation procedure based on data received from the imaging sensor 205.

[0080] FIG. 8 illustrates an exemplary automated intubation method using a method comprising inserting a detachable blade 801 into a patient's airway 802. Adjacent to the detachable blade, a bend 803 and a tube 804 disposed longitudinally on the bend are inserted into the patient's airway. The method further includes collecting airway data from at least one imaging sensor 805 disposed on the bend. The collected airway data is then communicated to at least one processing circuit 806, which utilizes machine learning models and the airway data to recognize at least one anatomical structure and predict at least one target path for insertion of the tube. The target path is then used by the processing circuit to generate and communicate control signals to at least one actuation unit 807 to actuate three-dimensional movement of the tube.

[0081] Specifically, once the detachable blade, flexure, and tube are directly or indirectly connected to the body, the detachable blade 801, flexure 803, and tube are inserted by introducing the body 808 into the vicinity of the patient's mouth. Also, the processing circuit 806 and the actuation unit 807 are preferably located within the body.

[0082] The three-dimensional movement of the tube 804 placed on the flexure 803 includes bending movements of the tube in X and Y planes induced by the two-dimensional movement of the flexure 803, and movement of the tube in Z plane by a sliding mechanism of the actuation unit 807 (not shown in FIG. 8). The actuation of the flexure is enabled by the actuation unit connected to the flexure via a cord (not shown in FIG. 8). The method also comprises displaying the data communicated from the imaging sensor 805 on a user interface 809, and overlaying the recognized anatomical structure and the intended path of tube insertion on the user interface.

[0083] The location of the distal end of the tube may be confirmed by standard clinical methods such as, but not limited to, capnometry, x-ray, and ultrasound. These methods may be directly incorporated into the device or may be incorporated to provide indirect support of such methods. For example, with respect to capnometry, the presence of CO2 levels in the air may confirm correct placement of the tube within the patient. This qualitative or quantitative confirmation may be provided by sensors located directly on or within the device, such as a CO2 monitor, or through more indirect methods, such as color-changing PH-responsive strips placed within the field of view of an imaging sensor to provide confirmation of correct CO2 levels. Similarly, ultrasound transmitters and receivers may be incorporated into the device that can confirm that the distal end of the tube has been correctly placed. The techniques described above are just a few of many clinical approaches to confirm correct placement of an intubation tube, which will be apparent to those skilled in the art.

[0084] Once the desired position or location within the patient's airway is reached, the tube is set to release from the body 808 and bend 803 using a tube release switch or lever 810 located on the exterior surface of the body. Alternatively, a touch button (not shown in FIG. 8) may be provided on the user interface 809 to release or disconnect the tube.

[0085] FIG. 9 is an illustration of the use of a user interface 901 having a display screen 902 that displays data received from at least one imaging sensor. The display screen further displays an overlay of at least one recognized anatomical structure 903 and a target path of insertion 905 of a tube 904. An operator may manually adjust the target path of insertion 905 of a tube 904 displayed on the user interface. Alternatively, an overlay of the tube, bends, recognized anatomical structure 903, and target path of insertion 905 is displayed on the user interface as an augmented reality, virtual reality, or other form of overlay known to those skilled in the art that provides an effective visual guidance to an operator. The recognized anatomical structure overlay may include annotations or labels for the operator to quickly identify the structures during the procedure.

[0086] Additionally, the display screen 902 of the user interface 901 may have a pair of up and down touch buttons 906 for manually controlling operation and / or overriding automatic operation if needed, a system power on / off touch button 907, and a tube release touch button 908.

[0087] In one embodiment, a pair of up and down touch buttons 906 may be used to selectively control manual actuation in a selected operating plane X, Y, or Z. Touch buttons 909 provided on the display screen may be used to select the operating plane before providing input via the touch buttons 906. Although the touch buttons are shown in Figure 9 as being located outside the boundaries of the visual data received from the imaging sensor, it will be appreciated that the placement of the touch buttons may be altered to provide the operator with the best possible visual display.

[0088] Although the present invention is described in the context of surgical, insertion, or implantation assistance, the present invention may also be implemented to provide educational or academic uses such as training and demonstrations.

[0089] No statement in the specification should be construed as indicating any element other than the claims as essential to the practice of the invention.

[0090] It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit and scope of the present invention. It is not intended to limit the present invention to the specific form or forms included. On the contrary, it is intended to cover all modifications, alternative structures, and equivalents that are included within the spirit and scope of the present invention, as defined by the appended claims. Therefore, it is intended that the present invention covers the modifications and variations of the present invention if they come within the scope of the appended claims and their equivalents.

Claims

1. 1. An automated intubation system comprising: a main body; a flexible portion connected to the main body; a variable length flexure having at least a portion of the flexible portion; a distal end of the flexure; at least one imaging sensor; a processing circuit for predicting at least one target path for the distal end of the flexure and generating control signals, where the target path is predicted based on data received from the imaging sensor; a user interface for displaying at least one target path to an operator and also allowing the operator to select a target path; at least one actuation unit for receiving control signals from the processing circuit and actuating three-dimensional movement of the distal end of the flexure along the target path to a first position; the processing circuit for receiving and using data from the at least one imaging sensor to compare the actual movement of the distal end of the flexure to the target movement of the distal end of the flexure; and the processing circuit for then generating an additional control signal to move the distal end of the flexure to the first position if the target first position and actual first position do not match.

2. The automated intubation system of claim 1 , wherein the processing circuitry is capable of predicting at least one new target trajectory.

3. 2. The automatic intubation system of claim 1, wherein when the distal end of the bend reaches the first location; the processing circuit generates a second location along the target pathway.

4. The automated intubation system of claim 1 , wherein the processing circuitry continually generates new positions for the distal end of the bend along the target path based on the data received from at least one imaging sensor.

5. The automated intubation system of claim 1 , wherein the data received from at least one imaging sensor can be either image or position data.

6. 2. The automated intubation system of claim 1, wherein the processing circuitry utilizes a machine learning model to compare the data of the actual movement of the distal end of the articulation section received from an imaging sensor to the target movement of the distal end of the articulation section.

7. The automated intubation system of claim 1 , wherein the processing circuitry may be included within the device or hosted on a remote server.

8. The automated intubation system of claim 1 , wherein the movement of the distal end of the bend along the target path can be overridden by the operator.

9. 1. A method for automatically intubating a patient, comprising: The method includes the steps of: inserting a blade, a flexure, and a tube disposed on the flexure into the airway of the patient; collecting airway data using at least one imaging sensor disposed on the flexure; communicating the collected airway data to a processing circuit; predicting a target path for inserting the tube and generating a control signal using the processing circuit, where the target path is predicted based on at least one anatomical structure recognized by the processing circuit using the collected airway data; and displaying the target path via a user interface to present the at least one target path to an operator. and also allowing the operator to select a target path; communicating the control signal generated by the processing circuit to at least one actuation unit to actuate the three-dimensional movement of the tube; comparing, by the processing circuit, the actual movement of the distal end of the bend portion to the target movement of the distal end of the bend portion using data from at least one imaging sensor; and, if the target first position and actual first position do not match, generating an additional control signal to move the distal end of the bend portion to the first position.

10. 10. The method of claim 9, wherein the processing circuitry is operable to predict at least one new target pathway.

11. 10. The method of claim 9, wherein when the distal end of the bend reaches the first location; the processing circuit generates a second location along the target pathway.

12. 10. The method of claim 9, wherein the processing circuitry continually generates new positions for the distal end of the bend along the target path based on the data received from at least one imaging sensor.

13. 10. The method of claim 9, wherein the data received from at least one imaging sensor can be either image or position data.

14. 10. The automatic intubation method of claim 9, wherein the processing circuitry utilizes a machine learning model to compare the data of the actual movement of the distal end of the articulation section received from an imaging sensor with the target movement of the distal end of the articulation section.

15. 10. The method of claim 9, wherein the processing circuitry may be contained within the device or hosted on a remote server.

16. 10. The method of claim 9, wherein the movement of the distal end of the bend along the target path can be overridden by the operator.

Images