System and method for automatic intubation

Abstract

A system, method, and apparatus for automatically performing endotracheal intubation on a patient comprises: inserting a blade into the upper airway of a patient to contract an anatomical structure; inserting a tube arranged in a bend portion and a tube arranged in the bend portion into the airway of a patient; collecting airway data using at least one imaging sensor arranged in the bend portion; transmitting the collected airway data to a processing circuit; predicting an intended path for insertion of the tube, i.e., an intended path predicted based on at least one anatomical structure recognized by the processing circuit using the collected airway data, and generating a control signal using the processing circuit; displaying the intended path through a user interface to display at least one intended path to an operator and also enabling the operator to select the intended path; and transmitting the control signal generated by the processing circuit to at least one operating unit to operate the three-dimensional movement of the tube.

Description

Technology Field

[0001] The present invention relates to an automated system and method for inserting an invasive medical device into a patient, and in particular to an automated system and method for inserting an invasive medical device into a patient's body cavity using image-based guidance. Background Technology

[0002] This section describes the technical field in detail and explains the problems faced in the technical field. Therefore, statements in this section should not be interpreted as prior art.

[0003] The efficient implantation of medical devices into a patient's body is one of the most urgent needs in today's medical field. One reason for this growing need is the wide range of applications provided by invasive medical devices, ranging from chest implantation of pacemakers to ensure a proper heart rate to the insertion of urinary catheters. Another reason is the high number of complications and complexities encountered by medical professionals, physicians, and anesthesiologists during implantation procedures, which require immediate intervention to prevent disease and death.

[0004] One such use case for the implantation of invasive devices is endotracheal intubation, which is performed to keep a patient's airway open to support breathing. Endotracheal intubation (or ETI) is performed by using a laryngoscope to visualize the glottal opening and then inserting a tube through it. The physician can directly view the glottis with their own eyes after manipulating the anatomical structures of the upper airway with a laryngoscope that creates a "straight line of sight." Clear visualization of the glottal opening using a laryngoscope depends on several factors, such as facial structure, the mallampati score, dental status, and joint stiffness. Therefore, endotracheal intubation is a process that requires significant skill and training. Even with proper training, visualizing the glottal opening and inserting the tube can be challenging.

[0005] It is estimated that during pre-hospital care, approximately 81% of endotracheal intubations are performed by non-physicians, while 19% are performed by physicians. The unpredictable environment during pre-hospital care further exacerbates the complexity of successful intubation. The failure rate for the first attempt at endotracheal intubation is estimated to be high at 41%. Such delays in intubation for patients lead to severe consequences. Hypoxia can cause permanent brain damage within 4 minutes and death within 10 minutes.

[0006] An alternative intubation method using a video laryngoscope provides a much better view because it includes a camera at the tip of the laryngoscope, eliminating the need for a "straight line of sight." The camera projects an image onto a monitor, allowing the physician to manually insert the endotracheal tube while viewing the screen. However, this still requires significant dexterity and visuospatial awareness. It is also a difficult skill to learn. The failure rate for the first attempt using a video laryngoscope can also be high.

[0007] When intubation is not possible for the patient, several alternative methods are attempted, including specialized airway devices such as supraglottic ventilation devices, King's tube or Combitube, mask ventilation, and, in some cases, even emergency cricothyroidotomy, which involves incising the neck and trachea and inserting a tube through the opening. As expected, these procedures are not as effective as simple endotracheal intubation and can be much more invasive for patients with long-term sequelae.

[0008] Most intubation systems and methods guided by the latest technology have limitations that cause problems such as higher delays and failure rates during intubation. Therefore, there is a clear need to design systems and methods that can not only facilitate rapid and successful intubation but also operate with complete autonomy and minimal operator (or user) intervention. The terms operator and user can be used interchangeably.

[0009] Patients severely affected by serious respiratory infections, such as the COVID-19 virus, may develop respiratory distress requiring intubation and ventilation. Because healthcare providers are in very close contact with infected patients and come into direct contact with their saliva, they are at risk of contracting the disease themselves while following the standard of care for these patients. Furthermore, the transmission of the disease to healthcare providers is directly related, among other things, to the duration and extent of contact with the patient, making ETI a high-risk procedure for infection transmission.

[0010] The present invention aims, above all, to overcome the drawbacks of the prior art as described above. Prior art literature

[65535] U.S. Patent Application Publication No. 2015 / 0059736 U.S. Patent Application Publication No. 2002 / 0173799

[0011] References to "one embodiment," "at least one embodiment," "embodiment," "one example," "example," "for example," etc., indicate that the embodiment(s) or example(s) may include specific features, structures, characteristics, attributes, elements, or limitations, but not all embodiments or examples necessarily include such specific features, structures, characteristics, attributes, elements, or limitations. Furthermore, the repeated use of the phrase "in an embodiment" does not necessarily refer to the same embodiment.

[0012] In an embodiment of the present invention, an automatic system inserts an invasive medical device into a patient's body cavity. The automatic system includes a processing circuit that receives data from at least one data source, recognizes structures related to the patient's body cavity, and predicts an intended path for inserting the invasive medical device into the patient. The processing circuit further generates a control signal based on the intended path to actuate the three-dimensional movement of the invasive medical device and transmits it to at least one actuation unit.

[0013] The processing circuit unit can recognize structures related to the patient's body cavity using a machine learning model along with data received from data source(s), predict an intended path, generate control signals, and transmit them to an actuation unit to operate the three-dimensional movement of the invasive medical device. Once movement begins, the intended path becomes the path that guides the device to the invasive medical device. The creation of the machine learning model involves receiving or collecting training data in the form of a predetermined dataset to train at least one neural network. This form of neural network may be an edge-implemented deep neural network-based object detector widely known in the art. As is widely known to those skilled in the art, other forms of machine learning other than neural networks may be substituted. The predetermined dataset may be images and videos, but is not limited thereto.

[0014] The data source(s) may be imaging sensors. These sensors may include, but are not limited to, cameras, infrared cameras, sonic sensors, microwave sensors, photodetectors, or other objects known to those skilled in the art that may be used to achieve the same purpose. Data received from the imaging sensors may be displayed in a user interface to provide the operator with a view of the patient's body cavity. Additionally, intended paths and recognized structures may be overlaid on the data received from the imaging sensors in the user interface for effective visual guidance to the operator.

[0015] In an exemplary embodiment of the present invention, an automatic intubation system predicts an intended path for the insertion of a tube and generates a control signal for at least one operating unit. The intended path is predicted based on at least one anatomical structure recognized using data received from at least one imaging sensor. An overlay of the intended path and / or the recognized anatomical structure is also displayed on the user interface against the data received from the imaging sensor(s) for effective visual guidance during intubation. Additionally, the intended path displayed on the user interface is adjustable by the operator and / or invalidated by the operator if the operator is not satisfied with the intended insertion path. Subsequently, the operator can select the proposed or adjusted intended path that the system should follow during the intubation process.

[0016] Additionally, the overlay of the intended path may also be visualized on the user interface in the form of augmented reality and / or any other form that provides effective visual guidance to the operator.

[0017] In one preferred embodiment, the automatic intubation system comprises a main body, a bending part, a flexible part connecting the main body to the bending part, a housing unit arranged in the bending part and comprising at least one imaging sensor, an intubation tube arranged on the flexible part and the bending part, a circuit part, a user interface, a disposable and / or reusable sleeve having a blade at one end to contract an anatomical structure, and at least one actuating unit for operating the three-dimensional movement of the tube. The length of the bending unit is variable and may be located only at the tip of the flexible part or may completely cover the flexible part. In another embodiment, the bending part may be located within any part of the flexible part determined by several factors, including but not limited to the relevant use and the anatomical structure requiring exploration. Preferably, the disposable and / or reusable sleeve is removablely coupled to the main body. The imaging sensor(s) are preferably cameras, but sensors such as infrared, photodetectors, or other feasible means known to those skilled in the art may be used to achieve the same purpose.

[0018] In a preferred embodiment of the present invention, the circuit section, user interface, and operating unit are part of the main body. The circuit section further includes a processing circuit section, a power circuit section, and a communication circuit section.

[0019] In an alternative embodiment of the present invention, the circuit and user interface are arranged separately from the main body within at least one separate box.

[0020] The processing circuit is used to predict the intended path for inserting the tube and to generate control signals based on at least one recognized anatomical structure. The processing circuit is also used to recognize anatomical structures using data received from an imaging sensor and at least one pre-trained machine learning model. The actuating unit receives control signals from the processing circuit and acts the three-dimensional movement of the tube. The actuating unit uses a connection with a bending part to actuate the bending movement of the tube, particularly in the X and Y planes. The actuating unit also includes a sliding mechanism that acts the sliding movement of the tube in the Z plane by moving the bending part and its associated actuating unit on a rail track. Alternatively, the sliding mechanism acts the sliding movement of the tube in the Z plane by direct contact or proximity with the tube without displacing the bending part and its associated actuating unit. A person skilled in the art has also recognized that other three-dimensional coordinate systems, such as radial, pole, cylindrical, and spherical, may be used instead of the x, y, and z coordinates described in this application.

[0021] In another embodiment of the present invention, the processing circuit is used only to predict the intended path and generate control signals, and the recognition of anatomical structures using imaging sensor data and machine learning models is performed by a separate, independent processing circuit.

[0022] A machine learning model is part of computer vision software developed by training one or more neural networks on a labeled dataset of images, the labeled dataset of images being constructed by converting video collections of intubation procedures into image files and labeling anatomical structures on the image files. In an alternative embodiment, the creation of a machine learning model involves receiving or collecting training data in the form of a predetermined dataset to train at least one neural network. The predetermined dataset may be, but is not limited to, images, audio, and video recorded and collected during the procedure.

[0023] In another embodiment of the present invention, a control signal received by an operating unit to operate the three-dimensional movement of the tube is manually generated by a pair of up-and-down buttons arranged on the outer surface of the main body or by a touch button arranged on the user interface. Thus, the system provides a manual operation mode if requested by the operator. If the operator is not satisfied with the intended path, the operator may disable the automatic operation of the tube using the pair of up-and-down buttons and the touch button.

[0024] In another aspect of the present invention, a method for automatically inserting an invasive medical device into a patient's body cavity is provided, comprising the step of inserting an invasive medical device arranged in a bend portion and an invasive medical device arranged in the bend portion into the patient's body cavity. The method comprises the step of collecting airway data using an imaging sensor arranged in the bend portion and transmitting the collected airway data to a processing circuit to predict an intended insertion path of the invasive medical device and generate a control signal. Subsequently, the control signal is transmitted to at least one operating unit to operate the three-dimensional movement of the invasive medical device. It is preferable that the intended path is predicted by the processing circuit based on the recognition of at least one structure related to the body cavity using data transmitted from the imaging sensor.

[0025] Additionally, the prediction of the intended insertion path and the recognition of structures related to the body cavity can be performed by a processing circuit using a machine learning model in conjunction with data transmitted from an imaging sensor. The creation of a machine learning model involves receiving or collecting training data in the form of a predetermined dataset to train at least one neural network. The predetermined dataset may be images and videos, but is not limited thereto. It can be anticipated that the device disclosed in this patent may be used in body cavities other than the airway described in this application, or may be used to perform other tasks within such body cavities.

[0026] In an exemplary embodiment of the present invention, a method for automatically intubating a patient is provided by inserting a tube arranged in a bend portion and a tube arranged in the bend portion into the patient's airway. The method further includes the step of collecting airway data using an imaging sensor arranged in the bend portion and transmitting the collected airway data to a processing circuit to predict the intended insertion path of the tube and generating a control signal to actuate the three-dimensional movement of the tube. Preferably, the intended path is predicted by the processing circuit based on the recognition of at least one anatomical structure using data transmitted from the imaging sensor. The processing circuit utilizes data transmitted from the imaging sensor and a machine learning model to recognize the anatomical structure and predict the intended insertion path of the tube.

[0027] The method may also involve a step of displaying airway data on a user interface to emphasize a view of the airway to the operator. Additionally, it involves a step of overlaying the intended path and recognized anatomical structures on the data transmitted from the imaging sensor on the user interface to provide effective visual guidance to the operator.

[0028] Semi-automatic invasive device insertion systems offer advantages over fully automated systems. The commercialization of such systems requires regulatory approval from government agencies like the FDA, and the pathways for semi-automatic systems can be simpler and less complex. Additionally, the adoption of fully automated systems can introduce layers of legal liability to which a company might be vulnerable. Furthermore, no matter how advanced the technology may be, it is advisable to have a skilled professional supervise the procedure and manually invalidate it if necessary to ensure proper intubation. Compared to fully automated systems, semi-automatic systems can reduce technical hurdles in developing and producing deployable systems. Finally, the inherent presence of verification and control mechanisms and availability layers that enforce the correct pathway prevents injury and is safer for the patient.

[0029] In an alternative embodiment, a complementary sensor capable of providing real-time information regarding the patient's relevant clinical parameters, such as vital signs including but not limited to pulse and heart rate, respiratory rate, oxygen saturation level, body temperature, and blood pressure; and other laboratory results including but not limited to blood gas levels, glucose levels, and other results known to a person trained in the latest technology, may be integrated with the device.

[0030] In another embodiment, the operator can connect to the device remotely via the Internet and operate the device using a similar user interface.

[0031] Other embodiments and preferred features of the present invention will become apparent from the following description and claims, along with their corresponding advantages. Brief explanation of the drawing

[0032] Various aspects and embodiments of the present invention are better understood by referring to the following detailed description. To better understand the present invention, the detailed description should be read carefully in conjunction with the drawings. FIG. 1 illustrates an exemplary architecture of an automatic system for inserting an invasive medical device into a patient according to the present invention; FIG. 2 illustrates an exemplary embodiment of an automatic intubation system according to the present invention; FIG. 3 illustrates an assembly of a main body, a disposable sleeve, and a tube of an automatic intubation system according to the present invention; FIG. 4 illustrates an alternative embodiment of an automatic intubation system according to the present invention; FIG. 5 illustrates the configuration of a curved portion according to the present invention; FIG. 6 illustrates an exemplary architecture of an automatic intubation system according to the present invention; FIG. 7 illustrates a flowchart for generating a machine learning model according to the present invention; FIG. 8 illustrates the use of a representative automatic intubation method according to the present invention; FIG. 9 illustrates the use of a user interface according to the present invention. Specific details for implementing the invention

[0033] The present disclosure is best understood by referring to the detailed drawings and descriptions provided in this application. Various embodiments have been described with reference to the drawings. However, because methods and systems may extend beyond the described embodiments, those skilled in the art will readily recognize that the detailed description provided in this application in relation to the drawings is for illustrative purposes only. For example, a number of alternatives and suitable approaches may be derived to implement the functions of any details described in this application according to the teachings provided and the requirements of a particular use case. Accordingly, any approach may extend beyond the specific implementation choices in the following embodiments.

[0034] The method of the present invention may be implemented by performing or executing selected steps or operations manually, automatically, or in a combination thereof. The term “method” means a method, means, technique, and procedure for achieving a given task, and includes, but is not limited to, methods, means, techniques, and procedures known to practitioners of the art to which the present invention pertains, or methods, means, techniques, and procedures easily developed from known methods, means, techniques, and procedures. The descriptions, examples, methods, and materials set forth in the claims and specification should not be construed as restrictive, but rather as illustrative only. A person skilled in the art will conceive many other possible variations within the scope of the technology described in this application.

[0035] While carefully reading the description of the exemplary embodiment of the best form of the present invention (hereinafter referred to as "exemplary embodiment"), the inventor should consider the exemplary embodiment as the best form for carrying out the invention at the time of filing the patent application in accordance with the inventor's belief. Since a person skilled in the art can recognize substantially equivalent structures or substantially equivalent operations and achieve the same results in the same manner or in a different manner, the exemplary embodiment should not be construed as limiting the present invention to one embodiment.

[0036] Discussion of a species (or specific item) refers to the genus (classification of the item) to which the species belongs and related species of this genus. Similarly, enumeration of the genus refers to species known in the art. Furthermore, as technology advances, numerous additional alternatives may arise to achieve the embodiments of the present invention. Such advancements are included in their respective genuses and should be recognized as functionally or structurally equivalent to the embodiments shown or described.

[0037] Unless otherwise explicitly stated, conjunctions (e.g., "or," "and," "comprising," or "including") should be interpreted in an inclusive sense rather than an exclusive one.

[0038] As understood by those skilled in the art, various structures and devices are depicted in block diagrams to avoid obscuring the invention. In the following description, it should be noted that, unless otherwise specified, operations with similar names are performed in a similar manner.

[0039] The foregoing descriptions and definitions are provided for the purpose of clarification and are not restrictive. Unless otherwise indicated, words and phrases should follow their ordinary and obvious meanings.

[0040] The present invention may be better understood by reviewing the drawings, and FIG. 1 is an example of an exemplary architecture of an automatic system (100) for inserting an invasive medical device into a patient's body cavity. The system includes a bending part (101), an imaging sensor (102), an invasive medical device (103), at least one operating unit (104), a user interface (105), and a circuit part (106). The circuit part further includes a processing circuit part (106a) for generating a control signal based on input from at least one imaging sensor and a machine learning model, a communication circuit part (106b) for providing data / signal communication between different components of the system, and a power circuit part (106c). The operating unit includes a sliding mechanism (107) for providing movement to the invasive medical device in the Z plane.

[0041] The processing circuit (106a) may be a combination of a single processor, a logic circuit, a dedicated controller that performs all functions, or a process assistance unit that depends on the functional requirements of the system. In an exemplary embodiment, the processing circuit comprises two independent process assistance units (106aa and 106ab). The process assistance unit (106aa) is computer vision software that utilizes data received from an imaging sensor (102) and machine learning techniques to perform at least one function (106aa1, 106aa2, ... 106aaN) for automating the intubation process. The function includes recognizing structures around and inside the patient's body cavity and predicting an intended path for inserting an invasive medical device (103) inside the patient. Alternatively, the processing circuit (106aa) predicts an intended path based on input from the imaging sensor, sample history data received remotely from the operating unit of a plurality of devices, or a machine learning model. The system further stores an intended path in memory (not shown in the system) to maintain a log of device operation for regulatory purposes. The log of the device may be shared with a remote device for monitoring and control purposes. Additional information, such as images from one or more imaging sensors, as well as states and decision points that may be shared with a remote server for further machine learning model improvement or for other purposes such as regulation or training, may be stored or shared. This information may be stored locally on the device or in a remote storage such as a server or in the cloud. The process assistance unit (106ab) generates a control signal based on the intended path predicted by the process assistance unit (106aa).Subsequently, a control signal generated by the process assist unit (106ab) is transmitted from the processing circuit to the operating unit (104) via the communication circuit (106b), and based on this, the operating unit operates at least one of the bending part (101) and the sliding mechanism (107) to provide three-dimensional movement to the invasive medical device. The process assist unit (106ab) may also be an integrated part of the operating unit (104), and the control signal may be received by the operating unit (104) via a wireless or wired communication circuit. The processing circuit (106aa) may also be remotely connected to the operating unit (104) via a network or wireless medium to transmit the control signal. The communication circuit may also be an integrated part of the operating unit. Each of the previously described functions may be combined with other functions within a single function unit for each of the previously described functions and all functions.

[0042] The communication circuit section (106b) may also be distributed to act as an element of bidirectional data / signal transmission in the complete system. The communication circuit section may be wired or wireless. The power circuit section (106c) distributes power to all units of the system. The power circuit section includes a rechargeable battery or a directly regulated power supply.

[0043] The actuation unit (104) may be a rotary motor, a linear motor, and / or a combination of a rotary motor and a linear motor. In an exemplary embodiment, a plurality of actuation units (A1, A2, ... An) independently actuate the bending part (101) and the sliding mechanism (107) to provide three-dimensional movement. Alternatively, the bending part (101) and the sliding mechanism (107) may be actuated together using a single actuation unit. The system can track the movement of an invasive medical device, calculate deviations by comparing them to an intended path, and correct the movement. Correction can be performed automatically or through manual intervention. Data of the actual movement may be transmitted to a remote device for monitoring purposes.

[0044] The user interface (105) communicates bidirectionally with the processing circuit (106a).

[0045] The user interface is preferably a display device for displaying data received from the imaging sensor (102) and an overlay of the intended path and / or recognized structure from the processing circuit for the data received from the imaging sensor to assist the operator in effective visual guidance. Alternatively, the user interface may be any device capable of enabling interaction between the operator and an automatic system, such as audio input / output, gesture-capable input, augmented reality-capable system, and / or projection device. The user interface may also be a head-up display or head-mounted display to support interaction in the form of virtual reality. The user interface (105) may be used to select a modified intended path generated by the operator by selecting a proposed intended path or by invalidating a proposed path and modifying a proposed intended path.

[0046] FIG. 2 is an example of an exemplary embodiment of an automatic intubation system (200) comprising a main body (201), a flexible part (202) connecting the main body to a bending part (203), and a housing unit (204) attached to the bending part. The housing unit further supports at least one imaging sensor (205), at least one guide light (206), and at least one exit channel (207). Preferably, the imaging sensor is a wide CMOS camera and the guide light is an LED light that turns on automatically when the system is turned on. Alternatively, independent control switches for the guide light and the imaging sensor may also be provided.

[0047] The main body further includes at least one operating unit (208) that converts a control signal received from a processing circuit into a three-dimensional movement for advancing the tube(s) in the patient's body cavity. The operating unit (208) may be a rotary motor, a linear motor, and / or a combination of a rotary motor and a linear motor. Optionally, the outer surface of the main body (201) has at least one button or handle (209) for manually controlling the operation, a light source (210) for indicating the power status of the automatic system (200), a switch (211) for turning the automatic system on or off, at least one port (212) for suction, and a tube release switch or lever (213) for separating the tube from the main body.

[0048] In one embodiment, the actuating unit (208) further comprises a sliding mechanism (214). The sliding mechanism may be an integral part of the actuating unit or a separate unit connected to the actuating unit. The sliding mechanism may be a movable base plate connected to the actuating unit via a rack and pinion mechanism (not shown), where the pinion is connected to the actuating unit for rotational motion and the rack is connected to the movable base plate to convert rotational motion into vertical motion and / or displacement. A person skilled in the art will be well aware of other methods or mechanisms for connecting the actuating unit to the movable base plate to achieve the same sliding mechanism. The primary purpose of the sliding mechanism is to provide Z-plane movement to the tube. The use of a sliding mechanism activating unit (208) is not required in this disclosure, and as disclosed below, a number of electromechanical systems may be used to provide Z-plane movement for an invasive medical device.

[0049] Alternatively, two independent operating units may be used to operate the bending part (203) and the sliding mechanism (214). The processing circuit (shown in FIG. 1) may transmit control signals for X and Y plane movement to the operating unit controlling the movement of the bending part, and control signals for Z plane movement to the operating unit associated with the sliding mechanism.

[0050] Alternatively, there are numerous different arrangements of actuation units for the three-dimensional movement of the tube, which will be readily apparent to a person skilled in the art. These may include rotary, gear, coil, or screw-based actuation units, as well as the use of free-floating actuation units. Careful attention must be paid to the accuracy of the movement in the X and Y planes, as well as the magnitude of the movement required in the Z plane.

[0051] A user interface (215) is also attached to the main body (201) and displays data received from the imaging sensor (205). Preferably, the user interface is a display device attached to the main body. Alternatively, the user interface is a touchable display device comprising at least one button for triggering operation, a button for releasing the tube, and a power button (not shown). The user interface may be any device capable of enabling interaction between the automatic system and the operator, such as audio input, audio output, or gesture-capable input. In another embodiment, the user interface may be composed of an intelligent agent that provides necessary operator feedback.

[0052] The main body (201) also includes a circuit section (216) that further includes a processing circuit section, a communication circuit section, and a power circuit section.

[0053] The bend portion (203) is connected to the operating unit (208). Preferably, the bend portion (203) is connected to the operating unit (208) via at least one cord (not shown in FIG. 2). The cord(s) are connected to the operating unit and pass through the flexible portion to reach and connect to the bend portion, thereby acting as a bending motion and / or movement of the bend portion. Alternatively, the cord(s) may be replaced with any feasible mechanical link, such as a thread, wire, cable, and chain. A person skilled in the art will be well aware of other methods or means for connecting the operating unit to the bend portion to provide two-dimensional movement in the X and Y planes to the bend portion (203).

[0054] FIG. 3 is an example of an assembly of a tube (301), a sleeve (302), and a main body (201) of an automatic intubation system (200). The tube may be arranged longitudinally in a flexible portion (202) and a bending portion (203). Alternatively, the tube may be arranged partially in the flexible portion and partially in the bending portion. Generally, the flexible portion provides a view of the airway through imaging sensor(s) that pass through the tube and are supported by a housing unit (204). The tube may be an endotracheal tube, including, but is not limited to, an oral, nasal, cuffed, cuffless, pre-formed reinforced, double-lumen endotracheal tube, or any custom-fit tube.

[0055] The sleeve (302) may be mechanically connected to the body (201) to detachably connect the blade (303) to the body, preferably through a snug fit connection. Other feasible mechanical connections known to those skilled in the art may also be used to achieve the same purpose. A detachable blade (303) at one end of the sleeve (302) is provided to retract anatomical structures during an intubation procedure. The sleeve may be manufactured from disposable and / or reusable materials.

[0056] The blade (303) is designed to improve the efficiency of the blade to provide better visibility during the intubation process and may have a shape similar to the blade of a conventional video laryngoscope. The blade may additionally have an integrated path to guide the tube during the initial stages of intubation. The path may be an open tunnel through which the tube can pass, or it may be formed in the blade using an indentation, a handrail, a groove, or a combination thereof.

[0057] The tube (301) may come into contact with the sliding mechanism (214) when arranged in the flexible portion and the bend portion. The contact between the sliding mechanism and the tube enables displacement of the tube along the flexible portion (202) and / or the bend portion (203) in the Z plane when the operating unit (208) operates the sliding mechanism.

[0058] Alternatively, the sliding mechanism (208) displaces the bend portion (203) and the associated actuating unit in the Z plane to insert and retract the bend portion into the patient's trachea. The actuating unit associated with the bend portion is arranged specifically on a rail guide (not shown) of the sliding mechanism so that the actuating unit associated with the sliding mechanism can displace it accordingly.

[0059] The tube (301) is connected to an operating unit (208) through its arrangement of at least one of a flexible portion (202) and a bending portion (203). The operating unit acts the bending portion to further actuate the bending movement of the tube in the X and Y planes. Simply put, the bending portion serves as a guide for the tube to navigate its direction inside the patient's airway.

[0060] FIG. 4 is an example of an alternative embodiment of an automatic intubation system (400) comprising a main body (401), a flexible part (402) connecting the main body to a bending part (403), and a housing unit (404) attached to the bending part or the flexible part. The housing unit may also support at least one imaging sensor (405), at least one guide light (406), and at least one exit channel (407). The exit channel (407) may be used to provide a channel when additional devices need to be inserted, such as for biopsy, aspiration, and irrigation. The main body further comprises at least one operating unit (408) which may be a rotary motor, a linear motor, and / or a combination of a rotary motor and a linear motor. Other types of motors will be readily apparent to a person skilled in the art. The outer surface of the main body (401) may have some or all of the following: at least one button or handle (409) for manually controlling the operation; a light source (410) for indicating the power status of the automatic system; a switch (411) for turning the automatic system on or off; at least one port (412) for suction; and a tube release switch or lever (413) for separating the tube from the main body and the bending part when the tube reaches a desired position or place. The operating unit (408) may further include a sliding mechanism (414).

[0061] The system further includes a user interface (415) and a circuit unit (416) arranged as a separate unit (417) outside the main body. The separate unit is connected to the main body via a cable (418). Alternatively, the user interface (415), the circuit unit (416), and the system are connected via a wireless connection (not shown). The wireless connection can be established via Bluetooth, Wi-Fi, Zigbee, telecommunications, NFC, or any other communication mode available at the time of system implementation. Wireless communication can also allow the device to be remotely controlled along with data transmission. The remotely connected processing circuit unit can also control multiple operating units from multiple devices at different times and may provide centralized control to hospital management and compliance departments. Communication between different units of the system can be protected by implementing a technology such as SSL.

[0062] FIG. 5 is an example of an exemplary embodiment of the configuration of the bending portion (203) of FIG. 2 comprising a plurality of independent vertebrae (501) stacked together and connected by rivets (502). The vertebrae are connected in such an arrangement that allows for partial and / or complete independent rotational movement of each vertebra relative to the rivet points. The rotational movement of each vertebra enables the bending of the bending portion. The vertebrae are connected to each other via cord(s) (503), one end of the cord(s) is connected to an operating unit (not shown in FIG. 5), and the other end is connected to a vertebra at the distal end of the bending portion. The vertebrae further comprise at least one eye-shaped ring (504) arranged internally. The cord(s) from the operating unit pass through the eye-shaped ring(s) and reach a connection point at the distal end vertebra. Alternatively, a mesh, a combination of the configuration described above and a mesh, or other feasible arrangements known to those skilled in the art may be used to achieve the same purpose.

[0063] FIG. 6 is an example of an exemplary architecture of an automatic intubation system (200) comprising a bending portion (203), an imaging sensor (205), a tube (301), at least one operating unit (208), a user interface (215), and a circuit portion (216). The circuit portion further includes a processing circuit portion (216a) for generating a control signal based on input from at least one imaging sensor, a communication circuit portion (216b) for providing data / signal communication between different components of the system, and a power circuit portion (216c). The operating unit includes a sliding mechanism (213) for providing movement to the tube in the Z plane.

[0064] The processing circuit (216a) may be a combination of a single processor, a logic circuit, a dedicated controller that performs all functions, or a process assistance unit that depends on the functional requirements of the system. In an exemplary embodiment, the processing circuit comprises two independent process assistance units (216aa and 216ab). The process assistance unit (216a) is computer vision software that utilizes data received from the imaging sensor (205) and machine learning techniques to perform at least one function (216aa1, 216aa2, ... 216aaN). The function includes the recognition of anatomical structures and the prediction of an intended path for the insertion of the tube (301) based on the recognition of at least one anatomical structure. The process assistance unit and / or the processing circuit interact with the imaging sensor (205) to receive data during the intubation procedure and perform the functions described above.

[0065] In one embodiment, the recognition of anatomical structures using imaging sensor data and machine learning techniques includes the detection of respiratory structures such as the tracheal opening, glottis, vocal cords, and / or the bifurcation between the esophagus and the trachea. In addition to or as an alternative to the detection of respiratory structures, other anatomical parts of the human body may also be detected and / or recognized.

[0066] Alternatively, the processing circuit (216aa) predicts an intended path based on input from an imaging sensor, sample history data received remotely from the operating unit of a plurality of devices, and a machine learning model. The system further stores the intended path in memory (not shown in the system) to maintain a log of device operation for regulatory purposes. The log of the device may be shared with the remote device for monitoring and control purposes. The process assistance unit (216ab) generates a control signal based on the intended path predicted by the process assistance unit (216aa). Subsequently, the control signal generated by the process assistance unit (216ab) is transmitted from the processing circuit to the operating unit (208) via the communication circuit (216b), and based thereon, the operating unit operates at least one of the bending part (203) and the sliding mechanism (214) to provide three-dimensional movement to the invasive medical device. The process assistance unit (216ab) may also be an integrated part of the operating unit (208), and the control signal is received by the operating unit via a wireless or wired communication circuit. In one scenario, the processing circuit (216aa) is remotely connected to the operating unit (208) via the Internet or a wireless medium to transmit control signals. The communication circuit may also be an integrated part of the operating unit.

[0067] The user interface (215) communicates bidirectionally with the processing circuit (106a).

[0068] The user interface is preferably a display device for displaying data received from an imaging sensor (205) and an overlay of an intended path received from a processing circuit unit to assist the operator. Additionally, the overlay of the intended path may also be visualized in the user interface in the form of augmented reality and / or any other form that provides effective visual guidance to the operator.

[0069] The user interface (215) may also be a touchable display device that allows the operator to adjust the intended path displayed on the user interface. Additionally, if the operator is not satisfied with the intended path of intubation, the intended path displayed on the user interface may be disabled by the operator. Furthermore, it may have touch buttons related to functions performed by buttons arranged on the outer surface of the main body, such as a button to trigger manual operation, a tube release button, and / or a system power off button. Alternatively, the user interface may be any device capable of enabling interaction between the automatic system and the operator, such as audio input, audio output, or gesture-capable input, or any other control scheme that may be enabled by an intelligent agent.

[0070] FIG. 7 is an exemplary flowchart for creating a machine learning model comprising the steps of collecting a plurality of intubation procedure videos from a conventional video laryngoscope (701) and separating the collection of intubation procedure videos based on the predicted difficulty level of the intubation procedure (702). The difficulty level may be automatically predicted in the form of a conventional Malampati score or a customized intubation difficulty scale using a fusion of a computer vision model and a known machine learning algorithm. For easy retrieval and separation of videos based on the calculated score, the calculated or predicted difficulty score may be embedded in the metadata of the video. These videos may be supplemented with videos obtained from other sources, including the device described in this application. There are no limitations on the video sources used for the training videos disclosed in this application.

[0071] In step (703), the separated video is trimmed to exclude parts of the video containing obstructed and / or unclear views of anatomical structures related to the intubation procedure. This step removes avoidable noise from the video data before moving on to the extensive training process of the machine learning model.

[0072] In step (704), the trimmed video file is converted into an image file, which is labeled with anatomical structures, and in step (705), a labeled data set of images is constructed. This labeled data set of images serves as a training data set for training one or more neural networks in step (706) to generate a machine learning model. The generated machine learning model is used in or as part of a process assistance unit (216aa) (i.e., computer vision software) executed by the processing circuit unit (216a) of FIG. 6 to recognize at least one anatomical structure based on data received from the imaging sensor (205) during the intubation procedure.

[0073] FIG. 8 is an example of the use of a typical automated intubation method comprising inserting a detachable blade (801) into the patient's airway (802). Adjacent to the detachable blade, a bend portion (803) and a tube (804) arranged longitudinally along the bend portion are inserted into the patient's airway. This method further involves collecting airway data from at least one imaging sensor (805) arranged along the bend portion. The collected airway data is then transmitted to at least one processing circuit (806) that utilizes a machine learning model and airway data to recognize at least one anatomical structure and predict at least one intended path for the insertion of the tube. The intended path is then used by the processing circuit to generate a control signal and transmit it to at least one operating unit (807) to operate the three-dimensional movement of the tube.

[0074] In particular, since the detachable blade, the bending part, and the tube are directly or indirectly connected to the main body, the detachable blade (801), the bending part (803), and the tube are inserted by introducing the main body (808) near the patient's oral cavity. Additionally, it is preferable that the processing circuit part (806) and the operating unit (807) be located within the main body.

[0075] The three-dimensional movement of the tube (804) arranged in the bend portion (803) includes the bending movement of the tube in the X and Y planes guided by the two-dimensional movement of the bend portion (803) and the movement of the tube in the Z plane by the sliding mechanism of the operating unit (807) (not shown in FIG. 8). The operation of the bend portion is enabled by the operating unit connected to the bend portion via code(s) (not shown in FIG. 8). The method also includes displaying data transmitted from the imaging sensor(s) (805) on the user interface (809), and overlaying the recognized anatomical structure and the intended insertion path of the tube on the user interface.

[0076] The position of the distal end of the tube can be verified by standard clinical treatment methods, such as capnometry, X-rays, and ultrasound, but not limited to these. These methods may be integrated directly into the device or may be integrated to provide indirect support for them. For example, regarding capnometry, the presence of CO2 levels in the air can verify the correct placement of the tube within the patient. This qualitative or quantitative verification may be provided by sensors placed directly on or within the device, such as a CO2 monitor, or through more indirect methods, such as color-changing pH detection strips placed within the view of the imaging sensor to provide accurate verification of CO2 levels. Similarly, ultrasound transmitters and receivers may be integrated into the device to verify whether the distal end of the tube is correctly placed. The techniques described above are merely some of the many clinical approaches to verify the correct placement of an intubation tube that are obvious to a person skilled in the art.

[0077] When the tube reaches a desired location or place inside the patient's airway, it is set to be released from the main body (808) and the bend (803) using a tube release switch or lever (810) located on the outer surface of the main body. Alternatively, a touch button (not shown in FIG. 8) may be provided on the user interface (809) to release or detach the tube.

[0078] FIG. 9 is an example of the use of a user interface (901) comprising a display screen (902) for displaying data received from at least one imaging sensor. The display screen further displays an overlay of at least one recognized anatomical structure (903) and an intended insertion path (905) of a tube (904). The operator may also manually adjust the intended insertion path (905) of the tube (904) displayed in the user interface. Alternatively, the overlay of the tube, the bend, the recognized anatomical structure (903), and the intended insertion path (905) may be displayed in the user interface as an augmented reality, virtual reality, or other form of overlay known to a person skilled in the art to provide effective visual guidance to the operator. The overlay of the recognized anatomical structure may also include annotations or labels for rapid identification of the structure by the operator during the procedure.

[0079] Additionally, the display screen (902) of the user interface (901) may include a pair of up and down touch buttons (906) for manually controlling operation and / or disabling automatic operation if necessary, a system power on / off touch button (907), and a tube release touch button (908).

[0080] In one embodiment, a pair of up and down touch buttons (906) may be used to selectively control manual operation on a selected work plane X, Y, or Z. A touch button (909) provided on a display screen may be used to select a work plane before providing input via the touch button (906). Although FIG. 9 depicts the touch buttons being arranged outside the boundaries of the visual data received from the imaging sensor, it should be understood that the arrangement of the touch buttons may be changed to provide the operator with the best possible visual representation.

[0081] Although the present invention has been described in the context of assistance for surgery, insertion, or implantation, the invention may also be practiced to realize educational or academic uses, such as training and demonstration.

[0082] No language in the specification should be interpreted as indicating any unclaimed element as essential to carrying out the invention.

[0083] It will be apparent to those skilled in the art that various modifications and variations may be made to the present invention without departing from the spirit and scope of the invention. There is no intention to limit the present invention to the specific forms or forms enclosed. On the contrary, the intent of the present invention is to include all modifications, alternative configurations, and equivalents within the spirit and scope of the invention, as defined in the appended claims. Accordingly, the present invention is intended to include modifications and variations of the invention where they fall within the scope of the appended claims and their equivalents.

Claims

Claim 1 An automatic intubation system comprising: a tube; a flexible portion disposed longitudinally within the tube; a bending portion forming at least a part of the distal end of the flexible portion; a housing unit disposed at the distal end of the bending portion and comprising at least one imaging sensor; a processing circuit portion that predicts at least one intended path for insertion of the tube and generates a control signal; and a user interface that displays at least one intended path and also allows at least one of selecting at least one intended path or modifying at least one intended path. An automatic intubation system comprising at least one operating unit receiving a control signal for operating a three-dimensional movement of a tube according to at least one selected or modified intended path, wherein the at least one intended path is configured to be predicted based on data received from at least one imaging sensor, historical data received from at least one operating unit, and at least one anatomical structure recognized using a machine learning model, wherein the machine learning model is configured to collect a plurality of intubation procedure videos; separate the plurality of intubation procedure videos based on the predicted difficulty level of each intubation procedure of the plurality of intubation procedure videos; trim the separated intubation procedure videos to exclude a portion of the video containing an unobstructed view and / or an unclear view of at least one anatomical structure; convert the trimmed video into at least one image file; construct a labeled dataset of a plurality of images by assigning a label representing at least one anatomical structure to at least one converted image file; and train one or more neural networks using the labeled dataset of a plurality of images. Claim 2 An automatic intubation system according to claim 1, wherein at least one operating unit receives a control signal from a processing circuit unit through at least one communication circuit unit. Claim 3 An automatic intubation system according to claim 1, wherein at least one operating unit is connected to a bending portion to operate the bending movement of the tube in the X plane and the Y plane. Claim 4 An automatic intubation system according to claim 1, wherein at least one operating unit comprises a sliding mechanism for operating the sliding movement of a tube in the Z plane. Claim 5 The automatic intubation system of claim 1 further comprises at least one button for triggering at least one operating unit, a switch for releasing a tube, and at least one port, wherein the at least one port is for providing a channel for at least one of an instrument, suction, or irrigation. Claim 6 An automatic intubation system according to claim 1, wherein the processing circuit is configured to recognize at least one anatomical structure based on data received from at least one imaging sensor and history data received from at least one operating unit using a machine learning model, subsequently predict at least one intended path, and generate a control signal based on at least one selected or modified intended path. Claim 7 delete Claim 8 An automatic intubation system according to claim 1, wherein the automatic intubation system is connected to a network and configured to be controlled by a remote operator. Claim 9 delete Claim 10 An automatic intubation system according to claim 1, wherein the user interface is configured to display an overlay of at least one anatomical structure and an overlay of at least one intended path for data received from at least one imaging sensor. Claim 11 An automatic intubation system according to claim 1, configured such that the selection or modification of at least one intended path displayed on a user interface is performed by an operator. Claim 12 An automatic intubation system according to claim 1, wherein the operation of three-dimensional movement of a tube along at least one selected or modified intended path is configured to be invalidated by the operator through a user interface when the operator is not satisfied with at least one intended path. Claim 13 An automatic intubation system according to claim 1, further comprising a main body; and a detachable blade connected to the main body through a disposable and / or reusable sleeve, wherein a flexible portion is connected to the main body. Claim 14 An automatic intubation system according to claim 1, wherein the housing unit further comprises at least one of a guide light and an exit channel. Claim 15 delete Claim 16 delete Claim 17 delete Claim 18 delete Claim 19 delete Claim 20 delete

Images