System and method for automatic cannulation

Abstract

A system, method, and apparatus for automatically performing tracheal intubation in a patient, comprising: inserting a blade into the upper airway of the patient to pull anatomy; inserting a curved portion and a tube disposed on the curved portion into the airway of the patient; collecting airway data using at least one imaging sensor disposed on the curved portion; communicating the collected airway data to processing circuitry; predicting an expected path for the tube insertion using the processing circuitry and generating a control signal, wherein the expected path is predicted based on at least one anatomical structure identified by the processing circuitry using the collected airway data; displaying the expected path via a user interface to display at least one expected path to an operator and also to allow the operator to select an expected path; and communicating the control signal generated by the processing circuitry to at least one actuation unit to actuate three-dimensional motion of the tube.

Description

Technical Field

[0001] The present invention relates to an automated system and method for inserting an invasive medical device into a patient’s body, and more specifically, to an automated system and method for inserting an invasive medical device into a patient’s body cavity using image-based guidance.

[0002] This section describes the technical field in detail and discusses the problems encountered in the field. Therefore, the statements in this section should not be construed as prior art. Background Technology

[0003] Effectively implanting medical devices into patients is one of the greatest needs in the medical field today. One reason for this need is the wide range of applications offered by invasive medical devices, from inserting pacemakers into the chest to ensure the heart beats at an appropriate rate to inserting urinary catheters. Another reason is the numerous complications and complex situations that medical staff, doctors, and anesthesiologists encounter during implantation, requiring immediate intervention to prevent illness and death.

[0004] One such application of invasive devices is endotracheal intubation, used to keep a patient's airway open to support breathing. Endotracheal intubation (or ETI) involves using a laryngoscope to visualize the opening of the glottis and then inserting a tube through that opening. After manipulating the anatomy of the upper airway with a laryngoscope, the physician can see the glottis directly through their eyes, forming a "straight-line view." Clear visualization of the glottic opening using a laryngoscope depends on several factors, such as facial structure, Mallampati score, dental condition, and joint stiffness. Therefore, endotracheal intubation is a process that requires considerable skill and training. Even with proper training, it can be difficult to visualize the glottis opening and insert the tube.

[0005] It is estimated that approximately 81% of endotracheal intubations during pre-hospital care are performed by non-physicians, compared to 19% by physicians. The unpredictable environment during pre-hospital care further complicates successful intubation. The failure rate on the first attempt at endotracheal intubation is estimated to be as high as 41%. This delay in intubation can have serious consequences. Hypoxia can cause permanent brain damage within 4 minutes and death within 10 minutes.

[0006] Alternative methods of intubation using video laryngoscopes offer a better field of view because they incorporate a camera at the tip of the laryngoscope, thus eliminating the need for a "straight-line view." The camera projects an image onto a monitor, which the doctor then uses to manually insert the endotracheal tube. This still requires considerable manual dexterity and visuospatial awareness. These are also difficult skills to learn. The failure rate on the first attempt using a video laryngoscope can also be high.

[0007] When intubation is not possible, several alternative methods are attempted, including supraglottic ventilation, specialized airway devices (such as King's tube or Combitube), mask ventilation, and in some cases, even emergency cricothyrotomy—which involves making an incision in the neck and trachea through which a tube is inserted. As expected, these procedures are less effective than simple endotracheal intubation and can be more invasive and have long-term consequences for the patient.

[0008] Most existing guided intubation systems and methods have limitations, leading to problems such as high latency and high failure rates during intubation. Therefore, there is a genuine need to design a system and method that not only facilitates rapid and successful intubation but also operates completely autonomously, minimizing operator (or user) intervention. Interchangeable operator and user accessibility is also required.

[0009] Patients with severe respiratory infections (such as COVID-19) may experience respiratory distress, requiring intubation and ventilation. Because healthcare workers are in close proximity to infected patients and come into direct contact with their saliva, they themselves are at risk of contracting the disease while adhering to standard care guidelines for such patients. Furthermore, the extent and duration of healthcare worker contact with infected patients directly correlates with the infection rate, making ETI a high-risk procedure for spreading infection.

[0010] One of the objectives of this invention is to overcome the shortcomings of the prior art, such as those described above. Summary of the Invention

[0011] References such as “an embodiment,” “at least one embodiment,” “embodiment,” “an example,” “example,” “for example,” etc., indicate that an embodiment or example may include a particular feature, structure, characteristic, property, element, or limitation, but not every embodiment or example must include that particular feature, structure, characteristic, property, element, or limitation. Furthermore, repeated use of the phrase “in an embodiment” does not necessarily refer to the same embodiment.

[0012] In an aspect of the invention, an automated system inserts an invasive medical device into a patient's body cavity. The automated system includes processing circuitry that receives data from at least one data source to identify structures associated with the patient's body cavity and predict a desired path for inserting the invasive medical device into the patient's body. The processing circuitry also generates control signals based on the desired path and transmits the control signals to at least one actuation unit to actuate three-dimensional movement of the invasive medical device.

[0013] The processing circuitry utilizes a machine learning model and data received from a data source to identify structures associated with the patient's body cavity, predict a desired path, generate control signals, and transmit these control signals to an actuation unit to actuate the three-dimensional movement of the invasive medical device. The desired path will be the path the device will follow once movement begins, guiding the invasive medical device. The generation of the machine learning model includes receiving or collecting training data in the form of a predetermined dataset to train at least one neural network. This neural network can be an object detector based on an edge-based deep neural network, as is well known in the art. Other forms of machine learning besides neural networks are possible alternatives, as is well known to those skilled in the art. The predetermined dataset can be, but is not limited to, images and videos.

[0014] The data source may be an imaging sensor. These sensors may include, but are not limited to, cameras, infrared cameras, acoustic sensors, microwave sensors, photodetectors, or other sensors known to those skilled in the art that may be used to achieve the same purpose. Data received from the imaging sensor can be displayed on a user interface to provide the operator with a view of the patient's body cavity. Furthermore, the intended path and identified structures can be overlaid on the data received from the imaging sensor on the user interface for effective visual guidance of the operator.

[0015] In an exemplary embodiment of the invention, the automated cannulation system predicts the intended path of the inserted catheter and generates control signals to at least one actuation unit. The intended path is predicted based on at least one anatomical structure identified using data received from at least one imaging sensor. The superposition of the intended path and / or the identified anatomical structure is also displayed on the data received from the imaging sensor on the user interface for effective visual guidance during cannulation. The intended path displayed on the user interface can also be adjusted by the operator and / or changed by the operator if they are not satisfied with the intended insertion path. The operator can then select a suggested or adjusted intended path for the system to follow during cannulation.

[0016] In addition, the overlay of the expected path can be displayed on the user interface in the form of augmented reality and / or any other form that provides effective visual guidance to the operator.

[0017] In a preferred embodiment, the automated cannulation system includes: a body; a curved portion; a flexible component connecting the body and the curved portion; a housing unit disposed on the curved portion, the housing unit including at least one imaging sensor; a catheter for cannulation disposed on the flexible component and the curved portion; circuitry; a user interface; a disposable and / or reusable cannula with a blade at one end for traction of anatomical structures; and at least one actuation unit for actuating three-dimensional movement of the catheter. The length of the curved unit is variable and may be located only at the tip of the flexible portion or may completely cover the flexible portion. In other embodiments, the curved portion may be located within any part of the flexible component, depending on several factors, including but not limited to the intended use and the anatomical structures requiring navigation. Preferably, the disposable and / or reusable cannula is detachably coupled to the body. The imaging sensor is preferably a camera, although sensors such as infrared, photodetectors, or other feasible methods known to those skilled in the art may be used to achieve the same purpose.

[0018] In a preferred embodiment of the invention, the circuitry, user interface, and actuation unit are part of the main body. The circuitry also includes processing circuitry, power supply circuitry, and communication circuitry.

[0019] In an alternative embodiment of the invention, the circuitry and user interface are arranged separately from the main body in at least one separate box.

[0020] The processing circuitry is used to predict the intended path of catheter insertion based on at least one identified anatomical structure and generate control signals. The processing circuitry is also used to identify the anatomical structure using data received from an imaging sensor and at least one pre-trained machine learning model. The actuation unit receives the control signals from the processing circuitry to actuate the three-dimensional movement of the catheter. The actuation unit specifically uses a connection with a curved portion to actuate the curved movement of the catheter in the X and Y planes. The actuation unit also includes a sliding mechanism to actuate the sliding movement of the catheter in the Z plane by moving the curved portion and its associated actuation unit along a track. Alternatively, the sliding mechanism actuates the sliding movement of the catheter in the Z plane through direct contact or adjacency with the catheter without moving the curved portion and its associated actuation unit. Those skilled in the art will also recognize that other three-dimensional coordinate schemes, such as radial coordinates, polar coordinates, cylindrical coordinates, and spherical coordinates, can be used instead of the x, y, and z coordinates described herein.

[0021] In another embodiment of the invention, the processing circuit is used only to predict the expected path and generate control signals, while the identification of anatomical structures using imaging sensor data and machine learning models is performed by a separate, independent processing circuit.

[0022] A machine learning model is part of computer vision software developed by training one or more neural networks on a labeled image dataset, wherein the labeled image dataset is constructed by converting a collection of intubation procedure videos into image files and labeling anatomical structures on the image files. In an alternative embodiment, the generation of the machine learning model includes receiving or collecting training data in the form of a predetermined dataset to train at least one neural network. The predetermined dataset may be, but is not limited to, images, audio, and video recorded and collected during the procedure.

[0023] In another embodiment of the invention, control signals for actuating the three-dimensional movement of the catheter, received by the actuation unit, are manually generated via a pair of up / down buttons arranged on the outer surface of the main body or a touch button arranged on the user interface. Therefore, the system provides a manual actuation mode if needed by the operator. If the operator is not satisfied with the predetermined path, they can also use the pair of up / down buttons and the touch button to control automatic actuation.

[0024] In another aspect of the invention, a method for automatically inserting an invasive medical device into a patient's body cavity is provided, comprising inserting a curved portion and an invasive medical device disposed on the curved portion into the patient's body cavity. The method includes collecting airway data using an imaging sensor disposed on the curved portion, and transmitting the collected airway data to processing circuitry to predict a desired insertion path of the invasive medical device and generate a control signal. The control signal is then transmitted to at least one actuation unit to actuate three-dimensional movement of the invasive medical device. Preferably, the processing circuitry predicts the desired path using data transmitted from the imaging sensor based on the identification of at least one structure associated with the body cavity.

[0025] Furthermore, by utilizing machine learning models and data transmitted from imaging sensors, the processing circuitry can perform predictions of the intended insertion path and identification of structures associated with the body cavity. The generation of the machine learning model includes receiving or collecting training data in the form of a predetermined dataset to train at least one neural network. The predetermined dataset may be, but is not limited to, images and videos. It is foreseeable that the device disclosed in this patent can be used in different body cavities besides the airways described herein, or to perform different tasks in any of these body cavities.

[0026] In an exemplary embodiment of the present invention, a method for automatically intubating a patient by inserting a curved portion and a catheter disposed on the curved portion into the patient's airway is provided. The method further includes collecting airway data using an imaging sensor disposed on the curved portion, transmitting the collected airway data to processing circuitry to predict the intended path of the inserted catheter, and generating control signals for actuating the three-dimensional motion of the catheter. Preferably, the processing circuitry predicts the intended path based on the identification of at least one anatomical structure, using data transmitted from the imaging sensor. The processing circuitry utilizes a machine learning model and data transmitted from the imaging sensor to identify anatomical structures and predict the intended path of the inserted catheter.

[0027] The method may also include displaying airway data on a user interface to highlight the airway view to the operator. Furthermore, it includes overlaying the intended path and identified anatomical structures onto the data transmitted from the imaging sensor on the user interface to provide effective visual guidance to the operator.

[0028] Semi-automated invasive device insertion systems offer advantages over fully automated systems. Commercialization of such systems will require regulatory approval from government agencies like the FDA, while the path for semi-automated systems may be simpler and less complex. Furthermore, owning a fully automated system can create a layer of legal liability that could affect the company. Additionally, while the technology may be excellent, it is best for the process to be supervised by trained professionals and manually controlled when necessary to ensure proper intubation. The technical hurdles in developing and manufacturing deployable systems may be reduced when comparing semi-automated to fully automated systems. Finally, built-in validation and control mechanisms, along with an availability layer for implementing the correct pathway, will prevent harm and enhance patient safety.

[0029] In an alternative embodiment, complementary sensors may be integrated with the device, which can provide real-time information on relevant clinical parameters of the patient, such as vital signs, including but not limited to pulse and heart rate, respiratory rate, oxygen saturation, temperature, and blood pressure; as well as other laboratory results, but not limited to blood gas levels, glucose levels, and other results that a person trained in the prior art would know.

[0030] In other embodiments, an operator can connect to the device remotely via the Internet and operate the device using a similar user interface.

[0031] Other embodiments and preferred features of the present invention, as well as corresponding advantages, will become apparent from the following description. Attached Figure Description

[0032] A better understanding of the various aspects and embodiments of the present invention will be achieved by referring to the following detailed description. For a better understanding of the invention, the detailed description should be read in conjunction with the accompanying drawings.

[0033] Figure 1 An exemplary architecture of an automated system for inserting an invasive medical device into a patient's body, according to the present invention, is shown;

[0034] Figure 2 An exemplary embodiment of the automatic cannulation system according to the present invention is shown;

[0035] Figure 3 The assembly of the main body, disposable cannula, and catheter of the automated intubation system according to the present invention is shown;

[0036] Figure 4 An alternative embodiment of the automatic cannulation system according to the present invention is shown;

[0037] Figure 5 The configuration of the curved portion according to the invention is shown;

[0038] Figure 6 An exemplary architecture of the automated cannulation system according to the present invention is shown;

[0039] Figure 7 A flowchart illustrating the generation of a machine learning model according to the present invention is shown;

[0040] Figure 8 The use of a representative automatic cannulation method according to the present invention is illustrated; and

[0041] Figure 9 The use of the user interface according to the present invention is illustrated. Detailed Implementation

[0042] This disclosure can be best understood by referring to the detailed accompanying drawings and description herein. Various embodiments have been discussed with reference to the drawings. However, those skilled in the art will readily understand that the detailed description provided herein with reference to the drawings is for illustrative purposes only, as the methods and systems can be extended beyond the described embodiments. For example, the teachings presented and the needs of a particular application can lead to a variety of alternatives and suitable methods to implement the functionality of any details described herein. Therefore, in the following embodiments, any method can be extended beyond certain implementation options.

[0043] The methods of the present invention can be implemented by manual, automatic, or a combination thereof, performing or carrying out selected steps or tasks. The term "method" refers to the manner, means, technique, and procedure for accomplishing a given task, including but not limited to those manner, means, technique, and procedure known to those skilled in the art to which this invention pertains, or those readily available from known manner, means, technique, and procedure. The descriptions, examples, methods, and materials presented in the claims and specification should not be construed as limiting but merely illustrative. Many other possible variations will be anticipated by those skilled in the art within the scope of the techniques described herein.

[0044] In reading the description of exemplary embodiments of the preferred mode of the present invention, hereinafter referred to simply as "exemplary embodiments," the exemplary embodiments should be considered, from the inventor's point of view, as the best mode for carrying out the invention at the time of patent filing. Since those skilled in the art will recognize that substantially equivalent structures or substantially equivalent actions can yield the same results in the same or different ways, the exemplary embodiments should not be construed as limiting the invention to one embodiment.

[0045] Discussions of species (or specific items) involve the genus (item category) to which the species belongs and related species within that genus. Similarly, descriptions of genuses also involve species known in the art. Furthermore, as technology advances, numerous additional alternatives to the aspects of the invention may emerge. Such modifications are contained within their respective genuses and should be considered functionally or structurally equivalent to the aspects shown or described.

[0046] Unless otherwise explicitly stated, conjunctions (such as “or,” “and,” “including,” or “contains”) should be interpreted as inclusive rather than exclusive.

[0047] Those skilled in the art will understand that various structures and devices are depicted in the block diagrams so as not to obscure the invention. It should be noted that in the following discussion, unless otherwise stated, actions with similar names are performed in a similar manner.

[0048] The foregoing discussion and definitions are provided for clarification purposes and are not restrictive. Unless otherwise stated, words and phrases should be interpreted in their ordinary, straightforward sense.

[0049] The invention can be better understood by referring to the accompanying drawings, in which... Figure 1This is an illustration of an exemplary architecture of an automated system 100 for inserting an invasive medical device into a patient's body cavity. The system includes a bending portion 101, an imaging sensor 102, an invasive medical device 103, at least one actuation unit 104, a user interface 105, and circuitry 106. The circuitry further includes: processing circuitry 106a, which generates control signals based on inputs from at least one imaging sensor and a machine learning model; communication circuitry 106b, which provides data / signal communication between different components of the system; and power supply circuitry 106c. The actuation unit includes a sliding mechanism 107 to provide movement of the invasive medical device in the Z-plane.

[0050] Depending on the system's functional requirements, the processing circuit 106a may be a combination of a single processor, logic circuitry, a dedicated controller performing all functions, or a processing auxiliary unit. In an exemplary embodiment, the processing circuit includes two independent processing auxiliary units 106aa and 106ab. The processing auxiliary unit 106aa is computer vision software that utilizes machine learning techniques and data received from the imaging sensor 102 to perform at least one function (106aa1, 106aa2...106aaN) for the automation of the intubation process. These functions include identifying structures around and inside the patient's body cavity, and predicting the expected path for inserting the invasive medical device 103 into the patient's body. Optionally, the processing circuit 106aa predicts the expected path based on input from the imaging sensor, historical sample data remotely received from the actuators of multiple devices, or a machine learning model. For management purposes, the system also stores the expected path in memory (not shown in the system) for maintaining device operation logs. The device logs can be shared with remote devices for monitoring and control purposes. Other information, such as images from one or more imaging sensors and status and decision points that can be shared with a remote server, can be stored or shared to further improve the machine learning model or for other purposes, such as regulatory or training purposes. This information can be stored locally on the device or on a remote storage device, such as a server or in the cloud. The processing auxiliary unit 106ab generates control signals based on the expected path predicted by the processing auxiliary unit 106aa. The control signals generated by the processing auxiliary unit 106ab are then transmitted from the processing circuitry to the actuation unit 104 via the communication circuitry 106b, based on which the actuation unit actuates at least one of the bending portion 101 and the sliding mechanism 107 to provide three-dimensional motion to the invasive medical device. The processing auxiliary unit 106ab may also be an integrated part of the actuation unit 104, and the control signals may be received by the actuation unit 104 via wireless or wired communication circuitry. The processing circuitry 106aa may also be remotely connected to the actuation unit 104 via a network or wireless medium to send control signals. The communication circuitry may also be an integrated part of the actuation unit. For each and all of the above functions, each of the above functions can be combined with another function within a single functional unit.

[0051] The communication circuit 106b can also be distributed throughout the system to serve as a component for bidirectional data / signal transmission. The communication circuit can be wired or wireless. The power supply circuit 106c distributes power to all units of the system. The power supply circuit includes a rechargeable battery or a DC regulated power supply.

[0052] Actuation unit 104 may be a rotary motor, a linear motor, and / or a combination of a rotary motor and a linear motor. In an exemplary embodiment, multiple actuation units (A1, A2...An) independently actuate the bending portion 101 and the sliding mechanism 107 to provide three-dimensional motion. Alternatively, a single actuation unit may be used to actuate the bending portion 101 and the sliding mechanism 107 integrated with each other. The system can track the motion of the invasive medical device and compare it to an expected path to calculate deviations and calibrate the motion. Calibration can be performed automatically or by manual intervention. Data on the actual motion can be transmitted to a remote device for monitoring purposes.

[0053] User interface 105 communicates bidirectionally with processing circuitry 106a. The user interface is preferably a display device to display data received from imaging sensor 102, as well as the recognition structure and / or expected path from processing circuitry superimposed on the data received from the imaging sensor, to assist the operator in effective visual guidance. Optionally, the user interface can be any device capable of enabling operator interaction with the automation system, such as audio input / output, gesture-enabled input, augmented reality-enabled systems, and / or projection devices. The user interface can also be a head-up display or a head-mounted display to support interaction in the form of virtual reality. User interface 105 can be used to select a suggested expected path or change a suggested path, and to select a modified expected path created by the operator by modifying the suggested expected path.

[0054] Figure 2 An exemplary embodiment of the automated cannulation system 200 includes: a main body 201; a flexible member 202 connecting the main body to a curved portion 203; and a housing unit 204 attached to the curved portion. The housing unit also supports at least one imaging sensor 205, at least one guide light 206, and at least one exit channel 207. Preferably, the imaging sensor is a wide CMOS camera, and the guide light is an LED that automatically turns on when the system is powered on. Optionally, independent control switches for the guide light and the imaging sensor may also be provided.

[0055] The body also includes at least one actuation unit 208 for converting control signals received from the processing circuitry into three-dimensional motion to advance the catheter into the patient's body cavity. The actuation unit 208 may be a rotary motor, a linear motor, and / or a combination of rotary and linear motors. Optionally, the outer surface of the body 201 has: at least one button or knob 209 for manual actuation; a light source 210 indicating the power status of the automation system 200; a switch 211 for turning the automation system on or off; at least one port 212 for aspiration; and a catheter release switch or lever 213 for disconnecting the catheter from the body.

[0056] In one embodiment, the actuation unit 208 further includes a sliding mechanism 214. The sliding mechanism may be an integral part of the actuation unit or a separate unit connected to the actuation unit. The sliding mechanism may be a movable base plate connected to the actuation unit via a rack and pinion mechanism (not shown), wherein the pinion is connected to the actuation unit for rotational motion, and the rack is connected to the movable base plate for converting the rotational motion into vertical motion and / or displacement. Those skilled in the art will recognize other methods or mechanisms for connecting the actuation unit to the movable base plate to achieve the same sliding mechanism. The primary purpose of the sliding mechanism is to provide Z-plane motion for the catheter. This disclosure does not require the use of the sliding mechanism actuation unit 208; as described below, multiple electromechanical systems can be used to provide Z-plane motion to invasive medical devices.

[0057] Optionally, two independent actuation units can be used to actuate the bending portion 203 and the sliding mechanism 214. Processing circuitry (such as...) Figure 1 As shown, control signals for X-plane and Y-plane motion can be sent to the actuation unit that controls the motion of the bending part, and control signals for Z-plane motion can be sent to the actuation unit associated with the sliding mechanism.

[0058] Alternatively, it will be apparent to those skilled in the art that actuation units can be arranged in various different ways for the three-dimensional movement of the conduit. These may include actuation units based on rotation, gears, coiling, or helical motion, as well as free-floating actuation units. Appropriate attention must be paid to allow for the required motion accuracy in the X and Y planes and the required range of motion in the Z plane.

[0059] User interface 215 is also attached to body 201 to display data received from imaging sensor 205. Preferably, the user interface is a display device attached to the body. Alternatively, the user interface is a touch-enabled display device including: at least one button for triggering actuation; a button for releasing the conduit; and a power button (not shown). The user interface can be any device capable of enabling an operator to interact with the automation system, such as an audio input, audio output, or gesture-enabled input. In another embodiment, the user interface may include an intelligent agent that provides necessary operator feedback.

[0060] The main body 201 also includes circuit 216, which further includes processing circuit, communication circuit and power supply circuit.

[0061] The bent portion 203 is connected to the actuation unit 208. Preferably, the bent portion 203 is connected via at least one rope. Figure 2(Not shown) is connected to actuation unit 208. A rope is connected to the actuation unit, passes through the flexible portion, reaches and connects to the curved portion to actuate bending motion and / or movement of the curved portion. Optionally, the rope may be replaced by any feasible mechanical link, such as wire, cable, or chain. Those skilled in the art will know other methods or means for connecting the actuation unit to the curved portion to provide two-dimensional motion in the X and Y planes to the curved portion 203.

[0062] Figure 3 This is an illustration of the assembly of the main body 201 of the automated intubation system 200 with the catheter 301 and cannula 302. The catheter may be arranged longitudinally on the flexible member 202 and the bend 203. Optionally, the catheter may be partially arranged on the flexible member and partially arranged on the bend. Typically, the flexible member passes through the catheter to provide a view of the airway via an imaging sensor supported by the housing unit 204. The catheter is, but is not limited to, an endotracheal catheter, which may include oral, nasal, cuffed, non-cuffed, pre-formed reinforced, double-lumen endotracheal catheters, or any custom-made catheter.

[0063] The cannula 302 may be mechanically connected to the body 201 to detachably connect the blade 303 to the body, preferably via a sliding fit. Other feasible mechanical connections known to those skilled in the art may also be used to achieve the same purpose. The detachable blade 303 at one end of the cannula 302 is used to pull anatomical structures during cannulation. The cannula may be made of disposable and / or reusable materials.

[0064] Blade 303 is designed to improve blade effectiveness for better visibility during intubation, and its shape can be similar to that of blades used in conventional video laryngoscopes. The blade may additionally have an integrated channel to guide the catheter during the initial stages of intubation. The channel may be an open channel through which the catheter can pass, or the channel may be formed on the blade using notches, fences, grooves, or combinations thereof.

[0065] When arranged on the flexible member and the curved portion, the conduit 301 can contact the sliding mechanism 214. When the actuating unit 208 actuates the sliding mechanism, the contact between the conduit and the sliding mechanism allows the conduit to be displaced in the Z-plane along the flexible member 202 and / or the curved portion 203.

[0066] Optionally, the sliding mechanism 208 moves the curved portion 203 and the associated actuation unit in the Z-plane to insert and pull the curved portion into the patient's trachea. The actuation unit associated with the curved portion is specifically disposed on a guide rail (not shown) of the sliding mechanism, such that the actuation unit associated with the sliding mechanism can move it accordingly.

[0067] The catheter 301 is connected to the actuation unit 208 via its arrangement on at least one of the flexible member 202 and the curved portion 203. The actuation unit actuates the curved portion to further actuate the bending movement of the catheter in the X and Y planes. Simply put, the curved portion serves as a guide for the catheter's direction of travel within the patient's airway.

[0068] Figure 4 This illustration shows an alternative embodiment of the automated cannulation system 400, which also includes: a body 401; a flexible member 402 connecting the body to a curved portion 403; and a housing unit 404 attached to the curved portion or the flexible member. The housing unit may also support at least one imaging sensor 405, at least one guide light 406, and at least one outlet channel 407. The outlet channel 407 can be used to provide access when additional equipment needs to be inserted, such as for biopsy, aspiration, and flushing. The body also includes at least one actuation unit 408, which may be a rotary motor, a linear motor, and / or a combination of a rotary motor and a linear motor. Other types of motors will be apparent to those skilled in the art. The outer surface of the body 401 may have some or all of the following: at least one button or knob 409 for manual actuation; a light source 410 for indicating the power status of the automation system; a switch 411 for turning the automation system on or off; at least one port 412 for suction; and a catheter release switch or lever 413 for disconnecting the catheter from the body and the bend when the catheter reaches the desired position or location. The actuation unit 408 may also include a sliding mechanism 414.

[0069] The system also includes a user interface 415 and circuitry 416, which are arranged externally as a separate unit 417. The separate unit is connected to the main body via cable 418. Optionally, the user interface 415, circuitry 416, and system are connected wirelessly (not shown). Wireless connectivity can be established via Bluetooth, Wi-Fi, Zigbee, telecommunications, NFC, or any other communication mode available in the system implementation. Wireless communication also enables the device to be remotely controlled as data is transmitted. The remotely connected processing circuitry can also control multiple actuation units in multiple devices at different times and can provide centralized control to hospital management and compliance departments. Communication between different units of the system can be secured by implementing technologies such as SSL.

[0070] Figure 5 yes Figure 2An exemplary embodiment of the curved portion 203 configuration shown includes a plurality of independent vertebrae 501 stacked on top of each other and connected by rivets 502. The vertebrae are connected in such a manner that each vertebrae can rotate about a rivet point portion and / or completely independently. The rotational movement of each vertebrae enables the curved portion to bend. The vertebrae are connected to each other via ropes 503, one end of which is connected to an actuating unit (…). Figure 5 (Not shown in the diagram), and the other end is connected to the vertebral body at the distal end of the curved portion. The vertebral body also includes at least one eye ring 504 disposed on the inner side. A rope from the actuation unit passes through the eye ring and reaches the connection point of the distal vertebral body. Optionally, a mesh or a combination of the above-described construction and a mesh, or other feasible arrangements known to those skilled in the art, may be used to achieve the same purpose.

[0071] Figure 6 This is an illustration of an exemplary structure of an automated cannulation system 200, including a bending portion 203, an imaging sensor 205, a catheter 301, at least one actuation unit 208, a user interface 215, and circuitry 216. The circuitry further includes: processing circuitry 216a, which generates control signals based on input from at least one imaging sensor; communication circuitry 216b, which provides data / signal communication between different components of the system; and power supply circuitry 216c. The actuation unit includes a sliding mechanism 213 to provide movement of the catheter in the Z-plane.

[0072] Depending on the system's functional requirements, the processing circuit 216a may be a combination of a single processor, logic circuitry, a dedicated controller performing all functions, or a processing auxiliary unit. In an exemplary embodiment, the processing circuit includes two independent processing auxiliary units 216aa and 216ab. The processing auxiliary unit 216a is computer vision software that utilizes machine learning techniques and data received from the imaging sensor 205 to perform at least one function (216aa1, 216aa2...216aaN). These functions include the identification of anatomical structures and predicting the expected path for catheter 301 insertion based on the identification of at least one anatomical structure. The processing auxiliary unit and / or the processing circuit interact with the imaging sensor 205 to receive data and perform the aforementioned functions during cannulation.

[0073] In one embodiment, identifying anatomical structures using imaging sensor data and machine learning techniques includes detecting respiratory structures such as the tracheal opening, glottis, vocal cords, and / or the bifurcation between the esophagus and trachea. Other anatomical parts of the human body may also be detected and / or identified, either in addition to or instead of detecting respiratory structures.

[0074] Optionally, the processing circuit 216aa predicts a desired path based on input from the imaging sensor, historical sample data remotely received from the actuation units of multiple devices, and a machine learning model. For management purposes, the system also stores the desired path in memory (not shown in the system) for maintaining device operation logs. The device logs can be shared with remote devices for monitoring and control purposes. The processing auxiliary unit 216ab generates a control signal based on the desired path predicted by the processing auxiliary unit 216aa. The control signal generated by the processing auxiliary unit 216ab is then transmitted from the processing circuit to the actuation unit 208 via the communication circuit 216b, based on which the actuation unit actuates at least one of the bending portion 203 and the sliding mechanism 214 to provide three-dimensional motion to the invasive medical device. The processing auxiliary unit 216ab may also be an integrated part of the actuation unit 208, and the control signal is received by the actuation unit via a wireless or wired communication circuit. In one case, the processing circuit 216aa is remotely connected to the actuation unit 208 via the Internet or a wireless medium to send control signals. The communication circuit can also be an integrated part of the actuation unit.

[0075] User interface 215 communicates bidirectionally with processing circuitry 106a. The user interface is preferably a display device to display data received from imaging sensor 205 and to display overlays of identified anatomical structures and / or expected paths received from processing circuitry, thereby assisting the operator. Furthermore, the overlay of the expected path may also be displayed on the user interface in augmented reality and / or any other form that provides effective visual guidance to the operator.

[0076] User interface 215 may also be a touch-enabled display device on which the expected path is shown, allowing the operator to adjust it. If the operator is not satisfied with the expected path of the intubation, the expected path displayed on the user interface can also be changed by the operator. Furthermore, it may have touch buttons associated with functions performed by buttons arranged on the outer surface of the body, such as buttons that trigger manual actuation, catheter release buttons, and / or system power-off buttons. Optionally, the user interface may be any device that enables the operator to interact with the automation system, such as an audio input, audio output, or gesture-enabled input, or any other control scheme that can be enabled by a smart agent.

[0077] Figure 7This is an illustrative flowchart for generating a machine learning model, comprising step 701 of collecting a large collection of intubation procedure videos from existing video laryngoscopes, and step 702 of separating the intubation procedure videos into a set based on the predicted difficulty level of the intubation procedure. Using a fusion of computer vision models and known machine learning algorithms, the difficulty level can be automatically predicted in the form of a traditional Malampatti score or a customized intubation difficulty level. The calculated or predicted difficulty score can be embedded in the video's metadata to facilitate easy retrieval and separation of videos based on the calculated score. These videos can be supplemented with videos obtained from other sources, including those from the devices described herein. There are no limitations on the video sources disclosed herein for training videos.

[0078] In step 703, the separated video is trimmed to remove portions containing obscured and / or unclear views of anatomical structures related to the intubation procedure. This step removes avoidable noise from the video data before moving on to the extensive training process of the machine learning model.

[0079] In step 704, the trimmed video file is converted into image files. Then, in step 705, the image files are labeled with anatomical structures to construct a labeled image dataset. In step 706, this labeled image dataset is used as a training dataset to train one or more neural networks to generate a machine learning model. The generated machine learning model is then used in a dataset... Figure 6 The processing circuit 216a executes in or as part of the processing auxiliary unit 216aa (i.e., computer vision software) to identify at least one anatomical structure during cannulation based on data received from the imaging sensor 205.

[0080] Figure 8 This illustration shows the use of a representative automated intubation method, which includes inserting a removable blade 801 into a patient's airway 802. Adjacent to the removable blade, a curved portion 803 and a catheter 804 longitudinally arranged on the curved portion are inserted into the patient's airway. The method also includes collecting airway data from at least one imaging sensor 805 arranged on the curved portion. The collected airway data is then transmitted to at least one processing circuit 806, which uses a machine learning model and the airway data to identify at least one anatomical structure and predict at least one intended path for catheter insertion. The processing circuit then uses the intended path to generate a control signal and transmits the control signal to at least one actuation unit 807 to actuate three-dimensional movement of the catheter.

[0081] Specifically, since the removable blade, the curved portion, and the catheter are directly or indirectly connected to the body, the removable blade 801, the curved portion 803, and the catheter are inserted by bringing the body 808 near the patient's mouth. Furthermore, the processing circuitry 806 and the actuation unit 807 are preferably located within the body.

[0082] The three-dimensional motion of the conduit 804 arranged on the curved portion 803 includes the bending motion of the conduit in the X and Y planes guided by the two-dimensional motion of the curved portion 803, and the sliding mechanism of the actuating unit 807. Figure 8 The movement of the guide tube (not shown) in the Z-plane. This is achieved via a rope (…). Figure 8 (Not shown) An actuation unit connected to the curved portion is used to actuate the curved portion. The method also includes displaying data transmitted from the imaging sensor 805 on a user interface 809, and overlaying the identified anatomical structures and the intended path of the insertion catheter on the user interface.

[0083] The position of the distal catheter can be confirmed using standard clinical care methods, such as, but not limited to, carbon dioxide detection, X-rays, and ultrasound. These methods can be directly integrated into devices or integrated to provide indirect support for such methods. For example, regarding carbon dioxide detection, the level of carbon dioxide present in the air can confirm the accurate placement of the catheter in the patient. This qualitative or quantitative confirmation can be provided by sensors placed directly on or within the device (such as a carbon dioxide monitor), or by more indirect methods (such as a color-changing pH-sensitive strip placed within the field of view of an imaging sensor) to confirm the correct carbon dioxide level. Similarly, ultrasound transmitters and receivers can be integrated into devices that confirm the correct placement of the distal catheter. The techniques discussed above are just a few of the many clinical methods for confirming the correct placement of the catheter, which will be apparent to those skilled in the art.

[0084] Once the desired location or site is reached within the patient's airway, the catheter is released from the body 808 and the curved portion 803 using the catheter release switch or lever 810 located on the outer surface of the main body. Optionally, a touch button ( Figure 8 (Not shown in the image) It can also be set on the user interface 809 to release or disconnect the conduit.

[0085] Figure 9The illustration uses a user interface 901, which includes a display screen 902 for displaying data received from at least one imaging sensor. The display screen also shows an overlay of at least one identified anatomical structure 903 and the intended path 905 of the insertion catheter 904. The operator can also manually adjust the intended path 905 of the insertion catheter 904 displayed on the user interface. Optionally, the overlay of the catheter, bends, identified anatomical structure 903, and intended insertion path 905 may be displayed on the user interface in augmented reality, virtual reality, or other overlay forms known to those skilled in the art to provide effective visual guidance to the operator. The overlay of the identified anatomical structure may also include annotations or markings for the operator to quickly identify structures during the procedure.

[0086] In addition, the display screen 902 of the user interface 901 may include: a pair of up and down touch buttons 906 for manual control of actuation and / or control of automatic actuation if needed; a system power on / off touch button 907; and a conduit release touch button 908.

[0087] In one embodiment, a pair of up and down touch buttons 906 can be used to selectively control manual actuation in a selected work plane X, Y, or Z. Touch buttons 909 provided on the display screen can be used to select a work plane before input is provided via touch buttons 906. It should be understood that, although in Figure 9 The touch buttons are depicted as being positioned outside the boundaries of the visual data received from the imaging sensor, but the arrangement of the touch buttons can be changed to provide the operator with the best possible visual representation.

[0088] Although the invention has been explained in the context of assisting surgery, insertion or implantation, it can also be practiced for educational or academic purposes, such as training and demonstration.

[0089] Nothing in the specification should be construed as indicating any unclaimed element necessary for the practice of the invention.

[0090] It will be apparent to those skilled in the art that various modifications and variations can be made to this invention without departing from its spirit and scope. The invention is not intended to be limited to the specific forms appended. Rather, it is intended to cover all modifications, alternative constructions, and equivalents falling within the spirit and scope defined in the appended claims. Therefore, this invention is intended to cover the modifications and variations provided herein, provided they are within the scope of the appended claims and their equivalents.

Claims

1. An automatic cannulation system, comprising: catheter; A flexible component, which is longitudinally disposed within the conduit; The curved portion forms at least a portion of the distal end of the flexible component; A housing unit disposed at the distal end of the flexible member and including at least one imaging sensor; Processing circuitry is used to predict at least one expected path for insertion of the catheter and generate control signals; A user interface for displaying the at least one expected path and allowing at least one of the following: selecting one of the at least one expected path and modifying one of the at least one expected path; as well as At least one actuation unit is configured to receive the control signal to actuate the catheter in three dimensions along a selected or modified intended path. The at least one expected path is predicted based on data received from the at least one imaging sensor, historical sample data received from the at least one actuation unit, and at least one anatomical structure identified by a machine learning model. The machine learning model is generated through the following steps: Collect multiple videos of the intubation process; The set of intubation process videos is separated based on the prediction difficulty level of the intubation process; Trim the separated intubation process video to remove video portions containing unobstructed and / or unclear views of at least one anatomical structure; Convert trimmed video into image files; Anatomical structures are labeled on the converted image files to construct a labeled image dataset; as well as One or more neural networks are trained using the labeled image dataset.

2. The automatic cannulation system according to claim 1, wherein the actuation unit receives a control signal from the processing circuit via at least one communication circuit.

3. The automated cannulation system of claim 1, wherein the actuation unit is connected to the bending portion to actuate the bending motion of the catheter in the X and Y planes.

4. The automatic cannulation system according to claim 1, wherein the actuation unit includes one of a sliding mechanism and a rotating mechanism to actuate the sliding movement of the catheter in the Z-plane.

5. The automated cannulation system of claim 1, wherein the processing circuitry utilizes a machine learning model and the data received from the imaging sensor to identify at least one anatomical structure, and subsequently predicts a desired path and generates a control signal.

6. The automatic intubation system of claim 1, wherein the system is connectable to a network and can be controlled by a remote operator.

7. The automatic cannulation system according to claim 1, wherein the user interface is a display device.

8. The automated cannulation system of claim 1, wherein the user interface displays an overlay of identified anatomical structures and an overlay of the intended path on the data received from the imaging sensor.

9. The automatic cannulation system of claim 1, wherein the intended path displayed on the user interface can be modified by the operator.

10. The automated cannulation system of claim 1, wherein when the operator is dissatisfied with the intended path, the operator can change the actuation of the catheter's movement according to the intended path.

11. The automatic cannulation system of claim 1, wherein the housing unit further comprises at least one guide light or at least one outlet channel.

Images