Systems and methods for automated motion control of a cantual system

Abstract

A system, method and device for automatically performing tracheal intubation in a patient comprising: inserting a blade into the upper airway of the patient to pull an anatomical structure; inserting a curved portion and a catheter disposed on the curved portion into the airway of the patient; collecting airway data using at least one imaging sensor disposed on the curved portion; transmitting the collected airway data to a processing circuit; predicting, using the processing circuitry, an expected path for insertion of the catheter and generating a control signal, wherein the expected path is predicted based on at least one anatomical structure identified by the processing circuitry using the collected airway data; displaying the expected paths via a user interface to display at least one expected path to an operator, and also allowing the operator to select an expected path; and transmitting the control signal generated by the processing circuit to at least one actuation unit to actuate the three-dimensional movement of the catheter.

Description

Technical Field

[0001] The present invention relates to an automated system and method for inserting an invasive medical device into a patient's body, and more particularly, to an automated system and method for inserting an invasive medical device into a patient's body cavity using image-based guidance.

[0002] This section describes the technical field in detail and discusses the problems encountered in the technical field. Therefore, the statements in this section should not be interpreted as prior art. Background Art

[0003] One of the greatest needs in medicine today is to effectively implant medical devices into patients. One reason for this need is that invasive medical devices offer a wide range of applications, from inserting a pacemaker in the chest to ensure the heart beats at an appropriate rate to inserting a urinary catheter. Another reason is that during the implantation process, medical staff, physicians, and anesthesiologists encounter a multitude of complications and intricacies that require immediate action to prevent morbidity and mortality.

[0004] One such application for implanting invasive devices is endotracheal intubation, which is used to keep a patient's airway open to support breathing. Endotracheal intubation (or ETI) is performed by using a laryngoscope to visualize the opening of the glottis and then inserting a tube through the opening. The physician can directly visualize the glottis with their eyes after manipulating the anatomy of the upper airway with the laryngoscope, creating a "straight line of sight." Clearly visualizing the glottis opening using a laryngoscope depends on several factors, such as facial structure, Mallampati score, dental condition, and joint stiffness. Therefore, endotracheal intubation is a procedure that requires considerable skill and training. Even with proper training, it can be difficult to visualize the glottis opening and insert a tube.

[0005] It is estimated that approximately 81% of endotracheal intubations during prehospital care are performed by non-physicians and 19% by physicians. The unpredictable environment during prehospital care further complicates successful intubation. It is estimated that the failure rate on the first attempt when doing an endotracheal intubation is as high as 41%. This delay in intubating a patient has serious consequences. Lack of oxygen can cause permanent brain damage within 4 minutes and death within 10 minutes.

[0006] Alternative methods of intubation using video laryngoscopes offer a better view because they contain a camera on the tip of the laryngoscope, so a "straight line of sight" is not required. The camera projects an image onto a monitor, and the endotracheal tube can be inserted manually by the physician, looking at the monitor. This still requires a great deal of manual dexterity and visual-spatial cognition. These are also difficult skills to learn. First attempt failure rates with video laryngoscopes can also be high.

[0007] When patients cannot be intubated, several alternatives are tried, including supraglottic ventilation devices, special airway devices (such as the King's tube or Combitube), mask ventilation, and in some cases, even emergency cricothyrotomy—an incision made in the neck and trachea through which a tube is inserted. As expected, these procedures are less effective than simple endotracheal intubation and can be more traumatic for the patient, with long-term sequelae.

[0008] Most of the guided intubation systems and methods in the prior art have limitations that lead to problems such as high delays and failure rates during intubation. Therefore, there is a real need to design a system and method that not only facilitates rapid and successful intubation, but also works completely autonomously and with minimal intervention by the operator (or user). Operator and user may be used interchangeably.

[0009] Patients with severe respiratory infections, such as COVID-19, may experience respiratory distress, which may require intubation and ventilation. Because healthcare workers are in close proximity to infected patients and come in direct contact with the saliva of such patients, they are at risk of contracting the disease themselves while following the standard of care for such patients. Additionally, factors such as the duration and extent of their contact with patients directly correlate with the risk of contracting the disease, making ETI a high-risk procedure for spreading infection.

[0010] One of the objects of the present invention is to overcome the deficiencies in the prior art such as those mentioned above. Summary of the invention

[0011] References to "one embodiment," "at least one embodiment," "an embodiment," "an example," "example," "for example," etc. indicate that the embodiment or example may include a particular feature, structure, characteristic, property, element, or limitation, but not every embodiment or example must include the particular feature, structure, characteristic, property, element, or limitation. Furthermore, repeated use of the phrase "in an embodiment" does not necessarily refer to the same embodiment.

[0012] In aspects of the present invention, an automated system inserts an invasive medical device into a body cavity of a patient. The automated system includes processing circuitry that receives data from at least one data source to identify structures associated with the body cavity of the patient and predict an expected path for inserting the invasive medical device into the patient. The processing circuitry also generates a control signal based on the expected path and transmits the control signal to at least one actuation unit to actuate three-dimensional movement of the invasive medical device.

[0013] The processing circuit can use the machine learning model and the data received from the data source to identify structures associated with the patient's body cavity, predict the expected path, generate control signals and transmit the control signals to the actuation unit to actuate the three-dimensional movement of the invasive medical device. The expected path will be the path along which the device will guide the invasive medical device once the movement begins. The generation of the machine learning model includes receiving or collecting training data in the form of a predetermined data set to train at least one neural network. This neural network can be in the form of an object detector based on an edge-implemented deep neural network as is known in the art. As is well known to those skilled in the art, other forms of machine learning other than neural networks may be substituted. The predetermined data set can be, but is not limited to, images and videos.

[0014] The data source may be an imaging sensor. These sensors may include, but are not limited to, cameras, infrared cameras, acoustic sensors, microwave sensors, photoelectric detectors, or other sensors known to those skilled in the art may also be used to achieve the same purpose. The data received from the imaging sensor may be displayed on a user interface to provide an operator with a view of the patient's body cavity. In addition, the expected path and the identified structure may be superimposed on the data received from the imaging sensor on the user interface for effective visual guidance of the operator.

[0015] In an exemplary embodiment of the present invention, an automated intubation system predicts an expected path for inserting a catheter and generates a control signal to at least one actuation unit. The expected path is predicted based on at least one anatomical structure identified using data received from at least one imaging sensor. An overlay of the expected path and / or the identified anatomical structure is also displayed on the data received from the imaging sensor on a user interface for effective visual guidance during intubation. The expected path displayed on the user interface may also be adjusted by the operator and / or changed by the operator if the operator is not satisfied with the expected insertion path. The operator may then select a suggested or adjusted expected path for the system to follow during intubation.

[0016] Additionally, an overlay of the intended path may also be displayed on the user interface in the form of augmented reality and / or any other form that provides effective visual guidance to the operator.

[0017] In a preferred embodiment, the automatic intubation system comprises: a main body; a curved portion; a flexible member connecting the main body and the curved portion; a housing unit arranged on the curved portion, the housing unit comprising at least one imaging sensor; a catheter for intubation, which is arranged on the flexible portion and the curved portion; a circuit; a user interface; a disposable and / or reusable sleeve with a blade at one end of the sleeve for pulling the anatomical structure; and at least one actuating unit for actuating the three-dimensional movement of the catheter. The length of the curved unit is variable and may be only at the tip of the flexible portion, or may completely cover the flexible portion. In other embodiments, the curved portion may be located in any part of the flexible portion, which is determined by several factors, including but not limited to the relevant use and the anatomical structure that needs to be navigated. Preferably, the disposable and / or reusable sleeve is detachably connected to the main body. The imaging sensor is preferably a camera, although sensors such as infrared, photodetectors or other feasible methods known to those skilled in the art can be used to achieve the same purpose.

[0018] In a preferred embodiment of the invention, the circuit, the user interface and the actuation unit are part of the body. The circuit further comprises a processing circuit, a power supply circuit and a communication circuit.

[0019] In an alternative embodiment of the invention, the circuitry and the user interface are arranged in at least one separate box from the main body.

[0020] The processing circuit is used to predict the expected path of catheter insertion according to at least one identified anatomical structure and generate a control signal. The processing circuit is also used to use the data received from the imaging sensor and at least one pre-trained machine learning model to identify the anatomical structure. The actuation unit receives a control signal from the processing circuit to actuate the three-dimensional movement of the catheter. The actuation unit particularly uses the connection with the curved portion to actuate the bending movement of the tube in the X plane and the Y plane. The actuation unit also includes a sliding mechanism to actuate the sliding movement of the catheter in the Z plane by moving the curved portion and its associated actuation unit on the track. Optionally, the sliding mechanism actuates the sliding movement of the catheter in the Z plane by direct contact or abutment with the catheter, without moving the curved portion and its associated actuation unit. It is also recognized by those skilled in the art that other three-dimensional coordinate schemes, such as radial coordinates, polar coordinates, cylindrical coordinates, and spherical coordinates, can be used to replace the x, y, and z coordinates described herein.

[0021] In another embodiment of the present invention, the processing circuit is used only to predict the expected path and generate the control signal, while the use of imaging sensor data and machine learning models to identify anatomical structures is performed by a separate independent processing circuit.

[0022] The machine learning model is a portion of computer vision software developed by training one or more neural networks on a labeled image dataset, wherein the labeled image dataset is constructed by converting a collection of intubation procedure videos into image files and labeling anatomical structures on the image files. In an alternative embodiment, the generation of the machine learning model includes receiving or collecting training data in the form of a predetermined dataset to train at least one neural network. The predetermined dataset can be, but is not limited to, images, audio, and video recorded and collected during the procedure.

[0023] In yet another embodiment of the present invention, the control signal received by the actuation unit for actuating the three-dimensional movement of the catheter is manually generated by a pair of up and down buttons arranged on the outer surface of the body or a touch button arranged on the user interface. Therefore, if the operator needs, the system provides a manual actuation mode. If the operator is not satisfied with the predetermined path, the operator can also use the pair of up and down buttons and the touch button to control the automatic actuation.

[0024] In another aspect of the present invention, a method for automatically inserting an invasive medical device into a patient's body cavity is provided, which includes inserting a curved portion and an invasive medical device arranged on the curved portion into the patient's body cavity. The method includes collecting airway data using an imaging sensor arranged on the curved portion, and transmitting the collected airway data to a processing circuit to predict an expected path for insertion of the invasive medical device and generate a control signal. The control signal is then transmitted to at least one actuation unit to actuate the three-dimensional movement of the invasive medical device. Preferably, the processing circuit uses data transmitted from the imaging sensor to predict the expected path based on the identification of at least one structure associated with the body cavity.

[0025] In addition, by utilizing the machine learning model and the data transmitted from the imaging sensor, the prediction of the expected insertion path and the identification of structures associated with the body cavity can be performed by the processing circuit. The generation of the machine learning model includes receiving or collecting training data in the form of a predetermined data set to train at least one neural network. The predetermined data set can be, but is not limited to, images and videos. It is foreseeable that the device disclosed in this patent can be used in different body cavities other than the airway described herein, or perform different tasks in any of these body cavities.

[0026] In an exemplary embodiment of the present invention, a method for automatically intubating a patient by inserting a curved portion and a catheter disposed on the curved portion into the patient's airway is provided. The method also includes collecting airway data using an imaging sensor disposed on the curved portion, and transmitting the collected airway data to a processing circuit to predict an expected path for inserting the catheter and generate a control signal for actuating three-dimensional movement of the catheter. Preferably, the processing circuit uses data transmitted from the imaging sensor to predict the expected path based on identification of at least one anatomical structure. The processing circuit uses a machine learning model and data transmitted from the imaging sensor to identify the anatomical structure and predict the expected path for inserting the catheter.

[0027] The method may also include displaying the airway data on a user interface to highlight the airway view to the operator. In addition, it includes superimposing the expected path and the identified anatomical structure on the data transmitted from the imaging sensor on the user interface to provide effective visual guidance to the operator.

[0028] Semi-automated invasive device insertion systems have advantages over fully automated systems. Commercialization of such systems will require regulatory approval from government agencies such as the FDA, whereas the path to a semi-automated system may be simpler and less complex. Additionally, having a fully automated system has the potential to create a layer of legal liability that companies may be subject to. Furthermore, while the technology may be good, it is best to have a trained professional oversee the process and manually control it when necessary to ensure proper cannulation. Comparing semi-automated systems to fully automated systems may reduce the technical barriers to developing and producing a deployable system. Finally, built-in validation and control mechanisms and usability layers that enforce the correct path will prevent harm and be safer for patients.

[0029] In alternative embodiments, complementary sensors may be integrated with the device which can provide real-time information of relevant clinical parameters of the patient, such as vital signs, including but not limited to pulse and heart rate, respiratory rate, oxygen saturation, temperature, blood pressure; and other laboratory results, including but not limited to blood gas levels, glucose levels, and other results that would be known to a person trained in the art.

[0030] In other embodiments, an operator may remotely connect to the device via the Internet and may operate the device using a similar user interface.

[0031] Other embodiments and preferred features of the invention and corresponding advantages will become apparent from the following description and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Various aspects and embodiments of the present invention may be better understood by reference to the following detailed description.For a better understanding of the present invention, the detailed description should be read in conjunction with the accompanying drawings.

[0033] Figure 1 An exemplary architecture of an automated system for inserting an invasive medical device into a patient according to the present invention is shown;

[0034] Figure 2 An exemplary embodiment of an automatic intubation system according to the present invention is shown;

[0035] Figure 3 The main body, disposable sleeve and catheter assembly of the automatic intubation system according to the present invention are shown;

[0036] Figure 4 An alternative embodiment of an automated intubation system according to the present invention is shown;

[0037] Figure 5 shows the configuration of the bending portion according to the present invention;

[0038] Figure 6 An exemplary architecture of an automatic intubation system according to the present invention is shown;

[0039] Figure 7 A flowchart of generating a machine learning model according to the present invention is shown;

[0040] Figure 8 shows the use of a representative automated intubation method according to the present invention; and

[0041] Fig. 9 The use of a user interface according to the invention is shown. DETAILED DESCRIPTION

[0042] The present disclosure may be best understood with reference to the detailed drawings and detailed descriptions set forth herein. Various embodiments have been discussed with reference to the drawings. However, those skilled in the art will readily appreciate that the detailed descriptions provided herein with respect to the drawings are for purposes of explanation only, as the methods and systems may extend beyond the described embodiments. For example, the teachings presented and the needs of a particular application may generate a variety of alternatives and suitable methods to implement the functionality of any detail described herein. Therefore, in the following embodiments, any method may be extended beyond certain implementation options.

[0043] The method of the present invention can be implemented by performing or implementing selected steps or tasks manually, automatically, or a combination thereof. The term "method" refers to the manner, means, technique, and procedure for completing a given task, including but not limited to those known to practitioners in the field to which the present invention belongs or those readily obtained from known manners, means, techniques, and procedures. The descriptions, examples, methods, and materials presented in the claims and specification should not be construed as limiting but merely illustrative. Those skilled in the art will foresee many other possible variations within the scope of the technology described herein.

[0044] When reading the description of the exemplary embodiment of the best mode of the invention, hereinafter referred to as the "exemplary embodiment", the exemplary embodiment should be considered the best mode of practicing the invention at the time of filing the patent in accordance with the inventor's opinion. Since one of ordinary skill in the art can recognize substantially equivalent structures or substantially equivalent actions to achieve the same results in the same way or in a different way, the exemplary embodiment should not be interpreted as limiting the invention to one embodiment.

[0045] Discussion of a species (or a particular item) relates to the genus (category of item) to which the species belongs and to related species within the genus. Likewise, description of the genus also relates to species known in the art. In addition, as technology advances, many additional alternatives for implementing aspects of the present invention may occur. Such improvements are contained in their respective genus and should be considered functionally equivalent or structurally equivalent to the aspects shown or described.

[0046] Unless expressly stated otherwise, conjunctions such as "or", "and", "including" or "comprising" are to be construed as inclusive rather than exclusive.

[0047] Those skilled in the art will appreciate that various structures and devices are described in the block diagrams so as not to obscure the present invention. It should be noted that in the following discussion, actions with similar names are performed in a similar manner unless otherwise specified.

[0048] The foregoing discussion and definitions are provided for purposes of clarity and are not limiting. Unless otherwise indicated, words and phrases should be given their ordinary, plain meanings.

[0049] The present invention may be better understood by examining the accompanying drawings, in which Figure 11 is a diagram of an exemplary architecture of an automated system 100 for inserting an invasive medical device into a patient's body cavity. The system includes a bending portion 101, an imaging sensor 102, an invasive medical device 103, at least one actuation unit 104, a user interface 105, and a circuit 106. The circuit also includes: a processing circuit 106a that generates control signals based on inputs from at least one imaging sensor and a machine learning model; a communication circuit 106b that provides data / signal communication between different components of the system; and a power supply circuit 106c. The actuation unit includes a sliding mechanism 107 to provide motion to the invasive medical device in the Z plane.

[0050] According to the functional requirements of the system, the processing circuit 106a can be a combination of a single processor, a logic circuit, a dedicated controller that performs all functions, or a processing auxiliary unit. In an exemplary embodiment, the processing circuit includes two independent processing auxiliary units 106aa and 106ab. The processing auxiliary unit 106aa is a computer vision software that uses machine learning technology and data received from the imaging sensor 102 to perform at least one function (106aa1, 106aa2...106aaN) for the automation of the intubation process. These functions include identifying the structure around the patient's body cavity and the structure inside, and predicting the expected path for inserting the invasive medical device 103 into the patient's body. Optionally, the processing circuit 106aa predicts the expected path based on the input from the imaging sensor, the sample history data remotely received from the actuation unit of multiple devices, or the machine learning model. For management purposes, the system also stores the expected path for maintaining the device operation log in the memory (not shown in the system). The log of the device can be shared with the remote device for monitoring and control purposes. Other information, such as images from one or more imaging sensors and states and decision points that can be shared with a remote server, can be stored or shared to further improve the machine learning model or for other purposes, such as regulatory or training purposes. This information can be stored locally on the device or on a remote storage device, such as a server or cloud. The processing auxiliary unit 106ab generates a control signal based on the expected path predicted by the processing auxiliary unit 106aa. The control signal generated by the processing auxiliary unit 106ab is then transmitted from the processing circuit to the actuation unit 104 via the communication circuit 106b, based on which the actuation unit actuates at least one of the bending portion 101 and the sliding mechanism 107 to provide three-dimensional motion to the invasive medical device. The processing auxiliary unit 106ab can also be an integrated part of the actuation unit 104, and the control signal can be received by the actuation unit 104 via a wireless communication circuit or a wired communication circuit. The processing circuit 106aa can also be remotely connected to the actuation unit 104 via a network or wireless medium to send the control signal. The communication circuit can also be an integrated part of the actuation unit. For each and all of the functions described above, each function described above may be combined with another function within a single functional unit.

[0051] Communication circuits 106b may also be distributed throughout the system to serve as elements for bidirectional data / signal transmission. The communication circuits may be wired or wireless. Power supply circuits 106c distribute power to all units of the system. The power supply circuits include rechargeable batteries or DC regulated power supplies.

[0052] The actuation unit 104 may be a rotary motor, a linear motor, and / or a combination of a rotary motor and a linear motor. In an exemplary embodiment, a plurality of actuation units (A1, A2 ... An) independently actuate the bending portion 101 and the sliding mechanism 107 to provide three-dimensional motion. Alternatively, a single actuation unit may be used to actuate the bending portion 101 and the sliding mechanism 107 integrally with each other. The system may track the motion of an invasive medical device and compare it with an expected path to calculate deviations and calibrate the motion. Calibration may be done automatically or by manual intervention. Data of the actual motion may be sent to a remote device for monitoring purposes.

[0053] The user interface 105 is in bidirectional communication with the processing circuit 106a. The user interface is preferably a display device to display the data received from the imaging sensor 102, and the identification structure and / or expected path from the processing circuit are superimposed on the data received from the imaging sensor to assist the operator in effective visual guidance. Optionally, the user interface can be any device that enables the operator to interact with the automated system, such as an audio input / output terminal, a gesture-enabled input terminal, an augmented reality-enabled system, and / or a projection device. The user interface can also be a head-up display or a head-mounted display to support virtual reality form interaction. The user interface 105 can be used to select a suggested expected path or change a suggested path, and select a modified expected path created by the operator by modifying the suggested expected path.

[0054] Figure 2 2 is an exemplary embodiment of an automatic intubation system 200, which includes: a body 201; a flexible member 202, which connects the body to a curved portion 203; and a housing unit 204, which is attached to the curved portion. The housing unit also supports at least one imaging sensor 205, at least one guide light 206, and at least one exit channel 207. Preferably, the imaging sensor is a wide CMOS camera, and the guide light is an LED light that automatically turns on when the system is turned on. Optionally, independent control switches for the guide light and the imaging sensor may also be provided.

[0055] The body also includes at least one actuation unit 208 for converting control signals received from the processing circuit into a three-dimensional motion for advancing the catheter into the patient's body cavity. The actuation unit 208 can be a rotary motor, a linear motor, and / or a combination of a rotary motor and a linear motor. Optionally, the outer surface of the body 201 has: at least one button or knob 209, which manually controls the actuation; a light source 210, which indicates the power status of the automation system 200; a switch 211, which turns the automation system on or off; at least one port 212, which is used for suction; and a catheter release switch or lever 213, which disconnects the catheter from the body.

[0056] In one embodiment, the actuating unit 208 also includes a sliding mechanism 214. The sliding mechanism may be an integral part of the actuating unit or may be a separate unit connected to the actuating unit. The sliding mechanism may be a movable substrate connected to the actuating unit via a rack and pinion mechanism (not shown), wherein the pinion is connected to the actuating unit for rotational motion, and the rack is connected to the movable substrate for converting the rotational motion into vertical motion and / or displacement. Those skilled in the art will know other methods or mechanisms for connecting the actuating unit to the movable substrate to achieve the same sliding mechanism. The main purpose of the sliding mechanism is to provide Z-plane motion for the catheter. The present disclosure does not require the use of a sliding mechanism actuating unit 208, and as described below, a plurality of electromechanical systems may be used to provide motion in the Z-plane to the invasive medical device.

[0057] Optionally, two independent actuation units may be used to actuate the bending portion 203 and the sliding mechanism 214. The processing circuit (eg Figure 1 As shown in the figure, the control signal of X-plane movement and Y-plane movement can be sent to the actuation unit that controls the movement of the bending part, and the control signal of Z-plane movement can be sent to the actuation unit associated with the sliding mechanism.

[0058] Alternatively, it will be apparent to those skilled in the art that there are a variety of different arrangements of actuation units for three-dimensional movement of the catheter. These may include the use of rotational, geared, coiled or helical based actuation units as well as free floating actuation units. Appropriate care must be taken to allow for the accuracy of movement in the X and Y planes and the required amplitude of movement in the Z plane.

[0059] A user interface 215 is also attached to the body 201 to display data received from the imaging sensor 205. Preferably, the user interface is a display device attached to the body. Optionally, the user interface is a touch-enabled display device that includes: at least one button to trigger actuation; a button to release the catheter; and a power button (not shown). The user interface can be any device that enables an operator to interact with the automated system, such as an audio input, an audio output, or a gesture-enabled input. In another embodiment, the user interface can include an intelligent agent that provides the necessary operator feedback.

[0060] The body 201 also includes circuitry 216, which also includes processing circuitry, communication circuitry, and power supply circuitry.

[0061] The curved portion 203 is connected to the actuating unit 208. Preferably, the curved portion 203 is connected to the actuating unit 208 via at least one cable ( Figure 2The flexible portion 200 is connected to the actuation unit 208 by a cable (not shown). The cable is connected to the actuation unit and passes through the flexible portion to reach and connect to the bending portion to actuate the bending motion and / or movement of the bending portion. Alternatively, the cable can be replaced by any feasible mechanical link, such as a line, a wire, a cable and a chain. Those skilled in the art will know other methods or means for connecting the actuation unit to the bending portion to provide two-dimensional motion in the X plane and the Y plane to the bending portion 203.

[0062] Figure 3 2 is a diagram of assembling the main body 201 of the automatic intubation system 200 with the catheter 301 and the sleeve 302. The catheter can be arranged longitudinally on the flexible member 202 and the curved portion 203. Optionally, the catheter can be arranged partially on the flexible member and partially on the curved portion. Typically, the flexible member passes through the catheter to provide a view of the respiratory tract via an imaging sensor supported by the housing unit 204. The catheter is, but is not limited to, an endotracheal tube, which can include an oral, nasal, cuffed, uncuffed, preformed reinforced, double-lumen endobronchial tube, or any custom catheter.

[0063] The cannula 302 can be mechanically connected to the body 201 to removably connect the blade 303 to the body, preferably via a slip-fit ​​connection. Other feasible mechanical connections known to those skilled in the art can also be used to achieve the same purpose. The removable blade 303 at one end of the cannula 302 is used to pull the anatomical structure during the intubation process. The cannula can be made of disposable and / or reusable materials.

[0064] The blade 303 is designed to improve the effectiveness of the blade to provide better visibility during intubation and may be similar in shape to the blade of a conventional video laryngoscope. The blade may additionally have an integrated channel to guide the catheter during the initial stages of intubation. The channel may be an open channel through which the catheter may pass, or the channel may be formed on the blade using indentations, fences, grooves, or a combination thereof.

[0065] When arranged on the flexible member and the curved portion, the catheter 301 may contact the sliding mechanism 214. When the actuation unit 208 actuates the sliding mechanism, the contact of the catheter with the sliding mechanism enables the catheter to be displaced along the flexible member 202 and / or the curved portion 203 in the Z plane.

[0066] Optionally, the sliding mechanism 208 moves the bending part 203 and the associated actuating unit in the Z plane to insert and pull the bending part in the patient's trachea. The actuating unit associated with the bending part is particularly arranged on a guide rail (not shown) of the sliding mechanism so that the actuating unit associated with the sliding mechanism can move it accordingly.

[0067] The catheter 301 is connected to the actuation unit 208 via its arrangement on at least one of the flexible member 202 and the curved portion 203. The actuation unit actuates the curved portion to further actuate the bending movement of the catheter in the X plane and the Y plane. In simple terms, the curved portion serves as a guide for the catheter to guide the direction in the patient's airway.

[0068] Figure 4 400, which further includes: a body 401; a flexible member 402, which connects the body to a curved portion 403; and a housing unit 404, which is attached to the curved portion or the flexible member. The housing unit may also support at least one imaging sensor 405, at least one guide light 406, and at least one exit channel 407. The exit channel 407 may be used to provide a channel in cases where additional equipment needs to be inserted, such as for biopsy, suction, and flushing. The exit channel 407 may be used to provide a channel in cases where additional equipment needs to be inserted, such as for biopsy, suction, and flushing. The body also includes at least one actuating unit 408, which may be a rotary motor, a linear motor, and / or a combination of a rotary motor and a linear motor. Other types of motors will be apparent to those skilled in the art. The outer surface of the body 401 may have some or all of the following: at least one button or knob 409 that manually controls actuation; a light source 410 that indicates the power status of the automated system; a switch 411 that turns the automated system on or off; at least one port 412 that is used for suction; and a catheter release switch or lever 413 that disconnects the catheter from the body and curved portion when the catheter reaches the desired position or site. The actuation unit 408 may also include a sliding mechanism 414.

[0069] The system also includes a user interface 415 and a circuit 416, which is arranged outside the body as an independent unit 417. The independent unit is connected to the body via a cable 418. Optionally, the user interface 415, the circuit 416 and the system are connected by a wireless connection (not shown). A wireless connection can be established by Bluetooth, Wifi, Zigbee, telecommunications, NFC or any other communication mode available when the system is implemented. Wireless communication also enables the device to be remotely controlled as data is transmitted. The processing circuit of the remote connection can also control multiple actuation units in multiple devices at different times, and can also provide centralized control to hospital management and compliance departments. The communication between different units of the system can be protected by implementing technologies such as SSL.

[0070] Figure 5 yes Figure 2An exemplary embodiment of a bending portion 203 configuration is shown, which includes a plurality of independent vertebrae 501 stacked on top of each other and connected by rivets 502. The vertebrae are connected in such a way as to allow each vertebra to rotate partially and / or completely independently around the rivet points. The rotational movement of each vertebra enables the bending portion to bend. The vertebrae are connected to each other via cables 503, one end of which is connected to an actuation unit ( Figure 5 The vertebral body further comprises at least one eyelet 504 arranged on the inner side. The rope from the actuation unit passes through the eyelet and reaches the connection point of the distal vertebral body. Optionally, a grid or a combination of the above-mentioned structure and the grid, or other feasible arrangements known to those skilled in the art, can be used to achieve the same purpose.

[0071] Figure 6 2 is a diagram of an exemplary structure of an automatic intubation system 200, which includes a bending portion 203, an imaging sensor 205, a catheter 301, at least one actuation unit 208, a user interface 215, and a circuit 216. The circuit also includes: a processing circuit 216a, which generates a control signal based on input from at least one imaging sensor; a communication circuit 216b, which provides data / signal communication between different components of the system; and a power supply circuit 216c. The actuation unit includes a sliding mechanism 213 to provide movement of the catheter in the Z plane.

[0072] Depending on the functional requirements of the system, the processing circuit 216a can be a single processor, a logic circuit, a dedicated controller that performs all functions, or a combination of processing auxiliary units. In an exemplary embodiment, the processing circuit includes two independent processing auxiliary units 216aa and 216ab. The processing auxiliary unit 216a is a computer vision software that uses machine learning technology and data received from the imaging sensor 205 to perform at least one function (216aa1, 216aa2...216aaN). These functions include the identification of anatomical structures and the prediction of the expected path for the insertion of the catheter 301 based on the identification of at least one anatomical structure. The processing auxiliary unit and / or the processing circuit interact with the imaging sensor 205 to receive data and perform the aforementioned functions during the intubation process.

[0073] In one embodiment, identifying anatomical structures using imaging sensor data and machine learning techniques includes detecting respiratory structures, such as the tracheal opening, glottis, vocal cords, and / or the bifurcation between the esophagus and the trachea. In addition to or in lieu of detecting respiratory structures, other anatomical parts of the human body may also be detected and / or identified.

[0074] Optionally, the processing circuit 216aa predicts the expected path based on the input from the imaging sensor, the sample historical data remotely received from the actuation unit of multiple devices, and the machine learning model. For management purposes, the system also stores the expected path for maintaining the device operation log in the memory (not shown in the system). The log of the device can be shared with the remote device for monitoring and control purposes. The processing auxiliary unit 216ab generates a control signal based on the expected path predicted by the processing auxiliary unit 216aa. The control signal generated by the processing auxiliary unit 216ab is then transmitted from the processing circuit to the actuation unit 208 via the communication circuit 216b, based on which the actuation unit actuates at least one of the bending portion 203 and the sliding mechanism 214 to provide three-dimensional motion to the invasive medical device. The processing auxiliary unit 216ab can also be an integrated part of the actuation unit 208, and the control signal is received by the actuation unit through a wireless or wired communication circuit. In one case, the processing circuit 216aa is remotely connected to the actuation unit 208 via the Internet or a wireless medium to send a control signal. The communication circuit may also be an integral part of the actuation unit.

[0075] The user interface 215 is in bidirectional communication with the processing circuit 106a. The user interface is preferably a display device to display the data received from the imaging sensor 205 and to display an overlay of the identified anatomical structure and / or the expected path received from the processing circuit to assist the operator. In addition, the overlay of the expected path may also be displayed on the user interface in the form of augmented reality and / or any other form that provides effective visual guidance to the operator.

[0076] The user interface 215 may also be a touch-enabled display device that allows the operator to adjust the expected path displayed thereon. If the operator is not satisfied with the expected path of the cannula, the expected path displayed on the user interface may also be changed by the operator. In addition, it may also have touch buttons related to the functions performed by buttons arranged on the outer surface of the body, such as buttons that trigger manual actuation, catheter release buttons, and / or system power off buttons. Alternatively, the user interface may be any device that enables the operator to interact with the automated system, such as an audio input, an audio output, or a gesture-enabled input, or any other control scheme that may be enabled by an intelligent agent.

[0077] Figure 7is an illustrative flow chart for generating a machine learning model, which includes a step 701 of collecting a large number of intubation procedure videos from existing video laryngoscopes, and a step 702 of separating the collection of intubation procedure videos according to the predicted difficulty level of the intubation procedure. Using a fusion of computer vision models and known machine learning algorithms, the difficulty level can be automatically predicted in the form of a traditional Marampati score or a customized intubation difficulty level. The calculated or predicted difficulty score can be embedded in the metadata of the video so that the video can be easily retrieved and separated based on the calculated score. These videos can be supplemented with videos obtained from other sources, including the devices described herein. There is no limitation on the video source disclosed herein for training videos.

[0078] In step 703, the separated videos are pruned to remove portions of the video containing occlusions and / or unclear views of anatomical structures relevant to the intubation procedure. This step cleans up avoidable noise in the video data before moving on to the process of extensive training of the machine learning model.

[0079] In step 704, the trimmed video file is converted into an image file, and then in step 705, the image file is labeled with the anatomical structure to construct a labeled image dataset. In step 706, the labeled image dataset is used as a training dataset to train one or more neural networks to generate a machine learning model. The generated machine learning model is used in the Figure 6 The processing circuit 216a executed by the processing auxiliary unit 216aa (ie, computer vision software) or as part of the processing auxiliary unit 216aa to identify at least one anatomical structure during the intubation process based on data received from the imaging sensor 205.

[0080] Figure 8 801 is an illustration of the use of a representative automated intubation method, which includes inserting a detachable blade 801 into a patient's airway 802. Adjacent to the detachable blade, a curved portion 803 and a catheter 804 longitudinally arranged on the curved portion are inserted into the patient's airway. The method also includes collecting airway data from at least one imaging sensor 805 arranged on the curved portion. The collected airway data is then transmitted to at least one processing circuit 806, which uses a machine learning model and the airway data to identify at least one anatomical structure and predict at least one expected path for catheter insertion. The processing circuit then uses the expected path to generate a control signal and transmits the control signal to at least one actuation unit 807 to actuate the three-dimensional movement of the catheter.

[0081] Specifically, since the detachable blade, the curved portion and the catheter are directly or indirectly connected to the main body, the detachable blade 801, the curved portion 803 and the catheter are inserted by introducing the main body 808 near the patient's mouth. In addition, the processing circuit 806 and the actuation unit 807 are preferably located in the main body.

[0082] The three-dimensional movement of the catheter 804 arranged on the curved portion 803 includes the bending movement of the catheter in the X plane and the Y plane guided by the two-dimensional movement of the curved portion 803, and the sliding mechanism ( Figure 8 The movement of the catheter in the Z plane is guided by a rope (not shown). Figure 8 The method also includes displaying the data transmitted from the imaging sensor 805 on the user interface 809, and superimposing the identified anatomical structure and the expected path of the inserted catheter on the user interface.

[0083] The position of the distal end of the catheter can be confirmed by standard methods of clinical care, such as, but not limited to, carbon dioxide detection, X-rays, and ultrasound. These methods can be directly incorporated into the device, or combined to provide indirect support for such methods. For example, with respect to carbon dioxide detection, the level of carbon dioxide present in the air can confirm the accurate placement of the catheter in the patient's body. This qualitative or quantitative confirmation can be provided by a sensor (such as a carbon dioxide monitor) placed directly on or in the device, or by a more indirect method (such as a color-changing pH sensitive strip placed in the field of view of the imaging sensor) to provide confirmation of the correct carbon dioxide level. Similarly, an ultrasonic transmitter and receiver can be incorporated into the device, which can confirm the distal end of the catheter is correctly placed. The techniques discussed above are only some of the many clinical methods for confirming the correct placement of the cannula, which are obvious to those skilled in the art.

[0084] Once the desired location or position within the patient's airway is reached, the catheter is released from the body 808 and curved portion 803 using a catheter release switch or lever 810 located on the outer surface of the body. Optionally, a button ( Figure 8 A user interface 809 (not shown) may also be provided on the user interface 809 to release or disconnect the catheter.

[0085] Fig. 9901 is an illustration of using a user interface including a display screen 902 for displaying data received from at least one imaging sensor. The display screen also displays an overlay of at least one identified anatomical structure 903 and an expected path 905 for inserting a catheter 904. The operator may also manually adjust the expected path 905 for inserting the catheter 904 displayed on the user interface. Optionally, the overlay of the catheter, the curved portion, the identified anatomical structure 903, and the expected path 905 for insertion is displayed on the user interface in an augmented reality, virtual reality, or other overlay known to those skilled in the art to provide effective visual guidance to the operator. The overlay of the identified anatomical structure may also include annotations or markings for the operator to quickly identify the structure during the procedure.

[0086] In addition, the display screen 902 of the user interface 901 may include: a pair of up and down touch buttons 906 to manually control actuation and / or control automatic actuation if desired; a system power on / off touch button 907; and a catheter release touch button 908.

[0087] In one embodiment, a pair of up and down touch buttons 906 may be used to selectively control manual actuation in a selected work plane X, Y, or Z. Touch buttons 909 provided on the display screen may be used to select a work plane prior to providing input via touch buttons 906. It should be understood that although in Fig. 9 The touch buttons are depicted in FIG. 1 as being arranged outside the boundaries of the visual data received from the imaging sensor, but the arrangement of the touch buttons may be varied to provide the best possible visual representation to the operator.

[0088] Although the present invention has been explained in the context of assisting surgery, insertion, or implantation, the present invention may also be implemented for educational or academic purposes, such as training and demonstrations.

[0089] No language in the specification should be construed as indicating any non-claimed element as essential to the practice of the invention.

[0090] It is obvious to those skilled in the art that various modifications and variations may be made to the present invention without departing from the spirit and scope of the present invention. It is not intended to limit the present invention to the specific form attached. On the contrary, the present invention is intended to cover all modifications, alternative constructions and equivalents that fall within the spirit and scope defined in the appended claims. Therefore, the present invention is intended to cover the modifications and variations provided by the present invention as long as they are within the scope of the appended claims and their equivalents.

Claims

1. An automatic intubation system, include: main body; a flexible member connected to the body; a curved portion having a variable length and including at least a portion of a flexible member; a distal end of the curved portion; at least one imaging sensor; processing circuitry for predicting at least one expected path of the distal end of the curved portion and generating a control signal, wherein the expected path is predicted based on data received from an imaging sensor; a user interface for displaying at least one intended path to an operator and also allowing the operator to select an intended path; at least one actuation unit for receiving a control signal from the processing circuit to actuate a three-dimensional movement of the distal end of the bending portion along the intended path toward a first position; The processing circuit receives and uses data from at least one imaging sensor to compare actual movement of the distal end of the bending portion with expected movement of the distal end of the bending portion; and If the expected first position and the actual first position do not coincide, the processing circuit then generates an additional control signal to move the distal end of the curved portion to the first position.

2. The automatic intubation system according to claim 1, in, The processing circuitry is capable of predicting at least one new expected path.

3. The automatic intubation system according to claim 1, in, Once the distal end of the curved portion reaches the first position, the processing circuit generates a second position along the expected path.

4. The automatic intubation system according to claim 1, in, The processing circuitry continuously generates new positions of the distal end of the curved portion along the intended path based on data received from at least one imaging sensor.

5. The automatic intubation system according to claim 1, in, The data received from the at least one imaging sensor can be image data or position data.

6. The automatic intubation system according to claim 1, in, The processing circuit compares data received from an imaging sensor of actual movement of the distal end of the bending portion with data of expected movement of the distal end of the bending portion using a machine learning model.

7. The automatic intubation system according to claim 1, in, The processing circuitry can be contained in the device or hosted on a remote server.

8. The automatic intubation system according to claim 1, in, The movement of the distal end of the curved portion along a desired path can be modified by an operator.

9. A method for automatically intubating a patient, include: inserting a blade, a curved portion, and a catheter disposed on the curved portion into an airway of the patient; collecting airway data using at least one imaging sensor disposed on the curved portion; transmitting the collected airway data to a processing circuit; predicting, using the processing circuitry, an expected path for insertion of the catheter and generating a control signal, wherein the expected path is predicted based on at least one anatomical structure identified by the processing circuitry using the collected airway data; displaying the expected paths via a user interface to display at least one expected path to an operator and also allow the operator to select an expected path; transmitting control signals generated by the processing circuit to at least one actuation unit to actuate three-dimensional movement of the catheter; comparing, by the processing circuitry, actual movement of the distal end of the bending portion with expected movement of the distal end of the bending portion using data from at least one imaging sensor; and If the expected first position and the actual first position do not coincide, an additional control signal is generated by the processing circuit to move the distal end of the bending portion to the first position.

10. The automatic intubation method according to claim 9, in, The processing circuitry is capable of predicting at least one new expected path.

11. The automatic intubation method according to claim 9, in, Once the distal end of the curved portion reaches the first position, the processing circuit generates a second position along the expected path.

12. The automatic intubation method according to claim 9, in, The processing circuitry continuously generates new positions of the distal end of the curved portion along the intended path based on data received from at least one imaging sensor.

13. The automatic intubation method according to claim 9, in, The data received from the at least one imaging sensor can be image data or position data.

14. The automatic intubation method according to claim 9, in, The processing circuit compares data received from an imaging sensor of actual movement of the distal end of the bending portion with data of expected movement of the distal end of the bending portion using a machine learning model.

15. The automatic intubation method according to claim 9, in, The processing circuitry can be contained in the device or hosted on a remote server.

16. The automatic intubation method according to claim 9, in, Movement of the distal end of the curved portion along the intended path can be modified by an operator.

Images