Automatic tracheal intubation method, device and system based on artificial intelligence

Abstract

The invention provides an automatic tracheal intubation method, device and system based on artificial intelligence, which are used for introducing an artificial intelligence technology and recognizing and displaying related key parts in real time in a tracheal intubation process on the basis of image recognition, so that on one hand, highly-visual data support is provided for tracheal intubation operation, and the operation efficiency of the tracheal intubation is improved. On the other hand, intelligent full-automatic or semi-automatic tracheal intubation operation is executed through path planning, a good auxiliary effect is achieved, the intubation success rate can be effectively increased, the intubation time can be effectively shortened, and high-quality medical services and good medical experience are guaranteed.

Description

Technical Field

[0001] The present application relates to the field of medical technology, and specifically to an artificial intelligence-based automated endotracheal intubation method, device, and system. Background Art

[0002] In the fields of surgical anesthesia, emergency rescue and critical care, patients may be unable to breathe independently, suffer from reflux aspiration, hypoxemia or hypercapnia, and need to receive endotracheal intubation and mechanical ventilation and artificial respiration treatment. Therefore, endotracheal intubation is important in clinical work.

[0003] As an important emergency rescue measure, endotracheal intubation needs to be performed by doctors with rich experience. The process is that the doctor uses laryngoscope and other endotracheal intubation equipment to expose the glottis and finally insert the endotracheal tube into the trachea through the patient's mouth or nose. The purpose of endotracheal intubation is to establish an artificial airway, perform mechanical ventilation, and ensure the patient's breathing.

[0004] However, the inventors of the present application have found that endotracheal intubation is a clinical operation that requires extensive knowledge preparation and repeated practice. Intubation time for inexperienced doctors is longer and the success rate is significantly lower. In addition, in actual situations, there may be difficult airways caused by various physiological and pathological reasons, which can also hinder endotracheal intubation, resulting in prolonged intubation time and reduced success rate. Summary of the invention

[0005] The present application provides an artificial intelligence-based automated endotracheal intubation method, device and system, which are used to introduce artificial intelligence (AI) technology, based on image recognition, to identify and display the key parts involved in real time during the endotracheal intubation process, so as to provide highly visualized data support for the endotracheal intubation operation on the one hand, and to perform intelligent fully automatic or semi-automatic endotracheal intubation operations through path planning on the other hand, which plays a better auxiliary role, thereby effectively improving the intubation success rate and shortening the intubation time, and ensuring high-quality medical services and a good medical experience.

[0006] In a first aspect, the present application provides an automated tracheal intubation method based on artificial intelligence, the method comprising:

[0007] During the endotracheal intubation process, image data collected by the probe is obtained, wherein the probe is configured with at least one camera, and the image data is at least one of picture data and video data;

[0008] Input the image data into a pre-configured image recognition model to carry out corresponding key parts recognition processing, wherein the image recognition model is a deep learning model, which is obtained by training the initial model with labeled sample image data, and is used to recognize the key parts contained in the image data input into the model, and the key parts include lips, teeth, hard palate, soft palate, uvula, tongue, tonsils, epiglottis, vocal cords, glottis and trachea;

[0009] While displaying the image data, the key part recognition results output by the image recognition model are also displayed;

[0010] Based on the key part identification results, fully automatic or semi-automatic tracheal intubation operation is performed through the tracheal intubation mechanical auxiliary device.

[0011] In combination with the first aspect of the present application, in a first possible implementation manner of the first aspect of the present application, the method further includes:

[0012] Remote consultation is carried out based on image data and key part recognition results.

[0013] In combination with the first aspect of the present application, in a second possible implementation manner of the first aspect of the present application, the method further includes:

[0014] Remote operation is performed based on image data and key part recognition results.

[0015] In combination with the first aspect of the present application, in a third possible implementation manner of the first aspect of the present application, the image recognition model is further used for:

[0016] Based on the key part identification results, tracheal intubation path planning is performed;

[0017] While displaying the image data, the key part recognition results output by the image recognition model are also displayed, including:

[0018] While displaying the image data and the key part recognition results output by the image recognition model, the tracheal intubation path planning results are also displayed.

[0019] In combination with the third possible implementation of the first aspect of the present application, in a fourth possible implementation of the first aspect of the present application, based on the key part recognition result, tracheal intubation path planning processing is performed, including:

[0020] Based on the key part recognition results and historical key part recognition results, the subsequent key part conditions in the subsequent tracheal intubation process are predicted;

[0021] Based on the key part identification results and subsequent key part situation prediction results, tracheal intubation path planning is performed.

[0022] In combination with the third possible implementation of the first aspect of the present application, in a fifth possible implementation of the first aspect of the present application, based on the key part recognition result, tracheal intubation path planning processing is performed, including:

[0023] Based on the image data and key part recognition results, tracheal intubation path planning is performed.

[0024] In combination with the fifth possible implementation of the first aspect of the present application, in a sixth possible implementation of the first aspect of the present application, the method further includes:

[0025] Based on the image data, the path characteristics of the historical path of the endotracheal intubation are analyzed;

[0026] Based on the image data and key part recognition results, tracheal intubation path planning is performed, including:

[0027] Based on the image data, key part recognition results and path characteristics of the historical path of the tracheal intubation, the tracheal intubation path planning process is performed.

[0028] In a second aspect, the present application provides an automated tracheal intubation device based on artificial intelligence, the device comprising:

[0029] An acquisition unit, used to acquire image data collected by the probe during the tracheal intubation process, wherein the probe is configured with at least one camera, and the image data is at least one of picture data and video data;

[0030] A recognition unit, used for inputting image data into a pre-configured image recognition model to carry out corresponding key parts recognition processing, wherein the image recognition model is a deep learning model, the image recognition model is obtained by training an initial model with labeled sample image data, and the image recognition model is used for recognizing key parts contained in the image data input into the model, and the key parts include lips, teeth, hard palate, soft palate, uvula, tongue, tonsils, epiglottis, vocal cords, glottis and trachea;

[0031] A display unit, used to display the image data and the key part recognition results output by the image recognition model;

[0032] The operating unit is used to perform fully automatic or semi-automatic tracheal intubation operation based on the key part identification result through the tracheal intubation mechanical auxiliary device.

[0033] In conjunction with the second aspect of the present application, in a first possible implementation manner of the second aspect of the present application, the device further includes:

[0034] The remote consultation unit is used to perform remote operations based on image data and key part recognition results.

[0035] In conjunction with the second aspect of the present application, in a second possible implementation manner of the second aspect of the present application, the device further includes:

[0036] The remote operation unit is used to perform remote operation based on image data and key part recognition results.

[0037] In conjunction with the second aspect of the present application, in a third possible implementation of the second aspect of the present application, the image recognition model is further used for:

[0038] Based on the key part identification results, tracheal intubation path planning is performed;

[0039] While displaying the image data, the key part recognition results output by the image recognition model are also displayed, including:

[0040] While displaying the image data and the key part recognition results output by the image recognition model, the tracheal intubation path planning results are also displayed.

[0041] In combination with the third possible implementation of the second aspect of the present application, in a fourth possible implementation of the second aspect of the present application, the image recognition model is specifically used for:

[0042] Based on the key part recognition results and historical key part recognition results, the subsequent key part conditions in the subsequent tracheal intubation process are predicted;

[0043] Based on the key part identification results and subsequent key part situation prediction results, tracheal intubation path planning is performed.

[0044] In combination with the third possible implementation of the second aspect of the present application, in a fifth possible implementation of the second aspect of the present application, the image recognition model is specifically used for:

[0045] Based on the image data and key part recognition results, tracheal intubation path planning is performed.

[0046] In combination with the fifth possible implementation of the second aspect of the present application, in a sixth possible implementation of the second aspect of the present application, the image recognition model is further used for:

[0047] Based on the image data, the path characteristics of the historical path of the endotracheal intubation are analyzed;

[0048] Image recognition models, specifically for:

[0049] Based on the image data, key part recognition results and path characteristics of the historical path of the tracheal intubation, the tracheal intubation path planning process is performed.

[0050] In a third aspect, the present application provides an artificial intelligence-based automated tracheal intubation system for executing the method provided in the first aspect of the present application or any possible implementation of the first aspect of the present application.

[0051] In a fourth aspect, the present application provides a computer-readable storage medium, which stores multiple instructions, and the instructions are suitable for a processor to load to execute the method provided by the first aspect of the present application or any possible implementation of the first aspect of the present application.

[0052] From the above content, it can be concluded that the present application has the following beneficial effects:

[0053] Aiming at the goal of high-quality auxiliary assistance for tracheal intubation, this application introduces artificial intelligence technology, based on image recognition, to identify and display the key parts involved in the tracheal intubation process in real time. On the one hand, it provides highly visual data support for the tracheal intubation operation, and on the other hand, it performs intelligent fully automatic or semi-automatic tracheal intubation operations through path planning, which plays a better auxiliary role, thereby effectively improving the intubation success rate and shortening the intubation time, ensuring high-quality medical services and a good medical experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0055] Figure 1 A schematic diagram of a process flow of an automated tracheal intubation method based on artificial intelligence in this application;

[0056] Figure 2 A schematic diagram of the structure of an automated tracheal intubation device based on artificial intelligence in this application;

[0057] Figure 3 This is a structural schematic diagram of the equipment involved in the artificial intelligence-based automated tracheal intubation system of this application. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0059] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described here can be implemented in an order other than that illustrated or described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices. The naming or numbering of steps in this application does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The process steps that have been named or numbered can change the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved.

[0060] The division of modules in this application is a logical division. There may be other division methods when it is implemented in actual applications. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection between modules can be electrical or other similar forms, which are not limited in this application. In addition, the modules or submodules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed in multiple circuit modules, and some or all of the modules may be selected according to actual needs to achieve the purpose of the present application.

[0061] Before introducing the artificial intelligence-based automated tracheal intubation method provided by the present application, the background content involved in the present application is first introduced.

[0062] The artificial intelligence-based automated endotracheal intubation method, device and computer-readable storage medium provided in the present application can be applied to an artificial intelligence-based automated endotracheal intubation system, which is used to introduce artificial intelligence technology, based on image recognition, to identify and display the key parts involved in the endotracheal intubation process in real time, thereby providing highly visualized data support for the endotracheal intubation operation on the one hand, and performing intelligent fully automatic or semi-automatic endotracheal intubation operations through path planning on the other hand, thereby playing a better auxiliary role, thereby effectively improving the intubation success rate and shortening the intubation time, and ensuring high-quality medical services and a good medical experience.

[0063] The artificial intelligence-based automated tracheal intubation method mentioned in the present application may be executed by an artificial intelligence-based automated tracheal intubation device, or an artificial intelligence-based automated tracheal intubation system integrated with the artificial intelligence-based automated tracheal intubation device, wherein the artificial intelligence-based automated tracheal intubation device may be implemented in hardware or software, and the artificial intelligence-based automated tracheal intubation system includes, in addition to a processing device responsible for data processing, related tracheal intubation devices for specifically performing tracheal intubation operations and / or a display device responsible for content display functions, wherein the processing device may be different types of devices such as a physical host, a server, or even a user equipment (UE), and the UE may specifically be a terminal device such as a smart phone, a tablet computer, a laptop computer, a desktop computer, or a personal digital assistant (PDA), and the processing device may also be configured in the form of a device cluster.

[0064] Next, we will begin to introduce the artificial intelligence-based automated tracheal intubation method provided by this application.

[0065] First, see Figure 1 , Figure 1 A flow chart of an automated tracheal intubation method based on artificial intelligence in the present application is shown. The automated tracheal intubation method based on artificial intelligence provided in the present application may specifically include the following steps S101 to S104:

[0066] Step S101, during the endotracheal intubation process, image data collected by the probe is obtained, wherein the probe is configured with at least one camera, and the image data is at least one of picture data and video data;

[0067] It can be understood that for the clinical technology of endotracheal intubation, a corresponding miniature camera is configured at the front end of the endotracheal tube, so that corresponding pictures / videos can be collected and transmitted to the display device for real-time viewing by the doctor operating the endotracheal intubation, so as to manually control the movement of the endotracheal tube of the endotracheal intubation equipment in the patient's body, thereby assisting in the completion of the endotracheal intubation operation. Considering that these contents belong to the basic / common contents of the endotracheal intubation equipment, this application will not elaborate on them.

[0068] The number of cameras may be one or more. The number of cameras may not only take into account the image acquisition requirements at different positions at the front end of the endotracheal tube, but also the data usage requirements of the 2D / 3D imaging solution.

[0069] Furthermore, in specific operations, the acquisition and processing of image data here can be either real-time control of the tracheal intubation equipment to collect and process image data, or external reception of image data collected by the tracheal intubation equipment. This can be adjusted according to the specific equipment deployment plan of the artificial intelligence-based automated tracheal intubation system that implements the solution of the present application (involving whether the system directly includes the tracheal intubation equipment).

[0070] Similarly, the image data specifically acquired is picture data, video data, or a combination of the two, which can also be adjusted according to actual needs.

[0071] Step S102, inputting the image data into a pre-configured image recognition model to carry out corresponding key parts recognition processing, wherein the image recognition model is a deep learning model, the image recognition model is obtained by training an initial model with labeled sample image data, and the image recognition model is used to recognize the key parts contained in the image data input into the model, and the key parts include lips, teeth, hard palate, soft palate, uvula, tongue, tonsils, epiglottis, vocal cords, glottis and trachea;

[0072] It can be understood that, for the image recognition setting introduced in this application, an image recognition model is pre-configured through model training, which can also be called a tracheal intubation key part recognition model, which is obtained by training an initial model with pre-labeled sample image data.

[0073] Specifically, this application involves model training in advance:

[0074] Configure sample image data and annotate it;

[0075] The initial model is trained based on the labeled sample image data to obtain the image recognition model.

[0076] Among them, the sample image data (i.e., training samples) used to train the model corresponds to the model input after the model is trained and put into actual use, which is picture data, video data, or a combination of the two.

[0077] In the process of labeling sample image data, manual labeling can be performed manually, or automatic labeling can be performed by an automatic labeling tool configured with corresponding automatic labeling strategies / rules. Both are possible in actual situations.

[0078] Specific model training can be:

[0079] In each round of model training, a sample image data is input into the model so that the model can recognize the key parts contained therein and realize forward propagation. Then, based on the key part recognition results output by the model, the loss function is calculated in combination with the annotation results (i.e. the theoretical key part recognition results). The model parameters are optimized according to the loss function calculation results to realize back propagation. In this way, when the preset model training requirements such as training time, training times or recognition accuracy are met, the model training can be completed, or the image recognition model can be put into actual use.

[0080] Among them, it is understandable that for the model architecture, model training framework and loss function used in the model training process, we can adopt the existing solutions, make further optimization and improvement on the basis of the existing solutions, or adopt novel self-developed solutions. These are all possible in actual situations.

[0081] As for the key parts that may be involved in the tracheal intubation process targeted by the present application, they may specifically include lips, teeth, hard palate, soft palate, uvula, tongue, tonsils, epiglottis, vocal cords, glottis and trachea. These key parts are the specific parts that the inventors of the present application explored and determined when configuring the present application scheme, considering how to better assist the tracheal intubation operation by prompting the key parts, and have better practical value.

[0082] In addition, the key part recognition effect that can be achieved by the image recognition model, that is, the key part recognition result that can be output by the image recognition model, in addition to identifying the location of the key part in the picture, can also further involve the distance / relative position between the key part and the front end of the tracheal tube / camera, which is a relatively mature technology in 2D / 3D image recognition solutions.

[0083] In this way, when the image data that needs to be identified for the current tracheal intubation task is obtained, the corresponding key part identification processing can be performed through the pre-configured image recognition model. After the image recognition model completes the processing, it will obviously output the corresponding key part identification results.

[0084] Step S103, while displaying the image data, display the key part recognition result output by the image recognition model;

[0085] It can be understood that after obtaining the key part recognition results output by the image recognition model after completing the key part recognition processing, in addition to the real-time display of the image data previously acquired, the key part recognition results can also be displayed in real time, in a visual way, to the doctor in real time and intuitively display the current situation of the patient's body including key parts, so that the doctor can judge and make decisions on the patient's internal condition, tracheal intubation situation and subsequent treatment, which plays a good auxiliary role, reduces the dependence of tracheal intubation operation on the doctor's operating experience, and helps to promote the next step of tracheal intubation operation more stably, efficiently and accurately.

[0086] During the display process, the image data and the key part recognition results can be displayed separately, for example, in two different display areas, or on different display devices, or the image data and the key part recognition results can be displayed together, that is, the display content is superimposed together, for example, in the picture involved in the image data, the image area where the key part corresponding to the key part recognition result is located is continued to be identified, typically such as identifying the corresponding red detection box. In this way, the display strategy adopted for specific display can be flexibly adjusted according to pre-settings and real-time settings.

[0087] Among them, it can be understood that the display device responsible for the display function can be either a dedicated display device or a device including a display screen. In addition, it can be either an external device or a device included in the artificial intelligence-based automated tracheal intubation system of the present application, which is more flexible.

[0088] Step S104, based on the key part identification result, a fully automatic tracheal intubation operation or a semi-automatic tracheal intubation operation is performed through a tracheal intubation mechanical auxiliary device.

[0089] In addition, in terms of details, the image data and key part recognition results obtained by the present application scheme can not only provide doctors with visual auxiliary data, but also provide corresponding data support for automated tracheal intubation processing.

[0090] Correspondingly, the artificial intelligence-based automated tracheal intubation system of the present application can also perform fully automatic or semi-automatic tracheal intubation operations based on the key part recognition results through a tracheal intubation mechanical auxiliary device.

[0091] Among them, it can be understood that, with different degrees of automation, the tracheal intubation operations involved in the automated tracheal intubation processing that can be achieved by the tracheal intubation equipment in actual situations can also be divided into two types: fully automatic tracheal intubation operations and semi-automatic tracheal intubation operations.

[0092] It is easy to understand that the fully automatic tracheal intubation operation means that under normal circumstances, there is no need for a doctor to control the operation. The tracheal intubation device will autonomously perform tracheal intubation based on the planned tracheal intubation path. If an abnormal situation occurs, the doctor can intervene, such as pausing / exiting the autonomous processing of the tracheal intubation device.

[0093] Of course, how to define / quantify the degree of automation in detail to achieve the division into fully automatic intubation operation and semi-automatic intubation operation can be adjusted according to actual conditions. What remains unchanged is that most or even all of the fully automatic intubation operations are completed autonomously by the intubation equipment, while the semi-automatic intubation operations must be controlled by the doctor by entering the corresponding work instructions under normal circumstances.

[0094] As for the planning of the tracheal intubation path involved, it can be understood that in specific applications, it can be determined by manual planning or obtained by autonomous planning by the system. This is possible in actual situations and can be configured according to actual needs.

[0095] from Figure 1 It can be seen from the illustrated embodiments that, aiming at the high-quality auxiliary goal of tracheal intubation, the present application introduces artificial intelligence technology, based on image recognition, to identify and display the key parts involved in the tracheal intubation process in real time, thereby providing highly visualized data support for the tracheal intubation operation on the one hand, and performing intelligent fully automatic or semi-automatic tracheal intubation operations through path planning on the other hand, which plays a better auxiliary role, thereby effectively improving the intubation success rate and shortening the intubation time, and ensuring high-quality medical services and a good medical experience.

[0096] Continue to the above Figure 1 Each step of the illustrated embodiment and possible implementation methods thereof in practical applications are described in detail.

[0097] It can be understood that in addition to providing data assistance at the tracheal intubation site, the present application scheme can also remotely combine image data and key part recognition results output by the image recognition model to provide remote guidance on site to meet the application needs of remote consultation and remote operation. In addition, it can also meet the application needs of remote online teaching.

[0098] In this regard, as an exemplary embodiment, after step S102, the method of the present application may further include:

[0099] Remote consultation is carried out based on image data and key part recognition results.

[0100] Alternatively, remote operation can be performed based on image data and key part recognition results.

[0101] Among them, remote consultation is aimed at complex situations. It is understandable that one or more doctors with rich experience can understand the on-site situation online and provide remote guidance through consultation.

[0102] Remote operation involves remote control of on-site endotracheal intubation equipment. It can be understood that the endotracheal intubation equipment is configured to receive corresponding work instructions to perform corresponding endotracheal intubation operations. In this regard, these work instructions can be entered by on-site doctors or obtained remotely. In addition, remote operation can also be combined with remote consultation and used together, so as to adapt to the flexible and changeable application needs in actual situations.

[0103] As an example, remote consultation and remote operation can be achieved based on the high-speed, low-latency 5G communication network.

[0104] At the same time, regardless of whether the automated tracheal intubation operation is introduced, considering the safety of the tracheal intubation operation itself, the artificial intelligence-based automated tracheal intubation system of the present application can also be configured with corresponding multiple safety measures / settings.

[0105] Specifically, the system may have a variety of safety sensors, and at least one of infrared, temperature and pressure sensors may be configured in the part that may contact the patient. Once the sensor value or the value obtained by the sensor value analysis exceeds the corresponding threshold or an abnormality occurs, the operation will be stopped immediately, and a sound alarm signal and a light alarm signal will be issued to avoid accidental injury to the patient's oropharyngeal tissue; the system may have control and interactive equipment, and when the doctor determines that an error has occurred in the system, he can manually control and take over the system operation at any time to correct the system operation; the system may also include an emergency switch, which can immediately pause or terminate the tracheal intubation operation process in an emergency to ensure patient safety.

[0106] In addition, in addition to providing auxiliary data references for key parts in real time, the present application may also involve auxiliary data references for recommended paths, that is, it may involve the planning of tracheal intubation paths. It is easy to understand that this data processing has its complexity and can be executed by an image recognition model with powerful data processing capabilities.

[0107] In this case, the work content of the image recognition model may also involve tracheal intubation path planning and processing. Correspondingly, the image recognition model of the present application has a corresponding model design. For example, a model fusion solution can be adopted. The overall image recognition model includes a first model responsible for key part recognition and processing and a second model responsible for tracheal intubation path planning and processing. The model input of the second model is connected to the model output of the first model. For example, one model can be directly used to implement different processing functions. These are all possible in actual situations.

[0108] In this regard, as an exemplary embodiment, the image recognition model of the present application can also be used for:

[0109] Based on the key part identification results, tracheal intubation path planning is performed.

[0110] It can be understood that the key part recognition result corresponds to the key part in the current front field of view of the patient's internal endotracheal tube. In this way, combined with the current location of the endotracheal tube (involving the determination method based on the penetration length of the tube, the key part that has appeared based on the previous image recognition, the distance between the key part determined by image recognition, the relative position between the key part determined by image recognition, etc.), it is possible to plan how the endotracheal tube can better promote further endotracheal intubation operations according to the theoretical path.

[0111] Among them, the so-called theoretical path can also be understood as a standard path. In view of the differences in the internal conditions of different patients, the adapted standard path can be slightly adjusted, or there are different standard paths that have been slightly adjusted to adapt to different actual conditions.

[0112] Furthermore, in terms of details, on the basis of the standard path, further fine-tuning can be performed by considering the conditions of the identified key parts.

[0113] It is understandable that patients who need endotracheal intubation may have abnormal conditions such as edema, injury, foreign matter or secretions at the position where the endotracheal tube can pass or at the key position. At this time, the standard path under normal circumstances may not be suitable for the current situation. In this regard, we can fully consider how to better inspect, treat or use the endotracheal tube in combination with the situation of the key parts to achieve a more suitable and appropriate endotracheal intubation path planning, ensure the quality of endotracheal intubation, and ensure the health / safety of the patient.

[0114] In this case, while displaying the image data mentioned above, displaying the key part recognition results output by the image recognition model may specifically include:

[0115] While displaying the image data and the key part recognition results output by the image recognition model, the tracheal intubation path planning results are also displayed.

[0116] It can be understood that while providing auxiliary data display functions for both image data and key part identification results, an auxiliary data display function for tracheal intubation path planning results can also be provided. In this way, for doctors with little operating experience or poor condition, it can obviously provide more direct and effective data references, helping doctors to perform higher quality tracheal intubation operations in actual situations. At the same time, this can also provide input data support for the automated tracheal intubation processing mentioned above.

[0117] As for the specific display and processing of the tracheal intubation path planning results, please refer to the previous introduction of the first two, so no specific expansion description will be given here.

[0118] Furthermore, for the tracheal intubation path planning process, the present application can also introduce a prediction mechanism for the patient's internal conditions, which can help achieve a more adaptive or smoother tracheal intubation effect in some special cases.

[0119] Specifically, as an exemplary embodiment, the tracheal intubation path planning process based on the key part recognition result may include:

[0120] Based on the key part recognition results and historical key part recognition results, the subsequent key part conditions in the subsequent tracheal intubation process are predicted;

[0121] Based on the key part identification results and subsequent key part situation prediction results, tracheal intubation path planning is performed.

[0122] In this embodiment, it can be understood that for patients who need tracheal intubation, in some clinical cases, corresponding abnormalities will occur successively in different key parts. If such abnormalities in key parts with a causal relationship can be considered / captured, compared to only focusing on the currently identified key parts to plan the tracheal intubation path, a path planning effect that is more suitable for the actual situation can be made in advance. At the same time, this also plays a role in disease diagnosis to a certain extent, and can directly or indirectly indicate the patient's specific condition, which is also helpful for the development of clinical work.

[0123] In this regard, the current key part recognition results corresponding to the current tracheal intubation task and the corresponding historical key part recognition results in the past can be obtained to predict the subsequent key parts (i.e., the key parts that the tracheal tube has not yet reached and has not been checked) for use in the subsequent tracheal intubation path planning and processing.

[0124] It is worth adding that, for the same patient, the historical key part recognition results obtained here can be not only the historical key part recognition results corresponding to the past in the current tracheal intubation task (that is, the key part recognition results that have been processed in the current tracheal intubation task), but also the key part recognition results of the previous tracheal intubation tasks. It is understandable that for the same patient, factors such as his own physical characteristics, patient working environment or patient living habits may also lead to the need for tracheal intubation, resulting in the internal conditions of the same patient in different tracheal intubation tasks / processing. There may be commonalities. In this regard, this factor can be considered to achieve better and customized tracheal intubation path planning effects.

[0125] In addition, during the tracheal intubation path planning process, the present application scheme may also consider the image data acquired previously. It can be understood that the image data involves many pictures other than key parts of the tracheal tube in the process of entering the patient's body and finally reaching the trachea, and at a subtle level, there may be image features that are helpful for tracheal intubation path planning.

[0126] In this regard, as an exemplary embodiment, the tracheal intubation path planning process based on the key part recognition result may also include:

[0127] Based on the image data and key part recognition results, tracheal intubation path planning is performed.

[0128] It can be understood that the embodiment here is considered from the data processing level of image recognition, and comprehensively considers the image features that may be involved. This will also help improve the planning effect of the tracheal intubation path in practical applications.

[0129] In addition, based on the image data, we can also consider the path of the tracheal tube in the process of entering the patient's body during this tracheal intubation task. It is understandable that whether it is manually operated by the doctor or autonomously operated by the tracheal intubation equipment, the tracheal tube may also be affected by some factors / conditions in reality, resulting in movement in the patient's body. There are some deviations, or there are tendencies in certain aspects. Of course, this is a relatively rare situation in actual situations, but if this kind of situation can be captured, it will help to achieve better tracheal intubation path planning effects.

[0130] In this regard, as an exemplary embodiment, the method of the present application may further include:

[0131] Based on the image data, the path characteristics of the historical path of the endotracheal intubation are analyzed;

[0132] It can be understood that based on the dynamic / full-process image data, the path characteristics of the historical path of the tracheal intubation in this tracheal intubation task can be analyzed according to the corresponding characteristic indicators, such as deviation amplitude, deviation frequency, etc., to form a set of characteristic data that briefly characterizes the path situation.

[0133] Based on the image data and key part recognition results, tracheal intubation path planning is performed, including:

[0134] Based on the image data, key part recognition results and path characteristics of the historical path of the tracheal intubation, the tracheal intubation path planning process is performed.

[0135] With the introduction of path feature analysis and processing, it is obvious that the image data, key part recognition results and path characteristics of the historical path of tracheal intubation can be comprehensively considered to plan the tracheal intubation path more delicately, overcome the combined influence of many factors in the details under actual conditions, and promote higher quality tracheal intubation processing.

[0136] Furthermore, the path characteristics of the historical tracheal intubation path here can also be extended to different previous tracheal intubation tasks (which may involve different patients), which means that they need to be obtained from different image data analysis. This is more helpful to capture the impact of objective factors reflected in the actual path, and to obtain a more suitable tracheal intubation path planning effect.

[0137] In addition, as a supplement to the details, when the tracheal intubation path planning process is performed and the tracheal intubation path planning result is used, it is also possible to make subtle corrections in the details during the real-time / dynamic tracheal intubation process based on the deviation between the real-time control amount and the tracheal intubation path planning result, that is, it involves the adjustment and control of the movement amount, wherein the correction amplitude / adjustment amplitude can be positively correlated with the deviation, and the smaller the deviation, the smaller the adjustment.

[0138] In this way, it can be ensured that the actual movement of the endotracheal tube is as close as possible to the endotracheal intubation path planning result, and further ensured that the endotracheal intubation path planning result can be stably implemented.

[0139] The above is an introduction to the automated tracheal intubation method based on artificial intelligence provided in the present application. In order to facilitate better implementation of the automated tracheal intubation method based on artificial intelligence provided in the present application, the present application also provides an automated tracheal intubation device based on artificial intelligence from the perspective of functional modules.

[0140] See also Figure 2 , Figure 2This is a structural diagram of an automated tracheal intubation device based on artificial intelligence in this application. In this application, the automated tracheal intubation device 200 based on artificial intelligence may specifically include the following structure:

[0141] An acquisition unit 201 is used to acquire image data collected by the probe during the tracheal intubation process, wherein the probe is configured with at least one camera, and the image data is at least one of picture data and video data;

[0142] The recognition unit 202 is used to input the image data into a pre-configured image recognition model to carry out corresponding key parts recognition processing, wherein the image recognition model is a deep learning model, the image recognition model is obtained by training the initial model with labeled sample image data, and the image recognition model is used to recognize the key parts contained in the image data input by the model, and the key parts include lips, teeth, hard palate, soft palate, uvula, tongue, tonsils, epiglottis, vocal cords, glottis and trachea;

[0143] The display unit 203 is used to display the key part recognition results output by the image recognition model while displaying the image data; the operation unit 204 is used to perform fully automatic tracheal intubation operation or semi-automatic tracheal intubation operation based on the key part recognition results through a tracheal intubation mechanical auxiliary device.

[0144] In an exemplary embodiment, the apparatus further comprises:

[0145] The remote consultation unit 205 is used to perform remote consultation based on the image data and the key part recognition results.

[0146] In yet another exemplary embodiment, the apparatus further comprises:

[0147] The remote operation unit 206 is used to perform remote operation based on the image data and the key part recognition result.

[0148] In yet another exemplary embodiment, the image recognition model is further used to:

[0149] Based on the key part identification results, tracheal intubation path planning is performed;

[0150] While displaying the image data, the key part recognition results output by the image recognition model are also displayed, including:

[0151] While displaying the image data and the key part recognition results output by the image recognition model, the tracheal intubation path planning results are also displayed.

[0152] In yet another exemplary embodiment, the image recognition model is specifically used to:

[0153] The subsequent critical parts of the intubation process;

[0154] Based on the key part identification results and subsequent key part situation prediction results, tracheal intubation path planning is performed.

[0155] In yet another exemplary embodiment, the image recognition model is specifically used to:

[0156] Based on the image data and key part recognition results, tracheal intubation path planning is performed.

[0157] In yet another exemplary embodiment, the image recognition model is further used to:

[0158] Based on the image data, the path characteristics of the historical path of the endotracheal intubation are analyzed;

[0159] Image recognition models, specifically for:

[0160] Based on the image data, key part recognition results and path characteristics of the historical path of the tracheal intubation, the tracheal intubation path planning process is performed.

[0161] The present application also provides an artificial intelligence-based automated tracheal intubation system from the perspective of hardware structure, which may involve one device or multiple devices. For the convenience of description, they may be collectively referred to as devices involved in the artificial intelligence-based automated tracheal intubation system, see Figure 3 , Figure 3 A schematic diagram of the structure of the device involved in the automatic tracheal intubation system based on artificial intelligence of the present application is shown. Specifically, the device involved in the automatic tracheal intubation system based on artificial intelligence of the present application may include a processor 301, a memory 302 and an input and output device 303. The processor 301 is used to execute the computer program stored in the memory 302 to implement the following Figure 1 The steps of the automated tracheal intubation method based on artificial intelligence in the corresponding embodiment; or, the processor 301 is used to execute the computer program stored in the memory 302 to implement the following Figure 2 Corresponding to the functions of each unit in the embodiment, the memory 302 is used to store the processor 301 executing the above Figure 1 The computer program required for the automated tracheal intubation method based on artificial intelligence in the corresponding embodiment.

[0162] Exemplarily, the computer program may be divided into one or more modules / units, one or more modules / units are stored in the memory 302, and executed by the processor 301 to complete the present application. One or more modules / units may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the computer device.

[0163] The devices involved in the automated tracheal intubation system based on artificial intelligence may include, but are not limited to, a processor 301, a memory 302, and an input-output device 303. Those skilled in the art will appreciate that the illustration is merely an example of the devices involved in the automated tracheal intubation system based on artificial intelligence, and does not constitute a limitation on the devices involved in the automated tracheal intubation system based on artificial intelligence, and may include more or fewer components than shown in the illustration, or a combination of certain components, or different components, for example, the devices involved in the automated tracheal intubation system based on artificial intelligence may also include network access devices, buses, etc., and the processor 301, the memory 302, the input-output device 303, etc. are connected via a bus.

[0164] The processor 301 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the equipment involved in the automated endotracheal intubation system based on artificial intelligence, and uses various interfaces and lines to connect various parts of the entire device.

[0165] The memory 302 can be used to store computer programs and / or modules. The processor 301 implements various functions of the computer device by running or executing the computer programs and / or modules stored in the memory 302 and calling the data stored in the memory 302. The memory 302 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the equipment involved in the artificial intelligence-based automated tracheal intubation system, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0166] When the processor 301 is used to execute the computer program stored in the memory 302, the following functions can be implemented:

[0167] During the endotracheal intubation process, image data collected by the probe is obtained, wherein the probe is configured with at least one camera, and the image data is at least one of picture data and video data;

[0168] Input the image data into a pre-configured image recognition model to carry out corresponding key parts recognition processing, wherein the image recognition model is a deep learning model, which is obtained by training the initial model with labeled sample image data, and is used to recognize the key parts contained in the image data input into the model, and the key parts include lips, teeth, hard palate, soft palate, uvula, tongue, tonsils, epiglottis, vocal cords, glottis and trachea;

[0169] While displaying the image data, the key part recognition results output by the image recognition model are also displayed;

[0170] Based on the key part identification results, fully automatic or semi-automatic tracheal intubation operation is performed through the tracheal intubation mechanical auxiliary device.

[0171] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the above-described artificial intelligence-based automated tracheal intubation device, artificial intelligence-based automated tracheal intubation system and its corresponding units can refer to the following. Figure 1 The description of the automated tracheal intubation method based on artificial intelligence in the corresponding embodiment will not be repeated here.

[0172] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0173] To this end, the present application provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the present application as follows: Figure 1 The steps of the automated tracheal intubation method based on artificial intelligence in the corresponding embodiment, the specific operation can refer to Figure 1 The description of the automated tracheal intubation method based on artificial intelligence in the corresponding embodiment will not be repeated here.

[0174] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0175] Due to the instructions stored in the computer-readable storage medium, the present application can be executed. Figure 1 The steps of the automated tracheal intubation method based on artificial intelligence in the corresponding embodiment, therefore, the present application can be implemented as follows Figure 1 The beneficial effects that can be achieved by the automated tracheal intubation method based on artificial intelligence in the corresponding embodiment are detailed in the previous description and will not be repeated here.

[0176] The above is a detailed introduction to the artificial intelligence-based automated tracheal intubation method, device, artificial intelligence-based automated tracheal intubation system and computer-readable storage medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An automated tracheal intubation method based on artificial intelligence, characterized in that: The method comprises: During the endotracheal intubation process, image data collected by the probe is obtained, wherein the probe is configured with at least one camera, and the image data is at least one of picture data and video data; Input the image data into a pre-configured image recognition model to carry out corresponding key parts recognition processing, wherein the image recognition model is a deep learning model, the image recognition model is obtained by training an initial model with labeled sample image data, and the image recognition model is used to recognize the key parts contained in the image data input into the model, and the key parts include lips, teeth, hard palate, soft palate, uvula, tongue, tonsils, epiglottis, vocal cords, glottis and trachea; While displaying the image data, display the key part recognition results output by the image recognition model; Based on the key part identification result, a fully automatic tracheal intubation operation or a semi-automatic tracheal intubation operation is performed through a tracheal intubation mechanical auxiliary device.

2. The method according to claim 1, characterized in that The method further comprises: A remote consultation is performed based on the image data and the key part recognition results.

3. The method according to claim 1, characterized in that The method further comprises: Remote operation is performed based on the image data and the key part recognition result.

4. The method according to claim 1, characterized in that: The image recognition model is also used to: Based on the key part identification result, tracheal intubation path planning processing is performed; The display of the image data and the key part recognition result output by the image recognition model include: While displaying the image data and the key part recognition results output by the image recognition model, the endotracheal intubation path planning results are also displayed.

5. The method according to claim 4, characterized in that The tracheal intubation path planning process based on the key part identification result includes: Based on the key part recognition result and the historical key part recognition results, predict the subsequent key part conditions in the subsequent tracheal intubation process; The tracheal intubation path planning process is performed based on the key part identification result and the subsequent key part situation prediction result.

6. The method according to claim 4, characterized in that The tracheal intubation path planning process based on the key part identification result includes: The endotracheal intubation path planning process is performed based on the image data and the key part recognition result.

7. The method according to claim 6, characterized in that The method further comprises: Analyzing path characteristics of the historical path of the endotracheal tube based on the image data; The endotracheal intubation path planning process based on the image data and the key part recognition result includes: The tracheal intubation path planning process is performed based on the image data, the key part recognition result and the path characteristics of the tracheal intubation historical path.

8. An automated endotracheal intubation device based on artificial intelligence, characterized in that: The device comprises: An acquisition unit, used to acquire image data collected by the probe during tracheal intubation, wherein the probe is configured with at least one camera, and the image data is at least one of picture data and video data; A recognition unit, used for inputting the image data into a pre-configured image recognition model to carry out corresponding key parts recognition processing, wherein the image recognition model is a deep learning model, the image recognition model is obtained by training an initial model with labeled sample image data, and the image recognition model is used for recognizing key parts contained in the image data input by the model, the key parts including lips, teeth, hard palate, soft palate, uvula, tongue, tonsils, epiglottis, vocal cords, glottis and trachea; A display unit, used to display the image data and the key part recognition result output by the image recognition model; The operating unit is used to perform fully automatic or semi-automatic tracheal intubation operation based on the key part identification result through a tracheal intubation mechanical auxiliary device.

9. An automated endotracheal intubation system based on artificial intelligence, characterized in that: Used to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the method according to any one of claims 1 to 7.

Images