Method and system for processing tracheal intubation images, and method for evaluating the effectiveness of tracheal intubation.

Abstract

To provide a method and system for processing tracheal intubation images, and to solve the problem that it is difficult to provide real-time operation and training feedback using only subjective judgment and manual interpretation at the current stage. [Solution] The present invention relates to a method and system for processing tracheal intubation images, and a method for evaluating the effectiveness of tracheal intubation. The present invention first creates a database containing images of target structures and important steps in the tracheal intubation process, and then uses machine learning to build a system for automatically identifying the target structures using the images stored in the database. The time difference between any two structural targets in the intubation image is defined as the stage intubation time, and an intubation time series is established by linking the stage intubation times in chronological order, which serves as a method for evaluating the effectiveness of tracheal intubation, and also establishes a stage-by-stage effectiveness evaluation model for tracheal intubation.

Description

Technical Field

[0001] The present invention relates to a processing method and system, and particularly to a method and system for processing tracheal intubation images, as well as a method for evaluating the effect of tracheal intubation.

[0002] Tracheal intubation is a high-risk and technically demanding medical procedure. Furthermore, tracheal intubation needs to be completed within a short time, and if it cannot be completed within a few minutes, it may cause serious organ damage. Currently, it is limited to intubation tools and techniques, and the difficulty of intubation is only determined by the subjective judgment of the operator and clinical intubation results (such as such time, number of times, etc.), or identified by specific images. Therefore, the differences in the determination of intubation difficulty are very large. Not only can they not be integrated and communicated in research, but the training results and pass / fail of clinical education in the skills required by all doctors cannot be determined by an objective evaluation method.

[0003] In recent years, tools for assisting intubation with images have diversified and are also popular in clinical use. However, it can be seen from actual clinical practice that the past evaluation of intubation difficulty and intubation methods cannot be directly adopted and applied to the image technology for assisting intubation. The current implementation method is to use a video laryngoscope or a video stylet to synchronously take pictures during the intubation process. However, the images recorded by these methods are only provided for later learning or analyzing the differences in local structures, and no systematic structure and time-series analysis are performed on the images of the intubation process. Therefore, it is difficult to apply the analysis of intubation difficulty or evaluate the results of intubation education and training.

Summary of the Invention

Problems to be Solved by the Invention

[0004] This invention addresses the need for clinical procedures and training in high-risk, technically demanding endotracheal intubation by providing a method and system for processing endotracheal intubation images. It offers the integration of immediate segmentation, time-series processing, and analysis of images from the entire intubation process, and gradually establishes a method for evaluating the effectiveness of endotracheal intubation, thereby solving the problem of difficulty in providing immediate operational and training feedback with current subjective judgment and artificial interpretation alone. [Means for solving the problem]

[0005] To achieve the above objective, the present invention provides a method for processing tracheal intubation images, comprising: Step 1: creating a database containing a plurality of structural targets, storing at least one first intubation process image in the database, further storing these structural targets defined in the at least one first intubation process image, wherein these structural targets are selected from a group consisting of lips, navel, pharynx, glottis, endotracheal tube and black marking lines of the endotracheal tube; Step 2: using these defined structural targets to identify target objects identical to these structural targets in a second intubation process image, thereby obtaining a plurality of target objects identical to these structural targets in the second intubation process image; and confirming the time when these identified target objects appear in the second intubation process image, thereby determining the target objects of Step 3 provides step 1: obtaining step-by-step intubation time and intubation time series during the intubation process. The time points at which these target objects appear in the second intubation process image are defined as n time points, the time difference between any two target objects is defined as step-by-step intubation time, and a plurality of step-by-step intubation times are combined in the order in which these target objects appear in the second intubation process image to establish an intubation time series for these target objects in the second intubation process image. The intubation time series is a series composed of the time order in which these target objects appeared in the second intubation process image.

[0006] In one embodiment, the analysis was performed using the first intubation process image, which was zero on the intubation difficulty evaluation table. cormorant These structural targets are defined by doing so.

[0007] In one embodiment, these structural targets are a group consisting of the lips, navel, pharynx, glottis, endotracheal tube, and the black marking line of the endotracheal tube.

[0008] In one embodiment, the processing method involves confirming the point in time when these structural targets appear in the first intubation process image to obtain the stepwise intubation time and intubation time series of these structural targets; the point in time when these structural targets appear in the first intubation process image is defined as n points in time, the time difference between any two structural targets is defined as the stepwise intubation time, and multiple stepwise intubation times are cut and combined in chronological order to establish the intubation time series of these structural targets in the first intubation process image; and an intubation time series diagram of the first intubation process image and a time series analysis diagram of intubation capacity are drawn using the stepwise intubation time and intubation time series of these structural targets.

[0009] In one embodiment, the processing method further includes drawing an intubation time series diagram and an intubation capacity time series analysis diagram of the second intubation process images based on the step-by-step intubation time and intubation time series of these target objects.

[0010] To achieve the above objective, the tracheal intubation image processing system of the present invention includes a database and an electronic device. The database stores at least one first intubation process image, and a plurality of structural targets are defined from the at least one first intubation process image. The electronic device and the database are electrically connected, the electronic device includes one or more processing units and a storage unit, the one or more processing units and the storage unit are electrically connected, the storage unit stores one or more program instructions, and the one or more program instructions are operated by the one or more processing units, the one or more processing units cause the defined structural targets to identify targets in a second intubation process image, thereby obtaining a plurality of target objects in the second intubation process image that are the same as these structural targets; and by confirming the time when these identified target objects appear in the second intubation process image, the step intubation time and intubation time series of these target objects are obtained. The points in time when these target objects appear in the second intubation process image are defined as n points in time, the time difference between any two target objects is defined as the step-by-step intubation time, and the intubation time series of these target objects in the second intubation process image is established by cutting and combining multiple step-by-step intubation times in chronological order.

[0011] In one embodiment, the database is located in a storage unit or cloud device.

[0012] In one embodiment, the one or more processing units further determine the point in time when these structural targets appear in the first intubation process image, of Stepped intubation times and intubation time series are obtained during the intubation process; the points in time when these structural targets appear in the first intubation process image are defined as n points in time, the time difference between any two structural targets is defined as the stepped intubation time, and a plurality of stepped intubation times are combined in the order in which these structural targets appear in the first intubation process image to establish an intubation time series for these structural targets in the first intubation process image, and the intubation time series is a series composed of the order in which these structural targets appear in the first intubation process image.

[0013] In one embodiment, the one or more processing units draw an intubation time series diagram and an intubation capacity time series analysis diagram of the second intubation process images based on the step intubation time and intubation time series of these target objects.

[0014] To achieve the above objective, the method for evaluating the effectiveness of tracheal intubation according to the present invention includes the processing method described above; and the tracheal intubation effectiveness is evaluated by the stepwise intubation time and intubation time series of these target objects and these structural targets.

[0015] In one embodiment, the effectiveness of tracheal intubation is evaluated by comparing the step-by-step intubation time and intubation time series of the first intubation process image and the second intubation process image. of Evaluate the effectiveness of intubation at each stage. [Effects of the Invention]

[0016] In summary, the present invention's method for processing tracheal intubation images involves creating a database containing multiple structural targets, storing at least one first intubation process image in the database, and defining these structural targets using the at least one first intubation process image; identifying targets in a second intubation process image using these defined structural targets, thereby obtaining multiple target objects identical to these structural targets in the second intubation process image; and confirming the timing at which these identified target objects appear in the second intubation process image to obtain the step-by-step intubation time and intubation time series of these target objects. The timing at which these target objects appear in the second intubation process image is defined as n timings, the time difference between any two target objects is defined as the step-by-step intubation time, and multiple step-by-step intubation times are cut and combined in chronological order to establish processes such as the intubation time series of these target objects in the second intubation process image. As a result, the present invention, submitted in response to the demand for high-risk, technically demanding endotracheal intubation clinical procedures and training, provides a method and system for processing endotracheal intubation images that can provide step-by-step intubation time and intubation time series images of the entire intubation process. It gradually provides the integration of step-by-step intubation division, time-series processing and analysis, and establishes a method for evaluating the effectiveness of endotracheal intubation, thereby solving the problem that currently, subjective judgment and artificial interpretation alone make it difficult to immediately provide operational assistance and training feedback. [Brief explanation of the drawing]

[0017] [Figure 1A] This is a functional block diagram of a tracheal intubation image processing system according to one embodiment of the present invention. [Figure 1B] This is a flowchart of a method for processing tracheal intubation images according to one embodiment of the present invention. [Figure 2] This is another flowchart of the method for processing tracheal intubation images according to the present invention. [Figure 3A] This is a time-series diagram of the first intubation process images according to one embodiment of the present invention. [Figure 3B] This is a time-series analysis diagram of the intubation capacity of the first intubation process images according to one embodiment of the present invention. [Figure 4A] This is a time-series diagram of the second intubation process images according to one embodiment of the present invention. [Figure 4B] It is a time-series analysis diagram of the intubation ability of the second intubation process image of an embodiment of the present invention. [Figure 5] It is a flowchart of a method for evaluating the effect of tracheal intubation of an embodiment of the present invention.

Embodiments for Carrying Out the Invention

[0018] Using each drawing, better embodiments related to the method and system for processing tracheal intubation images of the present invention, as well as the method for evaluating the effect of tracheal intubation, will be described below. The same elements will be described with the same reference numerals.

[0019] All the tracheal intubation analysis cases recorded in this specification have been approved by the Human Experiment Committee and are basically tracked (IRB NCKUH B-ER-107-088). Also, the processing system in this specification is also referred to as an analysis system, and the processing method is also referred to as an analysis method. Furthermore, the first intubation process image and the second intubation process image in this specification are only for distinction, and all are the full-process images recorded during the tracheal intubation process.

[0020] FIG. 1A is a functional block diagram of a processing system for tracheal intubation images of an embodiment of the present invention, and FIG. 1B is a flowchart of a method for processing tracheal intubation images of an embodiment of the present invention.

[0021] As shown in FIGS. 1A and 1B, the processing system 1 for tracheal intubation images of this embodiment includes a database 11 and an electronic device 12.

[0022] The database 11 can store at least one first intubation process image, and a plurality of structural targets can be defined in the at least one first intubation process image. Specifically, in order to perform analysis and processing and establish an evaluation model and standard for the tracheal intubation effect, it is necessary to establish a database 11 of a plurality of structural targets first. These structural targets are defined by at least one first intubation process image, and it is preferable to define these structural targets from a plurality of first intubation process images.

[0023] In this embodiment, when a patient undergoes tracheal intubation surgery under general anesthesia, if they sign a consent form, the entire process of intubation, as recorded during the procedure, is saved in database 11. The information used for analysis is selected from database 11 and represents successful intubation cases performed by the anesthesiologist. All patients are intubated using a Trachway Blade, but this is not limited to the Trachway Blade. The entire intubation process is captured using a camera lens pre-installed on the tip of the blade, and after intubation, the process is analyzed using an Intubation Difficulty Scale (IDS). An evaluation of the intubation time is also performed. This establishes the identification of basic structures or characteristic images that lead to successful intubation, and enables the automation of integrated analysis of variations in the process flow interval.

[0024] In this embodiment, these structural targets are defined by analyzing the first intubation process image in database 11 with a difficulty rating of zero (i.e., an easy intubation case) on the intubation difficulty evaluation table and establishing a model. These structural targets can be selected as a group consisting of the lip, epiglottis, laryngopharynx, glottis, endotracheal tube, and the black marking line on the endotracheal tube (for example, the endotracheal tube has two ring-shaped black lines), but are not limited to this. In this embodiment, there are a total of six structural targets, namely a group consisting of the lip, epiglottis, laryngopharynx, glottis, endotracheal tube, and the black marking line on the endotracheal tube, but are not limited to this. In different embodiments, the structural target objects defined in the tracheal intubation images may differ, and the number may be six or more or six or less. The user may define a different number and structure from the above-mentioned structural target objects from the intubation process images, depending on the need for effectiveness evaluation.

[0025] In some embodiments, for example, 33 complete intubation process image examples are selected from a database 11 that stores images of the first intubation process. These are then reviewed by multiple veteran specialists to divide the intubation process into different stages in chronological order. Furthermore, each defined structural target object is marked and identified in the images. A total of six types of structural target objects are selected, including lips: 27 images, nasolabial folds: 173 images, pharynx: 366 images, glottis: 377 images, endotracheal tube: 24 images, and black marking lines on the endotracheal tube: 345 images, resulting in images of six types of structural target objects that appear in chronological order. These structural target object images are used to train a target object identification model based on YOLOv3 (Real-Time Object Detection), and are later provided for target object identification in other intubation images. In some embodiments, the data from the first intubation process images is first cut into single image data at 30 fps, and then image target object detection is performed. Since any two target objects can be combined within a single intubation time, the intubation images can be cut into a time-series view of the intubation process. This allows for the time-series analysis of each structural target object within the flow of the tracheal intubation procedure.

[0026] In other words, a database 11 containing images of target structures and important steps in the tracheal intubation process is created first, and then an artificial intelligence (AI) is trained on the images stored in database 11 to construct an AI identification system that can automatically identify the structures involved. Here, by dividing the images of the entire intubation process into stages, the timing at which each structural target appears in the image series and the intention of the operation are identified, and the intubation time series is obtained after cutting and combining the staged intubation time and these staged intubation times. For example, intubation process images can be defined as n structural targets (n≧2) progressing in time series, and since these n structural targets correspond to n different time points in the image, the time difference between any two adjacent or non-adjacent structural targets can be defined as the staged intubation time.

[0027] Taking n=6 as an example, since any two structural targets can be defined as a single stage intubation time, the entire intubation process can be divided into 5 intubation stages corresponding to at least 5 (6-1=5) stage intubation times in chronological order, and up to 15 (6×5 / 2=15) stage intubation times, with each intubation stage having its own time and operational significance. Furthermore, by cutting and combining the stage intubation times chronologically, the intubation time series of the entire intubation process can be obtained. When n=6, the structural targets appear in chronological order, that is, the first to sixth structural targets can be divided into at least 5 stage intubation times (e.g., t1~t5). Here, the step-by-step intubation time t1 corresponds to the time between the first and second structural target objects, the step-by-step intubation time t2 corresponds to the time between the second and third structural target objects, the step-by-step intubation time t3 corresponds to the time between the third and fourth structural target objects, the step-by-step intubation time t4 corresponds to the time between the fourth and fifth structural target objects, and the step-by-step intubation time t5 corresponds to the time between the fifth and sixth structural target objects, resulting in a total of five intubation stages. By sequentially cutting and combining these five step-by-step intubation times (t1, t2, t3, t4, t5), a time series of intubation for six structural target objects can be obtained. For example, by sequentially cutting and combining three step-by-step intubation times (t1+t2), t3, and (t4+t5), a similar intubation time series for these six structural targets can be obtained, corresponding to the intervals between the first and third structural targets, the third and fourth structural targets, and the fourth and sixth structural targets, respectively. In a different embodiment, when n=8, the entire intubation process can be divided into seven intubation stages corresponding to at least seven (8-1=7) step-by-step intubation times, and in the most cases, into 28 intubation stages corresponding to 28 (8×7 / 2=28) step-by-step intubation times. Furthermore, by cutting and combining some step-by-step intubation times in chronological order, an intubation time series for the entire intubation process can be obtained, and the rest can be inferred.

[0028] In this embodiment, an image labeling tool is used, and LabelImg labeling software is a target labeling tool. Based on the structure and characteristics of the target object, experts hold meetings to select structural targets, provide principles for defining each structural target, and define the above six structural targets through testing, verification, and modification. Furthermore, artificial intelligence is trained with these six structural targets, and after repeated verification and modification, subsequent targets in intubation images can be automatically identified by the AI ​​system with an accuracy rate that is almost perfect, thus enabling the evaluation of the automation effect of tracheal intubation images.

[0029] To achieve perfect accuracy in AI identification, it is necessary to repeatedly perform verification and correction processes on the AI ​​to verify the accuracy of target object identification. Here, in addition to defining multiple first intubation process images of these structural targets as described above and having the AI ​​system test (verify) them, further testing (verification) is performed using other intubation process images (even if IDS=0 or IDS≠0), and the AI's identification ability is continuously trained and corrected, so that the AI ​​system can achieve perfect accuracy in target object identification.

[0030] Furthermore, it is particularly worth explaining that the temporal significance of each anatomical structure in the intubation image during the tracheal intubation process is as follows: Lips: This marks the start of the intubation flow, and when the lips are no longer visible, it indicates that the camera lens has entered the oral cavity; Perineum: When the camera lens enters the oral cavity and passes over the tongue, it has already reached the base of the tongue accurately. For beginners, being able to slide the blade to the base of the tongue indicates that the basic movements are accurate and not too far from the midline; the central position of the blade affects the range of movement of the perineum; Glottis: After seeing the perineum, the position, force, and angle of the blade are adjusted to obtain the best view of the open glottis; Superior viewing angle of the larynx: Whether the glottis can be quickly seen from the superior viewing angle of the larynx is an indicator for the anesthesiologist to judge the difficulty of intubation, and this time could not be specifically marked in past intubation images; Arytenoid commissure, AC): The pharyngeal structure that is first exposed during the intubation process. Its shape differs from that of the esophageal opening, making it an important anatomical structure for distinguishing the esophagus from the tracheal opening; Endotracheal tube anterior end: The presence of the endotracheal tube in the field of view indicates that the endotracheal tube can be guided from the opening to the pharynx; Middle section of the endotracheal tube: This confirms that the endotracheal tube is aimed at the pharynx and controlled to smoothly reach the larynx; Black marking line on the endotracheal tube: When the black marking line disappears, it indicates that the intubation has reached its correct position and the intubation process is complete.

[0031] See also Figure 1A. The electronic device 12 is electrically connected to the database 11. The electronic device 12 is, but is not limited to, a computer, server, mobile phone, or tablet. In some embodiments, the electrical connection between the electronic device 12 and the database 11 is wireless or wired, and the wireless connection is, for example, via a Wi-Fi module, Bluetooth® module, or mobile communication network (3G, 4G, or 5G) to receive, store, and process the data stored in the database 11. The electronic device 12 includes one or more processing units 121 and a storage unit 122, and one or more processing units 121 and the storage unit 122 are electrically connected. Figure 1A shows an example of one processing unit 121 and one storage unit 122, and the above database 11 is located in the storage unit 122 or a cloud device; or the database 11 is located in an independent computer-readable storage medium (such as a solid-state drive, USB, or any form of memory, but not limited to these) or a memory chip. When database 11 is located on a cloud device, before electronic device 12 processes and analyzes the data, it first downloads the data stored in database 11 from the cloud device to storage unit 122, and then processes and analyzes it using processing unit 121. If database 11 is located on storage unit 122, downloading is not necessary. If database 11 is located on an independent computer-readable storage medium, the stored contents can be read by processing unit 121 by inserting the electronic device.

[0032] The processing unit 121 can access the data stored in the storage unit 122 and further includes the core control device of the electronic device 12, for example, including at least one central processing unit (CPU) and memory, or other control hardware, software, or firmware. The storage unit 122 is a non-transitory computer-readable storage medium, for example, including at least memory, memory cards, memory chips, disks, videotapes, computer tapes, or any combination thereof. In some embodiments, the memory includes read-only memory (ROM), flash memory, field-programmable gate arrays (FPGA), solid-state disks (SSD), and other forms of memory or combinations thereof.

[0033] The storage unit 122 stores at least application software, which includes one or more program instructions 1221. After creating the database 11, one or more program instructions 1221 of the application software stored in the storage unit 122 are executed by one or more processing units 121, and one or more processing units 121 at least identify the target objects in the second intubation process image by these defined structural target objects, thereby obtaining multiple target objects (step S02 in Figure 1B) identical to these structural target objects in the second intubation process image; and by confirming the time when these identified target objects appear in the second intubation process image, the step-by-step intubation time and intubation time series of these target objects are obtained. The points in time when these target objects appear in the second intubation process image are defined as n points in time, the time difference between any two target objects is defined as the step intubation time (the second intubation process image can obtain (n-1) or more step intubation times), and by cutting and combining multiple step intubation times in chronological order, the intubation time series of these target objects in the second intubation process image (process S03 in Figure 1B) is established.

[0034] Furthermore, one or more processing units 121 draw an intubation time series diagram and an intubation capability time series analysis diagram (step S04 in Figure 2) of the second intubation process image based on the step intubation times and intubation time series of these target objects. In addition, one or more processing units 121 obtain the step intubation times and intubation time series of these structural targets by confirming the time when these structural targets appear in the first intubation process image. The time when these structural targets appear in the first intubation process image is defined as n time points, and the time difference between any two structural targets is defined as the step intubation time (the first intubation process image can obtain (n-1) or more step intubation times). Furthermore, by cutting and combining multiple step intubation times in chronological order, the intubation time series of these structural targets in the first intubation process image (step S05 in Figure 2) is established. Furthermore, the intubation time series diagram and intubation capability time series analysis diagram (step S06 in Figure 2) of the first intubation process image are drawn based on the step intubation times and intubation time series of these structural targets.

[0035] Next, we will explain the above processes S02 through S06 in detail.

[0036] As shown in Figure 1B, the method for processing tracheal intubation images according to the present invention includes steps S01 to S03.

[0037] Step S01 involves creating a database 11 containing multiple structural targets, storing at least one first intubation process image in the database 11, and defining these structural targets using this at least one first intubation process image. As mentioned above, by first identifying and defining the structural targets of at least one (preferably multiple) first intubation process images, the subsequent identification criteria are established. In this embodiment, these structural targets are defined by performing an analysis using multiple first intubation process images with zero in the intubation difficulty evaluation table and establishing a model. Furthermore, these structural targets in this embodiment appear in chronological order and may include a total of six target structures (not limited to six), such as the lips, nasolabial fold, pharynx, glottis, endotracheal tube, and the black marking line on the endotracheal tube. These six structural targets, the first intubation process images, the intubation time at each stage, and the intubation time series can all be stored in the database 11.

[0038] Step S02 involves identifying the target objects in the second intubation process image using these defined structural target objects, thereby obtaining multiple target objects identical to these structural target objects in the second intubation process image. These identified target objects in the second intubation process image can also be stored in the database 11. Specifically, in order to evaluate the intubation effect of a later intubation image (i.e., the second intubation process image), target object identification in the intubation process is performed on the second intubation process image, and these target objects identified in the second intubation process image are made identical to these structural target objects in the first intubation process image, thereby enabling the evaluation of the effectiveness of each stage in the intubation process using the same criteria. Each stage represents a different operational definition and intention, and can be evaluated or improved independently. In some embodiments, by using the AI ​​system that has completed the above training to perform target object identification on a later intubation process image, multiple target objects with the same structure as the structural target objects (six or more, and when comparing, the same number of targets and the same stage are compared) are obtained. In this embodiment, the second intubation process image is, for example, an intubation process image obtained by another physician (a PGY resident could be an example, but is not limited to this) during an intubation practice in the anesthesiology department.

[0039] Step S03 involves confirming the timing at which these identified target objects appear in the second intubation process image, thereby obtaining the stepwise intubation times and intubation time series for these target objects. The timing at which these target objects appear in the second intubation process image is defined as n timing points, the time difference between any two target objects is defined as the stepwise intubation time, and the intubation time series for these target objects in the second intubation process image is established by cutting and combining multiple stepwise intubation times in chronological order. Since the second intubation process image is also a time-series image of the intubation process, the timing at which each target object (e.g., lips, nasolabial folds, pharynx, glottis, endotracheal tube and the black marking line on the endotracheal tube, etc., but not limited to these) appears in the second intubation process image in sequence is obtained. As described above, the stepwise intubation times for these target objects are obtained, and the intubation time series obtained by cutting and combining these stepwise intubation times is provided for subsequent evaluation.

[0040] In this embodiment, the time points at which these target objects appear sequentially in the second intubation process image can be defined as, for example, six time points, and the time difference between any two target objects can be defined as one step intubation time. A minimum of five step intubation times (corresponding to five steps) can be obtained from the second intubation process image, and a maximum of fifteen step intubation times (corresponding to fifteen steps) can be obtained. Furthermore, by cutting the image in the order in which the target objects appear and combining these step intubation times (intubation stages), an intubation time series of the second intubation process image can be established. In other words, assuming that the six (n=6) target objects—the lips, navel, pharynx, glottis, endotracheal tube, and the black marking line on the endotracheal tube—appear sequentially in the image at points t1, t2, ..., t6, then (t2-t1) is the stepwise intubation time from the lips to the navel, (t3-t2) is the stepwise intubation time from the navel to the pharynx, ..., (t6-t5) is the stepwise intubation time from the endotracheal tube to the black marking line on the endotracheal tube. Also, (t3-t1) is the stepwise intubation time from the pharynx to the lips, (t5-t2) is the stepwise intubation time from the navel to the endotracheal tube, and (t6-t1) is the stepwise intubation time from the lips to the black marking line on the endotracheal tube. By analogy, we can obtain stepwise intubation times for five or more steps (the maximum being 15 in this embodiment).

[0041] For example, the difference between the first glottis image and the first epiglottis image is 7 seconds, meaning that the time from the glottis to the glottis is 7 seconds, and subsequent calculations can be made by analogy. It is noteworthy that the time difference between two target objects (or two structural targets) is not limited to two target objects (or two structural targets) that are adjacent in time series, but the stepwise intubation time can also be calculated for two target objects (or two structural targets) that are not adjacent, generating a stepwise intubation time corresponding to the intubation stage. This allows for an evaluation of the effectiveness of the intubation stage, and for example, the time difference (t6-t4) when the tube exits the glottis (at time t4) alone or simultaneously (at time t6) and the black marking line on the endotracheal tube disappears can be calculated. After obtaining the stepwise intubation time (t6-t4) for that intubation stage, the effectiveness of that intubation stage can be evaluated, and the same method can be used for other intubation stages.

[0042] As shown in Figure 2, this is another flowchart of the method for processing tracheal intubation images according to the present invention. In Figure 2, in addition to steps S01 to S03 described above, the processing method further includes steps S04 to S06. The time order of steps S04 and S05 (and steps S06) is not limited; step S04 may be performed first, followed by step S05 (and steps S06); or step S05 (and steps S06) may be performed first, followed by step S04; or steps S04 and S05 (and steps S06) may be performed simultaneously.

[0043] Next, we will explain steps S05 and S06, and then step S04. Step S05 involves confirming the timing at which these structural targets appear in the first intubation process image, thereby obtaining the step intubation time and intubation time series of these structural targets; the timing at which these structural targets appear in the first intubation process image is defined as n timings, the time difference between any two structural targets is defined as the step intubation time, and multiple step intubation times are cut and combined in chronological order to establish the intubation time series of these structural targets in the first intubation process image. Similarly, since the first intubation process image is also a time-series image of the intubation process, just as in the process of obtaining the step-by-step intubation times and intubation time series of these target objects in the second intubation process image in step S03 above, a similar method is used to obtain, for example, six time points and, for example, five or more step-by-step intubation times (up to, for example, 15) corresponding to each structural target object in the first intubation process image. Furthermore, by cutting and combining these step-by-step intubation times in chronological order, the intubation time series of these structural targets in the first intubation process image is established.

[0044] Step S06 generates an intubation time-series diagram of the first intubation process images and an intubation capability time-series analysis diagram based on the step-by-step intubation times and intubation time-series of these structural targets. In this embodiment, using the database 11 described above, three veteran anesthesiologists jointly mark the six structural targets and their corresponding time-series points on the (first) intubation image data with zero IDS, establishing intubation time-series diagrams for multiple step-by-step intubation and the entire standard airway intubation process, for later comparison of effectiveness. Refer to the intubation time-series diagram of the first intubation process images shown in Figure 3A and the intubation capability time-series analysis diagram of the first intubation process images shown in Figure 3B for the results.

[0045] In addition, process S04 draws an intubation time series diagram and an intubation capability time series analysis diagram of the second intubation process images based on the stepwise intubation times and intubation time series of these target objects. Similarly, for example, from the six target objects identified from the second intubation process images obtained from tracheal intubation performed by a PGY resident, and their corresponding intubation time series (process S03), it is possible to draw a stepwise intubation time and intubation time series diagram of the second (beginner) intubation process images as shown in Figure 4A, and an intubation capability time series analysis diagram of the second (beginner) intubation process images as shown in Figure 4B.

[0046] In Figures 3A and 4A, the horizontal axis represents time (seconds), with A being the last lip image, B the first epiglottis image, C the first larynx image, D the last free glottis image, and E the last blackline image. Figures 3A and 4A provide the time taken for each different stage of the tracheal intubation image. It is important to note that not all targets (and their corresponding time points) necessarily need to appear in the current intubation time series diagram; users can choose to omit one or more targets from the diagram by judging the difference in technique.

[0047] In the embodiments shown in Figures 3A and 4A, there are six target objects, but only five time points and four intubation stages (four stage intubation times: A to B, B to C, C to D, D to E) are depicted. Each stage represents a different operational definition and intent, and can be evaluated or improved independently, for example, by evaluating the effectiveness of a single intubation stage or all intubation stages individually. Of course, in different embodiments, six(n) target objects and two adjacent target objects can also be calculated, resulting in five intubation stages (five stage intubation times). Alternatively, the present invention is not limited to methods such as adjusting the process according to the user's needs to generate five or more step-by-step intubation times, further evaluating the step-by-step intubation times between one target object and another (for example, intubation stages such as A to B, B to C, C to D, D to E, or integrating the intubation stage from B to D, the intubation stage from A to D, etc.) and their effects, or separating the operational meaning for each intubation stage and evaluating the effect of each stage individually, or further evaluating the effect of only one of those intubation stages.

[0048] Figure 5 is a flowchart of a method for evaluating the effectiveness of tracheal intubation according to one embodiment of the present invention.

[0049] The present invention further provides a method for evaluating the effectiveness of tracheal intubation, which can be applied to the tracheal intubation image processing system 1 and method described above. The tracheal intubation image processing system 1 and method have been described in detail above and will not be explained further. The tracheal intubation effectiveness evaluation method of the present invention can perform automated evaluation of tracheal intubation and includes the tracheal intubation processing method (or process) and effectiveness evaluation process described above.

[0050] As shown in Figure 5, the tracheal intubation process (or method) includes steps S01 to S06, which have been explained in detail above and will not be explained further. The effect evaluation step evaluates the tracheal intubation effect of the second intubation process image based on the intubation time series of these target objects and these structural targets. The evaluation of the tracheal intubation effect of the second intubation process image includes step S07 in Figure 5, and evaluates the intubation effect of each stage of the second intubation process image by comparing the stage intubation time, intubation time series diagram and intubation capacity time series analysis diagram of the first intubation process image and the second intubation process image. The stage intubation time, intubation time series diagram and intubation capacity time series analysis diagram of the first intubation process image are shown in Figures 3A and 3B above as examples, and the stage intubation time, intubation time series diagram and intubation capacity time series analysis diagram of the second intubation process image are shown in Figures 4A and 4B above as examples.

[0051] Furthermore, please refer to the step-by-step intubation time and intubation time series diagram in Figure 3A of the first intubation process image. Steps A to B represent the process and time it takes for the intubation track wave blade to accurately enter the oral cavity and reach below the base of the tongue (step-by-step intubation time is approximately 3 seconds); steps B to C represent the process and time it takes to move the cyst to expose the laryngeal structure (step-by-step intubation time is approximately 2.5 seconds); steps C to D represent the process and time it takes to adjust the blade to appropriately bias it to expose the most favorable glottal structure and guide the endotracheal tube to the larynx (step-by-step intubation time is approximately 4 seconds); and steps D to E represent the process and time it takes to slide the endotracheal tube from the larynx to the fixed position in the trachea (step-by-step intubation time is approximately 4.5 seconds). Therefore, the total time of tracheal intubation in steps A to E (difference in step-by-step intubation time) is approximately 14 seconds. Furthermore, as can be seen from the time-series analysis of intubation time and intubation ability in the first intubation process images in Figure 3B, the tracheal intubation ability of the veteran anesthesiologist is between 75% and 100% (i.e., located within the gray area shown in Figure 3B), indicating excellent intubation skills.

[0052] In Figure 4A, the meanings of A to E in the stepwise intubation time and intubation time series diagram of the second (beginner) intubation process image are as described above. In Figure 4A, the stepwise intubation time from A to B is approximately 6 seconds, from B to C is approximately 19 seconds, from C to D is approximately 30 seconds, and from D to E is approximately 32 seconds. Therefore, the total time for tracheal intubation from A to E is approximately 87 seconds. Comparing Figure 4A with Figure 3A, it is clear that the PGY resident took a very long time to complete the intubation process, especially from C to D and from D to E, indicating a lack of skill in tracheal intubation from C to E.

[0053] Furthermore, in the time-series analysis of intubation ability in the second (beginner) intubation process images in Figure 4B, intubation images at different times can draw different sets of lines, each set of lines forming a quadrilateral, with each set of lines representing one intubation process. As can be seen from Figure 4B, most of these quadrilateral lines are located in area Z (the gray area in Figure 4B is an area where intubation ability is lower than expected, for example, tracheal intubation ability is 75% or less), indicating that the PGY resident's initial intubation skills are insufficient and need to be strengthened. In addition, compared to Figure 3B, the Epiglottis 1st to Glottis 1st stage skills in Figure 4B are also insufficient, and the PGY resident's intubation process is intermittent (from C to D and D to E in Figure 4A), indicating that the C to E stage skills are immature. Moreover, the PGY resident also exhibits insufficient skills at other points in time. Based on the above analysis and comparison, it is possible to evaluate the technical skills of the PGY resident in tracheal intubation, identify areas where skills are lacking, and determine areas where further training will be necessary, thus achieving the objective of evaluating the effectiveness of automated tracheal intubation.

[0054] As can be seen from the above disclosure, the present invention's method for processing tracheal intubation images and the method for evaluating the effectiveness of tracheal intubation including the processing method can use the time interval between identified target objects and the time interval between them as an indicator for evaluating the effectiveness of tracheal intubation, corresponding to the operating time spent at each stage of the airway intubation process, and can also be used as a target for searching for risk factors at each stage. By comparing this with a standardized airway intubation time series, the learning effect of new doctors can be understood, and more detailed learning feedback for each stage can be provided. Furthermore, the present invention can analyze the degree of differentiation or learning curve when performing intubation with different tools, different levels of personnel, and different intubation difficulty scores.

[0055] The distinguishing feature of this invention is that conventional image analysis of tracheal intubation mainly focuses on the larynx (cormack grade), making it impossible to identify the effects in other parts. Most importantly, this invention features characteristic marking (target objects) through multiple structural anatomy and develops a time-series tracheal intubation processing system and method, freeing us from the traditional concept of success or failure with each intubation. The intubation process is divided into different stages based on the target object and time, and further divided into different stages and corresponding depletion times by automatically identifying the target object, thereby establishing a constructive effect evaluation model for each stage of tracheal intubation.

[0056] This invention addresses the high-risk, technically demanding clinical procedure and training requirements for endotracheal intubation by developing artificial intelligence-based processing of endotracheal intubation images, providing immediate time-series analysis and objective quantification evaluation of images throughout the entire procedure. Furthermore, based on the above processing (analysis) method, this invention establishes a new automated evaluation system for the effectiveness of endotracheal intubation, solving the problem of difficulty in providing immediate operational assistance and training feedback, which is currently difficult with subjective judgment and artificial interpretation. Moreover, this system and method can be used for the technical evaluation and development of individual learning processes for endotracheal intubation in the future, the development and verification of new intubation tools or techniques, and the evaluation of educational plans for simulated training.

[0057] In summary, the present invention's method for processing tracheal intubation images involves creating a database containing multiple structural targets, storing at least one first intubation process image in the database, and defining these structural targets using the at least one first intubation process image; identifying targets in a second intubation process image using these defined structural targets, thereby obtaining multiple target objects identical to these structural targets in the second intubation process image; and confirming the timing at which these identified targets appear in the second intubation process image to obtain the step-by-step intubation time and intubation time series of these target objects. The timing at which these target objects appear in the second intubation process image is defined as n timings, the time difference between any two target objects is defined as the step-by-step intubation time, and multiple step-by-step intubation times are cut and combined in chronological order to establish processes such as the intubation time series of these target objects in the second intubation process image. As a result, the present invention, submitted in response to the demand for high-risk, technically demanding endotracheal intubation clinical procedures and training, provides a method and system for processing endotracheal intubation images that can provide step-by-step intubation time and intubation time series images of the entire intubation process. It gradually provides the integration of step-by-step intubation division, time-series processing and analysis, and establishes an effective evaluation of endotracheal intubation, thereby solving the problem that currently, subjective judgment and artificial interpretation alone make it difficult to immediately provide operational assistance and training feedback.

[0058] The above are only some examples of the present invention and do not limit the present invention. Any modifications or changes made insofar as they do not deviate from the spirit and scope of the present invention should fall within the scope of the claims of the present invention. [Explanation of Symbols]

[0059] 1. Tracheal intubation image processing system 11 Databases 12 Electronic equipment 121 Cheriyuan 122 Memory source 1221 Program Instructions A Last image of lips B First image of the disliked C First image of the larynx D. The last glottal image without a tube. E The last black line image S01, S02, S03, S04, S05, S06, S07 process Z Area

Claims

1. Step 1 involves creating a database containing multiple structural targets, storing at least one first intubation process image in the database, and further storing the structural targets defined by the at least one first intubation process image, wherein these structural targets are selected from a group consisting of the lips, navel, pharynx, glottis, endotracheal tube and the black marking line of the endotracheal tube; Step 2 involves using these defined structural target objects to identify target objects identical to these structural target objects in the second intubation process image, thereby obtaining multiple target objects identical to these structural target objects in the second intubation process image; and, Step 3 involves confirming the point at which these identified target objects appear in the second intubation process image, thereby obtaining the step-by-step intubation time and intubation time series during the intubation process of these target objects. A method for processing tracheal intubation images, characterized in that the points in time when these target objects appear in the second intubation process image are defined as n points in time, the time difference between any two target objects is defined as the step intubation time, and a plurality of step intubation times are combined in the order in which these target objects appear in the second intubation process image to establish an intubation time series for these target objects in the second intubation process image, and the intubation time series is a series composed of the order in which these target objects appear in the second intubation process image.

2. The processing method according to claim 1, characterized in that these structural target objects are defined by performing an analysis using the first intubation process image, which is zero on the intubation difficulty evaluation table.

3. By confirming the point at which these structural targets appear in the first intubation process image, the stepwise intubation time and intubation time series of these structural targets during the intubation process can be obtained; The processing method according to claim 1, characterized in that the time points at which these structural targets appear in the first intubation process image are defined as n time points, the time difference between any two structural targets is defined as the step intubation time, and a plurality of step intubation times are combined in the order in which these structural targets appear in the first intubation process image to establish an intubation time series of these structural targets in the first intubation process image, and the intubation time series is a series composed of the time order in which these structural targets appear in the first intubation process image.

4. A database is provided which includes storing at least one first intubation process image, and further storing a plurality of structural targets defined by the said at least one first intubation process image, wherein these structural targets are selected from a group consisting of lips, navel, pharynx, glottis, endotracheal tube and black marking lines on the endotracheal tube, and A database and an electronic device electrically connected to it, The electronic device includes one or more processing units and a storage unit, the one or more processing units and the storage unit are electrically connected, one or more program instructions are stored in the storage unit, and when the one or more program instructions are operated by the one or more processing units, the one or more processing units cause the defined structural targets to identify target objects identical to the structural targets in the second intubation process image, thereby obtaining multiple target objects identical to the structural targets in the second intubation process image; and by confirming the time when these identified target objects appear in the second intubation process image, the step-by-step intubation time and intubation time series in the intubation process of these target objects are obtained. A tracheal intubation image processing system characterized in that the points in time when these target objects appear in the second intubation process image are defined as n points in time, the time difference between any two target objects is defined as the step intubation time, and a plurality of step intubation times are combined in the order in which these target objects appear in the second intubation process image to establish an intubation time series for these target objects in the second intubation process image, and the intubation time series is a series composed of the order in which these target objects appear in the second intubation process image.

5. The processing system according to claim 4, characterized in that the database is located in a storage unit or cloud device.

6. The processing system according to claim 4, characterized in that these structural target objects are defined by performing an analysis using the first intubation process image, which is zero on the intubation difficulty evaluation table.

7. The processing system according to claim 4, wherein one or more processing units further confirm the time at which these structural targets appear in the first intubation process image, thereby obtaining the step-by-step intubation time and intubation time series in the intubation process of these structural targets; the processing system according to claim 4, wherein the processing system establishes an intubation time series for these structural targets in the first intubation process image by defining the time at which these structural targets appear in the first intubation process image as n time points, defining the time difference between any two structural targets as the step-by-step intubation time, and combining a plurality of step-by-step intubation times in the time order in which these structural targets appear in the first intubation process image, and the intubation time series is a series composed of the time order in which these structural targets appear in the first intubation process image.

8. The processing method described in claim 3, and A method for evaluating the effectiveness of tracheal intubation, characterized by comprising evaluating the effectiveness of tracheal intubation using the stepwise intubation time and intubation time series of the aforementioned target objects and the aforementioned structural target objects, and the second intubation process image.

9. The method for evaluating the effectiveness of tracheal intubation according to claim 8, characterized in that the evaluation of the effectiveness of tracheal intubation is ensured by comparing the stepwise intubation time in the first intubation process image and the intubation time series in the second intubation process image.

Images