Double-cavity tracheal intubation assisting method and system based on YOLOv5

By improving the YOLOv5 algorithm, combining the CA attention mechanism and BiFPN structure, the dual-lumen tracheal intubation assist system was developed, which solved the problem of identifying left and right bronchial positions in dual-lumen bronchial intubation, and improved the accuracy and safety of intubation. The application scenarios are expanded for devices with different computing resources.

CN118969233APending Publication Date: 2024-11-15THE NAVAL MEDICAL UNIV OF PLA
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202410806050.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-21
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

During dual-cavity bronchial intubation, it is difficult for doctors to accurately identify the location of left and right bronchials, which affects the accuracy and safety of intubation. Especially for inexperienced doctors, it is more difficult and in complex or non-standard cases. The accuracy of identification cannot be guaranteed.

Method used

Using the dual-lumen tracheal intubation assisted method based on YOLOv5, the object detection algorithm is improved by introducing CA attention mechanism, Focal EIoU Loss and BiFPN structures, and the recognition accuracy and positioning ability of left and right bronchials and augmentations in the bronchoscopy images are developed. An auxiliary system for real-time identification and analysis, providing diagnostic and therapeutic decision support.

Benefits of technology

It significantly improves the safety and success rate of dual-cavity tracheal intubation surgery, provides important operational guidance, reduces complication risk, improves the quality of medical services and patient satisfaction, and is suitable for equipment with different computing resources, expands the Application scenario.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118969233A_ABST
    Figure CN118969233A_ABST
Patent Text Reader

Abstract

The invention provides a double-lumen tracheal intubation auxiliary method and system based on YOLOv5. Bronchoscope images are collected, classified and marked, and a data set is expanded; 80% of the marked data set is used for carrying out improved YOLOv5 model training; testing the trained model by using the remaining 20% of the data set, and measuring the performance of the model on a bronchoscope target detection task by using mAP0.5 as an evaluation index; an image in the airway is collected, the trained model is used for real-time recognition and analysis, and a key structure in the bronchoscope image is accurately recognized; and an identification and analysis result is displayed through a result visual interface so as to assist a doctor to make diagnosis and treatment decisions. According to the method, the recognition precision and the positioning capability of the model on the key structure in the bronchoscope image are remarkably improved. The precise identification and positioning are the key of effective double-cavity tracheal intubation, and are beneficial to reducing the surgical risk and improving the surgical success rate.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of medical auxiliary diagnosis, and in particular to a double-lumen endotracheal intubation auxiliary method and system based on YOLOv5. Background Art

[0002] In the medical field, double-lumen endobronchial intubation technology is an important clinical operation, especially when lung surgery is performed or separate lung ventilation is required. This technology can effectively isolate the surgical area from healthy lung tissue by creating two independent ventilation chambers, significantly reducing the risks and complications during surgery. It provides safety protection for surgery while reducing the negative impact of surgery on the patient's lung function. In addition, when treating diseases such as lung cancer, double-lumen endobronchial intubation can also effectively increase the radiation dose and drug concentration in the treatment area by independently controlling the ventilation volume of both lungs, thereby enhancing the treatment effect.

[0003] Although double-lumen endobronchial intubation has many advantages, it still faces many challenges in actual operation. One of the most critical challenges is how to efficiently and accurately locate and insert the double-lumen tube. During the intubation process, doctors need to accurately identify the positions of the left and right bronchi, which is very difficult in actual operation. Due to the different body positions of the patients and the rotation operations performed by the doctors to better enter the airway, it is easy to cause confusion in the positions of the left and right bronchi, which poses a challenge to the accuracy and safety of intubation.

[0004] For doctors with less experience, it is more difficult to accurately identify the left and right bronchi. Without sufficient experience, it is difficult for them to judge the left and right by the anatomical characteristics of the bronchi. For experienced doctors, although they can use anatomical knowledge to assist in identification, they cannot guarantee 100% identification accuracy in complex or non-standard cases. Therefore, developing a system that can automatically identify the location of the left and right bronchi is extremely important for improving the accuracy and efficiency of double-lumen intubation and reducing surgical risks.

[0005] In this context, further optimizing double-lumen endobronchial intubation technology and related equipment has become an urgent clinical problem that needs to be solved. An auxiliary system that can automatically identify the location of the left and right bronchi and accurately guide the double-lumen tube into the target bronchus can not only provide important operational guidance for doctors, but also significantly improve the success rate of intubation and reduce the risk of complications. In addition, the system can also provide junior doctors with opportunities for learning and training, accelerate their skills improvement, and have a far-reaching impact on improving the overall quality of medical services. Therefore, the development of such a system is not only technically innovative, but also of great significance to improving clinical medical safety and efficiency. Summary of the invention

[0006] In order to overcome the shortcomings of the prior art, the present invention proposes a double-lumen endotracheal intubation auxiliary method and system based on YOLOv5. Through the careful improvement and optimization of the YOLOv5 target detection algorithm, significant beneficial effects are brought about, especially in the two key areas of accuracy and speed. It can not only improve the safety and success rate of double-lumen endotracheal intubation surgery, but also provide clinicians with a powerful auxiliary tool, which is helpful to improve the quality of medical services and patient satisfaction.

[0007] To achieve the above object, the present invention provides a double-lumen endotracheal intubation auxiliary method based on YOLOv5, comprising:

[0008] Step S1: Collect bronchoscopic images from clinical practice, classify and annotate these images according to structures such as left and right bronchi and carina, and use data augmentation methods to expand the dataset;

[0009] Step S2: Use 80% of the labeled data set to train the improved YOLOv5 model, including introducing the CA attention mechanism, replacing CIoU Loss with Focal EIoU Loss as the border loss function, and introducing the BiFPN structure in the Neck part of the model to improve the accuracy and recognition ability of small objects;

[0010] Step S3: Use the remaining 20% ​​of the dataset to test the trained model, and use mAP0.5 as the evaluation index to measure the performance of the model on the bronchoscopic object detection task;

[0011] Step S4: collecting images in the airway and using the trained model for real-time recognition and analysis to accurately identify key structures in the bronchoscopic image;

[0012] Step S5: Display the results of identification and analysis through a result visualization interface to assist doctors in making diagnosis and treatment decisions.

[0013] Further, step S1 is specifically as follows:

[0014] Step S11: collecting bronchoscopic images of 500 patients, totaling about 3,000 images;

[0015] Step S12: renumbering these images to protect patient privacy, and selecting 451 images including the left and right bronchi and carina;

[0016] Step S13: Three experienced physicians identified the left and right bronchi according to their different anatomical structures and annotated them using the LabelImg tool in the PASCAL VOC format;

[0017] Step S14: perform cropping and occlusion processing on part of the original image to ensure the feature invariance of the lesion area, reduce the amount of calculation, and enhance the diversity and robustness of the data set.

[0018] Further, step S2 is specifically as follows:

[0019] Step S21: Use 80% of the samples in the data set to train the neural network model;

[0020] Step S22: Introduce the CA attention mechanism to enhance the representation ability of features through coordinate information embedding and coordinate attention generation steps;

[0021] Step S23: Using Focal EIoU Loss instead of CIoU Loss as the border loss function to optimize the training efficiency and accuracy of the model;

[0022] Step S24: Introducing the BiFPN structure into the Neck network structure, and improving the network detection accuracy through bidirectional feature flow and efficient feature map fusion method;

[0023] Step S25: Use the remaining 20% ​​of the dataset to test the model, and use mAP0.5 as the evaluation indicator to measure the model performance.

[0024] Further, step S3 is specifically as follows:

[0025] Step S31: Select a high-resolution and high-sensitivity visualization device to ensure that the quality of the acquired bronchoscope image meets the requirements of subsequent processing.

[0026] Step S32: Adjust the acquisition environment (such as lighting, angle) to ensure that the image is clear and the details are discernible, so as to reduce the error in model recognition.

[0027] Step S33: In clinical anesthesia practice, the airway images are collected in real time, and the image data is instantly transmitted to the storage system to facilitate subsequent real-time processing and analysis.

[0028] Step S34: Perform necessary preprocessing on the collected image, such as denoising, adjusting contrast, and formatting, to ensure that the image input to the recognition module is standardized.

[0029] Further, step S4 is specifically as follows:

[0030] Step S41: when starting the system, load the trained neural network model and prepare for the image recognition task;

[0031] Step S42: Process each frame of the collected image in real time, including image resizing and normalization, to adapt to the model input requirements;

[0032] Step S43: the model performs deep feature extraction on the processed image and uses the improved YOLOv5 architecture to identify key structures in the image, such as the left and right bronchi and the carina;

[0033] Step S44: Analyze the accuracy and reliability of the recognition results based on the model output to determine whether re-collection or further processing is required.

[0034] Further, step S22 is specifically as follows:

[0035] Step S221: Apply pooling kernels of different directions (horizontal and vertical) to the input feature map X to generate feature maps of size (H, 1) and (1, W), respectively. This operation helps encode direction-aware spatial information on each channel.

[0036] Step S222: cascading the two feature maps generated in step S221, and transforming the cascaded feature maps through 1x1 convolution, wherein the parameter r is used to control the downsampling ratio to optimize the size of the module; the generated coordinate attention feature map can capture the long-range dependency of the input feature map along the spatial direction, thereby enhancing the model's understanding of the location information;

[0037] Step S223: Through operations such as segmentation, convolution, and normalization, the coordinate attention feature map is multiplied with the input feature map to generate the final attention weight to enhance the representation ability of useful features.

[0038] Furthermore, the formula in step S22 is as follows:

[0039] For input X, the output expression of the cth channel with height H is:

[0040]

[0041] A similar output expression for the cth channel with width W is:

[0042]

[0043] The final CA attention output expression is:

[0044] f=δ(F1([z H , z W ]))

[0045]

[0046] Among them, f is the feature map of spatial information in the horizontal and vertical directions, F1, F H and F W is a 1x1 convolution operation, σ is the activation function, and y is the final output.

[0047] Furthermore, the loss function in step S23 is as follows:

[0048] L=IoU γ L EIoU

[0049] Among them, IoU is an indicator to measure the degree of overlap between the predicted border and the true border, and its value ranges from 0 to 1. The higher the value, the more accurate the prediction. In EIoU, it is used to calculate the loss to optimize the model's prediction of the border. EIoU is an improved version of IoU. It takes into account the width and height difference of the border instead of a simple aspect ratio, which helps to more accurately reflect the actual shape and size of the border, thereby improving the regression accuracy. The Focal mechanism is designed to solve the problem of sample imbalance, especially in the target detection task, where the number of background classes (easy to classify samples) is much larger than that of foreground classes (difficult to classify samples). The Focal mechanism adjusts the loss function, reduces the weight of easy-to-classify samples, and increases the weight of difficult-to-classify samples, so that the model pays more attention to difficult-to-identify samples. The Focal in the loss function is somewhat different from the traditional Focal Loss. The traditional Focal Loss mainly achieves attention to difficult-to-classify samples by reducing the weight of easy-to-classify samples. According to the above formula: the higher the IoU, the greater the loss, which is equivalent to weighting the predicted value, which helps to improve the regression accuracy. The parameter γ adjusts the contribution of easy-to-classify and difficult-to-classify samples in the loss. Increasing the γ value can make the model pay more attention to difficult-to-classify samples and reduce the attention to easy-to-classify samples, thereby solving the problem of class imbalance. It is used to control the degree of attention the model pays to difficult and easy samples.

[0050] A system for a double-lumen endotracheal intubation auxiliary method based on YOLOv5 is applicable to the double-lumen endotracheal intubation auxiliary method based on YOLOv5, and includes a data preparation module, a model training and optimization module, an image acquisition module, an image recognition module and a result display module.

[0051] Furthermore, the data preparation module includes:

[0052] The image collection submodule is responsible for collecting bronchoscopic image data from clinical partner hospitals;

[0053] The preprocessing submodule preprocesses the collected images, including image renumbering, cropping, and occlusion processing, to protect patient privacy and optimize subsequent processing;

[0054] In the annotation submodule, physicians with clinical experience accurately annotate the left and right bronchi and carina in the image using professional tools in a standard format;

[0055] The model training and optimization module includes:

[0056] The neural network design submodule is used to design the improved YOLOv5 model architecture, including the customization of the backbone network and the Neck network structure;

[0057] The attention mechanism integration submodule is used to integrate the CA attention mechanism into the backbone network to improve the model's ability to locate useful feature information;

[0058] The loss function optimization submodule uses Focal EIoU Loss instead of traditional CIoU Loss to optimize the model's ability to regress bounding boxes and handle the imbalance of difficult and easy samples.

[0059] The feature fusion submodule introduces the BiFPN structure to optimize the model's information retention ability during the feature fusion process and enhance the recognition and positioning of small targets;

[0060] The image acquisition module comprises:

[0061] The acquisition device configuration submodule is used to select and configure high-resolution visualization acquisition devices to ensure image quality;

[0062] The real-time acquisition submodule is used to acquire airway images in real time in clinical scenarios and provide immediate input for the recognition module;

[0063] The image recognition module comprises:

[0064] The real-time processing submodule is used to process the acquired images in real time, including pre-processing operations such as resizing and normalization;

[0065] The recognition and analysis submodule is used to use the trained model to perform real-time recognition on the pre-processed image and identify the key structures in the image;

[0066] The result display module includes:

[0067] The interface design submodule is used to design an intuitive and easy-to-use result visualization interface to display the recognition results;

[0068] The interactive function submodule is used to develop interactive functions, such as image zooming, adjusting brightness, viewing historical results, etc., to improve user experience;

[0069] The result feedback submodule is used for users to provide feedback on the recognition results to further optimize the model performance.

[0070] Compared with the prior art, the present invention has the following beneficial effects:

[0071] 1. The present invention provides a double-lumen endotracheal intubation auxiliary method and system based on YOLOv5. By introducing advanced technologies such as CA attention mechanism, Focal EIoU Loss and BiFPN structure, the method significantly improves the recognition accuracy and positioning ability of the model for key structures (such as left and right bronchi and carina) in bronchoscope images. This precise recognition and positioning is the key to effective double-lumen endotracheal intubation, which helps to reduce surgical risks and improve surgical success rates.

[0072] 2. The present invention provides a double-lumen endotracheal intubation auxiliary method and system based on YOLOv5. By carefully constructing and preprocessing a bronchoscopic image dataset and using data annotated by professional physicians for training, the high adaptability and pertinence of the training model are ensured, further enhancing the accuracy of the model in actual clinical applications.

[0073] 3. The present invention provides a double-lumen endotracheal intubation auxiliary method and system based on YOLOv5, which performs lightweight processing on the improved YOLOv5 model, not only maintaining the high recognition accuracy of the model, but also significantly improving the running speed of the model. This is crucial for medical applications that require real-time processing and recognition of images, ensuring the ability to respond quickly and make decisions.

[0074] 4. The present invention provides a double-lumen endotracheal intubation auxiliary method and system based on YOLOv5. The lightweight model reduces the demand for computing resources and reduces the dependence on hardware, making the model not only suitable for high-performance computer systems, but also can be deployed on relatively limited computing resources, such as portable devices and visual double-lumen devices, greatly expanding its application scenarios and practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0076] Figure 1 This is a schematic diagram of the working principle of the present invention.

[0077] Figure 2 This is a schematic diagram of the network structure of the present invention

[0078] Figure 3 The diagram of the left and right bronchi and carina of the present invention is marked

[0079] Figure 4 The CA module structure diagram of the present invention is

[0080] Figure 5 The BiFPN network structure of the present invention is shown in FIG.

[0081] Figure 6 Schematic diagram of some representative image recognition results of the present invention

[0082] Figure 7 The inference effect diagram of the present invention is shown in FIG.

[0083] Figure 8 This is a diagram showing the system interface of the present invention. DETAILED DESCRIPTION

[0084] The technical solution of the present invention will be more clearly and completely explained below through description of preferred embodiments of the present invention in combination with the accompanying drawings.

[0085] Terminology explanation:

[0086] 1.YOLOv5: "You Only Look Once version 5", is a popular object detection algorithm used to quickly identify and classify objects in images in real time.

[0087] 2.mAP0.5: "mean Average Precision at IoU threshold 0.5", the average precision, an evaluation indicator when the intersection-over-union (IoU) threshold is 0.5, used to measure the performance of the target detection model.

[0088] 3.CA Attention Mechanism: "Coordinate Attention", coordinate attention mechanism, a technology used to enhance the network's attention to specific spatial location features.

[0089] 4.Focal EIoU Loss: It combines the Focal Loss and EIoU (Enhanced Intersection over Union) loss functions, aiming to improve the model’s prediction of bounding boxes while solving the sample imbalance problem.

[0090] 5.CIoU Loss: "Complete Intersection over Union Loss", a loss function used to optimize bounding box prediction in object detection, which takes into account the overlap, center point distance and aspect ratio of the bounding box.

[0091] 6.BiFPN: "Bidirectional Feature Pyramid Network", a network structure used to improve the efficiency and accuracy of feature extraction, which enhances model performance by fusing features of different scales.

[0092] 7.CSPDarknet53: A deep learning architecture used as the backbone network of YOLOv5, which effectively balances speed and accuracy and is suitable for real-time object detection tasks.

[0093] 8.PASCAL VOC: "Pattern Analysis, Statistical Modelling and Computational Learning Visual Object Classes", a public computer vision dataset, commonly used for training and evaluation of visual tasks such as object detection and image classification.

[0094] 9. LabelImg: An open source image annotation tool used to manually annotate objects in images for machine learning projects and generate the label data needed to train the model.

[0095] These terms cover key concepts such as algorithms, evaluation indicators, network structures, loss functions, etc. in the field of target detection, and are the basis for understanding and implementing the double-lumen endotracheal intubation auxiliary system based on YOLOv5.

[0096] like Figure 1 As shown, the present invention is specifically:

[0097] Construction of the dataset; the obtained bronchoscopes are divided into multiple categories, marked by drawing one or more borders, and finally constructed by data enhancement;

[0098] Construction of a neural network model; 80% of the sample data of the constructed data set is used to train the neural network model. After the training is completed, the remaining 20% ​​of the data set is used for testing and deployment, and mAP0.5 is used as an evaluation indicator to measure the performance of the model in the bronchoscopic target detection task. The neural network model is an improved YOLOv5 model; the improvement measures are as follows: (1) The CA attention mechanism is introduced into the backbone network to locate useful feature information and suppress useless information; (2) Focal EIoU Loss is used instead of the original CIoU Loss as the border loss function, and the width and height difference of the border is replaced by the width and height ratio, which also solves the problem of imbalance between difficult and easy samples; (3) The BiFPN structure is introduced into the Neck network structure to reduce the loss of information in feature fusion, thereby improving the network detection accuracy and enhancing the recognition and positioning of small targets;

[0099] Image acquisition module; using visualization equipment to acquire images within the airway;

[0100] Recognition module: Use the trained model to perform real-time recognition on the collected images;

[0101] The result visualization interface displays the bronchoscopic image recognition results in real time.

[0102] As a specific implementation method, Figure 2 As shown in the figure, at the input end, the collected bronchoscopic dataset is used as the input of the model. In the process of inputting into the model, the original bronchoscopic image is first subjected to data enhancement operations, including occlusion, cropping, rotation, etc. At the backbone end, the original YOLOv5 model uses CSPDarknet53 as its backbone network. In order to locate the useful feature information in the bronchoscopic image and suppress the useless information, this part introduces the CA attention mechanism to increase the network's attention to important features, thereby improving the model performance. At the neck end, the BiFPN structure is introduced. It is improved on the basis of FPN (Feature Pyramid Network). By introducing bidirectional (top-down and bottom-up) information flow, the performance of the feature pyramid network is improved. It is mainly used to process feature information of different scales, making the improved algorithm more robust and accurate. To improve the loss function, Focal EIoU Loss is used instead of the original CIoULoss as the bounding box loss function. By reducing the weight of easy-to-classify samples, the model pays more attention to difficult-to-classify samples, thereby improving the detection performance of minority categories (left main bronchus, right main bronchus, and carina).

[0103] As a specific embodiment of a double-lumen endotracheal intubation auxiliary system based on YOLOv5, the bronchoscopic data set is obtained from the clinical anesthesia practice of a tertiary hospital in Shanghai. The bronchoscopic images of 500 patients were collected and organized to construct a bronchoscopic data set, which contains about 3,000 images taken during bronchoscopic examinations. First, these original data images need to be renumbered to protect the patient's privacy information. Then, it is necessary to select bronchoscopic images containing the left and right bronchi and carina, a total of 451 images; in order to ensure the feature invariance of the lesion area and reduce the amount of calculation, this article cropped and occluded some of the original images. Figure 3 ; For each bronchoscopic data containing the left and right bronchi and carina, three physicians with many years of clinical intubation experience identified the left and right bronchi according to their different anatomical structures, and used the LabelImg tool to annotate the left and right bronchi and carina under the bronchoscopy in the PASCAL VOC public dataset format;

[0104] Construction of a neural network model; 80% of the sample data of the constructed data set is used to train the neural network model. After the training is completed, the remaining 20% ​​of the data set is used for testing and deployment, and mAP0.5 is used as an evaluation indicator to measure the performance of the model in the bronchoscopic target detection task. The neural network model is an improved YOLOv5 model; the improvement measures are as follows: (1) The CA attention mechanism is introduced into the backbone network to locate useful feature information and suppress useless information; (2) Focal EIoU Loss is used instead of the original CIoU Loss as the border loss function, and the width and height difference of the border is replaced by the width and height ratio, which also solves the problem of imbalance between difficult and easy samples; (3) The BiFPN structure is introduced into the Neck network structure to reduce the loss of information in feature fusion, thereby improving the network detection accuracy and enhancing the recognition and positioning of small targets;

[0105] As a specific implementation steps are as follows:

[0106] (1) The CA attention mechanism is introduced into the backbone network to locate useful feature information and suppress useless information. The CA module includes two main steps: coordinate information embedding and coordinate attention generation, such as Figure 4 As shown. First, for feature aggregation, the module encodes each channel in the horizontal and vertical directions using pooling kernels of size (H, 1) and (1, W), thereby generating a pair of direction-aware attention feature maps. Next, in the coordinate attention generation step, the module first cascades the two feature maps generated in the previous step, and then transforms them using a 1x1 convolution, where the parameter r controls the downsampling ratio to control the size of the module. The coordinate attention feature map generated in this way can effectively capture the long-range dependencies of the input feature map along the spatial direction, thereby retaining the position information, and is multiplied with the input feature map to improve the representation ability of the feature map. Finally, after operations such as segmentation, convolution, and normalization, the final attention weights are generated.

[0107] Specifically, for input X, the output expression of the cth channel with height H is:

[0108]

[0109] A similar output expression for the cth channel with width W is:

[0110]

[0111] The final CA attention output expression is:

[0112] f=δ(F1([z H , z W ]))

[0113]

[0114] Among them, f is the feature map of spatial information in the horizontal and vertical directions, F1, F H and F W is a 1x1 convolution operation, σ is the activation function, and y is the final output.

[0115] The above parameters are explained as follows:

[0116] 1. Input X: represents the input feature map of a certain layer in the network, which contains the spatial information of the image. In the CA attention mechanism, input X will be used to generate a direction-aware attention feature map.

[0117] 2. H and W: represent the height and width of the input feature map X. In the CA mechanism, the input feature map is pooled in the horizontal and vertical directions using pooling kernels of size (H, 1) and (1, W) to encode spatial information.

[0118] 3.f: Feature maps representing spatial information in the horizontal and vertical directions, which are obtained through specific aggregation operations (such as pooling and convolution).

[0119] 4.F1,F H , and F W : These are 1x1 convolution operations that process and transform feature maps. F1 is used for the final attention weight generation, while F H and F W Used for processing horizontal and vertical feature maps respectively.

[0120] 5.y: represents the final output of the CA attention mechanism, which is obtained by applying the processed attention weights to the original input X to enhance useful features and suppress irrelevant information.

[0121] (2) Focal EIoU Loss is used instead of the original CIoU Loss as the border loss function, and the width and height difference of the border is used to replace the width and height ratio. This also solves the problem of imbalance between difficult and easy samples. In this way, the model can be better optimized and the training accuracy can be improved. The loss function is as follows:

[0122] L=IoU γ L EIoU

[0123] Among them, the parameter γ is used to control the degree of attention paid by the model to difficult and easy samples. The Focal in this loss function is somewhat different from the traditional Focal Loss. The traditional Focal Loss mainly pays attention to difficult-to-classify samples by reducing the weight of easy-to-classify samples. According to the above formula: the higher the IoU, the greater the loss, which is equivalent to weighting the predicted value, which helps to improve the regression accuracy.

[0124] (3) Introducing the BiFPN structure into the Neck network structure enables it to reduce information loss in feature fusion, thereby improving network detection accuracy and enhancing the recognition and positioning of small targets; Figure 5 This is a schematic diagram of the iFPN network structure. Compared with the traditional FPN structure, BiFPN has the following advantages: first, higher feature map quality. BiFPN combines bidirectional feature flow to better retain the spatial information of the feature map and reduce the information transmission process; second, higher computational efficiency. BiFPN adopts an efficient feature map fusion method to effectively improve the efficiency of target detection without significantly increasing the computational cost; finally, stronger scalability. BiFPN can easily adapt to input feature maps of different resolutions and feature depths, making it suitable for a variety of different target detection tasks.

[0125] Image acquisition module: collects a large amount of bronchoscope data, annotates it through professional doctors, trains the data into artificial intelligence models, and uses visualization equipment to collect images in the airway;

[0126] Recognition module: Use the trained model to perform real-time recognition of collected images.

[0127] In order to verify the performance of the improved YOLOv5 target detection algorithm on the custom data set, and analyze the experimental results. The specific experimental process includes the following steps: First, use the improved YOLOv5 target detection algorithm to train the custom bronchoscopy data set and evaluate the performance of the model. Secondly, compare and analyze the experimental results with the YOLOv5 algorithm before improvement. Finally, compare and evaluate the improved model with the mainstream target detection algorithms. In order to verify the recognition effect of the improved YOLOv5 algorithm on the custom bronchoscopy data, this experiment selected some representative samples from the test set for display, such as Figure 6As shown; the experimental results show that for normal bronchoscopic images, the improved YOLOv5 algorithm can accurately identify the left and right bronchi and the carina, and the confidence of the left and right bronchi is above 0.80, which has a high confidence; for the rotated bronchoscopic images, the improved YOLOv5 algorithm can accurately identify the rotated bronchoscopic images, especially the confidence of the left bronchus recognition reaches 0.82; for small target images, since the camera is located in the main trachea, far away from the left and right bronchi and the carina, the improved YOLOv5 algorithm can still accurately identify the positions of the left and right bronchi, and its recognition confidence is only slightly reduced; d and e are the recognition effects under two lighting environments, respectively. Since the visible double cavity needs light source illumination when shooting in the human airway, and during the shooting process, liquids such as airway secretions will cause reflections that affect the field of view, ultimately leading to the generation of strong light and dark light environments. In these two lighting environments, the improved YOLOv5 algorithm accurately identified the left and right bronchi and carina, especially in the dark light environment, where it is difficult for the naked eye to identify the left and right bronchi, but the model can still accurately locate and identify them, but the confidence level has dropped to about 0.6; f and g are target detection in complex scenes; f is due to the refraction and reflection of light on airway secretions, resulting in rainbow-like occlusion, but since the occlusion is small, it does not interfere with the recognition and positioning of the model. g is partial occlusion caused by secretions adhering to the camera, and the occlusion area of ​​this part is larger than that of f. Therefore, even if the model accurately identifies the left and right bronchi and carina, the lower part of the right bronchus is severely occluded, and the size of the border cannot be determined, making the recognition confidence level very low. This shows that the target detection confidence level using the improved YOLOv5 algorithm is high, and the recognition and positioning is also very accurate. In addition, for rotated bronchoscopic images and bronchoscopic images under special environments, the algorithm can also accurately identify the left and right bronchi;

[0128] As a specific implementation method, the model trained by the server is deployed to the local visual double-lumen device, so that it can assist the anesthesiologist in intubation during clinical intubation. In addition, the improved YOLOv5 model not only needs to be able to quickly identify and locate the left and right bronchi and carina in bronchoscope images and double-lumen intubation videos, but also needs to perform real-time identification and detection during intubation using the visual double-lumen. Therefore, a double-lumen tracheal intubation assistance system is built by combining PyQt5 with the lightweight YOLOv5 model. The system can quickly detect bronchoscope images, double-lumen intubation videos, and real-time bronchoscope images during visual double-lumen intubation, and can display the detection results on a graphical GUI interface. The real-time monitoring results are shown in the above system interface display diagram, and the real-time effect reasoning diagram is shown in the above system interface display diagram. Figure 7 shown.

[0129] As a specific implementation, a system for a double-lumen endotracheal intubation auxiliary method based on YOLOv5, the data preparation module includes:

[0130] The image collection submodule is responsible for collecting bronchoscopic image data from clinical partner hospitals;

[0131] The preprocessing submodule preprocesses the collected images, including image renumbering, cropping, and occlusion processing, to protect patient privacy and optimize subsequent processing;

[0132] In the annotation submodule, physicians with clinical experience accurately annotate the left and right bronchi and carina in the image using professional tools and in a standard format;

[0133] Model training and optimization modules include:

[0134] The neural network design submodule is used to design the improved YOLOv5 model architecture, including the customization of the backbone network and the Neck network structure;

[0135] The attention mechanism integration submodule is used to integrate the CA attention mechanism into the backbone network to improve the model's ability to locate useful feature information;

[0136] The loss function optimization submodule uses Focal EIoU Loss instead of traditional CIoU Loss to optimize the model's ability to regress bounding boxes and handle the imbalance of difficult and easy samples.

[0137] The feature fusion submodule introduces the BiFPN structure to optimize the model's information retention ability during the feature fusion process and enhance the recognition and positioning of small targets;

[0138] The image acquisition module includes:

[0139] The acquisition device configuration submodule is used to select and configure high-resolution visualization acquisition devices to ensure image quality;

[0140] The real-time acquisition submodule is used to acquire airway images in real time in clinical scenarios and provide immediate input for the recognition module;

[0141] The image recognition module includes:

[0142] The real-time processing submodule is used to process the acquired images in real time, including pre-processing operations such as resizing and normalization;

[0143] The recognition and analysis submodule is used to use the trained model to perform real-time recognition on the pre-processed image and identify the key structures in the image;

[0144] The result display module includes:

[0145] The interface design submodule is used to design an intuitive and easy-to-use result visualization interface to display the recognition results;

[0146] The interactive function submodule is used to develop interactive functions, such as image zooming, adjusting brightness, viewing historical results, etc., to improve user experience;

[0147] The result feedback submodule is used for users to provide feedback on the recognition results to further optimize the model performance.

[0148] The system interface is shown in the figure Figure 8 shown.

[0149] The above specific implementations are only descriptions of the preferred implementations of the present invention, and do not limit the protection scope of the present invention. Without departing from the design concept and spirit of the present invention, various modifications, substitutions and improvements made by ordinary technicians in this field to the technical solution of the present invention based on the text description and drawings provided by the present invention should all fall within the protection scope of the present invention. The protection scope of the present invention is determined by the claims.

Claims

1. A double-lumen endotracheal intubation auxiliary method based on YOLOv5, characterized in that: include: Step S1: Collect bronchoscopic images from clinical practice, classify and annotate the image structures, and use data augmentation methods to expand the dataset; Step S2: Use 80% of the labeled data set to train the improved YOLOv5 model, including introducing the CA attention mechanism, replacing CIoU Loss with Focal EIoU Loss as the border loss function, and introducing the BiFPN structure in the Neck part of the model to improve the accuracy and recognition ability of small objects; Step S3: Use the remaining 20% ​​of the dataset to test the trained model, and use mAP0.5 as the evaluation index to measure the performance of the model on the bronchoscopic object detection task; Step S4: collecting images in the airway and using the trained model for real-time recognition and analysis to accurately identify key structures in the bronchoscopic image; Step S5: Display the results of identification and analysis through a result visualization interface to assist doctors in making diagnosis and treatment decisions.

2. A double-lumen endotracheal intubation auxiliary method based on YOLOv5 according to claim 1, characterized in that: Step S1 is specifically as follows: Step S11: collecting bronchoscopic images of 500 patients, totaling about 3,000 images; Step S12: renumbering these images to protect patient privacy, and selecting 451 images including the left and right bronchi and carina; Step S13: Three physicians identified the left and right bronchi based on their different anatomical structures and annotated them using the LabelImg tool in the PASCAL VOC format; Step S14: perform cropping and occlusion processing on part of the original image.

3. A double-lumen endotracheal intubation auxiliary method based on YOLOv5 according to claim 1, characterized in that: Step S2 is specifically as follows: Step S21: Use 80% of the samples in the data set to train the neural network model; Step S22: Introduce the CA attention mechanism to enhance the representation ability of features through coordinate information embedding and coordinate attention generation steps; Step S23: Using Focal EIoU Loss instead of CIoU Loss as the border loss function to optimize the training efficiency and accuracy of the model; Step S24: Introducing the BiFPN structure into the Neck network structure, and improving the network detection accuracy through bidirectional feature flow and efficient feature map fusion method; Step S25: Use the remaining 20% ​​of the dataset to test the model, and use mAP0.5 as the evaluation indicator to measure the model performance.

4. A double-lumen endotracheal intubation auxiliary method based on YOLOv5 according to claim 1, characterized in that: Step S3 is as follows: Step S31: Selecting a high-resolution and high-sensitivity visualization device; Step S32: adjusting the acquisition environment to ensure that the image is clear and the details are discernible, so as to reduce the error in model recognition; Step S33: real-time acquisition of airway images in clinical anesthesia practice, and instant transmission of image data to a storage system for subsequent real-time processing and analysis; Step S34: Perform necessary preprocessing on the collected image to ensure that the image input to the recognition module is standardized.

5. A double-lumen endotracheal intubation auxiliary method based on YOLOv5 according to claim 1, characterized in that: Step S4 is specifically as follows: Step S41: when starting the system, load the trained neural network model and prepare for the image recognition task; Step S42: Process each frame of the collected image in real time to adapt to the model input requirements; Step S43: the model performs deep feature extraction on the processed image and uses the improved YOLOv5 architecture to identify key structures in the image; Step S44: Analyze the accuracy and reliability of the recognition results based on the model output to determine whether re-collection or further processing is required.

6. A double-lumen endotracheal intubation auxiliary method based on YOLOv5 according to claim 3, characterized in that: Step S22 is specifically as follows: Step S221: applying pooling kernels in different directions to the input feature map X to generate feature maps of sizes (H, 1) and (1, W) respectively; Step S222: cascade the two feature maps generated in step S221, and transform the cascaded feature maps through 1x1 convolution, where the parameter r is used to control the downsampling ratio; the generated coordinate attention feature map captures the long-range dependency of the input feature map along the spatial direction; Step S223: Multiply the coordinate attention feature map with the input feature map to generate the final attention weight to enhance the representation ability of useful features.

7. A double-lumen endotracheal intubation auxiliary method based on YOLOv5 according to claim 6, characterized in that: The formula in step S22 is as follows: For input X, the output expression of the cth channel with height H is: A similar output expression for the cth channel with width W is: The final CA attention output expression is: f=δ(F l ([z H ,z W ])) Among them, f is the feature map of spatial information in the horizontal and vertical directions, F1, F H and F W is a 1x1 convolution operation, σ is the activation function, and y is the final output.

8. A double-lumen endotracheal intubation auxiliary method based on YOLOv5 according to claim 3, characterized in that: The loss function in step S23 is as follows: L=IoU γ L EIoU Among them, IoU is an indicator to measure the degree of overlap between the predicted border and the true border. Its value ranges from 0 to 1. The higher the value, the more accurate the prediction. EIoU is an improved version of IoU, which takes into account the width and height difference of the border instead of a simple aspect ratio. The parameter γ adjusts the contribution of easy-to-classify and difficult-to-classify samples in the loss, and is used to control the model's attention to easy and difficult samples.

9. A system for a double-lumen endotracheal intubation assisting method based on YOLOv5, applicable to a double-lumen endotracheal intubation assisting method based on YOLOv5 according to any one of claims 1 to 8, characterized in that: It includes data preparation module, model training and optimization module, image acquisition module, image recognition module and result display module.

10. The system of the double-lumen endotracheal intubation auxiliary method based on YOLOv5 according to claim 9, characterized in that: The data preparation module includes: The image collection submodule is responsible for collecting bronchoscopic image data from clinical partner hospitals; The preprocessing submodule preprocesses the collected images, including image renumbering, cropping, and masking to protect patient privacy and optimize subsequent processing; In the annotation submodule, physicians with clinical experience accurately annotate the left and right bronchi and carina in the image using professional tools and in a standard format; The model training and optimization module includes: The neural network design submodule is used to design the improved YOLOv5 model architecture, including the customization of the backbone network and the Neck network structure; The attention mechanism integration submodule is used to integrate the CA attention mechanism into the backbone network to improve the model's ability to locate useful feature information; The loss function optimization submodule uses Focal EIoU Loss instead of traditional CIoU Loss to optimize the model's ability to regress bounding boxes and handle the imbalance of difficult and easy samples. The feature fusion submodule introduces the BiFPN structure to optimize the model's information retention ability during the feature fusion process and enhance the recognition and positioning of small targets; The image acquisition module comprises: The acquisition device configuration submodule is used to select and configure high-resolution visualization acquisition devices to ensure image quality; The real-time acquisition submodule is used to acquire airway images in real time in clinical scenarios and provide immediate input for the recognition module; The image recognition module comprises: The real-time processing submodule is used to process the acquired images in real time, including resizing and normalization preprocessing operations; The recognition and analysis submodule is used to use the trained model to perform real-time recognition on the pre-processed image and identify the key structures in the image; The result display module includes: The interface design submodule is used to design an intuitive and easy-to-use result visualization interface to display the recognition results; The interactive function submodule is used to develop interactive functions, such as image zooming, adjusting brightness, and viewing historical results, to improve user experience; The result feedback submodule is used for users to provide feedback on the recognition results to further optimize the model performance.

Citation Information

Cited By

  • Improved yolov8n model training method and device for surgical knotting action video recognition

    CN120126050A

  • Auxiliary image segmentation method for bronchoscopy

    CN120259344A

  • An image segmentation method for assisting bronchoscopy

    CN120259344B