Intelligent bronchial lumen part identification method, electronic equipment and storage medium

Abstract

The invention discloses an intelligent bronchus lumen part identification method, electronic equipment and a storage medium, and the method comprises the steps: collecting each bronchus lumen part image under real bronchus endoscopy, and carrying out the data labeling, data preprocessing and data division, so as to construct a bronchus lumen part identification data set; designing and constructing a feature extraction backbone network and a loss function; training the network by using the training data set to obtain a trained bronchial lumen part identification model; and classifying and identifying the acquired images in the real-time bronchial endoscopy video, and displaying an identification result. On the basis of the deep learning technology, automatic identification of the bronchus lumen part is achieved, real-time bronchoscope position information is provided for bronchoscope operators, operation errors such as false detection and missing detection caused by manual inspection are effectively relieved, the bronchoscope inspection process is standardized, the endoscopic inspection quality is improved, and the operation efficiency is improved. And the workload of operators is effectively reduced.

Description

Technical Field

[0001] This invention belongs to the field of pattern recognition and computer image processing technology, and particularly relates to a method, computer device and storage medium for intelligent bronchial lumen location recognition based on deep learning. Background Technology

[0002] Accurate diagnosis of tracheal and bronchial lesions is a significant need in respiratory clinical practice. Bronchoscopy plays a crucial role in respiratory disease screening, providing direct imaging of tracheal and bronchial lesions, thus offering vital evidence for early detection and accurate assessment. Bronchoscopy is irreplaceable in respiratory clinical diagnosis and treatment, widely used in interventions for respiratory diseases such as lung cancer, tracheobronchial tuberculosis, and chronic obstructive pulmonary disease, greatly improving the accuracy and timeliness of clinical diagnosis and treatment, and becoming an indispensable tool in respiratory medicine.

[0003] Currently, bronchoscopy faces numerous challenges in clinical application, primarily including a relative scarcity of specialized physicians, uneven distribution of advanced bronchoscopic equipment, and quality differences among different examination units. Furthermore, the entire examination process heavily relies on the operator's clinical experience and skills. Due to the complexity of bronchial anatomy and the high similarity of images of different bronchial levels, errors in bronchoscope placement are prone to occur, leading to the omission of certain bronchial segments or airway cavities, ultimately seriously affecting patient safety and diagnostic accuracy. Therefore, improving the precision of bronchoscopy, reducing human error, ensuring comprehensive examination, and enhancing examination quality have become urgent problems to be solved.

[0004] With the rapid development of deep learning technology, artificial intelligence technology has demonstrated performance comparable to that of endoscopy experts in fields such as medical image analysis, diagnosis of related diseases, lesion detection, and quality control of endoscopic procedures, showing significant application potential, especially in solving key problems existing in current clinical work of respiratory endoscopy.

[0005] However, current deep learning algorithms for automatic bronchial lumen recognition mostly use general target recognition algorithms (such as VGG, GoogleNet, ResNet, etc.), without conducting specific research and adaptation improvements for the bronchus as a specific target. Their accuracy and efficiency are both insufficient. For example, existing algorithms cannot effectively address the fine-grained requirements of bronchial lumen recognition, and the convolutional neural networks used are inefficient in terms of convolution operations. Furthermore, existing algorithms lack effective training strategies to address the small amount of bronchial training data. In addition, most existing algorithms use conventional classification losses, lacking improvements to address the problem of "small inter-class distance and large intra-class distance" in bronchial lumen recognition scenarios. Therefore, how to use more advanced deep learning technologies for targeted improvements is a current challenge in the intelligent bronchial lumen recognition industry. Based on the above analysis, the problems and shortcomings of existing technologies are as follows: (1) The current methods for identifying the lumen of artificial bronchus have low automation and rely heavily on doctors' experience, resulting in poor operation standards, many blind spots in the examination, and low accuracy. (2) Existing automatic bronchial lumen identification algorithms are not accurate and efficient, and lack consideration for fine-grained identification tasks and efficiency of convolution design. (3) The existing automatic bronchial lumen recognition algorithm has insufficient loss function design and lacks consideration for the problem of "small inter-class distance and large intra-class distance" in the bronchial lumen recognition scenario; (4) Existing automatic bronchial lumen identification algorithms lack solutions to the problem of small training data and have insufficient training strategies. Summary of the Invention

[0006] To address the problems existing in the prior art, this invention provides an intelligent bronchial lumen location identification method, electronic device, and storage medium to achieve high-precision and high-efficiency automated identification of bronchial lumen images and output the identification results in real time.

[0007] This invention is implemented as follows: a method for intelligent identification of bronchial lumen locations, comprising the following steps: Step 1: Collect real, reliable images of various bronchial lumen locations under bronchial endoscopy with sample diversity, and perform standardized data annotation. Step 2 involves preprocessing and dividing the images of the bronchial lumen into data, including cropping, size normalization, and data amplification, to construct a dataset for bronchial lumen identification. Step 3: Based on depthwise separable convolution, attention mechanism, and additive angular margin, design and construct the bronchial feature extraction backbone network and loss function. Specifically, this step combines the ECA efficient channel attention mechanism module to adaptively adjust the weight of each channel, and integrates the SimAM attention module to improve the network's adaptive perception of the importance of spatial regions in bronchial lumen images through parameterless optimization. Combined with a specific loss function ArcLoss, it can efficiently extract key features in bronchial lumen images and improve the distinguishability between different types of bronchial lumen images. Step 4: Train the entire network using the training set to obtain the trained bronchial lumen location recognition network model. Step 5: Using the obtained model, classify and identify the acquired images in the real-time bronchial endoscopy video, and display the identification results to assist doctors in achieving accurate bronchial lumen localization and diagnosis.

[0008] According to this invention, by employing a deep learning-based automatic bronchial lumen identification method, high-precision algorithms can analyze bronchoscopic images in real time to accurately identify the current bronchial position of the lens, thereby achieving precise positioning and navigation during the bronchoscopy procedure. This technology not only effectively assists in standardizing the bronchoscopy procedure and reducing reliance on physicians' clinical experience, but also significantly reduces the workload of physicians and improves diagnostic and treatment efficiency. Furthermore, the application of this technology can greatly improve the quality of bronchoscopic examinations, optimize the safety and accuracy of the examination procedure, and provide patients with more accurate and efficient diagnostic support.

[0009] Furthermore, in step one, a bronchial endoscope is used to capture real video recordings of bronchial examinations, and patient privacy information is removed. Images of the bronchial lumen are then extracted at a certain frame rate and labeled by multiple professional physicians, categorizing the lumen images into specific classes to form a raw dataset of bronchial lumen images, including: First, to increase data diversity, various types of bronchial endoscopes were used to collect real bronchial examination videos from patients with different genders, ages, heights, and weights, and patient privacy information was removed. Then, image frames were extracted at a sampling frequency of n frames per second (e.g., n=3), and bronchial lumen images corresponding to 31 bronchial lumen categories were extracted. Next, images containing clear bronchial lumen structures were selected, and images that could not be accurately identified due to image quality issues or lesions affecting the identification of bronchial lumen sites were removed. Furthermore, for each selected bronchial lumen image, at least three qualified bronchial endoscopy operators were required to label and classify it. Only when at least two-thirds of the doctors' labels were consistent or reached a consensus would the image be included in the specific category dataset to ensure that the generated dataset has high-quality annotation information.

[0010] The 31 specific bronchial lumens involved are categorized from 1 to 31 as follows: glottis, trachea, carina, left main bronchus, left upper lobe, proper bronchus of left upper lobe, apical posterior segment of proper bronchus of left upper lobe, anterior segment of proper bronchus of left upper lobe, lingular segment of left upper lobe, superior lingular segment of left upper lobe, inferior lingular segment of left upper lobe, left lower lobe, dorsal segment of left lower lobe, anterior basal segment of left lower lobe, lateral basal segment of left lower lobe, posterior basal segment of left lower lobe, right main bronchus, right upper lobe, apical segment of right upper lobe, posterior segment of right upper lobe, anterior segment of right upper lobe, middle segment of right lung, middle lobe of right lung, lateral segment of right middle lobe, medial segment of right middle lobe, right lower lobe, dorsal segment of right lower lobe, medial basal segment of right lower lobe, anterior basal segment of right lower lobe, lateral basal segment of right lower lobe, and posterior basal segment of right lower lobe.

[0011] Furthermore, the data preprocessing in step two includes cropping, standardizing image sizes, and data augmentation to increase data diversity, constructing a bronchial lumen site identification dataset, specifically including: For each bronchial lumen image included in Step 1, to improve data quality and model training effectiveness, a black border removal algorithm was used to crop each image, retaining only the square region containing the bronchoscope's field of view. This is the core region, ensuring that the extracted image region accurately reflects the actual structural features of the bronchial lumen. All images were then uniformly resized to a standard size, typically 1000 pixels × 1000 pixels. Simultaneously, the sample size for each category was balanced to maintain a relatively uniform sample size across categories, ensuring representativeness and fairness in the training process.

[0012] Collecting real bronchial lumen images is difficult and limited in quantity. Before loading the training set data into the neural network model for training, random data augmentation is required to increase data diversity. Compared with traditional data augmentation, this method can significantly save storage space and effectively increase data diversity, thus avoiding overfitting during model training to some extent. Simulating actual bronchoscopy, the bronchoscope lens typically rotates and shifts continuously. Therefore, random data augmentation includes a combination of random rotation, translation, and scaling of the bronchial lumen images. Furthermore, different brands of bronchoscopes vary in image brightness and saturation; random data augmentation also adjusts the brightness and saturation of the images randomly. Simultaneously, Gaussian blur data augmentation is used to simulate motion blur caused by the endoscope moving inside the human bronchus.

[0013] Based on this, the dataset is further divided into training, validation, and test sets in a ratio (typically 8:1:1). The training set is used for model learning and training, the validation set is used for hyperparameter tuning and performance evaluation, and the test set is used for final model evaluation and validation to ensure the scientific nature of the training process and the reliability of the model's generalization ability.

[0014] Furthermore, in step three, a bronchial feature extraction backbone network is designed and constructed based on the characteristics of the bronchial lumen image to extract bronchial lumen image features, including: The bronchial feature extraction backbone network constructed in this invention is built using ConvNeXtBlock modules containing depthwise separable convolutions as its core components. The complete bronchial feature extraction backbone network includes: a preconvolutional layer, several stacked ConvNeXt Block modules (this invention uses a stacking configuration of (3, 3, 9, 3)), a downsampling layer, a global average pooling layer, a normalization layer, a fully connected layer, and a Softmax layer. Each ConvNeXt Block module is further composed of depthwise separable convolutional layers, batch normalization layers, GELU activation function layers, and linear layers.

[0015] To further enhance the information representation capability during feature extraction, this invention embeds an efficient channel attention module (ECA) and a parameter-free spatial attention module (SimAM) into the ConvNext Block module. This helps the designed network model focus more on important information in the image, improving the network model's performance and efficiency (for distinction, this invention names the modified ConvNeXt Block as ECA-SimAM ConvNeXt Block). The ECA module adaptively adjusts the weights of each channel, thereby strengthening the focus on key information channels. The SimAM module, through parameter-free optimization, enhances the network's adaptive perception of the importance of spatial regions, especially in key areas of bronchial anatomy, thus effectively enhancing the network's local region focusing capability.

[0016] The SimAM module mimics biological neural mechanisms to uncover the importance of neurons. Unlike traditional spatial or channel attention modules, it is a simple and efficient parameter-free attention mechanism. SimAM considers the importance of both channel and spatial dimensions simultaneously, generating 3D attention weights for feature maps without introducing additional parameters. This allows for a comprehensive and efficient evaluation of feature weights, thereby improving the performance of the base model. The SimAM attention mechanism is embedded within the ConvNeXtBlock module; specifically, it is added after the first depthwise separable convolutional layer within each ConvNeXt Block module.

[0017] The ECA attention module allows the network to adaptively learn the importance of each channel, thereby improving the representation ability of features while maintaining good computational efficiency. To enhance the model's attention to key channels, avoid excessive model complexity, and further improve the model's classification performance, the ECA attention mechanism is embedded in the ConvNeXt Block module. Specifically, the ECA module is added after the second 1×1 convolutional layer inside each ConvNeXt Block module.

[0018] After extracting the backbone network, this invention introduces the ArcLoss loss function based on additive angular margins as the loss function. By optimizing the intra-class aggregation and inter-class discriminability of features, it significantly improves the distinguishability between bronchial lumen images of different categories. The ArcLoss function effectively addresses the challenge of small differences between different bronchial lumen categories by strengthening intra-class compactness and increasing inter-class angular margins, thereby improving the model's recognition performance in fine-grained classification tasks. The calculation formula for the ArcLoss loss function is as follows: In the formula, Indicates the number of samples. Indicates the first The true label of each sample; It is the first The angle corresponding to the true category of each sample It is the first Individual samples and other categories Angle; This is the angular interval, used to increase the angular distance between the correct and incorrect categories; This is a scaling factor used to amplify the cosine value of the angle.

[0019] Furthermore, in step four, the constructed training dataset is used to train the overall network model, resulting in a trained bronchial lumen location recognition network model; including: The bronchial lumen location recognition dataset constructed in step two is used as input to the network for learning. The training dataset is used to train the model and determine parameters such as weights and biases. The validation dataset is used for model optimization. During training, the model's state and convergence are examined, and suitable hyperparameters are selected. The test set is used for model evaluation and application after training. The designed feature extraction network model is pre-trained using the large benchmark dataset ImageNet-21k, and the pre-trained model parameters are used as initial weights to initialize the model. Then, the pre-trained model is transferred to the bronchial lumen recognition training dataset for transfer learning. Existing deep network learning experience is transferred to accelerate the model's convergence process, speeding up convergence in the early stages of training. Then, a transfer learning strategy is used, with the training set of the constructed bronchial lumen location recognition dataset as the model's input data for network training. The validation set is used for hyperparameter selection and model optimization, and finally, the test set is used for final model evaluation and performance validation, resulting in the final bronchial lumen location recognition network model. In the backpropagation of the model, the PyTorch deep learning framework is used for automatic differentiation calculation to update the model weights using stochastic batch gradient descent (SGD). Specifically, the AdamW optimizer with decoupled weight decay is used during training to ensure the stability of parameter updates and effectively avoid overfitting. To further optimize the training effect, the MultiStepLR learning rate adjustment strategy is used to optimize the learning rate. The learning rate is gradually reduced according to the preset learning rate change nodes, thereby improving the convergence efficiency and stability during training. For example, the initial learning rate is set to 4e-3, the milestones are set to [30, 60, 90], and the scaling factor gamma is set to 0.1, so that the learning rate decreases in subsequent convergence processes. In addition, to alleviate the overfitting phenomenon during model training, the Drop-path regularization method is added to alleviate overfitting, with a drop probability set to 0.2. This method enhances the generalization ability of the network by randomly discarding some paths in the network. After the model training is completed, the model with the highest accuracy on the validation set is selected as the bronchial lumen site recognition network model.

[0020] Furthermore, step five involves classifying and recognizing the images acquired in real-time during the bronchial endoscopy examination video, and outputting the recognition results, including: First, in a real-world bronchoscopy video stream, each frame is automatically extracted in real time and input into a preprocessing module. In this module, image cropping algorithms, such as black border removal, are used to precisely extract the region containing only the bronchial lumen, removing background and irrelevant parts. This ensures that subsequent analysis focuses on key structures within the bronchoscopic field of view, and on the relevance and validity of the image content. Next, the cropped image is resized to fit the input requirements of the neural network, typically 224×224 pixels, to meet the input requirements of the bronchial lumen site recognition network model and ensure the input image conforms to network processing standards. To ensure the stability and efficiency of model training, the image data is standardized to a uniform numerical range, and the original RGB images are converted into tensor format, which meets the computational requirements of the neural network, adapting to the deep learning framework. After this preprocessing step, the processed image is input into the deployed deep neural network model for feature extraction and predictive classification. The model extracts features and analyzes them for classification and prediction, identifying any bronchial lumen region in the image in real time. It then generates and outputs the predicted location of the bronchoscope. Finally, this prediction is sent to the designed software front-end for display. The displayed image includes both the original bronchoscope image and textual and graphical representations of the lumen location, providing real-time position monitoring and feedback. This process, combined with real-time bronchoscope examination video, effectively marks and displays the bronchial lumen location, assisting doctors in monitoring the specific position of the bronchoscope within the bronchi. This, in turn, helps doctors make more accurate diagnostic and treatment decisions, improving diagnostic accuracy and safety.

[0021] Another object of the present invention is to provide an electronic device that applies the intelligent bronchial lumen location identification method described above. The intelligent bronchial lumen location identification electronic device includes a bronchial image acquisition device, a display, a graphics processor, a communication interface, a memory, a central processing unit, and a communication bus.

[0022] The bronchial image acquisition device, display, graphics processor, communication interface, memory, and central processing unit communicate with each other through the communication bus. The bronchial image acquisition device is used to acquire images of the bronchial lumen in real time; The display is used to show the real-time results of bronchial lumen location identification; The graphics processor is used for inference calculations in the neural network; The memory is used to store computer programs; The central processing unit is used to execute the computer program stored in the memory to implement the intelligent bronchial lumen location identification method.

[0023] Another object of the present invention is to provide an electronic device comprising a memory and a processor, the memory storing a computer program, which, when executed by the processor, causes the processor to perform the following steps: Images of various bronchial lumen locations were acquired during real-world bronchoscopic examinations, ensuring comprehensive data coverage of 31 different anatomical structures. Next, the acquired bronchial lumen images underwent data preprocessing, including cropping and resizing, to improve image quality and eliminate external interference, constructing a bronchial lumen location recognition dataset to provide a high-quality data foundation for subsequent model training. Subsequently, a feature extraction backbone network was designed and constructed, optimizing the feature extraction process based on salient features of the bronchial lumen images to improve recognition accuracy. Based on the aforementioned dataset, deep learning methods were used to train the overall network model, resulting in an efficient and accurate bronchial lumen location recognition network model. Finally, the trained model was applied to real-time bronchoscopic examination videos, automatically classifying and recognizing real-time acquired bronchial images, and visualizing the recognition results. This achieves intelligent bronchial lumen location recognition, assisting physicians in improving diagnostic accuracy and efficiency.

[0024] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the following steps: Images of various bronchial lumen locations were acquired during real-world bronchoscopic examinations, ensuring comprehensive data coverage of 31 different anatomical structures. Next, the acquired bronchial lumen images underwent data preprocessing, including cropping and resizing, to improve image quality and eliminate external interference, constructing a bronchial lumen location recognition dataset to provide a high-quality data foundation for subsequent model training. Subsequently, a feature extraction backbone network was designed and constructed, optimizing the feature extraction process based on salient features of the bronchial lumen images to improve recognition accuracy. Based on the aforementioned dataset, deep learning methods were used to train the overall network model, resulting in an efficient and accurate bronchial lumen location recognition network model. Finally, the trained model was applied to real-time bronchoscopic examination videos, automatically classifying and recognizing real-time acquired bronchial images, and visualizing the recognition results. This achieves intelligent bronchial lumen location recognition, assisting physicians in improving diagnostic accuracy and efficiency.

[0025] Combining all the above technical solutions, the advantages and positive effects of this invention are as follows: This invention proposes a complete deep learning-based bronchial lumen location recognition technology. A novel bronchial feature extraction network structure is designed based on the ConvNeXt Block module. By adding ECA attention and SimAM attention modules, the model has stronger learning capabilities, focusing more on important regions in bronchial lumen images, thus improving model performance and efficiency. Furthermore, the Arcloss loss function is used to normalize the feature vector output by the network, and additive angular margin is introduced to further optimize intra-class compactness and inter-class variability, enabling the model to more accurately distinguish different categories of bronchial lumen images. This technology enables accurate real-time monitoring of the bronchoscope's position within the bronchus, providing effective quality control support for bronchial endoscopy. With this technology, doctors can obtain the lens's position information in real time through the terminal screen when operating a bronchoscope equipped with this technology, effectively avoiding blind spots, improving the accuracy and safety of bronchoscopy, optimizing the clinical diagnostic process, and enhancing examination efficiency and quality. Attached Figure Description

[0026] Figure 1 This is a flowchart of the intelligent bronchial lumen location identification method provided in the embodiments of the present invention.

[0027] Figure 2 This is a schematic diagram of the overall framework for intelligent bronchial lumen location identification provided in an embodiment of the present invention.

[0028] Figure 3 This is a schematic diagram of actual bronchial endoscopy examination image data provided in the embodiments of the present invention.

[0029] Figure 4 This is a schematic diagram of various bronchial lumen images after preprocessing, provided in an embodiment of the present invention.

[0030] Figure 5 This is a schematic diagram of bronchial lumen image data amplification provided in an embodiment of the present invention.

[0031] Figure 6 This is a schematic diagram of the bronchial feature extraction backbone network structure provided in an embodiment of the present invention.

[0032] Figure 7 This is a schematic diagram of the ECA-SimAM ConvNeXt Block module structure provided in an embodiment of the present invention.

[0033] Figure 8 This is a schematic diagram of the SimAM attention module structure provided in an embodiment of the present invention.

[0034] Figure 9This is a schematic diagram of the ECA attention module structure provided in an embodiment of the present invention.

[0035] Figure 10 This is a schematic diagram of the structure of the intelligent bronchial lumen location identification electronic device provided in an embodiment of the present invention.

[0036] Figure 11 This is a schematic diagram of the structure of a computer-readable storage medium for intelligent bronchial lumen location identification provided in an embodiment of the present invention. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0038] To address the problems existing in the prior art, the present invention provides a method for intelligent identification of bronchial lumen locations based on deep learning, an electronic device, and a storage medium. The present invention will be described in detail below with reference to the accompanying drawings.

[0039] like Figure 1 As shown, the intelligent bronchial lumen location identification method provided in this embodiment of the invention includes the following steps: S101: Collect real, reliable images of various bronchial lumen locations under bronchial endoscopy with diverse sample data, and perform standardized annotation. S102, perform data preprocessing and data segmentation on bronchial lumen images by cropping, size normalization, and data amplification, and construct a bronchial lumen location recognition dataset; S103, based on ConvNeXt Block, ECA, SimAM and Arcloss, constructs a bronchial feature extraction backbone network and loss function; S104, The entire network is trained using a transfer learning strategy and training data to obtain a trained bronchial lumen location recognition network model; S105 classifies and identifies images acquired in real time during bronchial endoscopy examinations and displays the recognition results in text and graphic formats.

[0040] This invention is based on the ConvNeXt Block module, an efficient convolution operation module, and introduces the ECA efficient channel attention module and the SimAM parameterless spatial attention module. It designs an additive angular margin loss function ArcLoss and combines data augmentation and MultiStepLR training strategies to specifically address the shortcomings of existing deep learning algorithms in the field of automatic bronchial lumen identification, and designs a high-precision and high-efficiency intelligent bronchial lumen identification method.

[0041] Based on the above process, this invention can analyze bronchoscopic images in real time using a high-precision algorithm to accurately identify the current bronchial position of the lens, thereby achieving precise positioning and navigation during the bronchoscopy procedure. This technology not only effectively assists in standardizing the bronchoscopy procedure and reducing reliance on doctors' clinical experience, but also significantly reduces doctors' workload and improves diagnostic and treatment efficiency. Furthermore, the application of this technology can greatly improve the quality of bronchoscopic examinations, providing patients with more accurate and efficient diagnostic services.

[0042] The technical solution of the present invention will be further described below with reference to specific embodiments.

[0043] Example 1: Accurate diagnosis and identification of tracheal and bronchial lesions has always been a major challenge in respiratory clinical practice. Bronchoscopy, through high-resolution imaging, can clearly present the lesion status of the trachea and bronchi, providing intuitive imaging support, thus providing important evidence for the early detection and accurate assessment of lesions, and playing an important role in respiratory disease screening.

[0044] The clinical application of bronchoscopy currently faces numerous challenges, including a relative shortage of qualified physicians, uneven geographical distribution of advanced bronchoscopic equipment and techniques, and inconsistent quality of bronchoscopic examinations. This procedure heavily relies on the operator's clinical experience and skills, especially when dealing with the complex anatomical structure of the bronchi, where the physician's ability to identify the different levels of bronchi and their image features is crucial. Due to the high morphological similarity of the bronchial lumens at each level, the precise positioning of the bronchoscope is often difficult to determine, leading to the omission or misidentification of some bronchial segments. This results in incomplete airway examinations, missed lesions, and other serious problems, affecting diagnostic accuracy and treatment outcomes. These issues not only increase the diagnostic risks for patients but also exacerbate the uneven distribution of medical resources and the disparities in medical quality.

[0045] With the rapid development of deep learning technology, deep learning algorithms have demonstrated performance comparable to that of endoscopic experts in disease diagnosis, lesion detection, and quality control of endoscopic procedures, and hold promise for solving key problems in current clinical respiratory endoscopy. Therefore, researching how to use deep neural networks to efficiently and accurately identify images of various bronchial lumen locations not only provides a new solution and approach for quality control in bronchoscopy but also expands the application of deep learning. Therefore, this invention utilizes deep learning technology combined with attention mechanisms to design an intelligent method for identifying bronchial lumen locations.

[0046] This invention provides a method for intelligent identification of bronchial lumen locations. Figure 2This is a schematic diagram of the overall framework of the intelligent bronchial lumen location identification method provided in an embodiment of the present invention. The method specifically includes the following five steps: Step one: To increase data diversity, various types of bronchoscopes were used to capture real bronchial examination videos of patients with different genders, ages, heights, and weights undergoing bronchial endoscopy, and patient privacy information was removed. Bronchial lumen images were periodically cropped at a set frequency of 3 frames per second, filtering out images with unclear lumens or obvious lesions. The cropped images were labeled by multiple experienced physicians based on their anatomical and morphological characteristics. For each bronchial lumen image, it was labeled and classified by three or more professional bronchoscope operators. The image was included in a specific category dataset only after at least two-thirds of the physicians agreed (e.g., at least two out of three physicians agreed). Each frame of the bronchial lumen in the image was precisely classified into its corresponding category. Each image in the image dataset has high annotation accuracy, ensuring its reliability and accuracy in medical applications. Real bronchial lumen images under bronchial endoscopy are shown below. Figure 3 As shown.

[0047] The 31 specific bronchial lumens involved are categorized from 1 to 31 as follows: glottis, trachea, carina, left main bronchus, left upper lobe, proper bronchus of left upper lobe, apical posterior segment of proper bronchus of left upper lobe, anterior segment of proper bronchus of left upper lobe, lingular segment of left upper lobe, superior lingular segment of left upper lobe, inferior lingular segment of left upper lobe, left lower lobe, dorsal segment of left lower lobe, anterior basal segment of left lower lobe, lateral basal segment of left lower lobe, posterior basal segment of left lower lobe, right main bronchus, right upper lobe, apical segment of right upper lobe, posterior segment of right upper lobe, anterior segment of right upper lobe, middle segment of right lung, middle lobe of right lung, lateral segment of right middle lobe, medial segment of right middle lobe, right lower lobe, dorsal segment of right lower lobe, medial basal segment of right lower lobe, anterior basal segment of right lower lobe, lateral basal segment of right lower lobe, and posterior basal segment of right lower lobe.

[0048] Step 2: Perform data preprocessing on the bronchial lumen images to construct a bronchial lumen location recognition dataset.

[0049] Specifically, in step one, for each bronchial lumen image included in each category, it is first cropped using a black border removal algorithm, retaining only the core area containing the bronchoscopic view, and then adjusted to a uniform 1000 pixel × 1000 pixel size to ensure image standardization and consistency. To avoid data bias and improve the model's generalization ability, a data balancing strategy is adopted to ensure a relatively balanced number of samples in each category. To effectively evaluate model performance and ensure full utilization of the dataset, the overall dataset is randomly divided into training, validation, and test sets in an 8:1:1 ratio. After the above processing, the various bronchial lumen images present a standardized image style while ensuring annotation accuracy and sample diversity, such as... Figure 4 As shown.

[0050] Collecting real bronchial lumen images is difficult and limited in quantity. Random data augmentation is necessary before loading the training data into the neural network model for training. Compared to traditional data augmentation, this method significantly saves storage space and effectively increases data diversity, thus avoiding overfitting during model training to some extent. In actual bronchoscopy, the bronchoscope lens is constantly rotating and shifting. Therefore, random data augmentation includes a combination of random rotation, translation, and scaling of the bronchial lumen image. The random rotation angle ranges from -90° to 90°, the random scaling range is [0.75, 1.25], and the random translation scale is [0.12, 0.12]. Different brands of bronchoscopes vary in image brightness and saturation; therefore, random data augmentation also performs random brightness and saturation adjustments. The random brightness adjustment range is [0.7, 1.3], and the random contrast variation range is [0.7, 1.5]. Simultaneously, Gaussian blur data augmentation was used to simulate motion blur caused by the movement of an endoscope inside the human bronchus, with the standard deviation of the Gaussian distribution set to a range of (5, 20). For a single bronchial lumen image, the bronchial lumen image after random data augmentation is shown below. Figure 5 As shown.

[0051] Step 3: Design and construct the bronchial feature extraction backbone network and loss function.

[0052] The bronchial feature extraction backbone network constructed in this invention uses ConvNeXtBlock containing depthwise separable convolutions as its core component. The processing flow of the complete backbone network is as follows: First, a pre-convolutional layer (Conv 2d) with a 4×4 convolutional kernel and a stride of 4 is passed. Then, a layer normalization layer is used for data normalization. Next, several ConvNext Block modules are stacked (this invention uses a stacking setting of (3, 3, 9, 3)) and a downsampling layer are passed. Then, a global average pooling layer (GAP) is used to integrate the feature maps. Finally, a normalization layer (Layer Normal), a fully connected layer (Linear), and a Softmax layer are used to output the classification results of 31 classes. The specific structure of the bronchial feature extraction backbone network is as follows: Figure 6 As shown.

[0053] Each ConvNext Block module is further composed of depthwise separable convolutional layers (DepthwiseConv2d), batch normalization layers (Layer Norm), GELU activation function layers, and linear layers.

[0054] To further enhance the information representation capability during feature extraction, this invention embeds an efficient channel attention module (ECA) and a parameter-free spatial attention module (SimAM) into the ConvNext Block module. This helps the designed network model focus more on important information in the image, improving the performance and efficiency of the network model. (The modified ConvNeXt Block is named ECA-SimAM ConvNeXt Block, as shown below.) Figure 7 (As shown). Among them, the ECA module can adaptively adjust the weight of each channel, thereby strengthening the focus on key information channels, while the SimAM module improves the network's adaptive perception of the importance of spatial regions through parameterless optimization, especially in key areas of bronchial lumen anatomy, thereby effectively enhancing the network's local region focusing ability.

[0055] The SimAM module mimics biological neural mechanisms to uncover the importance of neurons. Unlike traditional spatial or channel attention modules, it offers a simple and efficient parameter-free attention mechanism. SimAM considers the importance of both channel and spatial dimensions simultaneously, generating 3D attention weights for feature maps without introducing additional parameters. This comprehensive and efficient evaluation of feature weights improves the performance of the underlying model.

[0056] The SimAM attention mechanism is embedded into the ConvNeXt Block module. Specifically, the SimAM module is added after the first depthwise separable convolutional layer. The SimAM attention module defines the following energy function, calculating the weights based on the relationship between the activation level of each neuron and the inhibition of surrounding neurons: Among them, the energy function Neuron t The minimum energy, This is the regularization coefficient. and These represent the mean and standard deviation for each channel.

[0057] Calculation yields ⊙ ; in, This represents the output feature map. Represents the input feature map, It is the energy function of each channel. The set of elements, where sigmoid is the activation function, and ⊙ represents element-wise multiplication. The SimAM module structure is as follows: Figure 8 As shown.

[0058] The ECA attention module allows the network to adaptively learn the importance of each channel, thereby improving feature representation while maintaining good computational efficiency. To enhance the model's attention to key channels, avoid excessive model complexity, and further improve the model's classification performance, the ECA attention mechanism is embedded in the ConvNeXt Block module.

[0059] Specifically, the ECA module is added after the second 1×1 convolutional layer. The ECA module takes the input 3D feature map with dimensions H×W×C and compresses it into a 1×1×C shape using global average pooling (GAP). C The one-dimensional vector is considered as a global feature representation for each channel, and the size of the convolution kernel is . k The value of the vector is adaptively determined by the number of channels C. Next, the vector is processed through a one-dimensional convolutional layer to obtain the calculated channel attention weights. These weights are then non-linearly mapped using the Sigmoid activation function and multiplied channel-by-channel with the original feature map to finally obtain the enhanced bronchial lumen feature representation. The ECA module structure is as follows: Figure 9 As shown.

[0060] After extracting the backbone network, this invention introduces the ArcLoss loss function based on additive angular margins as the loss function. By optimizing the intra-class aggregation and inter-class discriminability of features, it significantly improves the distinguishability between bronchial lumen images of different categories. The ArcLoss function effectively addresses the challenge of small differences between different bronchial lumen categories by strengthening intra-class compactness and increasing inter-class angular margins, thereby improving the model's recognition performance in fine-grained classification tasks. The calculation formula for the ArcLoss loss function is as follows: In the formula, N Indicates the number of samples. Indicates the first i The true label of each sample; It is the angle corresponding to the true category of the i-th sample. It is the first i Individual samples and other categories j Angle; m The angle interval is used to increase the correct class step 4. The training data is used to train the overall network to obtain the trained bronchial lumen location recognition network model.

[0061] Specifically, the bronchial lumen identification dataset constructed in step two is input into the network for learning. The training dataset is used to train the model, determine parameters such as weights and biases, and the validation dataset is used for model optimization, checking the model's state and convergence during training, and selecting suitable hyperparameters. The test set is used for model evaluation and application after training. The designed feature extraction network model is pre-trained using the large benchmark dataset ImageNet-21k, and the pre-trained weights are used to initialize the model. Then, the pre-trained model is transferred to the bronchial lumen identification training dataset for transfer learning. In the model's backpropagation, the PyTorch deep learning framework is used for automatic differentiation calculation to update the model weights using stochastic batch gradient descent (SGD). The AdamW optimizer with decoupled weight decay is used, and the learning rate is optimized using the MultiStepLR learning rate adjustment strategy. The initial learning rate is set to 4e-3, the milestones are set to [30, 60, 90], and the scaling factor gamma is set to 0.1, causing the learning rate to decrease during subsequent convergence. To mitigate overfitting, a drop-path regularization method was incorporated, with the drop probability set to 0.2. After training, the model with the highest accuracy on the validation set was selected as the bronchial lumen site recognition network model.

[0062] Step 5: Classify and identify the images acquired in real time from the bronchial endoscopy video, and output and display the identification results.

[0063] Specifically, in real bronchoscopy video sources, the system automatically extracts each frame of image and inputs it into the preprocessing module. First, image processing algorithms are used to crop the image, precisely extracting only the region containing the bronchial lumen, removing background and irrelevant parts to ensure that subsequent analysis focuses on key structures within the bronchoscope's field of view. Next, the cropped image is resized to a uniform 224×224 to meet the input requirements of the bronchial lumen location recognition network model. To ensure the stability and efficiency of model training, the image data is standardized, adjusted to a uniform numerical range, and the original RGB images are converted into tensor format, which meets the computational requirements of neural networks, adapting to the deep learning framework. After this preprocessing step, the processed image is input into the built neural network for feature extraction and predictive classification. The network analyzes the feature information in the image to generate a real-time prediction of the bronchoscope's location and dynamically feeds this result back to the video feed, providing real-time location monitoring and feedback. This process can effectively assist doctors in real-time monitoring of the positioning of the bronchoscope in the bronchus, improving the accuracy and safety of diagnosis.

[0064] This embodiment of the experiment demonstrates the following aspects: To demonstrate the effectiveness of the method of this invention, two sets of experiments were conducted. The first set compared the recognition accuracy of the method of this invention with other similar methods in the published literature. Please refer to Table 1. The results clearly show that the four indicators of Accuracy, macro-P, macro-R, and macro-F1 of the method of this invention are superior to all other deep learning methods, such as VGG-16, GoogleNet, ResNet-50, MobileNet_v3, EfficientNet-b0, ViT-B, Swim-T, and ConvNeXt. The experimental results prove that the use of the method of this invention can significantly improve the automatic recognition effect of bronchial lumen sites.

[0065] The second set of experiments consisted of ablation experiments on the techniques employed in this invention, including data augmentation, ECA, SimAM, and the ArcLoss loss function. As shown in Table 2, after introducing techniques such as data augmentation, ECA, SimAM, and the ArcLoss loss function, the four indicators of the method—Accuracy, macro-P, macro-R, and macro-F1—were all improved, thus demonstrating the effectiveness of the techniques employed in this invention.

[0066] Table 1. Accuracy comparison between the present invention and other similar methods in published literature. Table 2. Effects of the techniques employed in this invention, including data augmentation, ECA, SimAM, and ArcLoss loss function. In summary, this invention provides an intelligent method for identifying bronchial lumen locations. By improving the model's network structure and adding ECA and SimAM attention modules, the model's learning ability is enhanced, significantly improving its feature extraction and learning capabilities. This allows for more precise focusing on key regions in bronchial lumen images, improving the perception of important features. Data augmentation further mitigates the limited amount of data available for bronchial endoscopy images. Simultaneously, the ArcLoss loss function optimizes the intra-class compactness and inter-class variability of bronchial lumen image features, thereby improving the recognition accuracy and classification precision for different image categories. This technology not only automates quality control during bronchoscopy but also effectively improves the diagnostic performance of bronchial endoscopy in practical operation. When using a bronchoscope integrated with this technology, doctors can obtain the precise position information of the lens in real time and intuitively through the terminal screen, which greatly reduces blind spots during the examination, improves the overall level of bronchoscopy, and reduces the workload of doctors during the operation. It has demonstrated significant clinical value and ease of operation in practical applications.

[0067] Example 2: Based on the above embodiment one, please refer to Figure 10 , Figure 10 This is a schematic diagram of the structure of an intelligent bronchial lumen location identification electronic device provided in an embodiment of the present invention. This embodiment provides an intelligent bronchial lumen location identification electronic device, which includes a bronchial image acquisition device, a display, a graphics processor, a communication interface, a memory, a central processing unit, and a communication bus. The bronchial image acquisition device, display, graphics processor, communication interface, memory, and central processing unit communicate with each other via the communication bus. The bronchial image acquisition device is used to acquire images of the bronchial lumen in real time; The display is used to show the real-time results of bronchial lumen location identification; The graphics processor is used for inference calculations in the neural network; The memory is used to store computer programs; When the central processing unit executes the computer program stored in the memory, the computer program, when executed by the processor, performs the following steps: Step 1: Open the bronchial image acquisition device to acquire real-time video during the bronchial endoscopy examination. After removing the patient's privacy information, extract the bronchial lumen image at a certain frame rate to form the original dataset of bronchial lumen images.

[0068] Specifically, in step 1 of this embodiment, a bronchial examination video is collected using a bronchial image acquisition device, after removing patient privacy information. The video is divided into image frames at a sampling frequency of 3 frames per second. Black border removal is performed to extract corresponding bronchial lumen images, excluding images with unclear lumens or obvious lumens lesions. Each bronchial lumen image is labeled and categorized by three professional bronchial endoscopy operators. With the consent of at least two doctors, the image is included in a specific dataset of 31 categories.

[0069] Step 2: Perform data preprocessing on the bronchial lumen images to construct a bronchial lumen location recognition dataset.

[0070] Specifically, in step 2 of this embodiment, for each bronchial lumen image included in step 1, only the square region containing the bronchoscopic field of view is cropped and retained. All images are adjusted to a size of 1000 pixels × 1000 pixels. Since collecting real bronchial lumen images is difficult and limited in number, data augmentation is required before inputting the data into the neural network model for training to increase data diversity. This includes rotating, translating, scaling, adjusting brightness, changing saturation, and adding Gaussian noise to the images. Finally, the dataset is divided into a training set, a validation set, and a test set in an 8:1:1 ratio.

[0071] Step 3: Construct the feature extraction backbone network and loss function.

[0072] Specifically, the feature extraction backbone network constructed in step 3 of this embodiment is built using the ConvNeXt Block module as its core component. The complete bronchial feature extraction backbone network comprises: a preconvolutional layer, a normalization layer, a stack of four ECA-SimAM ConvNeXt Block modules, three downsampling layers, a global average pooling layer, a normalization layer, a fully connected layer, and a Softmax layer. This embodiment adds ECA attention and SimAM attention modules to the ConvNeXt Block module, enhancing the model's learning ability and significantly improving its feature extraction and learning capabilities. This allows for more precise focusing on key regions in bronchial lumen images, improving the perception of important features in bronchial lumen images. Simultaneously, this invention employs the ArcLoss loss function to optimize the intra-class compactness and inter-class variability of bronchial lumen image features, thereby improving the recognition accuracy and classification accuracy of different image categories and addressing the problem of small inter-class variability in bronchial lumen images.

[0073] Step 4: Use the training data to train the entire network to obtain a trained bronchial lumen location recognition network model.

[0074] Specifically, in step 4 of this embodiment, the designed feature extraction network model is pre-trained using the large benchmark dataset ImageNet-21k, and the pre-trained weights are used to initialize the model. Then, the pre-trained model is transferred to the bronchial lumen recognition training dataset for transfer learning. The AdamW optimizer with decoupled weight decay is selected, and the MultiStepLR strategy is used to optimize the learning rate. Drop-path regularization is added to alleviate overfitting. The bronchial lumen location recognition dataset constructed in step 2 is input into the network for learning. The training set is used to train the model and determine parameters such as weights and biases. The validation set is used for model optimization, selecting suitable hyperparameters for network training. The test set is used for evaluating and using the model after training.

[0075] Step 5: Classify and identify the images acquired in real time from the bronchial endoscopy video, and output the identification results.

[0076] Specifically, in step 5 of this embodiment of the invention, a real bronchial endoscopy video source is acquired by a bronchial image acquisition device, and the image of each frame is automatically extracted. The image is input into the preprocessing module for preprocessing, and the area containing only the bronchial lumen image is cropped out. The image is then adjusted to the size of the network input in step 3 and input into the constructed neural network for feature extraction and prediction classification. The real-time prediction result is output in the video frame.

[0077] The electronic device provided in this embodiment can execute the above-described intelligent bronchial lumen location identification method embodiment and the above-described intelligent bronchial lumen location identification device embodiment. The implementation principle and technical effect are similar, and will not be described again here.

[0078] Example 3: Based on the above embodiment 2, please refer to Figure 11 , Figure 11 This is a schematic diagram of a computer-readable storage medium provided in an embodiment of the present invention. The computer-readable storage medium provided in this embodiment stores a computer program, which, when executed by a processor, performs the following steps: Step 1: Open the bronchial image acquisition device to acquire real-time video during the bronchial endoscopy examination. After removing the patient's privacy information, extract the bronchial lumen image at a certain frame rate to form the original dataset of bronchial lumen images.

[0079] Specifically, in step 1 of this embodiment, a bronchial examination video is collected using a bronchial image acquisition device, after removing patient privacy information. The video is divided into image frames at a sampling frequency of 3 frames per second. Black border removal is performed to extract corresponding bronchial lumen images, excluding images with unclear lumens or obvious lumens lesions. Each bronchial lumen image is labeled and categorized by three professional bronchial endoscopy operators. With the consent of at least two doctors, the image is included in a specific dataset of 31 categories.

[0080] Step 2: Perform data preprocessing on the bronchial lumen images to construct a bronchial lumen location recognition dataset.

[0081] Specifically, in step 2 of this embodiment, for each bronchial lumen image included in step 1, only the square region containing the bronchoscopic field of view is cropped and retained. All images are adjusted to a size of 1000 pixels × 1000 pixels. Since collecting real bronchial lumen images is difficult and limited in number, data augmentation is required before inputting the data into the neural network model for training to increase data diversity. This includes rotating, translating, scaling, adjusting brightness, changing saturation, and adding Gaussian noise to the images. Finally, the dataset is divided into a training set, a validation set, and a test set in an 8:1:1 ratio.

[0082] Step 3: Construct the feature extraction backbone network and loss function.

[0083] Specifically, the feature extraction backbone network constructed in step 3 of this embodiment is built using the ConvNeXt Block module as its core component. The complete bronchial feature extraction backbone network comprises: a preconvolutional layer, a normalization layer, a stack of four ECA-SimAM ConvNeXt Block modules, three downsampling layers, a global average pooling layer, a normalization layer, a fully connected layer, and a Softmax layer. This embodiment adds ECA attention and SimAM attention modules to the ConvNeXt Block module, giving the model stronger learning capabilities and significantly enhancing its feature extraction and learning abilities. This allows for more precise focusing on key regions in bronchial lumen images, improving the perception of important features in bronchial lumen images. Simultaneously, this invention employs the ArcLoss loss function to optimize the intra-class compactness and inter-class variability of bronchial lumen image features, thereby improving the recognition of different image categories and addressing the problem of small inter-class variability in bronchial lumen images of different categories.

[0084] Step 4: Use the training data to train the entire network to obtain a trained bronchial lumen location recognition network model.

[0085] Specifically, in step 4 of this embodiment, the designed feature extraction network model is pre-trained using the large benchmark dataset ImageNet-21k, and the pre-trained weights are used to initialize the model. Then, the pre-trained model is transferred to the bronchial lumen recognition training dataset for transfer learning. The AdamW optimizer with decoupled weight decay is selected, and the MultiStepLR strategy is used to optimize the learning rate. Drop-path regularization is added to alleviate overfitting. The bronchial lumen location recognition dataset constructed in step 2 is used as input to the network for learning. The training set is used to train the model and determine parameters such as weights and biases. The validation set is used for model optimization, selecting suitable hyperparameters for network training. The test set is used for evaluating and using the model after training.

[0086] Step 5: Classify and identify the images acquired in real time from the bronchial endoscopy video, and output the identification results.

[0087] Specifically, in step 5 of this embodiment of the invention, a real bronchial endoscopy video source is acquired by a bronchial image acquisition device, and the image of each frame is automatically extracted. The image is input into the preprocessing module for preprocessing, and the area containing only the bronchial lumen image is cropped out. The image is then adjusted to the size of the network input in step 3 and input into the constructed neural network for feature extraction and prediction classification. The real-time prediction result is output in the video frame.

[0088] This embodiment provides a computer-readable storage medium that can execute the above-described intelligent bronchial lumen location identification method embodiment and the above-described intelligent bronchial lumen location identification electronic device embodiment. The implementation principle and technical effects are similar, and will not be described again here.

[0089] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented, in whole or in part, as a computer program product, the computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0090] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for intelligent identification of bronchial lumen locations, characterized in that, The intelligent bronchial lumen location identification method includes: Step 1: Collect real, reliable images of various bronchial lumen locations under bronchial endoscopy with sample diversity, and perform standardized data annotation. Step 2 involves preprocessing and dividing the images of the bronchial lumen into data, including cropping, size normalization, and data amplification, to construct a dataset for bronchial lumen identification. Step 3: Construct the bronchial feature extraction backbone network and loss function based on depthwise separable convolution, attention mechanism, and additive angular margin; Step 4: Train the entire network using the training set to obtain the trained bronchial lumen location recognition network model. Step 5: Using the obtained model, classify and identify the acquired images in the real-time bronchial endoscopy video, and display the identification results in real time in the form of text and graphics.

2. The intelligent bronchial lumen location identification method as described in claim 1, characterized in that, Step one involves acquiring authentic, reliable, and diverse images of various bronchial lumen locations obtained through bronchial endoscopy, and then performing standardized data annotation. Specifically, this includes: First, for patients undergoing bronchial endoscopy with different genders, ages, heights, and weights, various types of bronchial endoscopes were used to record real bronchial examination videos, and patient privacy information was removed. Then, image frames are extracted at a sampling frequency of n frames per second, and bronchial lumen images corresponding to 31 bronchial lumen categories are extracted. These 31 lumen categories are: glottis, trachea, carina, left main bronchus, left upper lobe, proper bronchus of left upper lobe, posterior segment of apical bronchus of left upper lobe, anterior segment of bronchus of left upper lobe, lingular segment of left upper lobe, superior lingular segment of left upper lobe, inferior lingular segment of left upper lobe, left lower lobe, and posterior segment of left lower lobe. Segment, left lower lobe anterior basal segment, left lower lobe lateral basal segment, left lower lobe posterior basal segment, right main bronchus, right upper lobe, right upper lobe apical segment, right upper lobe posterior segment, right upper lobe anterior segment, right upper lobe middle segment, right middle lobe, right middle lobe lateral segment, right middle lobe medial segment, right lower lobe, right lower lobe dorsal segment, right lower lobe medial basal segment, right lower lobe anterior basal segment, right lower lobe lateral basal segment, right lower lobe posterior basal segment; Finally, images containing clear bronchial lumen structures were selected, and images that could not be accurately identified due to image quality issues or lesions affecting the identification of bronchial lumen sites were removed. Furthermore, for each selected bronchial lumen image, at least three qualified bronchodilators were required to label and classify it, and the image was only included in the specific category dataset if at least two-thirds of the physicians' labels were consistent or reached a consensus.

3. The intelligent bronchial lumen location identification method as described in claim 1, characterized in that, The data preprocessing and data segmentation in step two, including cropping, size normalization, data amplification, and data partitioning, to construct a bronchial lumen location identification dataset, specifically include: First, for each bronchial lumen image included in step one, the image is cropped using a black border removal algorithm, retaining only the square region containing the bronchoscope's field of view, and all cropped images are uniformly adjusted to a standard size. Then, before inputting the data into the neural network model for training, data augmentation techniques are used to increase data diversity. Data augmentation operations include random image rotation, translation, scaling, brightness adjustment, saturation changes, and the addition of Gaussian noise. Finally, the processed site recognition dataset is divided into training, validation, and test sets proportionally.

4. The intelligent bronchial lumen location identification method as described in claim 1, characterized in that, Step three involves designing and constructing the bronchial feature extraction backbone network, specifically including: The bronchial feature extraction backbone network is constructed with ConvNeXt Block modules containing depthwise separable convolutions as the core component. The backbone network includes preconvolutional layers, stacked ConvNeXt Block modules, downsampling layers, global average pooling layers, normalization layers, fully connected layers, and Softmax layers. Based on the ConvNeXt Block module of the feature extraction backbone network, an efficient channel attention module ECA and a parameterless spatial attention module SimAM are embedded. The ECA module adaptively adjusts the weight of each channel to strengthen the attention to key information channels, while the SimAM module improves the network's adaptive perception of the importance of spatial regions through parameterless optimization, so as to effectively enhance the network's local region focusing ability. The SimAM module is embedded as follows: It is added after the first depthwise separable convolutional layer within each ConvNeXt Block module; SimAM is achieved through the design of an energy function. The weights are determined by calculating the activation level of each neuron in relation to the inhibition of surrounding neurons. ⊙ ,in, This represents the output feature map. Represents the input feature map, It is the energy function of each channel. The set of elements, where sigmoid is the activation function, and ⊙ represents element-wise multiplication; The ECA module is embedded as follows: an ECA module is added after the second 1×1 convolutional layer inside each ConvNeXt Block module; the ECA module compresses the input 3D feature map into a 1D vector through global average pooling (GAP); then, the vector is processed through a 1D convolutional layer to obtain the calculated channel attention weights, and the channel attention weights are non-linearly mapped using the Sigmoid activation function, and then multiplied with the original feature map channel by channel to finally obtain the enhanced bronchial lumen feature representation.

5. The intelligent bronchial lumen location identification method as described in claim 1, characterized in that, The design and construction of the loss function in step three specifically includes: The loss function is the ArcLoss loss function based on additive angular margins. It improves the distinguishability between different categories of bronchial lumen images by optimizing the intra-class aggregation and inter-class discriminability of bronchial image features, thus addressing the problem of small inter-class differences in bronchial lumen classification. The calculation formula for the ArcLoss loss function is as follows: In the formula, Indicates the number of samples. Indicates the first The true label of each sample; It is the first The angle corresponding to the true category of each sample It is the first Individual samples and other categories Angle; This is the angular interval, used to increase the angular distance between the correct and incorrect categories; This is a scaling factor used to amplify the cosine value of the angle.

6. The intelligent bronchial lumen location identification method as described in claim 1, characterized in that, Step four involves training the entire network using a training set to obtain a trained bronchial lumen location recognition network model, specifically including... We use model parameters pre-trained on the large benchmark dataset ImageNet-21k as initial weights to accelerate the model convergence process by passing on existing deep network learning experience. Then, we use a transfer learning strategy to train the network using the training set of the constructed bronchial lumen location recognition dataset as the input data. We then use the validation set for hyperparameter selection and model optimization, and finally use the test set for final model evaluation and performance verification, thus obtaining the final bronchial lumen location recognition network model. During training, the AdamW optimizer with decoupled weight decay is used to ensure the stability of parameter updates and effectively avoid overfitting; the MultiStepLR learning rate adjustment strategy is adopted to gradually reduce the learning rate according to the preset learning rate change nodes, thereby improving the convergence efficiency and stability during training. The Drop-path regularization method is added to enhance the network's generalization ability by randomly discarding some paths in the network.

7. The intelligent bronchial lumen location identification method as described in claim 1, characterized in that, Step five involves classifying and recognizing the images acquired in real-time during the bronchial endoscopy video and outputting the recognition results, specifically including: First, each frame of the image is automatically extracted in real time from the actual bronchial endoscopy video stream and fed into a preprocessing module. The preprocessing module crops the image, retaining only the region containing the bronchial lumen. Then, the cropped image is adjusted to fit the size and format required by the neural network input and input into the deployed deep neural network model. The model identifies any bronchial lumen region in the image in real time through feature extraction and classification prediction, and outputs the corresponding prediction result. Finally, the prediction result is sent to the corresponding software front-end for display. The displayed screen includes both the original bronchial endoscopy image and the textual and graphical lumen region recognition results.

8. An electronic device employing the intelligent bronchial lumen location identification method as described in any one of claims 1 to 7, characterized in that, The intelligent bronchial lumen location identification electronic device includes a bronchial image acquisition device, a display, a graphics processor, a communication interface, a memory, a central processing unit, and a communication bus. The bronchial image acquisition device, display, graphics processor, communication interface, memory, and central processing unit communicate with each other through the communication bus. The bronchial image acquisition device is used to acquire images of the bronchial lumen in real time; The display is used to show the real-time results of bronchial lumen location identification; The graphics processor is used for inference calculations in the neural network; The memory is used to store computer programs; The central processing unit is used to execute the computer program stored in the memory to implement the intelligent bronchial lumen location identification method.

9. The electronic device as claimed in claim 8, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the following steps: Images of various bronchial lumen locations were acquired during real bronchoscopic examinations, ensuring reliable image quality and sample diversity, and covering 31 different anatomical structures. Next, the acquired bronchial lumen images underwent data preprocessing to construct a bronchial lumen location recognition dataset. Subsequently, a bronchial feature extraction backbone network and loss function were constructed according to the procedures in claims 4 and 5, and the overall network model was trained using the aforementioned dataset according to the procedure in claim 6. Finally, the trained model was applied to real-time bronchoscopic examination videos to automatically classify and recognize the acquired bronchial images, and the recognition results were visualized.

10. A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the following steps: Thirty-one bronchial lumen images were acquired during a real bronchoscopy examination to ensure reliable image quality and sample diversity. Next, the acquired bronchial lumen images were preprocessed to construct a bronchial lumen site identification dataset. Subsequently, a feature extraction backbone network and loss function were designed and constructed according to the procedures of claims 4 and 5, and the overall network model was trained using the aforementioned dataset according to the procedure of claim 6. Finally, the trained model was applied to a real-time bronchoscopy video to automatically classify and identify the acquired bronchial images, and the identification results were visualized.

Images