An image-based lung bronchial airway segmentation method

Abstract

The application discloses a kind of based on image's lung bronchus airway segmentation method, realizes by inputting lung bronchus airway image to trained based on image's lung bronchus airway segmentation network;Based on image's lung bronchus airway segmentation network is constituted by coding network, neck network and decoding network;Coding network includes four coding modules, and wherein PWFR module is additionally arranged;Decoding network includes four decoding modules, and wherein AttdKD module and PWFR module are additionally arranged in sequence;The based on image's lung bronchus airway segmentation method is the airway segmentation method of whole and local attention, not only can strengthen the representation ability of effective feature, alleviate interclass imbalance phenomenon, improve airway segmentation continuity, and can realize to the accurate segmentation of airway main body and accurate identification of small airway, with good clinical application prospect.

Description

Technical Field

[0001] This invention relates to the field of medical image segmentation technology, and in particular to an image-based method for segmenting the lung, bronchus, and airway. Background Technology

[0002] Precise airway segmentation is a crucial prerequisite for preoperative planning and intraoperative navigation in bronchoscopic interventional treatment of lung diseases. The lung airway is a fine-grained structure, with the distal airways being small, narrow, complex, and tortuous, making manual annotation time-consuming, laborious, and highly subjective. To alleviate the burden on physicians and improve the quality and efficiency of airway annotation, automatic airway segmentation algorithms have been continuously updated and optimized, especially with the introduction of deep learning. These methods utilize multi-level feature learning and contextual understanding to select appropriate network architectures and training models, aiming to build efficient and accurate medical image analysis systems, freeing radiologists from tedious manual annotation processes. However, in practical applications, the distal small airways and pulmonary micronodules contain limited image information. During imaging, volume effects, noise, and artifacts blur detailed features, resulting in low recognition rates and difficulty in effectively guiding precise preoperative path planning and real-time intraoperative image guidance.

[0003] Traditional methods, such as region growing, adaptive thresholding, and fuzzy connectivity analysis, are effective for detecting thick tubular structures like the trachea and main bronchi. However, when the airway extends to level 5 or 6, the airway walls become thinner, narrower, and more tortuous, resembling the grayscale levels of surrounding tissues, severely limiting the applicability of traditional methods. With the support of numerous airway-annotated images in the EXACT'09 dataset, more and more researchers are inclined to use convolutional neural networks (CNNs) to learn highly robust and discriminative features. CNNs of different dimensions are also gradually being applied to airway segmentation tasks. For example, Charbonnier et al. used 2D CNNs in post-processing for airway leaks, while Yun et al. used 2.5D CNNs to extract more spatial features from adjacent slices, enhancing airway segmentation capabilities. Compared to lower dimensions, 3D CNNs have become the mainstream for airway feature extraction due to their predictive completeness and consistency, with some typical networks exhibiting excellent performance, such as AirwayNet, WingsNet, and NaviAirway. However, issues such as local discontinuous mapping and airway leakage remain major barriers to improving the airway segmentation performance of CNNs. These phenomena mainly occur in the peripheral lung region. The main reason for this is the severe inter-class imbalance between distal small airways and the background, and the blurred airway walls, which resemble the grayscale features of peripheral tissues. Another limitation is the high computational complexity of CNNs, which necessitates training on image patches. This makes it difficult for the network to detect subtle errors before stitching all patches into a complete image. Accurately identifying distal small airways is key to solving these problems.

[0004] Current work tends to design complex loss functions to achieve accurate identification of distal small airways and address issues such as airway discontinuity mapping and airway leakage. Examples include joint losses such as wBCE, Dice loss, Connectivity-Aware Surrogate, Local-Sensitive Distance, radial distance loss, and General Union loss. While optimized loss function design can indeed enhance the network's ability to screen for some small airways to a certain extent, these methods still do not fully consider the variability of airway structure caused by pathological factors such as bronchiectasis and bronchial wall thickening in real-world environments. This may reduce the model's clinical generalization ability.

[0005] Strengthening the ability to focus on distal airways is also an effective way to solve the discontinuity problem. Attention mechanisms, by weighting the "importance" of different feature representations, have shown good recognition ability for complex structures. Qin et al. introduced attention distillation to alleviate the gradient erosion problem, while Nan et al. designed a fuzzy attention layer to enhance the recognition ability of small airway branches. Ke et al. proposed a multi-scale feature method with attention supervision. Chen et al. introduced a transformer-based attention-guided network. Zeng et al. developed a multi-scale feature joint back attention network based on U-Net to improve the extraction ability of fine edge features. The Transformer model formally defines the attention mechanism, and with the strong performance of Self-Attention, Multi-head Attention, and Cross-Attention, Transformer has reached its performance peak. Tang et al. designed an adversarial transformer to solve the discontinuity of peripheral bronchioles, while Wu et al. proposed a context transformer for segmenting small airway branches. Other forms of transformers, such as U-Net transformer, Squeezeand-Expansion transformer, and hierarchical transformer, have also shown satisfactory performance in difficult-to-segment samples of medical images. Despite significant advancements in attention mechanisms for medical image segmentation, challenges remain, including poor model interpretability, unstable attention distribution, and overfitting. Airway leakage and local discontinuous mapping remain major challenges in optimizing airway segmentation performance, further exacerbated by the heterogeneity of image data across different medical institutions and the varying airway branching patterns among individuals.

[0006] To address the aforementioned issues, we propose an effective optimized network to improve the recognition of small distal airways and alleviate problems such as airway leakage and local discontinuous mapping. We infer that after re-extracting features based on their contribution, we perform attention-driven knowledge distillation, resulting in rich contextual feature maps in both channel and spatial dimensions. Compared to attention mechanisms that only focus on channels or space, this approach captures more important details of the segmented target. Our model was tested on the EXACT-09 and ATM datasets, and comprehensive analysis was performed using multiple evaluation metrics including TD, BD, and FPR. Significant segmentation performance improvements were achieved, with the most significant improvement being in the recognition of small distal airways. Summary of the Invention

[0007] The purpose of this invention is to provide an image-based method for lung bronchial airway segmentation that solves problems such as airway leakage and local discontinuous mapping during lung bronchial airway segmentation.

[0008] Therefore, the technical solution of the present invention is as follows:

[0009] A lung-bronchial-airway segmentation method is proposed, which is implemented by inputting lung-bronchial-airway images into a trained image-based lung-bronchial-airway segmentation network. The image-based lung-bronchial-airway segmentation network consists of an encoding network, a neck network, and a decoding network. The encoding network consists of a first encoding module, a second encoding module, a third encoding module, and a fourth encoding module connected in sequence; each encoding module consists of a first convolutional block module, a second convolutional block module, and a PWFR module connected in sequence. The neck network consists of a first convolutional block module, a second convolutional block module, and a PWFR module connected in sequence. The decoding network consists of a first decoding module, a second decoding module, a third decoding module, and a fourth decoding module connected in sequence; each decoding module consists of a first convolutional block module, a second convolutional block module, an AttdKD module, and a PWFR module connected in sequence. The output of the PWFR module of the first encoding module is connected to the first convolutional block module of the third decoding module. The output of the Block module is connected to the input of the fourth feature concatenation module, and the output of the fourth feature concatenation module is connected to the input of the fourth decoding module; the output of the PWFR module of the second encoding module and the output of the first Convolutional Block module of the second decoding module are connected to the input of the third feature concatenation module, and the output of the third feature concatenation module is connected to the input of the third decoding module; the output of the PWFR module of the third encoding module and the output of the first Convolutional Block module of the first decoding module are connected to the input of the first Convolutional Block module. The output of the Block module is connected to the input of the second feature stitching module, and the output of the second feature stitching module is connected to the input of the second decoding module; the output of the PWFR module of the fourth encoding module and the output of the PWFR module of the neck network are connected to the input of the first feature stitching module, and the output of the first feature stitching module is connected to the input of the first decoding module; the PWFR module consists of a channel splitter, a feature recalibration module, and an average point-level operation module connected in sequence; the AttdKD module consists of a teacher module and a student module; the teacher module includes a CAM module, a first convolutional thinning module, a SAM module, and a second convolutional thinning module connected in sequence, and the second input of the second convolutional thinning module is connected to the input of the first convolutional thinning module; the student module includes an element-wise multiplication module, and the second input of the first convolutional thinning module, the input of the CAM module, and the input of the element-wise multiplication module are all connected to the same output.

[0010] Furthermore, in the PWFR module, the number of channel segments N for the channel splitter is set to 3.

[0011] Furthermore, the specific operation steps of this image-based lung bronchus airway segmentation method are as follows:

[0012] Step 1: Construct an image-based lung-bronchial-airway segmentation network;

[0013] Step 2: Construct a lung bronchial airway image dataset for a deep learning network used for image segmentation;

[0014] Step 3: Enhance the lung, bronchus, and airway image dataset obtained in Step 2;

[0015] Step 4: Using the lung, bronchus, and airway image dataset obtained in Step 3, train the image-based lung, bronchus, and airway segmentation network constructed in Step 1 so that the network can achieve lung, bronchus, and airway segmentation.

[0016] Step 5: Input the collected lung bronchus airway images into the image-based lung bronchus airway segmentation network trained in Step 4, and output accurate lung bronchus airway segmentation status.

[0017] Furthermore, in step 2, the lung and bronchial airway image dataset adopts the Airway Tree Modeling dataset or a dataset constructed from self-acquired lung and bronchial airway CT images.

[0018] Furthermore, in step 3, the enhancement processing method is any one of random rotation and scaling, random flipping, random cropping, random brightness and contrast adjustment, elastic deformation, and random noise.

[0019] Further, in step 4, the training of the image-based lung-bronchial-airway segmentation network aims to minimize the distillation loss function in the AttdKD module; where the distillation loss function... The expression is:

[0020] ,

[0021] In the formula, N is the total number of samples in the current batch. and This is a spatial feature map of two adjacent layers. This represents the square of the Frobenius norm.

[0022] Furthermore, in step 4, the training parameters are set as follows: the image training size is set to 128*224*304, the batch size is set to 1, the step size is set to 64, and the initial learning rate is set to 0.003.

[0023] Compared with existing technologies, this image-based lung-bronchial-airway segmentation method alleviates inter-class imbalance, improves airway segmentation continuity, and can achieve accurate segmentation of the main airway and accurate identification of small airways, showing great promise for clinical application. Attached Figure Description

[0024] Figure 1 This is a flowchart of the image-based lung, bronchus, and airway segmentation method of the present invention;

[0025] Figure 2 This is a schematic diagram of the image-based lung-bronchial-airway segmentation network structure of the present invention.

[0026] Figure 3 This is a schematic diagram of the PWFR module structure of the image-based lung-bronchial-airway segmentation network of the present invention.

[0027] Figure 4 This is a schematic diagram of the AttdKD module structure of the image-based lung bronchus airway segmentation network of the present invention.

[0028] Figure 5 This is a schematic diagram of the confusion matrix for binary classification.

[0029] Figure 6 This is a visualization comparison of the method used in the embodiments of the present invention with other excellent algorithms;

[0030] Figure 7 This is a visualization of the results after adding each module of the method in an embodiment of the present invention. Detailed Implementation

[0031] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but the following embodiments are by no means intended to limit the present invention.

[0032] See Figure 1 The specific implementation of this image-based lung bronchus airway segmentation method is described below.

[0033] Step 1: Construct an image-based lung-bronchial-airway segmentation network.

[0034] This lung and bronchial airway image segmentation network is an improvement on the 3D-UNet network model. The network structure is reconstructed by adding a point-wise feature recalibration (PWFR) module to the encoding module of 3D-UNet, and an attention-driven knowledge distillation (AttdKD) module to the decoding module of 3D-UNet in addition to the PWFR module. At the same time, based on the actual segmentation effect of lung and bronchial airway images, the optimal number of encoding modules in the encoding network and the specific number of decoding modules in the decoding network are determined to achieve the best segmentation effect.

[0035] like Figure 3 As shown, the PWFR module is a new design module based on the segmentation requirements of this application. It consists of a channel segmenter, a feature recalibration module, and an averaging point-level operation module connected in sequence.

[0036] In the Channel Splitter, the number of channel splits N is set to 3. That is, the Channel Splitter first divides the channels of the input feature map into three subgroups: Sub-g FM 1, Sub-g FM 2 and Sub-g FM 3, in order to reduce computational complexity.

[0037] The Feature Recalibration module processes the feature maps of each subgroup separately; given the original feature map of any subgroup... This module automatically extracts the features with the highest and medium contributions from the corresponding group channels, while weakening the features with the lowest contribution. This allows different features to be weighted according to their contribution to the task, thus dividing them into the highest contribution group, the moderate contribution group, and the lowest contribution group. Figure 3 In the diagram, blue arrows indicate the flow of the highest-contributing features after weight allocation in the feature maps of each subgroup, green arrows indicate the flow of medium-contributing features, and orange arrows indicate the flow of the lowest-contributing features. Subsequently, in each group obtained by reassignment based on different contribution levels, the maximum (Max Point-wise), median (Median Point-wise), and minimum (Min Point-wise) values ​​of each group are extracted to obtain feature outputs that capture local features and diverse information representations. Its expression is:

[0038] ,

[0039] In the formula, , , , , and This refers to the height, width, and number of channels of the original feature map; These represent the highest contribution features in the three subgroups, respectively. These represent the medium contribution characteristics in the three subgroups, respectively. These represent the lowest contribution features in the three subgroups, respectively.

[0040] During the feature elimination and squeezing process of the aforementioned feature recalibration module, the spatial size of the feature map remains unchanged. Furthermore, during the training process, as effective features are continuously encouraged and redundant features are continuously suppressed, the key areas of the main airway are gradually given priority, while other invalid features such as noise signals are gradually ignored.

[0041] The Average Point-wise operation module acquires the feature output of the weight calibration module. The maximum value among the features with the highest contribution. The median value of the characteristic of moderate contribution level and the minimum value of the feature with the lowest contribution. The feature map is fused by combining features at different point levels to generate the final output feature map. The specific processing procedure of this module is expressed as follows:

[0042] ,

[0043] In the formula, and It is a non-linear activation function of Leaky ReLU and Sigmoid.

[0044] In the above processing, the convolution operator Channel reduction and channel recovery are respectively achieved through dimensions of [sizes to be filled in]. and The convolutional kernels and channel reorganization achieved through activation operations encourage effective feature channels and suppress redundant feature channels, thereby further enhancing the network's ability to learn highly discriminative features. Based on practical processing, this feature map is characterized by retaining the salient features of high-contribution regions (such as bright yellow) while suppressing redundant and unimportant parts, thus improving the effectiveness of feature representation.

[0045] This PWFR module differs from conventional methods that directly fuse multi-scale features. It aims to emphasize important features across different feature channels, eliminate weak semantic features, and learn more highly discriminative and useful features from a relatively small number of feature maps. This further highlights the dominant role of the airway during the learning process, alleviates the problem of local discontinuous mapping, and thus enhances the network's overall ability to represent the airway. Figure 3 As shown.

[0046] See Figure 4 The AttdKD module consists of a Teacher module and a Student module. The Teacher module includes a CAM module, a first convolution and refinement module, a SAM module, and a second convolution and refinement module connected in sequence, with the second input of the second convolution and refinement module connected to the input of the first convolution and refinement module. The Student module includes an element-wise multiplication module, with the second input of the first convolution and refinement module, the input of the CAM module, and the input of the element-wise multiplication module all connected to the same output. The CAM module and the SAM module are existing conventional modules, and their combined use in the AttdKD module constitutes the CBAM attention mechanism.

[0047] The AttdKD module processes the input feature map as follows: the feature map is simultaneously processed by both the Teacher and Student modules; whereby...

[0048] In the Teacher module, feature maps are used to generate high-quality feature representations through a complex attention mechanism (CBAM). Within this attention mechanism, the CAM module is a channel-attention module, and its channel-attention descriptor is defined as: That is, the intermediate feature map is subjected to global max pooling and global average pooling to aggregate the spatial information of the feature map. This is then fed into a shared multilayer perceptron to compress the spatial dimension of the input feature map while preserving channel information. Finally, it is activated by a sigmoid function to generate a channel attention map. Average pooling provides feedback to every pixel in the feature map, while max pooling only provides gradient feedback to the areas with the largest response during gradient backpropagation. Therefore, this descriptor highlights meaningful regions in the image, especially sensitive task-related regions such as distant fine bronchi, while blurring regions irrelevant to the segmentation task. The SAM module is a spatial attention module, and its Spatial-attention descriptor is represented as: In other words, the feature maps output by the channel attention module are pooled along the channel dimension to preserve spatial feature information. The pooled features are then concatenated and activated using a sigmoid function to obtain spatial attention weights.

[0049] In the Student path, feature maps are optimized and learned under the supervision of the Teacher using a lightweight structure; finally, the AttdKD module uses a distillation loss function. The feature maps output by the Student path are optimized by minimizing the attention loss to make the intermediate feature maps between adjacent layers more similar, gradually approaching the feature representation of the Teacher path, and significantly improving the feature extraction capability and performance of the lightweight model; specifically, the distillation loss function... The expression is:

[0050] ,

[0051] In the formula, N is the total number of samples in the current batch. and This is a spatial feature map of two adjacent layers. This represents the square of the Frobenius norm; since we note that the later layer acts as the teacher of the earlier layer, the "intensity of attention" will gradually permeate from the deep layer to the shallow layer.

[0052] In summary, unlike previous works that relied on prior knowledge and fine feature extraction or complex loss design, the AttdKD module in this application fully utilizes the attention mechanism in CBAM, combining channel and spatial information to condense feature distillation. The emphasized features eliminate background noise and fully aggregate spatial and channel knowledge to condense features, thereby enhancing the ability to focus on "micro-targets" such as distant fine airways under inter-class imbalance conditions and alleviating the problem of local discontinuous mapping. CBAM consists of a channel attention module and a spatial attention module. At different stages of the network, CBAM aggregates spatial and channel descriptors and multiplies them with the corresponding original feature maps, introducing cross-level and cross-spatial information into the original feature maps to create new, rich contextual feature maps. Furthermore, CBAM combines spatial and channel attention modules, enabling the spatial attention map to propagate along the channel and the channel attention map to propagate along the spatial dimension. Compared to attention mechanisms that only focus on the channel or space, it can better highlight the important features of the image, forcing the student network to imitate the important regions emphasized by the teacher, making it a strong candidate with significant distillation potential. In addition, according to performance comparison results, AttdKD does not increase network complexity and can be integrated into existing CNN networks to achieve efficient feature correction and performance improvement.

[0053] Specifically, the construction steps of the image-based lung bronchus airway segmentation network are described below.

[0054] Step 1.1 Based on the 3D-UNet encoding module, a PWFR module is added to the end of the encoding module of the network in this application to form the encoding module of the image-based lung bronchus airway segmentation network.

[0055] Based on the improved construction method in step 1.1 above, the encoding module of this application consists of a first Convolutional Block module, a second Convolutional Block module, and a PWFR module connected in sequence.

[0056] Step 1.2: Decoding module based on 3D-UNet. An AttdKD module and a PWFR module are added sequentially to the end of the decoding module of the network in this application to form a decoding module of the image-based lung bronchus airway segmentation network.

[0057] Based on the improved construction method in step 1.2 above, the decoding part of this application consists of a first Convolutional Block module, a second Convolutional Block module, an AttdKD module, and a PWFR module connected in sequence.

[0058] In steps 1.1 and 1.2 above, a PWFR module is integrated at the end of each sampling step to separate and eliminate weak features, select and enhance effective features, and "differentiate" spatial features at different locations before feature fusion to emphasize the dominant role of the airway in the network learning process. Specifically, this involves separating and eliminating weak features, selecting and enhancing effective features, and "differentiating" spatial features at different locations before feature fusion to emphasize the dominant role of the airway in the network learning process. When it is assumed that features at different locations in different channels contribute differently to airway recognition, the PWFR module selects features beneficial to airway representation and discards redundant features.

[0059] The AttdKD module is placed only in the decoder because the low-dimensional features generated by the encoder have low discriminative power and contribute little to attention extraction. Therefore, placing the AttdKD module in the downsampling process of the encoder is not very meaningful. However, placing it in the decoder allows for the generation of high-dimensional, highly discriminative features during the decoding process. The attention map obtained by the AttdKD module is more helpful in highlighting important regions such as distal small airways, maximizing its ability to extract detailed features. Specifically, without setting up two different models separately, the latter layer acts as a teacher, improving the feature mapping by considering channel and spatial information to shift attention to the former layer in the same model, thereby capturing richer detailed features to optimize the recognition performance of distal small airways.

[0060] Step 1.3: Construct an image-based lung-bronchial-airway segmentation network, such as... Figure 2 As shown, the network specifically consists of an encoding network, a neck network, and a decoding network; among which,

[0061] The encoding network consists of a first encoding module, a second encoding module, a third encoding module, and a fourth encoding module connected in sequence; each encoding module consists of a first convolutional block module, a second convolutional block module, and a PWFR module connected in sequence.

[0062] The neck network also consists of a first Convolutional Block module, a second Convolutional Block module, and a PWFR module connected in sequence;

[0063] The decoding network consists of a first decoding module, a second decoding module, a third decoding module, and a fourth decoding module connected in sequence; each decoding module consists of a first Convolutional Block module, a second Convolutional Block module, an AttdKD module, and a PWFR module connected in sequence.

[0064] The output of the PWFR module of the first encoding module and the output of the first Convolutional Block module of the third decoding module are respectively connected to the input of the fourth feature splicing module, and the output of the fourth feature splicing module is connected to the input of the fourth decoding module; the output of the PWFR module of the second encoding module and the output of the first Convolutional Block module of the second decoding module are respectively connected to the input of the third feature splicing module, and the output of the third feature splicing module is connected to the input of the third decoding module; the output of the PWFR module of the third encoding module and the output of the first Convolutional Block module of the first decoding module are respectively connected to the input of the second feature splicing module, and the output of the second feature splicing module is connected to the input of the second decoding module; the output of the PWFR module of the fourth encoding module and the output of the PWFR module of the neck network are respectively connected to the input of the first feature splicing module, and the output of the first feature splicing module is connected to the input of the first decoding module.

[0065] In this image-based lung-bronchial-airway segmentation network, four encoding modules progressively extract features from the feature map while simultaneously downsampling, continuously reducing the size of the feature map and increasing the number of channels. The neck network (Bottleneck) further optimizes the features, highlighting the airway region. After these optimization steps, the feature map enters the decoding network, where it undergoes progressive upsampling by four decoding modules, restoring the low-dimensional deep features of the feature map to high-dimensional spatial features. Simultaneously, the encoding and decoding modules establish skip connections and perform feature concatenation operations through a feature concatenation module, enhancing the combination of low-level and high-level features and improving segmentation details and boundary quality. Finally, the fourth decoding module outputs the processed feature map. Figure 2 The trend of feature map output by each decoding module shows that the processed feature map gradually concentrates (or focuses) towards the small bronchial part.

[0066] In summary, through steps 1.1 to 1.3 above, the lung and bronchial airway image segmentation network of this application is constructed, with the aim of being able to segment the lung and bronchial based on images.

[0067] See Figure 2 The image-based lung-bronchial-airway segmentation network structure and its processing procedure in this application are as follows:

[0068] The image input terminal has an image input size of 128*224*304;

[0069] The encoding network consists of a first encoding module, a second encoding module, a third encoding module, and a fourth encoding module connected in sequence. Each encoding module consists of a first Convolutional Block, a second Convolutional Block, and a PWFR module connected in sequence. The image size output by the first encoding module is 16*40*96*152, the image size output by the second encoding module is 32*20*48*76, the image size output by the third encoding module is 64*10*24*38, and the image size output by the fourth encoding module is 128*5*12*19.

[0070] The neck network is positioned between the encoding network and the decoding network, and it consists of a first Convolutional Block, a second Convolutional Block, and a PWFR module in sequence.

[0071] The decoding network consists of a first decoding module, a second decoding module, a third decoding module, and a fourth decoding module connected in sequence. Each decoding module is composed of a first convolutional block, a second convolutional block, an AttdKD module, and a PWFR module. The image size output by the first decoding module is 64*10*24*38, the image size output by the second decoding module is 32*20*48*76, the image size output by the third decoding module is 16*40*96*152, and the image size output by the fourth decoding module is 128*224*304, which is the segmentation result of the lung, bronchus, and airway image.

[0072] In the aforementioned encoding network, neck network, and decoding network, each Convolutional Block has the same architecture, consisting of sequentially connected 3×3×3 convolutional layers, instance normalization modules, and ReLU activation functions.

[0073] It should be noted that in the construction of the deep learning network used for image segmentation described above, the number of encoding modules in the encoding network and the number of decoding modules in the decoding network are both four. This is because, based on the actual segmentation effect of lung and bronchial airway images, when the number of the above two is less than four, the lung and bronchial airway images cannot be effectively segmented, resulting in poor final segmentation results. Conversely, when the number of the above two is greater than four, the lung and bronchial airway images will be over-segmented, which will also lead to poor final segmentation results.

[0074] Step 2: Construct a lung bronchial airway image dataset for a deep learning network for image segmentation.

[0075] Specifically, the implementation steps for step 2 are as follows:

[0076] In this application, the data used for training, validating and testing the image-based bronchial airway segmentation network are all from the Airway Tree Modeling dataset (hereinafter referred to as the ATM dataset). In this embodiment, 299 sets of CT images from the ATM dataset were selected and used.

[0077] Step 3: Enhance the lung, bronchus, and airway image dataset obtained in Step 2.

[0078] Specifically, the implementation steps for step 3 are as follows:

[0079] During training, data augmentation methods such as random rotation and scaling, random flipping, random cropping, random brightness and contrast adjustment, elastic deformation, and random noise are used.

[0080] Step 4: Using the lung, bronchus, and airway image dataset obtained in Step 3, train the image-based lung, bronchus, and airway segmentation network constructed in Step 1 to achieve the goal of lung, bronchus, and airway segmentation.

[0081] Specifically, the implementation steps for step 4 are as follows:

[0082] In step 4.1, the CT images in the lung, bronchus and airway image dataset constructed in step 3 are randomly selected and divided into CT images for training, CT images for validation, and CT images for testing in a ratio of 2.7:1:1.2.

[0083] Step 4.2: Input the training set obtained in Step 4.1 into the deep learning network for lung, bronchus, and airway image segmentation constructed in Step 1. Train the image segmentation network using the labeled images in the segmentation dataset as input and the image segmentation information as output. Simultaneously, during training, perform ablation experiments on the network after each training round using the validation set. Finally, select the weights corresponding to the round with the best recognition accuracy during training as the optimal weights and save them. Training is then complete. In this embodiment, the training parameters are shown in Table 1 below.

[0084] Table 1:

[0085] Training size 128*224*304 Batch size 1 Step length 64 Initial learning rate 0.003

[0086] Step 5: Input the collected lung and bronchial airway images into the image-based lung and bronchial airway segmentation network trained in Step 4, and the network will output accurate lung and bronchial airway segmentation status.

[0087] Furthermore, in order to evaluate the effectiveness of this method in segmenting lung bronchi, the segmentation accuracy of the method in this application was evaluated. Specifically, the evaluation metrics included precision, centerline-based Branchesdetected (BD), tree length detected (TD), voxel-wise-based False positive rate (FPR), and region-based Dice similarity coefficient (DSC).

[0088] like Figure 5The diagram shows a confusion matrix for binary classification. In the diagram, T and F refer to the positive and negative samples of the true values, while P and N refer to the positive and negative of the predicted values. Specifically, T represents a "correct" result, which means that the model prediction matches the actual situation; F represents a "wrong" result, which means that the model prediction does not match the actual situation; P usually refers to the category that the model predicts as "of interest". In lung-bronchial-airway segmentation, positive examples usually refer to the lung-bronchial-airway region; N refers to the category that the model predicts as "not of interest", which usually refers to the background region or non-lung-bronchial-airway region. There are four classification results in total: True Positive (TP): the model predicts a positive example (e.g., lung bronchial airway), and it is actually a positive example; True Negative (TN): the model predicts a negative example (e.g., non-lung bronchial airway or background), and it is actually a negative example; False Positive (FP): the model predicts a positive example (e.g., lung bronchial airway), but it is actually a negative example (e.g., non-lung bronchial airway or background); False Negative (FN): the model predicts a negative example (e.g., background or other non-lung bronchial airway portion), but it is actually a positive example (i.e., lung bronchial airway portion).

[0089] Precision measures the proportion of regions a model correctly predicts as "pulmonary bronchial airways." In other words, precision focuses on how many regions predicted by the model as pulmonary bronchial airways are actually pulmonary bronchi. Precision measures the model's accuracy in predicting positive cases (i.e., pulmonary bronchial airways). High precision means the model rarely misprescribes non-pulmonary bronchial airway regions as pulmonary bronchial regions, i.e., it has fewer false positives. Its expression is:

[0090] ,

[0091] Center-based Branches detected (BD) is used to evaluate whether the segmentation algorithm can detect the complete branching structure of the lung bronchi. BD represents the proportion of detected branches to the actual number of branches. BD reflects the segmentation algorithm's global detection capability of the lung bronchial tree, and its expression is:

[0092] ,

[0093] In the formula, It is the number of branches detected in the segmentation results. It represents the actual number of branches in the labeled data.

[0094] Tree length detected (TD) is used to measure the overall length detection capability of bronchial segmentation, which is the proportion of the detected tree structure length to the actual length in the segmentation result. TD helps evaluate the coverage capability and the ability to capture small branches of the segmentation algorithm. Its expression is:

[0095] ,

[0096] In the formula, It is the total length of all bronchial branches in the segmentation result; It is the total length of the actual bronchial tree in the labeled data.

[0097] Voxel Wise-based False Positive Rate (FPR), based on the voxel level, represents the proportion of voxels that are falsely detected as bronchi out of the total number of voxels in the segmentation result. FPR reflects the false positive rate of the algorithm; a lower FPR indicates a more accurate segmentation result. Its expression is:

[0098] ,

[0099] In the formula, FP (False Positive) is the number of voxels that are incorrectly segmented as bronchi, and TN (True Negative) is the number of voxels that are correctly segmented as non-bronchial tubes.

[0100] The Region-based Dice similarity coefficient (DSC) is a measure of regional overlap, reflecting the degree of overlap between the segmentation result and the actual bronchial region. A DSC value closer to 1 indicates that the segmentation result is closer to the actual annotation. Its expression is:

[0101] ,

[0102] In the formula, A is the voxel set of the bronchial region in the segmentation result, and B is the voxel set of the bronchial region in the actual annotation. It is the number of primes in the intersection of the segmented result and the real region. These represent the number of voxels in the segmentation result and the actual labeled data, respectively.

[0103] To verify the superiority of our method over existing technologies, we will evaluate other existing methods using the same metrics while assessing the aforementioned indicators, and then conduct a comparative analysis. Simultaneously, to further investigate the key components of the proposed segmentation framework, ablation studies were performed. Furthermore, based on model training as a baseline, we introduced the recently proposed FR, SE, and PE modules to compare the performance of our newly designed module, the PWFR module.

[0104] The specific test results of this ablation experiment are shown in Table 2 below.

[0105] Table 2:

[0106] Baseline 88.30±5.41 86.09±8.17 0.014±0.024 92.27±1.66 92.21±4.39 +PWFR 90.76±6.21 92.07±9.04 0.018±0.017 92.81±2.06 93.07±5.27 <![CDATA[+FR [1] ]]> 90.33±6.51 91.38±8.13 0.021±0.020 92.24±2.77 92.16±4.09 <![CDATA[+SE [2] ]]> 89.97±7.88 90.73±8.28 0.027±0.012 91.37±1.66 91.81±5.22 <![CDATA[+PE [3] ]]> 88.01±7.03 90.86±9.01 0.025±0.011 92.03±2.17 91.04±4.09 +AttdKD 92.84±6.55 90.09±8.17 0.027±0.021 93.34±1.66 93.48±4.39 <![CDATA[+DistKD [4] ]]> 90.06±8.06 89.63±7.54 0.034±0.022 90.11±3.08 93.01±3.94 <![CDATA[+LAD [5] ]]> 88.30±7.18 89.03±10.37 0.036±0.028 90.97±2.46 91.70±4.07 <![CDATA[+CIRKD [6] ]]> 89.97±6.01 88.34±9.33 0.034±0.024 91.01±2.31 91.44±3.64 Proposed 93.96±7.08 92.71±4.47 0.023±0.010 93.52±0.84 93.97±3.53

[0107] In Table 2, "Baseline" represents the baseline model, specifically the image-based lung-bronchial-airway segmentation network constructed in this application, after removing the PWFR module from each coding module and the PWFR and AttdKD modules from each decoding module; "+PWFR" indicates that a PWFR module is added to the end of each coding module based on the above baseline model; "+FR" indicates that an FR module is added to the end of each coding module based on the above baseline model; "+SE" indicates that an SE module is added to the end of each coding module based on the above baseline model; and "+PE" indicates that a PWFR module is added to the end of each coding module based on the above baseline model. The following are definitions of the terms: PE module is added to the end of each encoding module; "+AttdKD" indicates that an AttdKD module is added to the end of each decoding module based on the above benchmark model; "+DistKD" indicates that a DistKD module is added to the end of each decoding module based on the above benchmark model; "+LAD" indicates that an LAD module is added to the end of each decoding module based on the above benchmark model; "+CIRKD" indicates that a CIRKD module is added to the end of each decoding module based on the above benchmark model; "Proposed" indicates the image-based lung-bronchial-airway segmentation network constructed using this application.

[0108] As shown in Table 2, the benchmark model achieved 88.30% BD, 86.09% TD, 0.014% FPR, 92.27% DSC, and 92.21% Precision. The application of the PWFR module allowed for differentiated treatment of multi-scale spatial features, with continuous suppression of redundant features and emphasis on effective features, thus enhancing the representational ability of effective airway features. Furthermore, compared to the SE and PE modules, which treated spatial features "equally," the optimization provided by the PWFR module was more effective, particularly in improving TD performance. Although the FR module integrated spatial features from three dimensions through weighted combination, it still introduced a small number of low-weight redundant features, resulting in slightly inferior segmentation performance compared to the PWFR module. The application of the Baseline + AttdKD module achieved improvements of +5.14%, +4.65%, and +1.38% in BD, TD, and Precision, respectively. This can be attributed to the AttdKD module's use of channel and spatial information to condense features for feature distillation. The resulting feature map enhances the highlighting of small targets such as distal airways, which helps students more effectively mimic the teacher's attention and thus better segment small objects. Furthermore, the AttdKD module outperforms the DistKD, LAD, and CIRKD modules, primarily due to the introduction of an attention mechanism to improve the representation of local features, thereby enhancing the model's interpretability and understandability. It's also worth noting that with the powerful addition of the PWFR and AttdKD modules, the airway is fully emphasized, and while distal airways are given special attention, some irrelevant false positives are introduced, resulting in the baseline model achieving the optimal FPR.

[0109] The PWFR module recognizes that features at different locations within different channels contribute differently to airway identification. Therefore, its design primarily focuses on selecting features beneficial to airway representation while discarding redundant features, addressing the issue of discontinuous airway segmentation and highlighting the primary role of the airway. Compared to the baseline model, the PWFR module achieves 90.76% BD, 92.07% TD, and 92.81% DSC, with a particularly significant improvement in the TD metric, increasing by 6.95%. The AttdKD module, unlike the DistKD and CIRKD modules, uses a combination of spatial and channel descriptors to introduce cross-level and cross-spatial information into the original feature map. This aims to provide finer-grained knowledge guidance for learning distant, small airways, strengthening the focus on "micro-targets" and further emphasizing the primary role of the target airway. This is clearly reflected in the BD metric, achieving a significant improvement of 5.14%. On the TD model, the AttdKD module showed less improvement compared to the PWFR module (4.65% vs 6.95%), mainly due to the different areas of improvement each module focuses on. Furthermore, both the PWFR and AttdKD modules introduced additional false positives, with the latter introducing relatively more compared to the baseline model. This is primarily because the AttdKD module pays special attention to "micro-targets" such as small airways, inevitably introducing noise with similar appearance and size attributes, thus leading to a higher false positive rate.

[0110] like Figure 6 The figure shows a comparison of segmentation results from ablation experiments. Cases 24, 40, and 68 are three randomly selected CT images from the test set. It is evident from the figure that when only the UNet network is used for image segmentation, many small bronchial ends are not identified, meaning the highlighted areas are mainly concentrated in the main airways. However, as the PWER module is added sequentially to the UNet network, and then simultaneously with the PWER and AttKD modules, the network's learning focus continuously concentrates on the airway region, and the background area is significantly weakened. The network's attention clearly shifts towards the small bronchial ends, meaning the highlighted areas gradually transition to the small bronchial ends. In particular, the method of this invention shows a particularly strong focus on small airways in the coronal plane of Case 68, achieving a BD score of 96.16%. This indicates that almost all small airways are captured, demonstrating the superiority of the method of this invention in bronchial identification and segmentation.

[0111] In this embodiment, many state-of-the-art airway segmentation algorithms were reproduced and compared on the same dataset. Specifically, the performance of CNN networks with similar encoding and decoding structures, such as UNet and VNet, and some classic derivatives of UNet (UNet++, nnunet, etc.) in airway segmentation was compared below, and the results are shown in Table 3.

[0112] Table 3:

[0113] <![CDATA[Juarez 2018 [7] ]]> 77.91±13.82 82.81±11.40 0.014±0.013 91.21±3.49 92.33±4.47 <![CDATA[Juarez 2019 [8] ]]> 81.44±22.17 85.74±19.25 0.018±0.009 92.61±2.27 94.17±3.21 <![CDATA[Qin 2019 [9] ]]> 79.71±12.07 85.33±11.40 0.015±0.073 91.80±3.12 93.07±5.01 <![CDATA[Qin 2021 [1] ]]> 89.22±7.07 91.18±9.55 0.014±0.081 92.03±4.07 94.35±5.83 <![CDATA[Wang 2019

[10] ]]> 87.57±11.20 89.37±8.56 0.013±0.011 87.01±2.91 87.32±2.74 <![CDATA[Wingsnet

[11] ]]> 90.31±14.07 89.06±10.54 0.024±0.016 91.33±3.08 93.72±3.79 <![CDATA[Naviairway

[12] ]]> 85.74±9.50 87.34±7.15 0.027±0.034 90.77±2.57 90.11±4.07 <![CDATA[FANN

[13] ]]> 90.77±11.77 91.72±8.07 0.015±0.011 92.82±2.14 91.41±3.77 <![CDATA[CAS CNNs

[14] ]]> 91.29±3.13 92.41±4.21 0.021±0.010 92.38±2.77 93.45±2.83 This application 93.96±7.08 92.71±4.47 0.023±0.010 93.52±0.84 93.97±3.53

[0114] Juarez et al. (2018) proposed a 3D UNet based on joint wBCE and Dice loss for tracheal segmentation. Furthermore, by replacing the deepest convolutional layers of the UNet with a graph neural network (GNN), they improved BD and TD performance from 77.91% and 82.81% to 81.44% and 85.74%, respectively. However, in our individual test cases, the GNN's performance was not impressive, resulting in standard deviations of 22.17% and 19.25% on BD and TD, respectively, reflecting the model's instability despite achieving a very high precision (94.17%). Qin et al. designed AirwayNet, which identifies important airway regions by emphasizing the connectivity of airway voxels. Building upon this, they introduced attention distillation and feature recalibration to alleviate slope erosion and enhance sensitivity to tubular targets, achieving performance improvements of 11.93% and 6.86% on BD and TD, respectively. Wang et al. introduced 3D piecewise convolutional layers to capture the spatial information of slender structures and designed a radial distance loss to strengthen the weight of the tubular centerline during training, helping the network to focus more attention on small targets, thus achieving better TD (89.37%). However, the detection precision (87.32%) was appalling, and the coverage of the real airway was also far from satisfactory (low DSC score).

[0115] Wingsnet was proposed to address gradient erosion and dilation problems, with its General Union loss function focusing on preprocessing imbalances within airways. Compared to a previous study, Wingsnet showed similar performance in TD detection, but with significant improvements of 7.33% and 4.96% in precision and DSC scores, respectively. Furthermore, Wingsnet outperformed NaviAirway in all aspects. Nan et al. used a fuzzy attention neural network with a comprehensive loss function to predict airway regions, particularly for airway branch identification. While the detection accuracy was slightly lower than Wingsnet, it showed a small improvement in BD and TD values ​​and fewer false positive predictions. Zhang et al. designed Connectivity-Aware Surrogate Loss (CAS) and Local-Sensitive Distance (LSD) Loss to improve tracheal segmentation connectivity and achieved a considerable length detected rate (92.41%). However, the overemphasis on small targets led to oversegmentation, resulting in a high false positive rate.

[0116] Visualization comparison of airway segmentation results in practical applications of the above-mentioned airway segmentation algorithms, for example. Figure 7 As shown. Overall, the proposed method has considerable advantages, outperforming existing methods in multiple evaluation metrics. Specifically, UNet's performance is relatively poor, especially in predicting distant small airways. This is mainly because only the largest connectivity region is retained in the prediction results, and some small airways are treated as false positives, which also reflects the discontinuity of UNet in overall airway prediction. WingsNet and NaviAirway networks have problems such as local discontinuous mapping and airway leakage. CAS CNN aims to improve tracheal segmentation connectivity and achieves a considerable length detection rate (92.41%), but the overemphasis on small targets leads to oversegmentation, resulting in a high false positive rate. In contrast, this application fully emphasizes the airway as a whole, focusing on distant small airways, thus achieving the best performance across various metrics.

[0117] Furthermore, we used BD, TD, and FPR to evaluate the performance of the algorithm on the EXACT'09 dataset. Table 4 below shows a comparison experiment between the lung-bronchial-airway segmentation method of this application and other segmentation methods.

[0118] Table 4:

[0119] <![CDATA[Neko * ]]> 35.5±8.2 30.4±7.4 0.89±1.78 <![CDATA[UCCTeam * ]]> 41.6±9.0 36.5±7.6 0.71±1.67 <![CDATA[FF_ITC * ]]> 79.6±13.5 79.9±12.1 11.92±13.16 <![CDATA[HybAir * ]]> 51.1±10.9 43.9±9.6 6.78±26.60 <![CDATA[MISLAB * ]]> 42.9±9.6 37.5±7.1 0.89±1.64 <![CDATA[NTNU * ]]> 31.3±10.4 27.4±9.6 3.60±3.37 <![CDATA[Qin et al. [1] ]]> 76.7±11.5 72.7±11.6 3.65±2.86 <![CDATA[Xu et al.

[15] ]]> 51.7±10.8 44.5±9.4 0.85±1.59 <![CDATA[Yun et al.

[16] ]]> 65.7±13.1 60.1±11.9 4.56±3.73 <![CDATA[Pinho et al.

[17] ]]> 32.1±6.9 26.9±6.9 3.63±4.92 <![CDATA[Born et al.

[18] ]]> 41.7±16.2 34.5±13.2 0.41±1.09 <![CDATA[Bauer et al.

[19] ]]> 63.0±10.4 58.4±13.2 1.44±2.06 <![CDATA[Irving et al .

[20] ]]> 43.5±19.1 36.4±17.1 1.27±2.91 <![CDATA[Feuerstein et al.

[21] ]]> 76.5±13.3 73.3±13.4 15.56±9.52 This application 81.7±13.1 79.9±10.3 5.1±3.6

[0120] As shown in Table 4, among all the participants listed on the official website, FF_ITC achieved the best BD (79.6%) and TD (79.9%). They employed a four-stage airway extraction method, where Hessian analysis was used to detect as many airway candidates as possible. Optimization with the minimum spanning tree did indeed bring better segmentation performance, but at the cost of a high false positive rate (FPR = 11.92%). Similarly, Feuerstein et al.'s segmentation results also showed significant leakage in peripheral branches, resulting in the highest FPR (15.56%), although their BD and TD scores were also relatively good. Clearly, our method has the best performance in connectivity metrics (81.7% BD and 79.9% TD), and more encouragingly, our FPR is not significantly higher than the previous two studies. This indicates that our method, while ensuring good detection performance of connected regions, also eliminates many "false targets" resembling small airways, and significantly improves airway leakage. Compared with other methods, our method is highly competitive in connectivity metrics. Ultimately, however, FPR is a drawback for our method in terms of performance. It shares a weakness with some previous work: in order to ensure overall airway segmentation, it focuses excessively on the recognition performance of small, distal airways, thus introducing many false positives.

[0121] Born et al. proposed the Vicinity-Sensitive 3D Region Growing method, which uses user-defined seed points in the airway to gradually grow and expand to the entire airway region. The improvement of this method lies in ensuring that the segmented region does not overflow the airway wall, thus mitigating oversegmentation. Therefore, it achieved the best false positive rate (0.41% FPR) among all participants, but this also reduces the ability to identify distal airways. Xu et al. innovated fuzzyconnectedness theory to measure airway wall thickness, preventing the segmented region from infiltrating into adjacent lung parenchyma beyond the blurred airway, largely eliminating false positive samples (0.85% FPR). However, both of these methods are unsatisfactory in predicting airway connectivity regions.

[0122] In summary, this invention provides an airway segmentation method that considers both the overall picture and local details. On one hand, by proposing a newly designed PWFR module, this invention enhances the representational ability of effective features, alleviates inter-class imbalance, and improves the continuity of airway segmentation. On the other hand, by adding the AttdKD module, this invention emphasizes attention to distal small airways. Experimental results on publicly available datasets based on this invention demonstrate that our proposed method can achieve accurate segmentation of the main airway structure and accurate identification of small airways, showing great promise for clinical applications.

[0123] References:

[0124] [1] Y. Qin et al., "Learning tubule-sensitive CNNs for pulmonaryairway and artery-vein segmentation in CT," IEEE Trans. Med Imaging., vol. 40, no. 6, pp. 1603-1617, Jun. 2021.

[0125] [2] W. Zhu et al., "AnatomyNet: Deep learning for fast and fully automated whole-volume segmentation of head and neck anatomy," Med. Phys., vol. 46, no. 2, pp. 576-589, Feb. 2019.

[0126] [3] AM Rickmann, AG Roy, I. Sarasua, N. Navab, and C Wachinger, "'Project \& Excite' Modules for Segmentation of Volumetric Medical Scans," in Proc. Med. Image Comput. Comput. Assist. Interv. - MICCAI 2019, Pt II., 2019.

[0127] [4] T. Huang,S. You,F,Wang,C. Qian,and C. Xu,“Knowledge distillationfrom a stronger teacher,” in Proc. Conf. Neural Inf. Process. Syst.,vol. 35,pp. 33716–33727,2022.

[0128] [5] T. Liu,C. Chen,X. Yang,and W. Tan,“Rethinking knowledgedistillation with raw features for semantic segmentation,” in Proc. IEEEWinter Conf. Appl. Comput. Vis.,pp. 1155–1164,2024.

[0129] [6] C. Yang,H. Zhou,Z. An,X. Jiang,Y. Xu,and Q. Zhang,“Cross-imagerelational knowledge distillation for semantic segmentation,” in Proc. IEEE / CVF Conf. Comput. Vis. Pattern Recognit.,pp. 12319–12328,2022.

[0130] [7] A. Garcia-Uceda. Juarez,H. A. W. M. Tiddens,and M. de. Bruijne,“Automatic Airway Segmentation in Chest CT Using Convolutional NeuralNetworks,” in Proc. Med. Image Comput. Comput. Assist. Interv. – MICCAI2018.,vol. 11046,pp. 234-242,2018.

[0131] [8] A. Garcia-Uceda. Juarez,R. Selvan,Z. Saghir,and M. de. Bruijne,“AJoint 3D UNet-Graph Neural Network-Based Method for Airway Segmentation fromChest CTs,” in Proc. Mach. Learn. Med. Imaging.,vol. 11861,pp. 123–130,2019.

[0132] [9] Y. Qin et al.,“AirwayNet: A Voxel-Connectivity Aware Approach forAccurate Airway Segmentation Using Convolutional Neural Networks,” in Proc.Int. Conf. Med. Image Comput. Comput. Assist.,vol. 11769,2019.

[0133]

[10] C. Wang et al.,“Tubular Structure Segmentation Using SpatialFully Connected Network with Radial Distance Loss for 3D Medical Images,” inProc. Int. Conf. Med. Image Comput. Comput. Assist.,vol. 11769,2019.

[0134]

[11] H. Zheng et al.,“Alleviating Class-Wise Gradient Imbalance forPulmonary Airway Segmentation,” IEEE Trans. Med Imaging.,vol. 40,no. 9,pp.2452-2462,Sep. 2021.

[0135]

[12] Wang,Andong et al.,“NaviAirway: a bronchiole-sensitive deeplearning-based airway segmentation pipeline for planning of navigationbronchoscopy,” IEEE Trans. Med Imaging.,2022.

[0136]

[13] Y. Nan et al.,“Fuzzy Attention Neural Network to TackleDiscontinuity in Airway Segmentation,” IEEE Trans. Neural Netw. Learn. Syst.,May. 2023.

[0137]

[14] M. Zhang,and Y. Gu,“Towards Connectivity-Aware Pulmonary AirwaySegmentation,” IEEE J. Biomed. Health Informat.,vol. 28,no. 1,pp. 321-332,Jan. 2024.

[0138]

[15] Z. Xu,U. Bagci,B. Foster,A. Mansoor,J. K. Udupa,and D. J.Mollura,“A hybrid method for airway segmentation and automated measurement ofbronchial wall thickness on CT,” Med. Image Anal.,vol. 24,no. 1,pp. 1–17,Aug.2015.

[0139]

[16] J. Yun et al.,“Improvement of fully automated airwaysegmentation on volumetric computed tomographic images using a 2.5dimensional convolutional neural net,” Med. Image Anal.,vol. 51,pp. 13-20,Jan. 2019.

[0140]

[17] R. Pinho,S. Luyckx and J. Sijbers,“Robust Region Growing BasedIntrathoracic Airway Tree Segmentation,” in Proc. Second Int. Workshop Pulm.Image Anal.,2009.

[0141]

[18] S Born,D. Lwamaru,M. Pfeifle,and D. Bartz,“Three-StepSegmentation of the Lower Airways with Advanced Leakage-Control,” in Proc.Second Int. Workshop Pulm. Image Anal.,2009.

[0142]

[19] C. W. Bauer,H,Bischof,and R Beichel,“Segmentation of AirwaysBased on Gradient Vector Flow,” in Proc. Int. Conf. Med. Image Comput.Comput. Assist. Interv.,2009.

[0143]

[20] B. Irving,P. Taylor,and A. Toddpokropek,“3D segmentation of theairway tree using a morphology based method,” in Proc. Second Int. WorkshopPulm. Image Anal.,2009.

[0144]

[21] M. Feuerstein,T. Kitasaka,and K. Mori,“Adaptive Branch Tracingand Image Sharpening for Airway Tree Extraction in 3-D Chest CT,” in Proc.Second Int. Workshop Pulm. Image Anal.,2009.

Claims

1. An image-based method for lung, bronchus, and airway segmentation, characterized in that, This is achieved by inputting lung and bronchial airway images into a trained image-based lung and bronchial airway segmentation network. The image-based lung and bronchial airway segmentation network consists of an encoding network, a neck network, and a decoding network. The encoding network consists of a first encoding module, a second encoding module, a third encoding module, and a fourth encoding module connected in sequence. Each encoding module consists of a first convolutional block module, a second convolutional block module, and a PWFR module connected in sequence. The neck network consists of a first convolutional block module, a second convolutional block module, and a PWFR module connected in sequence. The decoding network consists of a first decoding module, a second decoding module, a third decoding module, and a fourth decoding module connected in sequence. Each decoding module consists of a first convolutional block module, a second convolutional block module, an AttdKD module, and a PWFR module connected in sequence. The output of the PWFR module of the first encoding module and the output of the first convolutional block module of the third decoding module are connected to the input of the fourth feature stitching module, and the output of the fourth feature stitching module is connected to the input of the fourth decoding module. The output of the PWFR module of the second encoding module and the output of the first convolutional block module of the second decoding module are connected to the input of the first convolutional block module. The output of the Block module is connected to the input of the third feature concatenation module, and the output of the third feature concatenation module is connected to the input of the third decoding module; the output of the PWFR module of the third encoding module is connected to the first Convolutional module of the first decoding module. The output of the Block module is connected to the input of the second feature stitching module, and the output of the second feature stitching module is connected to the input of the second decoding module; the output of the PWFR module of the fourth encoding module and the output of the PWFR module of the neck network are connected to the input of the first feature stitching module, and the output of the first feature stitching module is connected to the input of the first decoding module; the PWFR module consists of a channel splitter, a feature recalibration module, and an average point-level operation module connected in sequence; the AttdKD module consists of a teacher module and a student module; the teacher module includes a CAM module, a first convolutional thinning module, a SAM module, and a second convolutional thinning module connected in sequence, and the second input of the second convolutional thinning module is connected to the input of the first convolutional thinning module; the student module includes an element-wise multiplication module, and the second input of the first convolutional thinning module, the input of the CAM module, and the input of the element-wise multiplication module are all connected to the same output.

2. The image-based lung bronchus airway segmentation method according to claim 1, characterized in that, In the PWFR module, the number of channel splits N for the channel splitter is set to 3.

3. The image-based lung bronchus airway segmentation method according to claim 1, characterized in that, The specific operating steps are as follows: Step 1: Construct an image-based lung-bronchial-airway segmentation network; Step 2: Construct a lung, bronchial, and airway image dataset for a deep learning network used for image segmentation; Step 3: Enhance the lung, bronchus, and airway image dataset obtained in Step 2; Step 4: Using the lung, bronchus, and airway image dataset obtained in Step 3, train the image-based lung, bronchus, and airway segmentation network constructed in Step 1 so that the network can achieve lung, bronchus, and airway segmentation. Step 5: Input the collected lung bronchus airway images into the image-based lung bronchus airway segmentation network trained in Step 4, and output accurate lung bronchus airway segmentation status.

4. The image-based lung bronchus airway segmentation method according to claim 1, characterized in that, In step 2, the lung and bronchial airway image dataset is either the Airway Tree Modeling dataset or a dataset constructed from self-acquired lung and bronchial airway CT images.

5. The image-based lung bronchus airway segmentation method according to claim 1, characterized in that, In step 3, the enhancement processing method is any one of random rotation and scaling, random flipping, random cropping, random brightness and contrast adjustment, elastic deformation, and random noise.

6. The image-based lung bronchus airway segmentation method according to claim 1, characterized in that, In step 4, the training of the image-based lung-bronchial-airway segmentation network aims to minimize the distillation loss function in the AttdKD module; where the distillation loss function... The expression is: ， In the formula, N is the total number of samples in the current batch. and This is a spatial feature map of two adjacent layers. This represents the square of the Frobenius norm.

7. The image-based lung bronchus airway segmentation method according to claim 1, characterized in that, In step 4, the training parameters are set as follows: the image training size is set to 128*224*304, the batch size is set to 1, the step size is set to 64, and the initial learning rate is set to 0.003.

Images