Image-based lung bronchial airway segmentation method

Abstract

The invention discloses a lung bronchial airway segmentation method based on an image. The method is realized by inputting a lung bronchial airway image into a trained lung bronchial airway segmentation network based on the image. The image-based lung bronchial airway segmentation network is composed of a coding network, a neck network and a decoding network. The coding network comprises four coding modules, and a PWFS module is additionally arranged in the four coding modules; the decoding network comprises four decoding modules, and an AttdKD module and a PWFR module are sequentially and additionally arranged in the four decoding modules; the lung bronchial airway segmentation method based on the image takes account of the whole airway segmentation method and pays attention to the local airway segmentation method, the characterization capability of effective features can be enhanced, the inter-class imbalance phenomenon can be relieved, the airway segmentation continuity can be improved, accurate segmentation of the airway main body and accurate recognition of the small airway can be achieved, and the accuracy of the lung bronchial airway segmentation is improved. Good clinical application prospects are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image segmentation, and particularly to an image-based lung bronchial airway segmentation method. Background Art

[0002] Accurate lung airway segmentation is a key prerequisite for preoperative planning and intraoperative navigation in bronchoscopic interventional treatment of lung diseases. The lung airway is a fine-grained structure with small and narrow distal airways, complex and tortuous shapes, making manual annotation time-consuming and laborious, and highly subject-dependent. To relieve the burden on doctors and improve the quality and efficiency of lung airway annotation, automatic airway segmentation algorithms have been continuously updated and optimized, especially with the introduction of deep learning. These methods use multi-level feature learning and context understanding to select appropriate network architectures and training models, aiming to build an efficient and accurate medical image analysis system to liberate radiology experts from the cumbersome manual annotation process. However, in practical applications, the distal small airways and pulmonary micro-nodules have little imaging information, and due to partial volume effects, noise, and artifacts during the imaging process, the detailed features are blurred, the recognition rate is low, and it is difficult to effectively guide preoperative accurate path planning and intraoperative real-time image guidance.

[0003] Traditional methods, such as region growing, adaptive thresholding, and fuzzy connectivity analysis, are effective for detecting thick tubular structures such as the trachea and main bronchi. However, when the airway extends to the 5th or 6th level, the airway wall becomes thinner, narrow and tortuous, and has a similar gray level to the surrounding tissues, severely limiting the applicability of traditional methods. With the support of a large number of airway-annotated images in the EXACT’09 dataset, more and more scholars tend to use convolutional neural networks (CNNs) to learn highly robust and discriminative features, and different-dimensional CNNs have also been gradually applied to the airway segmentation task. For example, Charbonnier et al. used 2D CNNs in the post-processing of airway leakage, and Yun et al. used 2.5D CNNs to extract more spatial features on adjacent slices to enhance the segmentation ability of the airway. Compared with lower dimensions, 3D CNNs have become the mainstream of airway feature extraction due to the integrity and consistency of their predictions, and some typical networks show very excellent performance, such as AirwayNet, WingsNet, and NaviAirway. However, problems such as local discontinuous mapping and airway leakage are still the main barriers restricting the performance improvement of CNNs in airway segmentation. Such phenomena mainly occur in the peripheral regions of the lungs. The main reason for this is the severe inter-class imbalance problem between the distal small airways and the background, and the airway wall is blurred and has similar gray features to the peripheral tissues; another limitation is that the computational complexity of CNNs is relatively high, which forces CNNs to be trained on image patches, making it difficult for the network to detect some small mistakes during the process before stitching all the patches into a complete image. How to accurately identify distal small airways is the key to solving the above problems.

[0004] Current work tends to design complex loss functions to achieve accurate identification of distal small airways and solve problems such as local discontinuous mapping and airway leakage of the airway. For example, by combining losses such as wBCE, Dice loss, Connectivity-Aware Surrogate, Local-Sensitive Distance, radial distance loss, and GeneralUnion loss. The optimized design of the loss function can indeed enhance the network's screening ability for some small airways to a certain extent, but these methods still do not fully consider the variability of airway structures caused by pathological factors such as patient bronchiectasis and bronchial wall thickening in the real environment, which may reduce the clinical generalization ability of the model.

[0005] Enhancing the ability to focus on the distal airway is also an effective way to address the discontinuity problem. The attention mechanism shows good recognition ability for complex structures by weighting the "importance" of different feature representations. Qin et al. alleviated the gradient erosion problem by introducing attention distillation. Nan et al. designed a fuzzy attention layer to enhance the recognition ability of small airway branches. Ke et al. proposed a method of attention-supervised multi-scale features. Chen et al. introduced an attention-guided network based on the transformer. Zeng et al. developed a multi-scale feature joint reverse attention network based on u-net, improving the ability to extract fine edge features. The Transformer model formally defined the attention mechanism. With the strong performance shown by Self-Attention, Multi-head Attention, and Cross-Attention, the Transformer has reached the peak of performance. Tang et al. designed an adversarial transformer to solve the discontinuity of peripheral bronchioles. Wu et al. proposed a context transformer for segmenting small airway branches. Other forms of transformers, such as U-Net transformer, Squeezeand-Expansion transformer, and hierarchical transformer, also show satisfactory performance in difficult-to-segment samples of medical images. Although the attention mechanism has made significant progress in medical image segmentation, challenges such as poor model interpretability, unstable attention distribution, and overfitting still exist. Problems such as airway leakage and local discontinuous mapping are still the main challenges in optimizing airway segmentation performance. The heterogeneity of imaging data between different medical institutions and different airway branching patterns between individuals further exacerbate this challenge.

[0006] To address the above situation, we propose an effective optimization network to improve the recognition ability of distal small airways and alleviate problems such as airway leakage and local discontinuous mapping. We infer that after re-extracting features according to the contribution degree and then performing attention-driven knowledge distillation, the context feature maps enriched in both the channel and spatial dimensions can capture more important details of the segmentation target compared to the attention mechanism that only focuses on the channel or space. Our model was tested for performance on the EXACT-09 dataset and the ATM dataset, comprehensively analyzed using multiple evaluation metrics such as TD, BD, and FPR, and achieved significant segmentation performance. The improvement in performance is more importantly reflected in the recognition ability of distal small airways. Summary of the Invention

[0007] The object of the present invention is to provide an image-based pulmonary bronchial airway segmentation method for solving problems such as airway leakage and local discontinuous mapping during pulmonary bronchial airway segmentation.

[0008] To this end, the technical solution of the present invention is as follows:

[0009] An image-based lung bronchial airway segmentation method is realized by inputting lung bronchial airway images into a trained image-based lung bronchial airway segmentation network. The image-based lung bronchial airway segmentation network consists of an encoding network, a neck network, and a decoding network. The encoding network is composed of a first encoding module, a second encoding module, a third encoding module, and a fourth encoding module connected in sequence. Each encoding module is composed of a first Convolutional Block module, a second Convolutional Block module, and a PWFS module connected in sequence. The neck network is composed of a first Convolutional Block module, a second Convolutional Block module, and a PWFS module connected in sequence. The decoding network is composed of a first decoding module, a second decoding module, a third decoding module, and a fourth decoding module connected in sequence. Each decoding module is composed of a first Convolutional Block module, a second Convolutional Block module, an AttdKD module, and a PWFR module connected in sequence. The output end of the PWFS module of the first encoding module and the output end of the first Convolutional Block module of the third decoding module are connected to the input end of the fourth feature splicing module, and the output end of the fourth feature splicing module is connected to the input end of the fourth decoding module. The output end of the PWFS module of the second encoding module and the output end of the first Convolutional Block module of the second decoding module are connected to the input end of the third feature splicing module, and the output end of the third feature splicing module is connected to the input end of the third decoding module. The output end of the PWFS module of the third encoding module and the output end of the first Convolutional Block module of the first decoding module are connected to the input end of the second feature splicing module, and the output end of the second feature splicing module is connected to the input end of the second decoding module. The output end of the PWFS module of the fourth encoding module and the output end of the PWFS module of the neck network are connected to the input end of the first feature splicing module, and the output end of the first feature splicing module is connected to the input end of the first decoding module. The PWFR module is composed of a channel splitter, a feature recalibration module, and an average point-level operation module connected in sequence. The AttdKD module consists of a teacher module and a student module. The teacher module includes a CAM module, a first convolutional refinement module, a SAM module, and a second convolutional refinement module connected in sequence, and the second input end of the second convolutional refinement module is connected to the input end of the first convolutional refinement module. The student module includes an element-wise multiplication module, and the second input end of the first convolutional refinement module, the input end of the CAM module, and the input end of the element-wise multiplication module are all connected to the same output end.

[0010] Further, in the PWFR module, the number N of channel divisions of the channel splitter is set to 3.

[0011] Furthermore, the specific operation steps of the image-based lung bronchial airway segmentation method are as follows

[0012] Step 1: Construct an image-based lung bronchial airway segmentation network;

[0013] Step 2: Construct a lung bronchial airway image dataset for the deep learning network used for image segmentation;

[0014] Step 3: Perform enhancement processing on the lung bronchial airway image dataset obtained in Step 2;

[0015] Step 4: Use the lung bronchial airway image dataset obtained in Step 3 to train the image-based lung bronchial airway segmentation network constructed in Step 1, so that the network can achieve lung bronchial airway segmentation;

[0016] Step 5: Input the collected lung bronchial airway images into the image-based lung bronchial airway segmentation network trained in Step 4, and output the accurate lung bronchial airway segmentation status.

[0017] Furthermore, in Step 2, the lung bronchial airway image dataset adopts the Airway Tree Modeling dataset or a dataset constructed from self-collected lung bronchial airway CT images.

[0018] Furthermore, in Step 3, the enhancement processing method is any one of random rotation and scaling, random flipping, random cropping, random brightness and contrast adjustment, elastic deformation, and random noise.

[0019] Furthermore, in Step 4, the training of the image-based lung bronchial airway segmentation network aims to minimize the distillation loss function in the AttdKD module; among them, the distillation loss function L AttdKD is expressed as:

[0020]

[0021] In the formula, N is the total number of samples in the current batch, A i ″- and A i ″ +1 are the spatial feature maps of two adjacent layers, represents the square of the Frobenius norm.

[0022] Furthermore, in Step 4, the training parameter settings include: the image training size is set to 128*224*304, the batch size is set to 1, the step size is set to 64, and the initial learning rate is set to 0.003.

[0023] Compared with the prior art, the image-based lung bronchial airway segmentation method alleviates the inter-class imbalance phenomenon, improves the continuity of airway segmentation, can achieve accurate segmentation of the main airway and accurate identification of small airways, and has good clinical application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a flowchart of the image-based lung bronchial airway segmentation method of the present invention;

[0025] Figure 2 is a schematic diagram of the network structure of the image-based lung bronchial airway segmentation network of the present invention;

[0026] Figure 3 is a schematic diagram of the PWFR module structure of the image-based lung bronchial airway segmentation network of the present invention;

[0027] Figure 4 is a schematic diagram of the AttdKD module structure of the image-based lung bronchial airway segmentation network of the present invention;

[0028] Figure 5 is a schematic diagram of a confusion matrix for binary classification;

[0029] Figure 6 is a comparison chart of the visualization effects of the method adopted in the embodiment of the present invention and other excellent algorithms;

[0030] Figure 7 is a visualization effect diagram of the results after adding each module of the method adopted in the embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0031] The following further describes the present invention in conjunction with the accompanying drawings and specific embodiments, but the following embodiments are by no means any limitation to the present invention.

[0032] Refer to Figure 1 , and the specific implementation of the image-based lung bronchial airway segmentation method is described as follows.

[0033] Step 1: Construct an image-based lung bronchial airway segmentation network.

[0034] The lung bronchial airway image segmentation network is improved based on the 3D-UNet as the network base model. By adding a Point-wise Feature Recalibration (hereinafter referred to as the PWFR module) in the encoding module of the 3D-UNet, and in addition to adding the PWFR module in the decoding module of the 3D-UNet, an Attention-driven Knowledge Distillation (hereinafter referred to as the AttdKD module) is added to realize the reconstruction of the network structure; at the same time, based on the actual segmentation effect of the lung bronchial airway image, the optimal number of construction of the encoding modules in the encoding network and the specific number of construction of the decoding modules in the decoding network are determined to achieve the optimal segmentation effect.

[0035] As Figure 3 shown, the PWFR module is a new designed module based on the segmentation requirements of this application, which is specifically composed of a channel splitter, a feature recalibration module, and an average point-level operation module connected in sequence.

[0036] In the Channel Splitter, the number of channel splits N is set to 3, that is, the Channel Splitter is used to first divide the channels of the input feature map into three subgroups: Sub-g FM 1, Sub-g FM 2, and Sub-g FM 3 to reduce the computational complexity.

[0037] The Feature Recalibration module is used to process the feature maps of each subgroup respectively; when a given original feature map F of any subgroup C×H×W is provided, this module automatically extracts the features with the highest and medium contribution degrees in the corresponding grouped channels, and weakens the features with the lowest contribution degree, so that different features are weighted according to their contributions to the task, so that different features can be divided into the Highest Contribution group, the Moderate Contribution group, and the Lowest Contribution group respectively; in Figure 3 , the blue arrow indicates the flow direction of the highest contribution features in the feature maps of each subgroup after weight assignment, the green arrow indicates the flow direction of the medium contribution features, and the orange arrow indicates the flow direction of the lowest contribution features; subsequently, in each group re-allocated according to different contribution degrees, the maximum value (Max Point-wise), the median value (Median Point-wise), and the minimum value (Min Point-wise) of each group are extracted respectively to obtain the feature output T m that captures local features and diverse information expressions, and its expression is:

[0038]

[0039] Wherein, 1 ≤ i ≤ H, 1 ≤ j ≤ W, 1 ≤ k, k + 1, k + 2 ≤ C, and H, W, and C refer to the height, width, and number of channels of the original feature map; respectively represent the highest contributing features in the three subgroups, respectively represent the medium contributing features in the three subgroups, respectively represent the lowest contributing features in the three subgroups;

[0040] In the feature rejection and squeezing process of the above feature recalibration module, the spatial size of the feature map remains unchanged; furthermore, during the training process, as the effective features are continuously encouraged and the redundant features are continuously suppressed, the key regions of the main airway are gradually given priority, and other invalid features such as noise signals are gradually ignored.

[0041] The Average Point-wise operation module obtains the feature output T of the feature recalibration module m , to fuse the maximum value among the features with the highest contribution degree the median value of the features with the medium contribution degree and the minimum value of the features with the lowest contribution degree otherwise, and generate the final output feature map by fusing different point-wise features; the expression of the specific processing process of this module is

[0042] S m = E(T m ) = r(N 2 * d(N 1 * T m ))

[0043] Wherein, r(·) and d(·) are the non-linear activation functions of Leaky ReLU and Sigmoid.

[0044] In the above processing process, the convolution operator (*) respectively realizes channel reduction and channel restoration through convolution kernels with sizes of N 1 and N 2 ; the channel reorganization realized through the excitation operation will encourage the effective feature channels and suppress the redundant feature channels, thereby further enhancing the network's learning ability for highly discriminative features. According to the actual processing, the feature map is characterized by retaining the significant features in the high contribution regions (such as bright yellow), while suppressing the redundant and unimportant parts, thereby improving the effectiveness of feature expression.

[0045] This PWFS module is different from the conventional method of directly fusing multi-scale features in the past. Its aim is to emphasize the important features on different feature channels, eliminate weak semantic features, and be able to learn more highly discriminative and useful features from relatively few feature maps, further highlighting the main position of the airway in the learning process and alleviating the problem of local discontinuous mapping, thereby enhancing the network's overall representation ability of the airway, as Figure 3 shown.

[0046] See Figure 4 , the AttdKD module consists of a Teacher module and a Student module; the Teacher module includes a CAM module, a first Convolution and Refinement module, a SAM module, and a second Convolution and Refinement module connected in sequence, and the second input end of the second Convolution and Refinement module is connected to the input end of the first Convolution and Refinement module; the Student module includes an Element-wiseMultiplication module, and the second input end of the first Convolution and Refinement module, the input end of the CAM module, and the input end of the Element-wiseMultiplication module are all connected to the same output end; among them, both the CAM module and the SAM module are existing conventional modules, and their combined use in the AttdKD module constitutes the CBAM attention mechanism.

[0047] The specific processing process of the AttdKD module for the input feature map is: the feature map enters the Teacher module and the Student module for processing simultaneously; among them,

[0048] In the Teacher module, the feature map generates a high-quality feature representation through a complex attention mechanism (CBAM); in the attention mechanism, the CAM module is a channel attention module, and its channel-attention descriptor is defined as: That is, the intermediate feature map is respectively subjected to global max pooling and global average pooling to aggregate the spatial information of the feature map, and then sent to a shared multi-layer perceptron to compress the spatial dimension of the input feature map and retain the channel information, and then generate a channel attention map through sigmoid activation. Among them, average pooling has feedback on each pixel point on the feature map, while max pooling only has gradient feedback at the place with the largest response in the feature map during the calculation of gradient backpropagation; therefore, this descriptor highlights the meaningful regions in the image, especially sensitive task-related regions such as distal small bronchi, and at the same time blurs the regions irrelevant to the segmentation task; and the SAM module is a spatial attention module, and its Spatial-attention descriptor is expressed as: That is, the feature map output by the channel attention module is pooled in the channel dimension, and the spatial feature information is retained. The pooled features are connected through concatenate and activated by sigmoid to obtain the spatial attention weights;

[0049] In the Student path, the feature map is optimized and learned under the supervision of the Teacher through a lightweight structure; finally, the AttdKD module optimizes the feature map output by the Student path through the distillation loss function L AttdKD That is, by minimizing the attention loss, the intermediate feature maps between adjacent layers are made more similar, gradually approaching the feature expression of the feature map in the Teacher path, and significantly improving the feature extraction ability and performance of the lightweight model; specifically, the distillation loss function L AttdKD The expression of is:

[0050]

[0051] In the formula, N is the total number of samples in the current batch, A i ″- and A i ″ +1 Are the spatial feature maps of two adjacent layers, Represents the square of the Frobenius norm; since it is noted that the latter layer acts as the teacher of the former layer, the "attention strength" will slowly penetrate from the deep layer to the shallow layer.

[0052] In summary, different from previous works that rely on prior knowledge and fine feature extraction or design complex losses, the AttdKD module of this application makes full use of the attention mechanism in CBAM, combines channel and spatial information to condense feature distillation. The emphasized features eliminate background noise, fully aggregate spatial knowledge and channel knowledge to condense features, so as to enhance the attention ability to "micro-targets" such as distal small airways under the state of class imbalance and alleviate the problem of local discontinuous mapping. CBAM consists of a channel attention module and a spatial attention module. At different stages of the network, CBAM aggregates spatial descriptors and channel descriptors, multiplies them with the corresponding original feature maps, introduces cross-layer and cross-space information into the original feature maps, and creates new rich context feature maps. Furthermore, CBAM combines spatial and channel dual attention modules, enabling the spatial attention map to propagate along the channels and the channel attention map to propagate along the spatial dimension. Compared with the attention mechanism that only focuses on channels or space, it can better highlight the important features of the image, forcing the student network to imitate the important regions emphasized by the teacher, making it a powerful candidate with significant distillation potential; in addition, according to the performance comparison results, AttdKD does not increase the network complexity and can be integrated into the existing CNN network to achieve efficient feature correction and improve performance.

[0053] Specifically, the construction steps of the image-based lung bronchial airway segmentation network are described as follows.

[0054] Step 1.1: Based on the encoding module of 3D-UNet, a PWFR module is added to the tail of the encoding module in the network of this application to form the encoding module of the image-based lung bronchial airway segmentation network.

[0055] Based on the improved construction method in the above Step 1.1, the encoding module of this application is composed of a first Convolutional Block module, a second Convolutional Block module, and a PWFS module connected in sequence.

[0056] Step 1.2: Based on the decoding module of 3D-UNet, an AttdKD module and a PWFR module are sequentially added to the tail of the decoding module in the network of this application to form the decoding module of the image-based lung bronchial airway segmentation network.

[0057] Based on the improved construction method in the above Step 1.2, the decoding part of this application is composed of a first Convolutional Block module, a second Convolutional Block module, an AttdKD module, and a PWFR module connected in sequence.

[0058] In the above Steps 1.1 and 1.2, by integrating the PWFR module at the end of each sampling, it is used to separate and eliminate weak features, select and strengthen effective features, and "treat differently" the spatial features at different positions before feature fusion, so as to emphasize the main position of the airway in the network learning process; specifically, separate and eliminate weak features, select and strengthen effective features, and "treat differently" the spatial features at different positions before feature fusion, so as to emphasize the main position of the airway in the network learning process. When assuming that features at different positions in different channels have different contribution degrees to airway recognition, the PWFR module will select features beneficial to airway representation ability and discard redundant features.

[0059] The AttdKD module is only placed in the decoder because the low-dimensional features generated by encoding have low discriminability and little contribution to attention extraction. Therefore, it is not meaningful to place the AttdKD module in the downsampling process of the encoding part; while setting it in the decoding part can generate high-dimensional and highly discriminative features during decoding. The attention map obtained by the AttdKD module is more helpful for highlighting important regions such as distal small airways, and can give full play to its ability in extracting detailed features to a greater extent. Specifically: without separately setting two different models, the latter layer plays the role of a teacher, and transfers attention to the previous layer in the same model by improving the feature map considering channel and spatial information to capture richer detailed features for optimizing the recognition performance of distal small airways.

[0060] Step 1.3. Construct an image-based lung bronchial airway segmentation network, as Figure 2 shown. The network is specifically composed of an encoding network, a neck network, and a decoding network. Among them,

[0061] The encoding network is composed of a first encoding module, a second encoding module, a third encoding module, and a fourth encoding module connected in sequence. Each encoding module is composed of a first Convolutional Block module, a second Convolutional Block module, and a PWFS module connected in sequence.

[0062] The neck network is also composed of a first Convolutional Block module, a second Convolutional Block module, and a PWFS module connected in sequence.

[0063] The decoding network is composed of a first decoding module, a second decoding module, a third decoding module, and a fourth decoding module connected in sequence. Each decoding module is composed of a first Convolutional Block module, a second Convolutional Block module, an AttdKD module, and a PWFR module connected in sequence.

[0064] The output end of the PWFS module of the first encoding module and the output end of the first Convolutional Block module of the third decoding module are respectively connected to the input end of the fourth feature splicing module, and the output end of the fourth feature splicing module is connected to the input end of the fourth decoding module. The output end of the PWFS module of the second encoding module and the output end of the first Convolutional Block module of the second decoding module are respectively connected to the input end of the third feature splicing module, and the output end of the third feature splicing module is connected to the input end of the third decoding module. The output end of the PWFS module of the third encoding module and the output end of the first Convolutional Block module of the first decoding module are respectively connected to the input end of the second feature splicing module, and the output end of the second feature splicing module is connected to the input end of the second decoding module. The output end of the PWFS module of the fourth encoding module and the output end of the PWFS module of the neck network are respectively connected to the input end of the first feature splicing module, and the output end of the first feature splicing module is connected to the input end of the first decoding module.

[0065] In this image-based pulmonary bronchial airway segmentation network, four encoding modules gradually extract the features of the feature maps while performing downsampling. The size of the feature maps continuously decreases, and the number of channels increases. The Bottleneck network further optimizes the features, highlighting the features of the airway region. After the above optimization steps, the feature maps enter the decoding network and are gradually upsampled through four decoding modules to restore the low-dimensional deep features of the feature maps to high-dimensional spatial features. At the same time, skip connections are made between the encoding modules and the decoding modules, and the concatenation operation is completed through the feature concatenation module, enhancing the combination of low-level features and high-level features and improving the details and boundary quality of the segmentation. Finally, the fourth decoding module outputs the processed feature maps. As Figure 2 can be seen from the changing trends of the feature maps output by each decoding module shown, the processed feature maps show a process of gradually concentrating (or focusing) towards the small bronchial parts.

[0066] In summary, through the above steps 1.1 to 1.3, the pulmonary bronchial airway image segmentation network of this application is constructed to complete the segmentation of the pulmonary bronchus based on images.

[0067] See Figure 2 , the structure and processing process of the image-based pulmonary bronchial airway segmentation network of this application are as follows:

[0068] Image input end, with an image input size of 128*224*304;

[0069] Encoding network, which is composed of a first encoding module, a second encoding module, a third encoding module, and a fourth encoding module connected in sequence; each encoding module is composed of a first Convolutional Block, a second Convolutional Block, and a PWFR module connected in sequence; the image size output by the first encoding module is 16*40*96*152, the image size output by the second encoding module is 32*20*48*76, the image size output by the third encoding module is 64*10*24*38, and finally the image size output by the fourth encoding module is 128*5*12*19;

[0070] The Bottleneck network is set between the encoding network and the decoding network and is composed of a first Convolutional Block, a second Convolutional Block, and a PWFR module in sequence;

[0071] A decoding network, which is composed of a first decoding module, a second decoding module, a third decoding module, and a fourth decoding module connected in sequence; each decoding module is composed of a first Convolutional Block, a second Convolutional Block, an AttdKD module, and a PWFR module; the image size output by the first decoding module is 64*10*24*38, the image size output by the second decoding module is 32*20*48*76, the image size output by the third decoding module is 16*40*96*152, and the image size output by the fourth decoding module is 128*224*304, which is the lung bronchial airway image segmentation result map;

[0072] In the above encoding network, neck network, and decoding network, each Convolutional Block has the same architecture, and is composed of a 3×3×3 convolutional layer, an instance normalization module, and a ReLU activation function connected in sequence.

[0073] It should be noted here that in the construction of the above deep learning network for image segmentation, the number of encoding modules in the encoding network and the number of decoding modules in the decoding network are both four. This is because according to the actual lung bronchial airway image segmentation effect, when the number of both is <4, the lung bronchial airway image cannot be effectively segmented, resulting in a poor final segmentation effect of the image. When the number of both is >4, the lung bronchial airway image will also have a poor final segmentation effect due to over-segmentation.

[0074] Step 2: Construct a lung bronchial airway image dataset for the deep learning network for image segmentation.

[0075] Specifically, the implementation steps of this step 2 are as follows:

[0076] In this application, the data for training, validating, and testing the lung bronchial airway segmentation network based on images all come from the Airway Tree Modeling dataset (hereinafter referred to as the ATM dataset). In this embodiment, 299 sets of CT images in the ATM dataset are selected and used.

[0077] Step 3: Perform enhancement processing on the lung bronchial airway image dataset obtained in step 2.

[0078] Specifically, the implementation steps of this step 3 are as follows:

[0079] During the training process, data enhancement methods such as random rotation and scaling, random flipping, random cropping, random brightness and contrast adjustment, elastic deformation, and random noise are used.

[0080] Step 4: Use the lung bronchial airway image dataset obtained in Step 3 to train the image-based lung bronchial airway segmentation network constructed in Step 1, so as to achieve the goal of lung bronchial airway segmentation.

[0081] Specifically, the implementation steps of Step 4 are as follows:

[0082] In Step 4.1, the CT images in the lung bronchial airway image dataset constructed in Step 3 are randomly selected and divided into CT images for training, CT images for validation, and CT images for testing according to the ratio of 2.7:1:1.2.

[0083] Step 4.2: Input the training set obtained by Step 4.1 into the deep learning network for lung bronchial airway image segmentation constructed in Step 1, and train the image segmentation network with the labeled images in the dataset as the input and the image segmentation information as the output. At the same time, during the training process, use the validation set to conduct ablation experiment verification on the network after each round of training. Finally, take the weights corresponding to the round with the best recognition accuracy during training as the optimal weights and save them. The training is completed. In this embodiment, the training parameters are shown in Table 1 below.

[0084] Table 1:

[0085] Parameter Name Parameter Value Training Size 128*224*304 Batch Size 1 Step Size 64 Initial Learning Rate 0.003

[0086] Step 5: Input the collected lung bronchial airway images into the image-based lung bronchial airway segmentation network trained in Step 4, and the accurate lung bronchial airway segmentation status can be output.

[0087] Furthermore, in order to evaluate the effectiveness of this method for lung bronchial segmentation, the segmentation accuracy of the method of this application is evaluated respectively. Specifically, the evaluation indicators include Precision, centerline-based Branchesdetected (BD), Tree length detected (TD), voxel wise-based False positive rate (FPR), and region-based Dice similarity coefficient (DSC).

[0088] As Figure 5The figure shows a schematic diagram of the confusion matrix for binary classification; in the figure, T and F refer to the positive and negative samples of the true values, while P and N are the positive and negative of the predicted values. Specifically, T represents the "correct" result, which means the model prediction is consistent with the actual situation, F represents the "incorrect" result, which means the model prediction does not match the actual situation, P usually refers to the category predicted by the model as "of interest". In the segmentation of pulmonary bronchial airways, the positive example usually refers to the pulmonary bronchial airway region, and N refers to the category predicted by the model as "not of interest", usually referring to the background region or non-pulmonary bronchial airway region. There are a total of four classification results; among them, True Positive (TP): The model predicts a positive example (such as a pulmonary bronchial airway), and it is actually a positive example; True Negative (TN): The model predicts a negative example (such as a non-pulmonary bronchial airway or background), and it is actually a negative example; False Positive (FP): The model predicts a positive example (such as a pulmonary bronchial airway), but it is actually a negative example (such as a non-pulmonary bronchial airway or background); False Negative (FN): The model predicts a negative example (such as a background or other non-pulmonary bronchial airway parts), but it is actually a positive example (i.e., the pulmonary bronchial airway part).

[0089] Precision is a measure of the proportion of the regions predicted by the model as "pulmonary bronchial airways" that are correct. In other words, precision focuses on how many of the regions predicted by the model as pulmonary bronchial airways are actually pulmonary bronchi. Precision measures the accuracy of the model when predicting the positive class (i.e., pulmonary bronchial airways). High precision means that the model rarely mispredicts non-pulmonary bronchial airway parts as pulmonary bronchial regions, that is, the model has fewer False Positives. Its expression is:

[0090]

[0091] Centerline-based Branches detected (BD) is used to evaluate whether the segmentation algorithm can detect the complete branch structure of the pulmonary bronchi, representing the proportion of the detected branch number to the actual branch number; BD reflects the global detection ability of the segmentation algorithm for the pulmonary bronchial tree, and its expression is:

[0092]

[0093] In the formula, N detected branches is the number of branches detected in the segmentation result, and N ground truth branches is the actual number of branches in the labeled data.

[0094] The overall length detection ability of bronchus segmentation is measured by Tree length detected (TD), that is, the ratio of the length of the tree-like structure detected in the segmentation result to the true length; TD helps to evaluate the coverage ability of the segmentation algorithm and the capture effect on tiny branches, and its expression is:

[0095]

[0096] In the formula, L detected is the total length of all bronchial branches in the segmentation result; L ground truth is the total length of the true bronchial tree in the labeled data.

[0097] Voxel Wise-based False positive rate (FPR), based on the voxel level, represents the ratio of the voxels misdetected as bronchi in the segmentation result to the total number of voxels; FPR reflects the false detection situation of the algorithm. The lower the FPR, the more accurate the segmentation result, and its expression is:

[0098]

[0099] In the formula, FP (False Positive) is the number of voxels missegmented as bronchi, and TN (True Negative) is the number of voxels correctly segmented as non-bronchi.

[0100] Region-based Dice similarity coefficient (DSC) is a regional overlap metric, which reflects the overlap degree between the segmentation result and the true bronchial region; the closer DSC is to 1, the closer the segmentation result is to the true annotation, and its expression is:

[0101]

[0102] In the formula, A is the set of voxels in the bronchial region of the segmentation result, B is the set of voxels in the bronchial region of the true annotation, |A∩B| is the number of intersecting voxels between the segmentation result and the true region, and |A| + |B| are the numbers of voxels in the segmentation result and the true annotation respectively;

[0103] To verify the superiority of the method of this application compared with the prior art, while evaluating the above indicators, we will also evaluate the same indicators for other prior art methods and conduct a comparative analysis. At the same time, in order to further study the key components in the proposed segmentation framework, an ablation study was carried out; in addition, based on the model training as the baseline, the FR module, SE module and PE module proposed in recent years were introduced respectively on this basis to compare the performance of the newly designed module in this application, that is, the PWFR module.

[0104] The specific test results of this ablation experiment are shown in Table 2 below.

[0105] Table 2:

[0106] Method BD TD FPR DSC Precision Baseline 88.30±5.41 86.09±8.17 0.014±0.024 92.27±1.66 92.21±4.39 +PWFR 90.76±6.21 92.07±9.04 0.018±0.017 92.81±2.06 93.07±5.27 <![CDATA[+FR [1] > 90.33±6.51 91.38±8.13 0.021±0.020 92.24±2.77 92.16±4.09 <![CDATA[+SE [2] > 89.97±7.88 90.73±8.28 0.027±0.012 91.37±1.66 91.81±5.22 <![CDATA[+PE [3] > 88.01±7.03 90.86±9.01 0.025±0.011 92.03±2.17 91.04±4.09 +AttdKD 92.84±6.55 90.09±8.17 0.027±0.021 93.34±1.66 93.48±4.39 <![CDATA[+DistKD [4] > 90.06±8.06 89.63±7.54 0.034±0.022 90.11±3.08 93.01±3.94 <![CDATA[+LAD [5] > 88.30±7.18 89.03±10.37 0.036±0.028 90.97±2.46 91.70±4.07 <![CDATA[+CIRKD [6] > 89.97±6.01 88.34±9.33 0.034±0.024 91.01±2.31 91.44±3.64 Proposed 93.96±7.08 92.71±4.47 0.023±0.010 93.52±0.84 93.97±3.53

[0107] In Table 2, "Baseline" represents the baseline model, specifically, the PWFR modules in each encoding module, as well as the PWFR module and AttdKD module in each decoding module, are removed from the image-based pulmonary bronchial airway segmentation network constructed in this application; "+PWFR" means adding a PWFR module to the tail of each encoding module based on the above baseline model; "+FR" means adding an FR module to the tail of each encoding module based on the above baseline model; "+SE" means adding an SE module to the tail of each encoding module based on the above baseline model; "+PE" means adding a PE module to the tail of each encoding module based on the above baseline model; "+AttdKD" means adding an AttdKD module to the tail of each decoding module based on the above baseline model; "+DistKD" means adding a DistKD module to the tail of each decoding module based on the above baseline model; "+LAD" means adding an LAD module to the tail of each decoding module based on the above baseline model; "+CIRKD" means adding a CIRKD module to the tail of each decoding module based on the above baseline model; "Proposed" means using the image-based pulmonary bronchial airway segmentation network constructed in this application.

[0108] From the test results in Table 2, it can be seen that the benchmark model achieved 88.30% BD, 86.09% TD, 0.014% FPR, 92.27% DSC, and 92.21% Precision; then, the application of the PWFR module makes the multi-scale spatial features "differentiated", the continuous suppression of redundant features and the emphasis on effective features, which enhances the representation ability of effective airway features; in addition, compared with the SE module and PE module that "treat" spatial features "equally", the optimization brought by the PWFR module is more effective, and the improvement in TD performance is particularly obvious. Although the FR module integrates spatial features from three dimensions through weighted combination, it still introduces a small number of low-weight redundant features, and the segmentation performance is slightly inferior to the PWFR module. The application of the Baseline+AttdKD module has achieved +5.14%, +4.65% and +1.38% improvements in BD, TD and Precision. This can be attributed to the AttdKD module combining channel and spatial information to condense features for feature distillation. The resulting feature map enhances the highlighting of tiny targets such as distal small airways, which helps students to more effectively imitate the teacher's attention and better segment small objects. In addition, the performance of the AttdKD module is also better than that of the DistKD module, the LAD module and the CIRKD module. This is mainly because the attention mechanism is introduced to improve the representation ability of local features and improve the interpretability and comprehensibility of the model. It cannot be ignored that with the strong addition of the PWFR module and the AttdKD module, the main body of the airway is fully emphasized. While the distal small airways are focused on, some irrelevant false positives are introduced, making the FPR of the baseline model the best.

[0109] The PWFR module believes that features at different positions in different channels have different degrees of contribution to airway recognition. Therefore, its design is mainly used to select features beneficial to airway characterization ability, discard redundant features, solve the problem of discontinuous airway segmentation, and highlight the main position of the airway. Compared with the baseline model, the PWFR module achieved 90.76% BD, 92.07% TD, and 92.81% DSC. Among them, the improvement in the TD index is particularly obvious, increasing by 6.95%. The AttdKD module is different from distillation methods such as the DistKD module and the CIRKD module. By aggregating spatial and channel dual descriptors, it introduces cross-level and cross-space information into the original feature map, aiming to provide more fine-grained knowledge guidance for the learning of distal small airways, strengthen the attention to "micro-targets", and further highlight the main position of the target airway. This is clearly reflected in the BD index, achieving a significant increase of 5.14%. In terms of TD, compared with the PWFR module, the improvement of the AttdKD module is smaller (4.65% vs 6.95%), which is mainly due to the different focuses of improvement introduced in each module. In addition, both the PWFR module and the AttdKD module introduce additional false positives compared with the baseline model, and the latter introduces relatively more. This is mainly because the AttdKD module pays special attention to "micro-targets" such as small airways, and inevitably introduces some noises with similar appearance, size and other attributes, resulting in a higher false positive rate.

[0110] As Figure 6 shown, the segmentation effect comparison of the ablation experiment is provided. Among them, case24, case40, and case68 shown in the figure are three sets of case CT images randomly selected from the test set. It can be clearly seen from the figure that when only the UNet network is used to segment the image, many small ends of the bronchi are not recognized, that is, the highlighted part is mainly concentrated on the main road; as the PWER module is sequentially added to the UNet network, and the PWER module and the AttKD module are added at the same time, the focus of network learning continuously gathers on the airway area, the background area is also greatly weakened, and the attention of the network is obviously transferred to the small ends of the bronchi, that is, the highlighted part gradually transitions to the small end part of the bronchi; among them, the method of the present invention pays particular attention to the network's attention to small airways in the coronal plane of Case 68, and also achieves a BD score of 96.16%, which indicates that almost all small airways are captured, proving the superiority of the method of the present invention in bronchial recognition and segmentation.

[0111] In this embodiment, many state-of-the-art airway segmentation algorithms were also reproduced and comparative analysis was carried out on the same dataset. Specifically, the performance of CNN networks with similar encoding-decoding structures such as UNet and VNet and some classic derivative networks of UNet (UNet++, nnunet, etc.) in airway segmentation was compared as follows. The results are shown in Table 3 below.

[0112] Table 3:

[0113]

[0114]

[0115] Juarez et al. in 2018 proposed a 3D UNet based on the combined loss of wBCE and Dice for lung trachea segmentation. In addition, by replacing the deepest convolutional layer of UNet with a graph neural network (GNN), BD and TD were increased from 77.91% and 82.81% to 81.44% and 85.74% respectively. However, in our individual test cases, the performance of GNN was not satisfactory, and standard deviations of 22.17% and 19.25% were obtained for BD and TD respectively, which also reflected the instability of the model to some extent, although it achieved a very perfect Precision (94.17%). Qin et al. designed AirwayNet to identify important regions of the airway by emphasizing the connectivity of airway voxels. On this basis, by introducing attention distillation and Feature Recalibration, the slope erosion problem was alleviated and the sensitivity to tubular targets was enhanced, achieving performance improvements of 11.93% and 6.86% for BD and TD respectively. Wang et al. introduced a three-dimensional slice-by-slice convolutional layer to capture the spatial information of slender structures and designed a radial distance loss to strengthen the weight of the tubular centerline during training, helping the network to focus more attention on small targets, thus achieving better TD (89.37%). However, the detection Precision (87.32%) was unbearable to look at, and the coverage rate with the real airway was also hard to describe (low DSC score).

[0116] Wingsnet was proposed to solve the problems of gradient erosion and dilation, and the General Union loss function in it focuses on preprocessing the phenomenon of intra-class imbalance. Compared with the previous study, Wingsnet has similar performance in TD detection, but has obvious improvements of 7.33% and 4.96% in terms of Precision and DSC scores. In addition, Wingsnet also has better performance than NaviAirway in all aspects. Nan et al. used a fuzzy attention neural network with a comprehensive loss function to predict the airway region, especially for the recognition of airway branches. Although the detection accuracy is slightly reduced compared with Wingsnet, there is a small increase in BD and TD values, and there are fewer false positive predictions. The Connectivity-Aware Surrogate Loss (CAS) and Local-Sensitive Distance (LSD) Loss designed by Zhang et al. aim to improve the connectivity of tracheal segmentation and achieve an impressive length detected rate (92.41%), but the overemphasis on small targets leads to over-segmentation problems, resulting in a relatively high false positive rate.

[0117] The visual comparison of the airway segmentation results of the above-mentioned various airway segmentation algorithms in practical applications is as Figure 7 shown. Generally speaking, the proposed method has considerable superiority and is better than the existing methods in terms of multiple evaluation indicators. Specifically, the performance of UNet is relatively poor, especially in the prediction of distal small airways. This is mainly because only the largest connected region is retained in the prediction results, and some small airways are treated as false positives, which also reflects the discontinuity of UNet in the overall prediction of airways. There are problems such as local discontinuous mapping and airway leakage in WingsNet and NaviAirway networks. CAS CNN aims to improve the connectivity of tracheal segmentation and achieve an impressive length detected rate (92.41%), but the overemphasis on small targets leads to over-segmentation problems, resulting in a relatively high false positive rate. However, in this application, the main body of the airway is fully emphasized, and the distal small airways are focused on, making all indicators the best.

[0118] In addition, we use BD, TD, and FPR to measure the performance of the algorithm on the EXACT’09 dataset. The following Table 4 shows the comparison experiment of the lung bronchial airway segmentation method of this application with other segmentation methods.

[0119] Table 4:

[0120] Method BD (%) TD (%) FPR (%) <![CDATA[Neko * > 35.5±8.2 30.4±7.4 0.89±1.78 <![CDATA[UCCTeam * > 41.6±9.0 36.5±7.6 0.71±1.67 <![CDATA[FF_ITC * > 79.6±13.5 79.9±12.1 11.92±13.16 <![CDATA[HybAir * > 51.1±10.9 43.9±9.6 6.78±26.60 <![CDATA[MISLAB * > 42.9±9.6 37.5±7.1 0.89±1.64 <![CDATA[NTNU * > 31.3±10.4 27.4±9.6 3.60±3.37 <![CDATA[Qin et al. [1] > 76.7±11.5 72.7±11.6 3.65±2.86 <![CDATA[Xu et al.

[15] > 51.7±10.8 44.5±9.4 0.85±1.59 <![CDATA[Yun et al.

[16] > 65.7±13.1 60.1±11.9 4.56±3.73 <![CDATA[Pinho et al.

[17] > 32.1±6.9 26.9±6.9 3.63±4.92 <![CDATA[Born et al.

[18] > 41.7±16.2 34.5±13.2 0.41±1.09 <![CDATA[Bauer et al.

[19] > 63.0±10.4 58.4±13.2 1.44±2.06 <![CDATA[Irving et al.

[20] > 43.5±19.1 36.4±17.1 1.27±2.91 <![CDATA[Feuerstein et al.

[21] > 76.5±13.3 73.3±13.4 15.56±9.52 This Application 81.7±13.1 79.9±10.3 5.1±3.6

[0121] As shown in Table 4, among all the participants announced on the official website, FF_ITC achieved the best BD (79.6%) and TD (79.9%). They adopted a 4-stage airway extraction method, in which Hessian analysis was used to detect as many airway candidates as possible. Optimized by the minimum spanning tree, it did bring better segmentation performance, but at the cost of sacrificing the false positive rate (FPR = 11.92%); similarly, in the segmentation results of Feuerstein et al., there were a large number of leaks in the peripheral branches, resulting in the highest FPR (15.56%), although the results in terms of BD and TD were also relatively considerable. Obviously, our method has the best performance in terms of connectivity metrics (81.7% BD and 79.9% TD), and more gratifyingly, our FPR is not too high compared with the previous two studies. This shows that our method not only ensures the detection performance of the connected region, but also eliminates a lot of "pseudo-targets" similar to small airways, and the airway leakage phenomenon has also been significantly improved. Compared with other methods, our method has strong competitiveness in terms of connectivity metrics. However, ultimately, FPR is a minus point for the performance competition of the method in this paper. It has the same weakness as some previous work: in order to ensure the overall segmentation of the airway, it pays too much attention to the recognition performance of distal small airways to a certain extent, thus introducing a lot of false positives.

[0122] Born et al. proposed the Vicinity-Sensitive 3D Region Growing method. By defining the position of the seed point in the airway by the user, it gradually grows and then expands to the entire airway region. The progress of this method is that it can ensure that the segmented region will not overflow the airway outer wall too much, weakening the over-segmentation phenomenon. Therefore, its false positive rate among all participants is the most perfect (0.41% FPR), but this will also reduce the ability to identify distal airways. Xu et al. innovated the fuzzy connectedness theory to measure the airway wall thickness, preventing the segmented region from penetrating into the adjacent lung parenchyma across the fuzzy airway, largely eliminating the appearance of false positive samples (0.85% FPR). However, the above two methods are not satisfactory in terms of the airway connectivity region prediction index.

[0123] In summary, the present invention provides an airway segmentation method that takes into account both the overall and local aspects. On the one hand, the present invention enhances the representation ability of effective features by proposing a new designed PWFR module, alleviates the inter-class imbalance phenomenon, and improves the continuity of airway segmentation. On the other hand, the present invention emphasizes the attention to distal small airways through the newly added AttdKD module. According to the experimental results of the present invention on the public dataset, the method we proposed can achieve accurate segmentation of the airway main body and accurate identification of small airways, and has good clinical application prospects.

[0124] References:

[0125] [1] Y.Qin et al., “Learning tubule-sensitive CNNs for pulmonary airway and artery-vein segmentation in CT,” IEEE Trans.Med Imaging., vol.40, no.6, pp.1603-1617, Jun.2021.

[0126] [2] W.Zhu et al., “AnatomyNet:Deep learning for fast and fully automated whole-volume segmentation of head and neck anatomy,” Med.Phys., vol.46, no.2, pp.576-589, Feb.2019.

[0127] [3] A.M.Rickmann, A.G.Roy, I.Sarasua, N.Navab, and C Wachinger, “‘Project\&Excite’Modules for Segmentation of Volumetric Medical Scans,” in Proc.Med.Image Comput.Comput.Assist.Interv.-MICCAI 2019, Pt II., 2019.

[0128] [4] T.Huang, S.You, F, Wang, C.Qian, and C.Xu, “Knowledge distillation from a stronger teacher,” in Proc.Conf.Neural Inf.Process.Syst., vol.35, pp.33716–33727, 2022.

[0129] [5]T. Liu, C. Chen, X. Yang, and W. Tan, “Rethinking knowledge distillation with raw features for semantic segmentation,” in Proc. IEEE Winter Conf. Appl. Comput. Vis., pp. 1155–1164, 2024.

[0130] [6]C. Yang, H. Zhou, Z. An, X. Jiang, Y. Xu, and Q. Zhang, “Cross-image relational knowledge distillation for semantic segmentation,” in Proc. IEEE / CVF Conf. Comput. Vis. Pattern Recognit., pp. 12319–12328, 2022.

[0131] [7]A. Garcia-Uceda Juarez, H. A. W. M. Tiddens, and M. de Bruijne, “Automatic Airway Segmentation in Chest CT Using Convolutional Neural Networks,” in Proc. Med. Image Comput. Comput. Assist. Interv.–MICCAI 2018., vol. 11046, pp. 234 - 242, 2018.

[0132] [8]A. Garcia-Uceda Juarez, R. Selvan, Z. Saghir, and M. de Bruijne, “A Joint 3D UNet-Graph Neural Network-Based Method for Airway Segmentation from Chest CTs,” in Proc. Mach. Learn. Med. Imaging., vol. 11861, pp. 123–130, 2019.

[0133] [9]Y. Qin et al., “AirwayNet: A Voxel-Connectivity Aware Approach for Accurate Airway Segmentation Using Convolutional Neural Networks,” in Proc. Int. Conf. Med. Image Comput. Comput. Assist., vol. 11769, 2019.

[0134]

[10] C. Wang et al., “Tubular Structure Segmentation Using Spatial Fully Connected Network with Radial Distance Loss for 3D Medical Images,” in Proc. Int. Conf. Med. Image Comput. Comput. Assist., vol. 11769, 2019.

[0135]

[11] H. Zheng et al., “Alleviating Class-Wise Gradient Imbalance for Pulmonary Airway Segmentation,” IEEE Trans. Med Imaging., vol. 40, no. 9, pp. 2452 - 2462, Sep. 2021.

[0136]

[12] Wang, Andong et al., “NaviAirway: a bronchiole-sensitive deep learning-based airway segmentation pipeline for planning of navigation bronchoscopy,” IEEE Trans. Med Imaging., 2022.

[13] Y. Nan et al., “Fuzzy Attention Neural Network to Tackle Discontinuity in Airway Segmentation,” IEEE Trans. Neural Netw. Learn. Syst., May. 2023.

[0137]

[14] M. Zhang, and Y. Gu, “Towards Connectivity-Aware Pulmonary Airway Segmentation,” IEEE J. Biomed. Health Informat., vol. 28, no. 1, pp. 321-332, Jan. 2024.

[0138]

[15] Z. Xu, U. Bagci, B. Foster, A. Mansoor, J. K. Udupa, and D. J. Mollura, “A hybrid method for airway segmentation and automated measurement of bronchial wall thickness on CT,” Med. Image Anal., vol. 24, no. 1, pp. 1–17, Aug. 2015.

[0139]

[16] J. Yun et al., “Improvement of fully automated airway segmentation on volumetric computed tomographic images using a 2.5 dimensional convolutional neural net,” Med. Image Anal., vol. 51, pp. 13-20, Jan. 2019.

[0140]

[17] R. Pinho, S. Luyckx and J. Sijbers, “Robust Region Growing Based Intrathoracic Airway Tree Segmentation,” in Proc. Second Int. Workshop Pulm. Image Anal., 2009.

[0141]

[18] S Born, D. Lwamaru, M. Pfeifle, and D. Bartz, “Three-Step Segmentation of the Lower Airways with Advanced Leakage-Control,” in Proc. Second Int. Workshop Pulm. Image Anal., 2009.

[0142]

[19] C.W. Bauer, H. Bischof, and R. Beichel, “Segmentation of Airways Based on Gradient Vector Flow,” in Proc. Int. Conf. Med. Image Comput. Comput. Assist. Interv., 2009.

[0143]

[20] B. Irving, P. Taylor, and A. Toddpokropek, “3D segmentation of the airway tree using a morphology based method,” in Proc. Second Int. Workshop Pulm. Image Anal., 2009.

[0144]

[21] M. Feuerstein, T. Kitasaka, and K. Mori, “Adaptive Branch Tracing and Image Sharpening for Airway Tree Extraction in 3-D Chest CT,” in Proc. Second Int. Workshop Pulm. Image Anal., 2009.

Claims

1. A method for segmenting lung bronchial airways based on an image, characterized in that: The image-based pulmonary bronchial airway segmentation network is input with a pulmonary bronchial airway image to achieve the following: the image-based pulmonary bronchial airway segmentation network is composed of an encoding network, a neck network and a decoding network; the encoding network is composed of a first encoding module, a second encoding module, a third encoding module and a fourth encoding module connected in sequence; each encoding module is composed of a first ConvolutionalBlock module, a second Convolutional Block module and a PWFS module connected in sequence; the neck network is composed of a first Convolutional Block module, a second Convolutional Block module and a PWFS module connected in sequence; the decoding network is composed of a first decoding module, a second decoding module, a third decoding module and a fourth decoding module connected in sequence; each decoding module is composed of a first Convolutional Block module, a second Convolutional Block module, an AttdKD module and a PWFR module connected in sequence; the output end of the PWFS module of the first encoding module and the output end of the first Convolutional Block module of the third decoding module are connected to the input end of the fourth feature splicing module, and the output end of the fourth feature splicing module is connected to the input end of the fourth decoding module; the output end of the PWFS module of the second encoding module is connected to the first Convolutional Block module of the second decoding module The output end of the Block module is connected to the input end of the third feature splicing module, and the output end of the third feature splicing module is connected to the input end of the third decoding module; the output end of the PWFS module of the third encoding module is connected to the first Convolutional The output end of the Block module is connected to the input end of the second feature splicing module, and the output end of the second feature splicing module is connected to the input end of the second decoding module; the output end of the PWFS module of the fourth encoding module and the output end of the PWFS module of the neck network are connected to the input end of the first feature splicing module, and the output end of the first feature splicing module is connected to the input end of the first decoding module; the PWFR module is composed of a channel splitter, a feature recalibration module and an average point-level operation module connected in sequence; the AttdKD module is composed of a teacher module and a student module; the teacher module includes a CAM module, a first convolution refinement module, a SAM module, and a second convolution refinement module connected in sequence, and the second input end of the second convolution refinement module is connected to the input end of the first convolution refinement module; the student module includes an element-by-element multiplication module, and the second input end of the first convolution refinement module, the input end of the CAM module, and the input end of the element-by-element multiplication module are all connected to the same output end.

2. The image-based lung bronchial airway segmentation method according to claim 1, characterized in that: In the PWFR module, the channel division number N of the channel divider is set to 3.

3. The image-based lung bronchial airway segmentation method according to claim 1, characterized in that: The specific steps are: Step 1: construct an image-based lung bronchial airway segmentation network; Step 2: construct a lung bronchial airway image dataset for a deep learning network for image segmentation; Step 3, performing enhancement processing on the lung bronchial airway image dataset obtained in step 2; Step 4: using the lung bronchial airway image dataset obtained in step 3, the image-based lung bronchial airway segmentation network constructed in step 1 is trained so that the network can achieve lung bronchial airway segmentation; Step 5: Input the collected lung bronchial airway images into the image-based lung bronchial airway segmentation network trained in step 4, and output an accurate lung bronchial airway segmentation state.

4. The image-based lung bronchial airway segmentation method according to claim 1, characterized in that: In step 2, the lung bronchial airway image dataset adopts the Airway Tree Modeling dataset, or a dataset constructed from self-collected lung bronchial airway CT images.

5. The image-based lung bronchial airway segmentation method according to claim 1, characterized in that: In step 3, the enhancement processing method is any of random rotation and scaling, random flipping, random cropping, random brightness and contrast adjustment, elastic deformation, and random noise.

6. The image-based lung bronchial airway segmentation method according to claim 1, characterized in that: In step 4, the training of the image-based lung bronchial airway segmentation network aims to minimize the distillation loss function in the AttdKD module; where the distillation loss function L AttdKD The expression is: Where N is the total number of samples in the current batch, A i ″- and A i ″ +1 is the spatial feature map of two adjacent layers, Represents the square of the Frobenius norm.

7. The image-based lung bronchial airway segmentation method according to claim 1, characterized in that: In step 4, the training parameter settings include: the image training size is set to 128*224*304, the batch size is set to 1, the step size is set to 64, and the initial learning rate is set to 0.003.

Images