Airway segmentation network based on multi-scale directional attention and local graph convolution fusion

Abstract

The invention discloses an airway segmentation network based on multi-scale directional attention and local graph convolution fusion, and relates to the field of medical image processing, a U-Net type three-dimensional encoder-decoder structure is adopted, space detail information is reserved and gradient propagation is promoted through jump connection, an encoder is used for extracting multi-level semantic features, and the decoder is used for extracting multi-level semantic features; a decoder gradually recovers spatial resolution through jump connection, a segmentation result is obtained through a convolution layer and a Sigmoid activation function, an airway segmentation network integrates a multi-scale direction attention mechanism and a local graph convolution attention mechanism, enhancement is carried out in combination with relative position coding, the identification capability of small airway branches is improved, and boundary false detection is reduced. According to the method, the recognition rate of the tail end fine bronchus is remarkably improved, breakage and leak detection are reduced, the connectivity and topological integrity of the airway tree are enhanced, the model is more sensitive to feature response of small airways, fuzzy boundaries and low-contrast areas, the continuity of the output airway model is higher, and the method can be directly used for bronchoscope robot navigation and path planning.

Description

Technical Field

[0001] This invention relates to the field of medical image processing, and in particular to an airway segmentation network based on the fusion of multi-scale directional attention and local graph convolution. Background Technology

[0002] Lung diseases are among the leading causes of death and disability worldwide [1,2], with early pulmonary nodules, chronic obstructive pulmonary disease (COPD), and interstitial lung disease severely impacting patients' respiratory function and quality of life. Complete identification and precise navigation of the airway structure play a crucial role in the diagnosis and treatment of these diseases. The human airway branches progressively from the trachea into the main bronchus, lobar bronchus, segmental bronchus, and finally the terminal bronchioles and terminal bronchi, such as… Figure 2 As shown, the airway has numerous branches, gradually decreasing diameter, and presents a highly complex three-dimensional tree-like structure. Due to factors such as lung respiratory motion, lesions, and individual anatomical differences, the airway lumen may deform or shift position, resulting in weak signals, blurred boundaries, and significant noise interference in the terminal bronchioles in the imaging data, leading to highly significant intra- and inter-class imbalances. Against this backdrop, bronchoscopy and its robot-assisted systems are widely used in the diagnosis and minimally invasive treatment of lung diseases. Especially in early lung nodule biopsy, intervention of distal lung lesions, and local treatment, robotic bronchoscopy can accurately reach the target lesion area in three-dimensional space. However, the reliability of robot navigation is highly dependent on the accuracy and integrity of airway segmentation. If the airway segmentation model fails to accurately identify the terminal bronchioles or produces structural breaks, it may lead to incorrect navigation path planning, surgical instruments entering non-target areas, increasing operational risks and surgical time. Therefore, constructing a three-dimensional airway model that can completely and continuously identify airway branches, especially high-generation bronchioles, is the foundation for achieving safe, reliable, and accurate navigation, and is also a key technical requirement for improving the efficiency and success rate of robotic bronchoscopy in clinical applications.

[0003] Traditional airway segmentation methods, such as manual delineation using medical imaging software like SNAP-ITK, airway tree extraction based on region growing algorithms [3], and segmentation strategies based on tubular structure detection [4], have been widely used in early studies, but they generally suffer from low segmentation efficiency and poor adaptability to complex scenes. When there is noise interference in the image or the adjacent tissue structure is relatively complex, the airway boundary is often blurred, making it difficult for the algorithm to accurately extract small branches. In addition, these methods generally rely on a large amount of manual annotation or rich prior knowledge, and have a low degree of automation, making it difficult to meet the needs of intelligent medical image analysis. With the rise of artificial intelligence technology, airway segmentation methods based on deep learning have gradually become the mainstream research direction.

[0004] The development of deep learning has greatly promoted the progress of medical image segmentation technology. For example, Wu et al. built a segmentation network based on the Transformer architecture to capture long-distance dependencies, thereby improving the coherence of the airway trunk and the consistency of the global context [5]. In addition, Tang et al. combined self-attention mechanism with adversarial learning and proposed an adversarial Transformer framework, which effectively repaired the breakage and omission problem in airway segmentation and significantly enhanced the integrity and topological coherence of the distal bronchioles [6]. Although these models have achieved good performance in specific tasks, their complex training process, high computational resource consumption, and strong data dependence still limit their clinical application. Therefore, ensuring segmentation accuracy while taking into account the lightweight and stability of the model is still an important research direction in the field of medical image segmentation. In this context, 3D U-Net, with its encoder-decoder symmetric structure, has outstanding performance in local detail modeling and global feature fusion. Its high running efficiency and flexible structural design make it one of the most representative benchmark networks in airway segmentation tasks. Qin et al. further proposed AirwayNet, which combines the U-Net framework with the voxel connectivity constraint mechanism to achieve collaborative modeling of local structures and global airway paths, thereby reducing computational complexity while enhancing the structural continuity and stability of the results [7]. Meng et al. designed a hybrid model combining centerline tracing and 3D U-Net, using the centerline tracing algorithm to pre-locate potential airway regions, reducing the dependence on complex localization modules and large-scale labeled data, and then using 3D U-Net to achieve accurate segmentation and remote airway reconstruction [8]. These improved schemes based on 3D U-Net fully utilize its efficient, robust and scalable characteristics, significantly improving the network's ability to perceive and adapt to complex anatomical structures.

[0005] Despite significant achievements by existing methods, current 3D U-Net and its variants still fall short in recognizing fine structures such as small bronchi. Airway segmentation tasks remain hampered by significant inter-class and intra-class imbalances: the ratio of foreground (airway) to background pixels is severely unbalanced, and structural differences between large and small airways further complicate the learning process. These issues often lead to airway breaks and incomplete bronchograms in the segmentation results, affecting the overall integrity and topological continuity of the airway tree and reducing the reliability of clinical diagnostic and navigation applications.

[0006] Therefore, those skilled in the art are dedicated to developing an airway segmentation network based on the fusion of multi-scale directional attention and local graph convolution. Summary of the Invention

[0007] In view of the above-mentioned deficiencies of the prior art, the technical problem to be solved by the present invention is that the existing three-dimensional airway segmentation algorithms have insufficient ability to identify terminal bronchioles, the models have poor sensitivity to small airway signals and blurred boundary regions, and it is difficult to take into account both global structure and local details.

[0008] To achieve the above objectives, this invention provides an airway segmentation network based on the fusion of multi-scale directional attention and local graph convolution. The airway segmentation network employs a U-Net-type 3D encoder-decoder structure, preserving spatial detail information and promoting gradient propagation through skip connections. The encoder extracts multi-level semantic features, and the decoder progressively restores spatial resolution through skip connections. The final segmentation result is obtained through convolutional layers and a sigmoid activation function. This airway segmentation network integrates multi-scale directional attention and local graph convolutional attention mechanisms. The multi-scale directional attention mechanism aggregates anisotropic contextual information along the three directions of depth, height and width, and models it in the multi-scale feature space. It calculates the feature difference map between the microstructure features and the original input and fuses them in a weighted manner. The local graph convolutional attention mechanism complements the multi-scale directional attention mechanism. It improves the ability to distinguish structures by modeling fine-grained local geometric relationships. Within the local neighborhood, it calculates attention weights based on the feature differences between the central voxel and its neighboring voxels, and combines this with relative position encoding for enhancement. This improves the ability to identify small airway branches and reduces false detections at boundaries.

[0009] Furthermore, the airway segmentation network includes a multi-scale directional attention module, a local graph convolutional attention module, and a parallel fusion module. The multi-scale directional attention module and the local graph convolutional module are executed in parallel. The parallel fusion module is embedded between the encoder and the decoder in each layer of the airway segmentation network. It concatenates the outputs of the multi-scale directional attention module and the local graph convolutional module, and then fuses the concatenated results through convolution to output fused features.

[0010] Furthermore, the multi-scale directional attention module utilizes the local graph convolutional attention mechanism to capture anisotropic contextual information of the airway structure along the three directions of depth, height, and width, aggregates the anisotropic contextual information, and performs modeling in the multi-scale feature space, thereby achieving unified modeling of global coherence and local precision.

[0011] Furthermore, the multi-scale directional attention module processing flow includes: multi-scale feature extraction, three-axis directional convolutional separation, and microstructure difference enhancement path, wherein, The multi-scale feature extraction process involves performing three layers of three-dimensional convolution downsampling operations at different scales on the input features to expand the receptive field and simultaneously acquire local and global contextual information. The three-axis convolutional separation performs convolutional separation operations along the depth, height, and width directions respectively to capture the directional sensitive features of the airway; The microstructure difference enhancement path employs a difference enhancement mechanism, which concatenates spatial attention features with enhanced microstructure features along the channel dimension, and performs feature fusion through convolutional layers and instance normalization layers.

[0012] Furthermore, the three-axis convolutional separation includes the following steps: S101: Perform convolutional separation operations on the input features along the depth, height, and width directions:

[0013]

[0014]

[0015] S102: Concatenate the output features from the three convolutional directions:

[0016] S103: Merge and normalize the convolutions, apply the sigmoid function to generate directional attention weights, and apply the output:

[0017]

[0018] in: The kernel size is 3D convolution, These represent the feature responses in the depth, height, and width directions, respectively. Feature response after feature concatenation For attention weights, For output features, Features after splicing As input features, For the Sigmoid function, " indicates element-wise multiplication.

[0019] Furthermore, the difference enhancement mechanism includes the following steps: S201: Calculate the difference strength between the input features and the downsampled features:

[0020] S 202: By using weighted fusion, subtle airways and local edge variations are highlighted, improving the network's response in low-contrast regions.

[0021] S203: Spatial attention features and enhanced microstructure features are concatenated along the channel dimension, and feature fusion is performed through convolutional layers and instance normalization layers. The fused output serves as the final feature of the multi-scale directional attention module.

[0022] in, Features of the intermediate layer of the downsampling path, To take the absolute value of each element, This is a diagram showing the intensity variations of local microstructures. The features after weighted fusion This represents the final feature of the multi-scale directional attention module.

[0023] Furthermore, the local graph convolutional attention module utilizes the local graph convolutional attention mechanism to model the geometric dependence and semantic association of local airway structures by combining local graph convolution and attention mechanisms, thereby enhancing the response capability to small airways and improving the overall segmentation accuracy and structural consistency.

[0024] Furthermore, the local graph convolutional attention module processing flow includes: neighborhood unrolling and graph construction, relative position encoding, and graph attention weighting, wherein, The neighborhood expansion and graph construction construct a neighborhood point set centered on each voxel in the feature space, forming a local graph structure, and capturing the spatial dependency and structural consistency between voxels in the local region. The relative position encoding defines a relative position vector for the spatial relationship between nodes, explicitly expressing the spatial relationship between each voxel in the neighborhood and the central voxel; The graph attention weighting method calculates the feature similarity between nodes using a lightweight multilayer perceptron and automatically focuses on neighborhood features that are more meaningful to the segmentation result through weighted aggregation.

[0025] Furthermore, in the local graph structure, nodes represent voxel features, and edges represent spatial connections between neighboring voxels. The local graph structure is as follows:

[0026] The relative position is encoded as follows:

[0027] in, It is a local graph structure. For the set of voxel nodes in the neighborhood, It is the set of connecting edges between nodes. For nodes The relative position encoding between them Representing nodes respectively The depth, height, and width coordinates, Representing nodes respectively The depth, height, and width coordinates.

[0028] Furthermore, in the graph attention weighting process, the feature similarity between nodes is calculated in the following way:

[0029] Weighted aggregation is calculated using the following formula:

[0030] in, For nodes eigenvectors, For nodes Relative to node The weighted aggregated feature vector, Represents a node Relative to node Attention weights For nodes The neighborhood set.

[0031] In a preferred embodiment of the present invention, compared with the prior art, the present invention has the following beneficial technical effects: 1. This invention integrates multi-scale directional attention mechanism and local graph convolutional attention mechanism. Targeting the multiple characteristics of airway structure in terms of spatial directionality, local topological connectivity and microscale structural details, it constructs a unified feature expression framework of "directional modeling + topology preservation + multi-scale microstructure enhancement", which realizes fine segmentation and complete structural reconstruction of complex airway systems.

[0032] 2. This invention significantly improves the identification rate of terminal bronchioles, reduces breakage and missed detection, and enhances the connectivity and topological integrity of the airway tree.

[0033] 3. This invention is more sensitive to the feature response of small airways, fuzzy boundaries, and low-contrast regions, achieving refined 3D reconstruction. The output airway model has stronger continuity and can be directly used for navigation and path planning of bronchoscopy robots.

[0034] The following will further explain the concept, specific structure, and technical effects of the present invention in conjunction with the accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the overall structure of the airway segmentation network according to a preferred embodiment of the present invention; Figure 2 This is a schematic diagram of the human airway according to a preferred embodiment of the present invention; Figure 3 This is a schematic diagram of the multi-scale directional attention module (HDA) structure according to a preferred embodiment of the present invention; Figure 4 This is a schematic diagram of the Local Convolutional Attention Module (LGA) structure according to a preferred embodiment of the present invention; Figure 5 This is a schematic diagram of the parallel fusion module CPB structure according to a preferred embodiment of the present invention. Detailed Implementation

[0036] The following description, with reference to the accompanying drawings, illustrates several preferred embodiments of the present invention to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms, and the scope of protection of the present invention is not limited to the embodiments mentioned herein.

[0037] In the accompanying drawings, components with the same structure are indicated by the same numerical designation, and components with similar structures or functions are indicated by similar numerical designations. The dimensions and thicknesses of each component shown in the drawings are arbitrary, and the present invention does not limit the dimensions and thicknesses of each component. To make the illustrations clearer, the thickness of some components has been appropriately exaggerated in the drawings.

[0038] To overcome the aforementioned challenges in existing technologies, this invention proposes a novel airway segmentation network—AFNet. By integrating multi-scale feature modeling and attention mechanisms, it enhances local structure recognition capabilities while maintaining global topological consistency, thereby effectively alleviating inter-class and intra-class imbalance problems and achieving accurate segmentation and coherent reconstruction of complex airway structures.

[0039] like Figure 1 As shown in the figure, the airway segmentation network based on multi-scale directional attention and local graph convolutional fusion provided in this embodiment of the invention adopts a U-Net-type 3D encoder-decoder structure. It preserves spatial detail information and promotes gradient propagation through skip connections. The encoder is used to extract multi-level semantic features, and the decoder gradually restores spatial resolution through skip connections. The final segmentation result is obtained through convolutional layers and a sigmoid activation function. The airway segmentation network integrates multi-scale directional attention and local graph convolutional attention mechanisms. 1) Multi-scale directional attention mechanism: Aggregate anisotropic contextual information along the three directions of depth, height and width, and model it in the multi-scale feature space. Calculate the feature difference map between the microstructure features and the original input, and fuse them in a weighted manner. 2) Local graph convolutional attention mechanism, which complements the multi-scale directional attention mechanism, improves the ability to distinguish structures by modeling fine-grained local geometric relationships. Within the local neighborhood, attention weights are calculated based on the feature differences between the central voxel and its neighboring voxels, and enhanced by relative position encoding, thereby improving the ability to identify small airway branches and reducing false detections at boundaries.

[0040] In this embodiment, the airway segmentation network includes a multi-scale directional attention module, a local graph convolutional attention module, and a parallel fusion module. The multi-scale directional attention module and the local graph convolutional module are executed in parallel. The parallel fusion module is embedded between the encoder and decoder of each layer of the airway segmentation network. It concatenates the outputs of the multi-scale directional attention module and the local graph convolutional module, and then fuses the concatenated results through convolution to output fused features.

[0041] In this embodiment, the multi-scale directional attention module utilizes the local graph convolutional attention mechanism to capture the anisotropic contextual information of the airway structure along the three directions of depth, height, and width, aggregates the anisotropic contextual information, and performs modeling in the multi-scale feature space to achieve unified modeling of global coherence and local precision.

[0042] The multi-scale directional attention module processing flow includes: multi-scale feature extraction, three-axis directional convolutional separation, and microstructure difference enhancement path, among which, 1) Multi-scale feature extraction: The input features are subjected to three layers of three-dimensional convolution downsampling operations at different scales to expand the receptive field and obtain local and global contextual information at the same time; 2) Three-axis convolution separation: Convolution separation operations are performed along the depth, height and width directions respectively to capture the directional sensitive features of the airway.

[0043] Specifically, three-axis convolutional separation includes the following steps: S101: Perform convolutional separation operations on the input features along the depth, height, and width directions:

[0044]

[0045]

[0046] S102: Concatenate the output features from the three convolutional directions:

[0047] S103: Merge and normalize the convolutions, apply the sigmoid function to generate directional attention weights, and apply the output:

[0048]

[0049] in: The kernel size is 3D convolution, These represent the feature responses in the depth, height, and width directions, respectively. Feature response after feature concatenation For attention weights, For output features, Features after splicing As input features, For the Sigmoid function, " indicates element-wise multiplication.

[0050] 3) Microstructure difference enhancement path: The difference enhancement mechanism is adopted to concatenate the spatial attention features and the enhanced microstructure features in the channel dimension, and then perform feature fusion through convolutional layers and instance normalization layers.

[0051] In this embodiment, the difference enhancement mechanism includes the following steps: S201: Calculate the difference strength between the input features and the downsampled features:

[0052] S 202: By using weighted fusion, subtle airways and local edge variations are highlighted, improving the network's response in low-contrast regions.

[0053] S203: Spatial attention features and enhanced microstructure features are concatenated along the channel dimension, and feature fusion is performed through convolutional layers and instance normalization layers. The fused output serves as the final feature of the multi-scale directional attention module.

[0054] in, Features of the intermediate layer of the downsampling path, To take the absolute value of each element, This is a diagram showing the intensity variations of local microstructures. The features after weighted fusion This represents the final feature of the multi-scale directional attention module.

[0055] In this embodiment, the local graph convolutional attention module utilizes the local graph convolutional attention mechanism. By combining local graph convolution and attention mechanisms, it models the geometric dependence and semantic association of local airway structures, enhances the response capability to small airways, and improves the overall segmentation accuracy and structural consistency.

[0056] The local graph convolutional attention module processing flow includes: neighborhood unrolling and graph construction, relative position encoding, and graph attention weighting. 1) Neighborhood Unfolding and Graph Construction: In the feature space, a set of neighborhood points is constructed with each voxel as the center to form a local graph structure, capturing the spatial dependencies and structural consistency between voxels in the local region; In a local graph structure, nodes represent voxel features, and edges represent spatial connections between neighboring voxels. The local graph structure is as follows:

[0057] The relative position encoding is:

[0058] in, It is a local graph structure. For the set of voxel nodes in the neighborhood, It is the set of connecting edges between nodes. For nodes The relative position encoding between them Representing nodes respectively The depth, height, and width coordinates, Representing nodes respectively The depth, height, and width coordinates.

[0059] 2) Relative position encoding: Define relative position vectors for the spatial relationships between nodes, explicitly expressing the spatial relationships between each voxel in the neighborhood and the central voxel; 3) Graph attention weighting: The similarity of features between nodes is calculated by a lightweight multilayer perceptron, and the neighborhood features that are more meaningful to the segmentation results are automatically focused through weighted aggregation.

[0060] In this embodiment, the feature similarity between nodes is calculated in the graph attention weighting process as follows:

[0061] Weighted aggregation is calculated using the following formula:

[0062] in, For nodes eigenvectors, For nodes Relative to node The weighted aggregated feature vector, Represents a node Relative to node Attention weights For nodes The neighborhood set.

[0063] Compared with existing technologies, the airway segmentation network based on multi-scale directional attention and local graph convolution fusion provided in this invention has the following advantages: 1. Existing 3D airway segmentation algorithms generally suffer from the following problems: insufficient ability to identify terminal bronchioles, often resulting in breaks or missed detections, leading to incomplete airway tree topology; poor sensitivity of the models to small airway signals and blurred boundary regions, making it difficult to simultaneously consider global structure and local details; leading to path planning errors, target deviations, and insufficient accessibility during end-effector navigation of bronchoscopic robots, affecting intraoperative accuracy and safety. This invention proposes a 3D airway segmentation network integrating multi-scale directional attention (HDA) and local graph convolutional attention (LGA) mechanisms, named Airway Feature Extraction Network (AFNet). This network addresses the multiple characteristics of airway structure in terms of spatial directionality, local topological connectivity, and microscale structural details by constructing a unified feature representation framework of "directional modeling + topology preservation + multi-scale microstructure enhancement," achieving fine segmentation and structurally complete reconstruction of complex airway systems.

[0064] 1) Multi-scale directional attention module (HDA): It adopts three-level scale (original, 2× downsampling, 4× downsampling) to perform convolutional separation along the depth, height and width directions to capture the longitudinal, lateral and radial structural features of the airway; at the same time, it introduces a microstructure difference enhancement path to highlight the local feature changes in the bronchiolar region through residual calculation, so that the model still has the ability to recognize in the weak signal region.

[0065] 2) Local Graph Convolutional Attention Module (LGA): Local graph structure modeling is achieved through 3×3×3 neighborhood unfolding, and relative position encoding is introduced to represent spatial association; a lightweight MLP is used to generate attention weights, and the feature differences within the neighborhood are weighted and fused to achieve structural continuity constraints in three-dimensional space.

[0066] 3) Parallel Fusion Module (CPB): Combines directional attention and graph convolutional attention in parallel, and forms a feature representation of "direction awareness + topology preservation" through 1×1 convolution adaptive fusion. This module is embedded in the U-Net encoding and decoding structure at multiple levels to achieve multi-level global semantic and local geometric fusion.

[0067] This invention significantly improves the recognition rate of terminal bronchioles (high-level bronchioles), reduces fragmentation and missed detection, and enhances the connectivity and topological integrity of the airway tree. The model is more sensitive to features of small airways, blurred boundaries, and low-contrast regions, achieving refined 3D reconstruction. The output airway model has stronger continuity and can be directly used for navigation and path planning in bronchoscopy robots. It reduces intraoperative navigation deviations caused by segmentation errors, shortens operation time, and improves equipment utilization efficiency. It reduces the dependence of model training and deployment on computing resources, facilitates integration into clinical robot platforms, and improves the accuracy of early lung lesion detection and interventional treatment. It enhances the clinical usability and safety of robotic bronchoscopy systems and has significant value for medical application promotion.

[0068] 2. To address the issue of lost features of local fine structures (such as small airway branches) during deep convolution in existing 3D airway segmentation networks, this invention proposes a multi-scale directional attention fusion module (HDA). Through multi-scale directional modeling and explicit microstructure enhancement, it achieves adaptive enhancement of small airway and edge features. This module calculates the average response along the three principal axes of the feature map and extracts directional activation maps using corresponding convolutional kernels. The directional activation maps at different scales are interpolated, aligned, and weighted before fusion to generate multi-scale spatial attention weights. The original features are then weighted by attention to obtain directional awareness features. Simultaneously, the microstructure path output undergoes difference enhancement. Finally, the two features are fused to strengthen the response to small airways. This invention preserves and enhances features of small airways and edge regions, effectively reducing the false negative rate (FN reduction of approximately 8–15%). Multi-scale fusion enhances the collaborative perception capability of deep semantics and shallow structure; explicit directional modeling significantly improves the model's sensitivity to airway orientation. In medical CT segmentation scenarios, it can improve the efficiency of automated detection and reduce manual correction time; it also helps with bronchoscopic surgical path planning and precise navigation of AI diagnostic systems.

[0069] 3. This invention addresses the significant topological fragmentation problem in existing airway segmentation methods. Conventional convolutional or self-attention modules only consider weighted averages within a fixed window, lacking modeling of the geometric and spatial relationships between neighboring points. To address this, we propose a Local Graph Convolutional Attention Mechanism (LGA) based on neighborhood relationships. This mechanism establishes difference-aware weighted graph aggregation within the 3D voxel neighborhood. By taking each central voxel as a node and extracting the feature vectors of its 27 surrounding neighboring points, difference features are calculated. These difference features are then concatenated with positional encodings, and attention weights are calculated using an attention mechanism-based dynamic programming (MLP). The final output is the neighborhood-weighted aggregation result. This process is equivalent to performing a relationship-aware graph convolution operation within the 3D neighborhood. This invention significantly enhances the network's ability to model topological consistency between airway tree branches; reduces breakage caused by fuzzy boundaries or local noise (BD improvement of approximately 10–20%); improves the model's robustness in complex anatomical structures (multi-branching, multi-directional) scenarios; effectively enhances the structural integrity and interpretability of medical image segmentation, contributing to clinical visualization and subsequent 3D printing reconstruction; and can be extended to various 3D medical tasks such as blood vessel segmentation, nerve fiber tracking, and bronchial navigation.

[0070] The present invention will now be described in detail with reference to preferred embodiments.

[0071] This preferred embodiment provides a three-dimensional segmentation network for automatic extraction of lung airways in chest CT images—the Airway Feature Attention Network (AFNet). Each layer of AFNet consists of a parallel fusion module (CPB), such as... Figure 1 As shown, AFNet is designed to enhance the ability to discriminate small branches while maintaining the integrity of the overall airway topology, thereby improving the detection capability of the terminal airway and reducing false detections. The network employs an encoder-decoder structure, using skip connections to preserve spatial details and promote gradient propagation. The final segmentation result is obtained through a 1×1×1 convolutional layer and a sigmoid activation function.

[0072] To capture both global coherence and local accuracy, AFNet integrates two complementary attention mechanisms: multi-scale directional attention and local graph convolutional attention, implemented by the multi-scale directional attention module and the local graph convolutional attention module, respectively.

[0073] Multi-scale Directional Attention (HDA) module: This module aggregates anisotropic contextual information along the depth, height, and width directions and models it in a multi-scale feature space. To highlight the small bronchial structure, the network designs a dedicated microstructure path. By calculating the feature difference map between the microstructure features and the original input and fusing them in a weighted manner, it enhances low-contrast branch regions while suppressing coarse structural noise.

[0074] Local Graph Convolutional Attention Module (LGA): This module complements HDA by improving structural discrimination by modeling fine-grained local geometric relationships. Within a 3×3×3 local neighborhood, LGA calculates attention weights based on the feature differences between the central voxel and its neighboring voxels, and combines this with relative position encoding for enhancement, thereby improving the ability to identify small airway branches and reducing false detections at boundaries.

[0075] I. Overall Model Structure

[0076] This embodiment employs a U-Net-type 3D encoder-decoder structure: the encoder extracts multi-level semantic features; the decoder gradually restores spatial resolution through skip connections; and a fusion module is embedded between each layer of encoder and decoder. The fusion module includes: a multi-scale directional attention module (HDA); and a local graph convolutional attention module (LGA); both operate in parallel and are adaptively fused at the output through a 1×1×1 convolution.

[0077] II. Multi-scale Directional Attention Module (HDA)

[0078] This module is one of the main innovations of this invention. It aims to fully capture the anisotropic contextual information of the airway structure in three-dimensional space, namely, the differential feature dependencies along the depth, height, and width directions, thereby achieving unified modeling of global coherence and local refinement. The HDA module, for example... Figure 3 As shown in the diagram. The design principle of this module is to simulate the human sensitivity to directional information when recognizing tubular structures, enabling the network to more accurately characterize the main pathways and branch extensions of the airways.

[0079] Let the input feature tensor be: ,in: Number of channels; These represent the depth, height, and width dimensions, respectively.

[0080] The module processing flow is as follows: (1) Multi-scale feature extraction Input features After three layers of processing at different scales:

[0081] in, and These represent 3D convolutional downsampling operations with strides of 2 and 4, respectively. This step is used to expand the receptive field to simultaneously acquire local and global contextual information.

[0082] (2) Convolution separation in three axes

[0083] To capture the orientation-sensitive features of the airway, convolutional separation operations are performed along the depth, height, and width directions, respectively:

[0084]

[0085]

[0086] in: : indicates that the convolution kernel size is 3D convolution; : Represents the feature responses in the depth, height, and width directions, respectively.

[0087] This design effectively reduces computational complexity and enhances the model's sensitivity to airway orientation by separating convolution kernels in different directions.

[0088] Then concatenate the output features from the three convolutional directions:

[0089] After fusion and normalization via 1×1×1 convolution, a sigmoid function is applied to generate directional attention weights:

[0090] The final output is:

[0091] in, For the Sigmoid function, "" indicates element-wise multiplication. Through the above steps, the module achieves adaptive weighting of features in different directions, enhancing directional response.

[0092] (3) Microstructural difference enhancement pathway

[0093] To improve the model's detection capability in terminal bronchioles and blurred boundary regions, a difference enhancement mechanism is introduced.

[0094] Calculate the difference strength between the input features and the downsampled features:

[0095] in: Features of the intermediate layer of the downsampling path; : Take the absolute value of each element; : This represents a graph showing the intensity variation of local microstructures.

[0096] Then, through weighted fusion:

[0097] This method highlights small airways and local edge variations, improving the network's response in low-contrast regions.

[0098] Finally, spatial attention features With enhanced microstructural features The features are concatenated along the channel dimension and then fused using a 1×1×1 convolutional layer and an instance normalization layer.

[0099] The fused output serves as the final feature of the HDA module, combining multi-scale global contextual information with enhanced sensitivity to microstructures.

[0100] III. Local Graph Convolutional Attention Module (LGA)

[0101] The LGA module effectively models the geometric dependencies and semantic relationships of local airway structures by combining local graph convolution and attention mechanisms. It enhances the model's responsiveness to small airways without disrupting the overall topology, reducing false negatives and false positives at boundaries, thereby significantly improving overall segmentation accuracy and structural consistency. The LGA module, for example... Figure 4 As shown.

[0102] The principle is as follows: (1) Neighborhood expansion and graph construction In the feature space, a 3×3×3 neighborhood point set is constructed with each voxel as the center, forming a local graph structure:

[0103] in: : The set of voxel nodes in the neighborhood; The set of edges connecting nodes. This neighborhood can be viewed as a small graph structure, where nodes represent voxel features and edges represent spatial connections between neighboring voxels. In this way, the module can capture the spatial dependencies and structural consistency between voxels in a local region.

[0104] (2) Relative position encoding

[0105] To characterize the spatial relationships between nodes, a relative position vector is defined:

[0106] in, Representing nodes respectively The depth, height, and width coordinates are used to explicitly express the spatial relationship between each voxel in the neighborhood and the central voxel.

[0107] Position encoding provides directional and distance information, enabling the network to understand the morphological changes of the airway structure in three-dimensional space, thereby improving the model's ability to identify complex bifurcated and curved regions.

[0108] (3) Graph attention weighting

[0109] Calculate feature similarity between nodes using a lightweight multilayer perceptron (MLP):

[0110] in: :node eigenvectors; : Represents a node Relative to node Attention weights.

[0111] The weighted aggregation formula is:

[0112] in, For nodes The neighborhood set.

[0113] This operation ensures the continuity of local topological information and mitigates the fragmentation of bronchioles. A larger weight indicates a higher contribution of that voxel to the feature update of the central voxel. Through this weighting method, the model can automatically focus on neighborhood features that are more meaningful to the segmentation results, such as edge transition regions or small branches, while assigning lower weights to irrelevant or noisy features.

[0114] IV. Parallel Fusion Module (CPB)

[0115] The directional attention module and the graph convolution module are executed in parallel, and the output results are as follows: and The two are then concatenated and fused using a 1×1×1 convolution:

[0116] The resulting fused features contain both directional and topological information, enabling joint modeling of local geometry and global semantics. The parallel fusion module (CPB) is as follows: Figure 5 As shown.

[0117] V. Dataset

[0118] In this embodiment, the proposed method was evaluated on the BAS dataset. The Binary AirwaySegmentation (BAS) dataset [9] contains 90 chest CT scans and their corresponding airway annotations, of which 70 are from the LIDC dataset

[10] and 20 are from the EXACT09 dataset

[11] . The planar resolution of the dataset is between 0.50 and 0.82 mm, and the slice thickness is between 0.50 and 1.80 mm. According to the partitioning strategy in reference [9], the dataset was divided into 50 samples for training, 20 samples for validation, and 20 samples for testing.

[0119] VI. Comparative Experiment

[0120] The proposed AFNet was evaluated on the Binary Airway Segmentation (BAS) dataset and compared with state-of-the-art airway segmentation methods.

[0121] As shown in Table 1, AFNet achieves 93.07% accuracy in TD and 90.10% accuracy in BD, surpassing most existing methods and demonstrating its powerful ability to capture distant and peripheral airway branches. Notably, compared to WingsNet [9], which is also designed to address class imbalance, AFNet improves BD by 1.40%, demonstrating its advantage in maintaining connectivity of distant branches. At the same time, AFNet achieves a better balance between sensitivity (high TD / BD) and specificity (high accuracy), detecting more airway branches while still achieving an accuracy of 92.17%.

[0122] Specifically, general medical image segmentation networks such as nnUNet

[12] and V-Net

[13] have high accuracy, but their TD and BD are relatively low. Among the specific methods for airway structure, Wang et al.

[14] enhanced airway detection capability by introducing tubular structure priors; Juarez et al.

[15] and Jin et al.

[16] used three-dimensional convolution to achieve a balance between detection rate and accuracy; AirwayNet [7] and its improved version

[17] significantly improved detection performance; while BronchiNet

[18] was relatively inadequate in complex airway topology. Nan et al.

[19] further improved the detection effect through fuzzy learning mechanism; NaviAirway

[20] introduced navigation and tracking concepts to improve branch recognition capability, but the overall performance was still lower than the latest methods.

[0123] Overall, achieving high topology metrics (TD and BD) while maintaining high accuracy remains challenging, as improving one metric often comes at the expense of the other.

[0124] To comprehensively evaluate the performance of the airway segmentation algorithm, a multi-level evaluation metric combining voxel-level and skeleton / branch-level metrics was adopted. The ground truth mask is set as follows: The predicted mask is ,in Represents the voxel index, and Based on the matching relationship between the prediction results and the true labels, the following four basic counts can be defined: True Positive (TP), False Positive (FP), True Negative (TN), and False Negative (FN).

[0125] Table 1 Comparison of results from various segmentation methods

[0126] Based on this, the following voxel-level evaluation index is calculated in this embodiment: 1) Precision The proportion of voxels predicted by the model as airways that are actually correctly classified:

[0127] To more comprehensively evaluate the model's performance in airway topology reconstruction and branch detection, a skeleton-based evaluation metric was further introduced.

[0128] The specific steps are as follows: First, apply a mask to the actual annotation. G Three-dimensional skeletonization was performed to obtain an airway skeleton voxel set. Subsequently, by removing bifurcation points and marking connected regions, the entire airway framework was decomposed into... Each of the following independent branches is denoted as... ,in .

[0129] 2) Tree Length Detected (TD)

[0130] The formula used to measure the coverage of the predicted results on the actual airway skeleton is as follows:

[0131] 3) Branch Detected Rate (BD)

[0132] This reflects the proportion of airway branches successfully identified by the model. For each branch... If the predicted proportion of voxels covering the branch skeleton in the mask satisfies

[0133] The branch is then considered to have been successfully detected (this threshold is set according to the ATM22 Challenge rules).

[0134] Ultimately, the BD metric is calculated as follows:

[0135] in, This indicates an indicator function that takes the value 1 when the condition is met and 0 otherwise.

[0136] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

[0137] References: [1] Momtazmanesh, S., Moghaddam, SS, Ghamari, S.-H., Rad, EM, Rezaei, N., Shobeiri, P., Aali, A., Abbasi-Kangevari, M., Abbasi-Kangevari, Z., Abdelmasseh, M., et al.: Global burden of chronic respiratory diseases and risk factors, 1990 2019: an update from the global burden of diseasestudy 2019. EClinicalMedicine 59 (2023) [2] Cheung, W.K., Pakzad, A., Mogulkoc, N., Needleman, S., Rangelov,B., Gud mundsson, E., Zhao, A., Abbas, M., McLaverty, D., Asimakopoulos, D.,et al.: Automated airway quantification associates with mortality inidiopathic pulmonary fibrosis. European radiology 33(11), 8228–8238 (2023) [3] Pinho, R., Luyckx, S., Sijbers, J.: Robust region growing basedintrathoracic air way tree segmentation. In: Proc. of Second InternationalWorkshop on Pulmonary Image Analysis, pp. 261–271 (2009) [4] Bauer, C., Pock, T., Bischof, H., Beichel, R.: Airway treereconstruction based on tube detection. In: Proc. of Second InternationalWorkshop on Pulmonary Image Analysis, pp. 203–213 (2009) [5] Wu, Y., Zhao, S., Qi, S., Feng, J., Pang, H., Chang, R., Bai, L.,Li, M., Xia, S., Qian, W., et al.: Two-stage contextual transformer-basedconvolutional neural network for airway extraction from ct images. ArtificialIntelligence in Medicine 143, 102637 (2023) [6] Tang, Z., Nan, Y., Walsh, S., Yang, G.: Adversarial transformerfor repairing human airway segmentation. IEEE Journal of Biomedical andHealth Informatics 27(10), 5015–5022 (2023) [7] Qin, Y., Chen, M., Zheng, H., Gu, Y., Shen, M., Yang, J., Huang,X., Zhu, Y.-M., Yang, G.-Z.: Airwaynet: a voxel-connectivity aware approachfor accurate airway segmentation using convolutional neural networks. In:International Conference on Medical Image Computing and Computer-assistedIntervention, pp. 212–220 (2019). Springer [8] Meng, Q., Roth, H.R., Kitasaka, T., Oda, M., Ueno, J., Mori, K.:Tracking and segmentation of the airways in chest ct using a fullyconvolutional network. In: Medical Image Computing and Computer-AssistedIntervention- MICCAI 2017: 20th International Conference, Quebec City, QC,Canada, September 11-13, 2017, Proceedings, Part II 20, pp. 198–207 (2017).Springer [9] Zheng, H., Qin, Y., Gu, Y., Xie, F., Yang, J., Sun, J., Yang, G.-Z.: Alleviating class-wise gradient imbalance for pulmonary airwaysegmentation. IEEE Transactions on Medical Imaging 40(9), 2452–2462 (2021).

[10] Armato III, SG, McLennan, G, Bidaut, L, McNitt-Gray, MF,Meyer, CR, Reeves, AP, Zhao, B, Aberle, DR, Henschke, CI, Hoffman,EA, et al onct scans. Medical physics 38(2), 915–931 (2011).

[11] Lo, P., Van Ginneken, B., Reinhardt, JM, Yavarna, T., De Jong,PA, Irving, B., Fetita, C., Ortner, M., Pinho, R., Sijbers, J., et al.:Extraction of airways from ct (exact'09). IEEE Transactions on MedicalImaging 31(11), 2093–2107 (2012).

[12] Isensee, F., Jaeger, P.F., Kohl, S.A., Petersen, J., Maier-Hein,K.H.: nnu-net: a self-configuring method for deep learning-based biomedicalimage segmentation. Nature methods 18(2), 203–211 (2021)

[13] Milletari, F., Navab, N., Ahmadi, S.-A.: V-net: Fullyconvolutional neural net works for volumetric medical image segmentation. In:2016 Fourth International Conference on 3D Vision (3DV), pp. 565–571 (2016).Ieee

[14] Wang, C., Hayashi, Y., Oda, M., Itoh, H., Kitasaka, T., Frangi,A.F., Mori, K.: Tubular structure segmentation using spatial fully connectednetwork with radial distance loss for 3d medical images. In: InternationalConference on Med ical Image Computing and Computer-assisted Intervention,pp. 348–356 (2019). Springer

[15] Garcia-Uceda Juarez, A., Tiddens, H.A., Bruijne, M.: Automaticairway segmentation in chest ct using convolutional neural networks. In:Image Analysis for Moving Organ, Breast, and Thoracic Images: ThirdInternational Workshop, RAMBO 2018, Fourth International Workshop, BIA 2018,and First International Workshop, TIA 2018, Held in Conjunction with MICCAI2018, Granada, Spain, September 16 and 20, 2018, Proceedings 3, pp. 238–250(2018). Springer

[16] Jin, D., Xu, Z., Harrison, A.P., George, K., Mollura, D.J.: 3dconvolutional neural networks with graph refinement for airway segmentationusing incomplete data labels. In: International Workshop on Machine Learningin Medical Imaging, pp. 141–149 (2017). Springer

[17] Qin, Y., Zheng, H., Gu, Y., Huang, X., Yang, J., Wang, L., Yao,F., Zhu, Y.-M., Yang, G.-Z.: Learning tubule-sensitive cnns for pulmonaryairway and artery-vein segmentation in ct. arXiv preprint arXiv:2012.05767(2020)

[18] Garcia-Uceda, A., Selvan, R., Saghir, Z., Tiddens, H.A.,Bruijne, M.: Automatic airway segmentation from computed tomography usingrobust and efficient 3-dconvolutional neural networks. Scientific Reports 11(1), 16001 (2021)

[19] Nan, Y., Del Ser, J., Tang, Z., Tang, P., Xing, X., Fang, Y.,Herrera, F., Pedrycz, W., Walsh, S., Yang, G.: Fuzzy attention neural networkto tackle discontinuity in airway segmentation. IEEE transactions on neuralnetworks and learning systems 35(6), 7391–7404 (2023)

[20] Wang, A., Tam, T.C.C., Poon, H.M., Yu, K.-C., Lee, W.-N.:Naviairway: a bronchiole-sensitive deep learning-based airway segmentationpipeline for planning of navigation bronchoscopy. Authorea Preprints (2022)

Claims

1. An airway segmentation network based on multi-scale directional attention and local graph convolution fusion, characterized in that, The airway segmentation network adopts a U-Net type three-dimensional encoder-decoder structure, retains spatial detail information and promotes gradient propagation through a skip connection, the encoder is used to extract multi-level semantic features, the decoder gradually recovers the spatial resolution through the skip connection, and the final segmentation result is obtained through a convolution layer and a Sigmoid activation function, the airway segmentation network fuses a multi-scale directional attention mechanism and a local graph convolution attention mechanism, wherein The multi-scale directional attention mechanism aggregates anisotropic context information along three directions of depth, height and width, and models in a multi-scale feature space, and the feature difference map between the microstructure features and the original input is calculated and fused in a weighted manner. The local graph convolution attention mechanism supplements the multi-scale directional attention mechanism, improves the structure discrimination ability by modeling fine-grained local geometric relationships, calculates the attention weight according to the feature difference between the center voxel and its neighbor voxels in the local neighborhood, and enhances it combined with the relative position coding, improves the recognition ability of small airway branches and reduces boundary false detection.

2. The airway segmentation network of claim 1, wherein, The airway segmentation network includes a multi-scale directional attention module, a local graph convolution attention module and a parallel fusion module, the multi-scale directional attention module and the local graph convolution module are executed in parallel, and the parallel fusion module is embedded between the encoder and the decoder of each layer of the airway segmentation network, splices the output results of the multi-scale directional attention module and the local graph convolution module, and outputs the fused features through convolution fusion.

3. The airway segmentation network of claim 2, wherein, The multi-scale directional attention module uses the local graph convolution attention mechanism to capture the anisotropic context information of the airway structure along three directions of depth, height and width, aggregate anisotropic context information, and model in a multi-scale feature space, realizing unified modeling of global coherence and local fineness.

4. The airway segmentation network of claim 3, wherein, The multi-scale directional attention module processing flow includes: multi-scale feature extraction, three-axis directional convolution separation and microstructure difference enhancement path, wherein The multi-scale feature extraction performs three-dimensional convolution downsampling operation on the input features through three layers of different scales, expands the receptive field, and simultaneously obtains local and global context information; The three-axis directional convolution separation performs convolution separation operation along the depth, height and width directions respectively, and captures the direction-sensitive features of the airway; The microstructure difference enhancement path adopts a difference enhancement mechanism, splices the spatial attention features and the enhanced microstructure features in the channel dimension, and performs feature fusion through a convolution layer and an instance normalization layer.

5. The airway segmentation network of claim 4, wherein, The three-axis directional convolution separation includes the following steps: S101: performing convolution separation operation on the input features along the depth, height and width directions: S102: splicing the three direction convolution output features: S103: convolution fusion and normalization, applying Sigmoid function to generate directional attention weight and applying output: wherein: is a three-dimensional convolution with a kernel size of respectively represent the feature responses in the depth direction, the height direction and the width direction, is the feature response after feature concatenation, is the attention weight, is the output feature, is the concatenated feature, is the input feature, is a Sigmoid function, denotes element-wise multiplication.​ 6. The airway segmentation network of claim 5, wherein, The difference enhancement mechanism includes the following steps: S201: calculating the difference intensity between the input features and the down-sampled features: S 202: Highlighting the small airway and local edge changes through weighted fusion to improve the network response in low-contrast areas: S203: The spatial attention features and the enhanced microstructure features are spliced in the channel dimension, and the feature fusion is performed through a convolution layer and an instance normalization layer, and the fused output is used as the final feature of the multi-scale directional attention module: wherein, is the intermediate layer feature in the down-sampling path, is the element-wise absolute value, is the intensity variation map of the local microstructure, is the weighted fused feature, is the final feature of the multi-scale direction attention module.

7. The airway segmentation network of claim 6, wherein, The local graph convolution attention module uses the local graph convolution attention mechanism to model the geometric dependence and semantic association of the local structure of the airway by combining local graph convolution and attention mechanism, enhance the response ability to small airway, and improve the overall segmentation accuracy and structural consistency.

8. The airway segmentation network of claim 7, wherein, The processing flow of the local graph convolution attention module includes neighborhood expansion and graph construction, relative position coding and graph attention weighting, wherein, The neighborhood expansion and graph construction construct a neighborhood point set in the feature space centering on each voxel to form a local graph structure, capturing the spatial dependence and structural consistency between voxels in the local region; The relative position coding defines a relative position vector for the spatial relationship between nodes to explicitly express the spatial relationship between each voxel in the neighborhood and the center voxel; The graph attention weighting calculates the feature similarity between nodes through a lightweight multi-layer perceptron and automatically focuses on the neighborhood features that are more meaningful to the segmentation result through a weighted aggregation method.

9. The airway segmentation network of claim 8, wherein, In the local graph structure, the nodes represent voxel features, and the edges represent the spatial connection relationship between neighborhood voxels, and the local graph structure is: The relative position coding is: wherein, is a local graph structure, is a set of voxel nodes within a neighborhood, is a set of connecting edges between nodes, is a node is a relative position encoding between nodes respectively represent depth, height, width coordinates of a node respectively represent depth, height, width coordinates of a node respectively represent depth, height, width coordinates of a node respectively represent depth, height, width coordinates of a node 10. The airway segmentation network of claim 9, wherein, In the graph attention weighting process, the feature similarity between nodes is calculated in the following way: The weighted aggregation is calculated in the following formula: wherein, is a feature vector of a node , is a weighted aggregated feature vector of a node relative to a node , denotes an attention weight of a node relative to a node , is a neighborhood set of a node .

Images