A graph-based method and apparatus for bronchial classification

Abstract

This invention discloses a bronchus classification method and apparatus based on graph structures. The method includes: acquiring lung image sample data; extracting a binarized segmentation mask for the bronchi to obtain a bronchus mask image; inputting this image into a convolutional neural network for prediction to obtain high-dimensional convolutional features and a first bronchus segmentation result; performing skeletonization processing on the bronchus mask image to obtain point-by-point coordinate features; combining the point-by-point coordinate features and the high-dimensional convolutional features and inputting them into a point-voxel graph neural network for prediction to obtain a second bronchus segmentation result; iteratively training the convolutional neural network and the point-voxel neural network based on the first and second bronchus segmentation results, and then using the trained convolutional neural network to segment actual lung images. Utilizing the point-voxel neural network to guide the learning process of the convolutional neural network through convolutional features can improve classification performance.

Description

Technical Field

[0001] This invention relates to the field of neural network technology, and in particular to a method and apparatus for classifying bronchi based on graph structures. Background Technology

[0002] Existing bronchial tree classification methods mainly include traditional keypoint detection, rule-based methods, and deep learning methods. Traditional keypoint detection primarily uses keypoints to classify bronchial lobes, but requires manually created features. Rule-based methods, such as bottom-up bronchial labeling methods, suffer from poor handling of individual variability. Deep learning methods, such as linear programming post-processing or two-stage convolutional neural networks, while offering improvements, still have limitations in handling bronchial topology. Therefore, current bronchial tree classification methods either rely on manually created features or require additional labels, and are inefficient in handling individual differences in bronchial structure, thus limiting improvements in classification accuracy and efficiency. Summary of the Invention

[0003] Therefore, it is necessary to provide a bronchial classification method, apparatus, computer equipment, and storage medium based on graph structure to address the above-mentioned technical problems and solve at least one of the problems existing in the prior art.

[0004] In a first aspect, the embodiments of this application provide a bronchial classification method based on graph structure, comprising:

[0005] Acquire lung image sample data, and extract the binarized segmentation mask of the bronchus from the lung image sample data to obtain the bronchus mask image;

[0006] The bronchial mask image is input into a convolutional neural network for prediction to obtain high-dimensional convolutional features and the first bronchial segmentation result.

[0007] The bronchial mask image is skeletonized, and point-by-point coordinate features are obtained based on the skeletonized bronchial mask image.

[0008] The point-by-point coordinate features and the high-dimensional convolutional features are combined to obtain combined features. The combined features are then input into a point voxel image neural network for prediction to obtain the second bronchus segmentation result.

[0009] Based on the first bronchus segmentation result and the second bronchus segmentation result, the convolutional neural network and the dot voxel neural network are iteratively trained, and the trained convolutional neural network is used to segment the actual lung image to be segmented.

[0010] In one embodiment, the process of skeletonizing the bronchial mask image and obtaining point-by-point coordinate features based on the skeletonized bronchial mask image includes:

[0011] The bronchial mask image is refined into a single-pixel-wide centerline using a preset skeletonization algorithm to obtain a centerline image.

[0012] The three-dimensional coordinates of the centerline voxels are extracted from the centerline image to obtain point cloud features;

[0013] Based on the point cloud features, each three-dimensional coordinate is normalized and the boundary points are extracted to obtain the point-by-point coordinate features.

[0014] In one embodiment, the normalization and boundary point extraction of each three-dimensional coordinate to obtain the point-by-point coordinate features includes:

[0015] The bounding box of each bronchial segment is cropped from the skeletal mask image after skeletonization, wherein each bronchial segment is composed of the centerline voxel within its corresponding bounding box;

[0016] The three-dimensional coordinates of each voxel in the centerline image are normalized relative to the shape of the bounding box of the corresponding bronchial segment.

[0017] The three-dimensional coordinates of several key points are extracted from each bronchial segment after normalization to obtain the point-by-point coordinate features.

[0018] In one embodiment, the skeletonization process of the bronchial mask image includes:

[0019] Step a: Normalize the bronchial mask image;

[0020] Step b: Determine whether each pixel in the normalized bronchial mask image meets the deletion criteria;

[0021] Step c: If so, delete the pixels that meet the deletion conditions to obtain a new bronchial mask image;

[0022] Step d: Repeat steps bc above until no pixels can be deleted, thus obtaining the centerline image.

[0023] In one embodiment, the step of combining the point-by-point coordinate features and the high-dimensional convolutional features to obtain combined features, and inputting the combined features into a point voxel image neural network for prediction, includes:

[0024] The point-by-point coordinate features and the high-dimensional convolutional features are combined to obtain the combined features;

[0025] The combined features are used as node features of the voxel graph and input into the voxel graph neural network for prediction.

[0026] In one embodiment of this application, after performing skeletonization processing on the bronchial mask image, the method further includes:

[0027] Based on the skeletonized bronchial mask image, the boundary points are extracted;

[0028] The bronchial mask image after skeletonization is traversed, and the bronchial mask image is divided into bronchial segments based on the type of the traversed boundary points.

[0029] Construct a voxel graph, with each bronchial segment as a node in the voxel graph.

[0030] In one embodiment of this application, before performing skeletonization processing on the bronchial mask image, the following steps are included:

[0031] Connectivity analysis was performed on the bronchial mask image to identify all independent connected regions;

[0032] Calculate the number of pixels in each connected region and determine the largest connected region;

[0033] The bronchial mask image is updated based on the largest connected region, and skeletonization processing is performed based on the updated bronchial mask image.

[0034] In one embodiment, after extracting the binarized segmentation mask of the bronchus from the lung image sample data to obtain the bronchus mask image, the process includes:

[0035] Connectivity analysis was performed on the bronchial mask image to identify all independent connected regions;

[0036] Calculate the number of pixels in each connected region and determine the largest connected region;

[0037] The mask of the bronchial mask image is updated based on the largest connected region.

[0038] In one embodiment, the bronchial mask image is input into a convolutional neural network for prediction to obtain high-dimensional convolutional features, including:

[0039] The three-dimensional coordinates of each bronchial segment are extracted from the bronchial mask image after skeletonization to obtain the point cloud features of each bronchial segment. Each feature point in the point cloud features is used to represent a key point in the corresponding bronchial segment.

[0040] For each bronchial segment, select a preset number of key points;

[0041] For each key point, determine the voxel closest to that key point;

[0042] By indexing the key points, the corresponding convolutional features are extracted from the skeletonized bronchial mask image.

[0043] The convolutional features corresponding to each key point are combined to obtain the high-dimensional convolutional features.

[0044] In one embodiment, the iterative training of the convolutional neural network and the dot-voxel neural network based on the first bronchus segmentation result and the second bronchus segmentation result includes:

[0045] Based on the first bronchus segmentation result and the true label, calculate the first loss value;

[0046] Based on the second bronchial segmentation result and the true label, the second loss value is calculated;

[0047] Calculate the total loss value based on the first loss value and the second loss value;

[0048] The convolutional neural network and the dot voxel neural network are iteratively trained based on the total loss value until the total loss value is less than a preset loss threshold, at which point the training is complete.

[0049] Secondly, a graph-based bronchial classification device is provided, comprising:

[0050] A bronchial mask image acquisition unit is used to acquire lung image sample data and extract the binarized segmentation mask of the bronchus from the lung image sample data to obtain a bronchial mask image.

[0051] The first prediction unit is used to input the bronchial mask image into a convolutional neural network for prediction, and obtain high-dimensional convolutional features and the first bronchial segmentation result.

[0052] The point-by-point coordinate feature acquisition unit is used to perform skeletonization processing on the bronchial mask image and obtain point-by-point coordinate features based on the skeletonized bronchial mask image.

[0053] The second prediction unit is used to combine the point-by-point coordinate features and the high-dimensional convolutional features to obtain combined features, and input the combined features into the point voxel image neural network for prediction to obtain the second bronchus segmentation result.

[0054] The training unit is used to iteratively train the convolutional neural network and the dot voxel neural network based on the first bronchus segmentation result and the second bronchus segmentation result, and to perform segmentation processing on the actual lung image to be segmented through the trained convolutional neural network.

[0055] Thirdly, a computer device is provided, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein the processor implements the above-described graph-based bronchial classification method when executing the computer-readable instructions.

[0056] Fourthly, a readable storage medium is provided, wherein when the computer-readable instructions are executed by one or more processors, the one or more processors cause the one or more processors to perform the graph-based bronchial classification method described above.

[0057] The aforementioned graph-based bronchus classification method, apparatus, computer device, and storage medium are implemented as follows: The method includes: acquiring lung image sample data; extracting a binarized segmentation mask for the bronchus from the lung image sample data to obtain a bronchus mask image; inputting the bronchus mask image into a convolutional neural network for prediction to obtain high-dimensional convolutional features and a first bronchus segmentation result; performing skeletonization processing on the bronchus mask image to obtain point-by-point coordinate features based on the skeletonized bronchus mask image; combining the point-by-point coordinate features and the high-dimensional convolutional features to obtain combined features; inputting the combined features into a point-voxel graph neural network for prediction to obtain a second bronchus segmentation result; iteratively training the convolutional neural network and the point-voxel neural network based on the first and second bronchus segmentation results, and segmenting the actual lung image to be segmented using the trained convolutional neural network. In this embodiment, extracting the relative position information represented by point cloud features can help model the inherent bronchus topology during the classification process. Meanwhile, high-dimensional voxel features can implicitly capture other types of tracheal attributes such as bronchial diameter, providing a powerful supplement to bronchial classification. Utilizing point-voxel map neural networks (PNNNs) to guide the learning process of convolutional neural networks (CNNs) through convolutional features can improve the classification performance of CNNs. CNNs can be used to classify bronchi at the pixel level. CNNs can generate coarse bronchial predictions and voxel-level features for further segment-level classification. Attached Figure Description

[0058] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is a schematic diagram of an application environment for a graph-based bronchial classification method according to an embodiment of the present invention;

[0060] Figure 2 This is a flowchart illustrating a bronchial classification method based on graph structure in one embodiment of the present invention;

[0061] Figure 3 This is a schematic flowchart of the point voxel map generation process in one embodiment of the present invention;

[0062] Figure 4 This is a schematic diagram of a bronchial classification device based on a graph structure according to an embodiment of the present invention;

[0063] Figure 5 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0064] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0065] The graph-based bronchial classification method provided in this embodiment can be applied to, for example... Figure 1 The bronchus classification network (BCNet) consists of two branches: one is a pixel-level convolutional neural network for bronchial labeling, such as UNet, or alternatively, a residual connection neural network, a dense connection neural network, or a U-shaped neural network based on self-attention; the other is a segment-level point-voxel graph neural network for bronchial labeling, such as the point-voxel neural network PV-GNN, or alternatively, a graph self-attention network GAT, a graph sampling aggregation network GraphSAGE, etc. Taking UNet and PV-GNN as examples, the structure-guided representation learning in BCNet means that UNet can benefit from the gradients propagated from the PV-GNN branch, because PV-GNN uses the convolutional features of the UNet branch to construct graph nodes.

[0066] The UNet network includes multiple convolutional modules. The UNet network can obtain pixel-level bronchus segmentation results and can calculate high-dimensional voxel features for subsequent segment-level bronchus labeling. That is, the penultimate layer of the UNet network can generate a three-dimensional feature map as a high-dimensional convolutional feature.

[0067] For lung images, point cloud features can be extracted first, generating point-by-point coordinate features. These point-by-point coordinate features are then combined with the high-dimensional convolutional features output by the UNet network to form the graph node features of the voxel map, which are then input into the PV-GNN network for prediction processing. Specifically, the PV-GNN network can include multiple network modules. Each network model can include Mean sage-convolution (MSC) layers, Graph Normalization (GN) layers, and linear rectified activation functions, such as the ReLU activation function.

[0068] Extracting relative positional information from point cloud features can help model inherent bronchial topology during classification. Simultaneously, high-dimensional voxel features can implicitly capture other tracheal attributes such as bronchial diameter, providing a powerful supplement to bronchial classification. Utilizing GNNs to guide the learning process of UNet through convolutional features can improve UNet's classification performance. By using UNet to classify bronchi at the pixel level, such as... Figure 1 As shown in the upper part, UNet can generate coarse bronchial predictions and voxel-level features for further fragment-level classification.

[0069] In one embodiment, such as Figure 2 As shown, a bronchial classification method based on graph structure is provided, which can be applied to... Figure 1 Taking the server-side as an example, the explanation includes the following steps:

[0070] In step S110, lung image sample data is acquired, and a binarized segmentation mask of the bronchus is extracted from the lung image sample data to obtain a bronchus mask image.

[0071] In this embodiment, CT scan images of the lungs can be read from a medical image database, and a sample dataset, a validation dataset, and a test dataset can be constructed. The sample data may include a large number of lung image samples. Lung image samples are arbitrarily selected from this dataset, and a bronchial binarization segmentation mask is manually annotated. The category label for each bronchial segment is then labeled based on the binarized bronchial mask, and input into the bronchial classification network BCNet for prediction. Iterative training is then performed based on the prediction results and the remaining lung image sample data.

[0072] The binarized bronchial mask is a binary representation of the bronchial region extracted from lung image sample data. The pixel values ​​of the lung image sample data can be normalized, for example, to 0 to 1, to mitigate the effects of lighting variations and noise. A pre-trained segmentation model, such as the UNet network, can then be used for initial bronchial segmentation.

[0073] In step S120, the bronchus mask image is input into a convolutional neural network for prediction to obtain high-dimensional convolutional features and the first bronchus segmentation result;

[0074] In this embodiment, the convolutional neural network can be a UNet network, used to predict the category of each bronchial branch. A three-dimensional feature map of each bronchial branch is generated from the penultimate layer of the UNet as the high-dimensional convolutional feature. Specifically, for each bronchial fragment, the K nearest voxels of K key points can be located, and K convolutional features are extracted from the feature map based on the exponents of the K voxels. The voxel convolutional features of the bronchial fragment can be defined as a combination of K features with a length of C·K, where C is the channel number of the feature map (default C = 24). Therefore, the size of the voxel convolutional features can be 240.

[0075] In this embodiment, the specific process for extracting K-convolutional features from the feature map is as follows: Point cloud features of bronchial segments are extracted from the bronchial mask image, where each point cloud feature represents a key point within the bronchial segment. For each bronchial segment, K key points can be selected, for example, points on the center line can be uniformly sampled. The nearest voxel to each key point is found by calculating the position index of the key point in the bronchial mask image. Then, the corresponding convolutional features are extracted from the feature map using the key point index. Combining the convolutional features of the K key points yields the feature vector of the bronchial segment, i.e., the high-dimensional convolutional features.

[0076] The feature map is a high-dimensional feature map generated by the UNet network.

[0077] In this embodiment, the UNet network is a commonly used convolutional neural network structure for image segmentation tasks. Specifically, it may consist of an encoder (downsampling path) and a decoder (upsampling path), connected by skip connections. The encoder part consists of multiple convolutional and pooling layers, progressively reducing the spatial resolution of the image while increasing the depth of the feature maps. Skip connections are used to concatenate the feature maps from the encoder in the decoder, combining low-level and high-level features. The decoder part may consist of multiple upsampling and convolutional layers, progressively restoring the spatial resolution of the image. The output layer may be a 1x1 convolutional layer, used to map the feature maps to the required number of classes to obtain the first segmentation result. The prediction process specifically involves normalizing the input bronchial mask image, scaling the pixel values ​​to a standard range, such as 0-1, and then performing prediction processing through the encoder and decoder to obtain the class probability of each pixel. Annotated bronchial mask images can be used as supervision signals. A loss function, such as cross-entropy loss, can be used to calculate the loss between the class probability and the true label, and training can be performed based on this loss. The network weights can be adjusted using backpropagation to minimize the error between the predicted results and the true label.

[0078] In step S130, the bronchial mask image is skeletonized, and point-by-point coordinate features are obtained based on the skeletonized bronchial mask image.

[0079] In this embodiment, skeletonization is a technique that converts objects in a binary image into single-pixel-wide skeletons while preserving their topological structure. Using a preset skeletonization algorithm, such as the Zhang-Suen algorithm and the Guo-Hall algorithm, boundary pixels of the bronchial mask image are gradually removed until only the centerline image remains, thus achieving skeletonization processing of the bronchial mask image.

[0080] In this embodiment, for the bronchial mask image after skeletonization, the three-dimensional coordinates of the voxels of the bronchial segments, i.e., the three-dimensional coordinates of the volume pixels, are first extracted to generate point cloud features. This bronchial mask image is a three-dimensional image; the foreground pixel value can be 1, and the background pixel value can be 0. The extracted point cloud features, i.e., the three-dimensional coordinates of the voxels, are normalized and segmented. For each segment, k key points are sampled. Therefore, for each bronchial segment, its point-by-point coordinate features can be defined as a list of the three-dimensional coordinates of these K points. For example, K is set to 10. Therefore, the size of the point-by-point coordinate features is 30.

[0081] It should be noted that point cloud features are three-dimensional point sets composed of voxels (volume pixels) of bronchial segments, which can contain more spatial distribution and density information. Point-by-point coordinate features are typically the three-dimensional coordinates of several key points extracted from the centerline of each bronchial segment, reflecting the geometry and structure of the bronchial segment. Point-by-point coordinate features provide information on the geometry and structure of the bronchial segment. These coordinates can be used to describe the geometric characteristics of the bronchial segment, such as its attitude, curvature, and orientation.

[0082] In step S140, the point-by-point coordinate features and the high-dimensional convolution features are combined to obtain combined features, and the combined features are input into the point voxel image neural network for prediction to obtain the second bronchus segmentation result.

[0083] In this embodiment, for the original lung sample data, a binarized bronchial mask image can be extracted, and the three-dimensional coordinates of key points can be extracted from the bronchial mask image as the point-by-point coordinate features. High-dimensional convolutional features are then predicted using a convolutional neural network. The bronchial mask image can be skeletonized to extract the bronchial centerline, and point cloud features can be extracted from the bronchial centerline. Based on the point cloud features, a voxel map is constructed. Each node in the voxel map identifies a key point of a bronchial segment, and the edges of the voxel map connect adjacent key points, forming a graph structure. The point-by-point coordinate features and the high-dimensional convolutional features are combined through feature concatenation and other methods to obtain combined features. These combined features are used as nodes in the voxel map, and prediction is performed using a voxel map neural network to obtain a second bronchial segmentation result, which is a line segment-level prediction result.

[0084] In this embodiment, the point voxel graph neural network can specifically consist of convolutional norm blocks and fully connected layers. A network module consists of a Mean sage-convolution (MSC) layer, a Graph Normalization (GN) layer, and a ReLU activation function. The MSC layer can use an average aggregation function to combine information from neighboring nodes to overcome inductive bias. First, the combined features are received through the input layer, then output through the network module, and finally, the class of each node is predicted through the output layer. It should be noted that residual connections are added after each convolutional block in this network module to alleviate the gradient vanishing problem. After each prediction, the loss between the predicted result and the true class can be calculated using a preset loss function. For example, cross-entropy loss (CE) and neighborhood consistency regularization (NCR) can be used as loss functions. Cross-entropy loss (CE) can be used to calculate the error between the predicted result and the true class, and neighborhood consistency regularization (NCR) is used to penalize local spatial changes and encourage neighboring nodes of the same class to be closer in the latent space.

[0085] It should be noted that during the training process, the point voxel image neural network can use the pre-labeled bronchial segment categories as supervision signals and perform iterative training using the aforementioned loss function. At the same time, the network weights can be adjusted through the backpropagation algorithm.

[0086] In step S150, based on the first bronchus segmentation result and the second bronchus segmentation result, the convolutional neural network and the dot voxel neural network are iteratively trained, and the trained convolutional neural network is used to segment the actual lung image to be segmented.

[0087] In this embodiment, based on the first bronchus segmentation result, a loss value between the first bronchus segmentation result and the true label can be calculated using a preset loss function, such as cross-entropy loss. Based on the second bronchus segmentation result, a loss value between the second bronchus segmentation result and the true label can also be calculated using a preset loss function, such as cross-entropy loss and neighborhood consistency regularization. The two loss values ​​are added together to obtain the total loss value. The convolutional neural network and the point-voxel neural network are iteratively trained based on the total loss value until a preset convergence condition is met, such as the total loss value being greater than a preset threshold, or the number of iterations reaching a preset number. The trained convolutional neural network and point-voxel neural network are then obtained. This trained convolutional neural network can then be used to segment actual lung images.

[0088] This application provides a graph-based bronchus classification method, comprising: acquiring lung image sample data; extracting a binarized segmentation mask for the bronchus from the lung image sample data to obtain a bronchus mask image; inputting the bronchus mask image into a convolutional neural network for prediction to obtain high-dimensional convolutional features and a first bronchus segmentation result; performing skeletonization processing on the bronchus mask image to obtain point-by-point coordinate features based on the skeletonized bronchus mask image; combining the point-by-point coordinate features and the high-dimensional convolutional features to obtain combined features; inputting the combined features into a point-voxel graph neural network for prediction to obtain a second bronchus segmentation result; iteratively training the convolutional neural network and the point-voxel neural network based on the first and second bronchus segmentation results, and using the trained convolutional neural network to segment the actual lung image to be segmented. In this application embodiment, extracting the relative position information represented by point cloud features can help model the inherent bronchus topology during the classification process. Meanwhile, high-dimensional voxel features can implicitly capture other types of tracheal attributes such as bronchial diameter, providing a powerful supplement to bronchial classification. Utilizing point-voxel map neural networks (PNNNs) to guide the learning process of convolutional neural networks (CNNs) through convolutional features can improve the classification performance of CNNs. CNNs can be used to classify bronchi at the pixel level. CNNs can generate coarse bronchial predictions and voxel-level features for further segment-level classification.

[0089] In one embodiment of this application, the process of skeletonizing the bronchial mask image and obtaining point-by-point coordinate features based on the skeletonized bronchial mask image includes:

[0090] The bronchial mask image is refined into a single-pixel-wide centerline using a preset skeletonization algorithm to obtain a centerline image.

[0091] The three-dimensional coordinates of the centerline voxels are extracted from the centerline image to obtain point cloud features;

[0092] Based on the point cloud features, each three-dimensional coordinate is normalized and the boundary points are extracted to obtain the point-by-point coordinate features.

[0093] Specifically, a binarized bronchial mask image is acquired. This bronchial mask image is three-dimensional, where objects are represented in white (pixel value 1) and the background in black (pixel value 0). It can be understood as a three-dimensional array, or matrix. Using a pre-defined skeletonization algorithm, such as the Zhang-Suen and Guo-Hall algorithms, boundary pixels of the objects are progressively removed until only the centerline image remains. In other words, the bronchial mask image is refined into a single-pixel-wide centerline image through skeletonization. The three-dimensional coordinates of the centerline voxels are extracted from the centerline image to generate point cloud features. The three-dimensional coordinates of the centerline voxels are then normalized, and boundary points are extracted to obtain point-by-point coordinate features.

[0094] In one embodiment of this application, the normalization and boundary point extraction of each three-dimensional coordinate to obtain the point-by-point coordinate features includes:

[0095] The bounding box of each bronchial segment is cropped from the skeletonized bronchial mask image, wherein each bronchial segment is composed of centerline voxels within its corresponding bounding box.

[0096] The three-dimensional coordinates of each voxel in the centerline image are normalized relative to the shape of the bounding box of the corresponding bronchial segment.

[0097] The three-dimensional coordinates of several key points are extracted from each bronchial segment after normalization to obtain the point-by-point coordinate features.

[0098] Specifically, based on the bronchial mask image after skeletonization, the bounding box of each bronchial segment can be cropped from the bronchial tree. Each segment consists of a centerline voxel within the bounding box. Then, the coordinates (X, Y, Z) of each voxel can be normalized to [0, 1] relative to the shape of the bounding box. This normalization helps to describe the pose and angle of a line segment. Assuming the length of the centerline is L, the centerline can be divided into K-1 intervals. The system is divided into K segments, and K points are collected from the start and end points of these K-1 segments. For each bronchial segment, its point-by-point coordinate feature can be defined as a list of three-dimensional coordinates of these K points. K is set to 10. Therefore, the size of the point-by-point coordinate feature is 30.

[0099] In one embodiment of this application, the skeletonization process of the bronchial mask image includes:

[0100] Step a: Normalize the bronchial mask image;

[0101] Step b: Determine whether each pixel in the normalized bronchial mask image meets the deletion criteria;

[0102] Step c: If so, delete the pixels that meet the deletion conditions to obtain a new bronchial mask image;

[0103] Step d: Repeat steps bc above until no pixels can be deleted, thus obtaining the centerline image.

[0104] The preset skeletonization algorithms can include the Zhang-Suen algorithm and the Guo-Hall algorithm. Taking the Zhang-Suen algorithm as an example, the specific steps for skeletonization of a bronchial mask image are as follows: A marker array can be constructed to mark pixels that need to be deleted. Then, each pixel in the bronchial mask image is traversed. For each pixel (X, Y), it is determined whether it meets the deletion condition. The deletion condition can include: pixel (x, y) is a foreground pixel (value 1); it has 2 to 6 foreground pixels in its 8-neighborhood; the transition from 0 to 1 is 1; at least one neighboring pixel is a background pixel and satisfies one or any combination of specific patterns. If the above conditions are met, pixel (X, Y) is marked for deletion, and all pixels marked for deletion are set to 0, i.e., they are treated as background pixels. Then, the above steps can be repeated, but with different deletion conditions than before, to delete pixels again until no pixels can be deleted, thus obtaining a centerline image with a width of one pixel.

[0105] In one embodiment of this application, the step of combining the point-by-point coordinate features and the high-dimensional convolutional features to obtain combined features, and inputting the combined features into a point voxel image neural network for prediction, includes:

[0106] The point-by-point coordinate features and the high-dimensional convolutional features are combined to obtain the combined features;

[0107] The combined features are used as node features of the voxel graph and input into the voxel graph neural network for prediction.

[0108] Specifically, point-by-point coordinate features and high-dimensional convolutional features can be combined through feature splicing to obtain richer combined features, which can then be used as node features of the voxel map and input into the voxel map neural network for bronchial segment classification.

[0109] In one embodiment of this application, before performing skeletonization processing on the bronchial mask image, the following steps are included:

[0110] Connectivity analysis was performed on the bronchial mask image to identify all independent connected regions;

[0111] Calculate the number of pixels in each connected region and determine the largest connected region;

[0112] The bronchial mask image is updated based on the largest connected region, and skeletonization processing is performed based on the updated bronchial mask image.

[0113] Specifically, for the binarized bronchial mask image, such as Figure 3 As shown in Figure (a), the mask is updated to its maximum connected region and defined as a point voxel map, which can be embedded using hybrid voxels and point-level features as nodes. Connectivity analysis is performed on the binary mask using a connected component labeling algorithm (such as Flood Fill or Connected Components Labeling) to label all independent connected regions. The number of pixels in each connected region is calculated, the size of each connected region is recorded, and the region with the largest number of pixels is found from all connected regions. A new binary bronchial mask is created, retaining only the pixels of the maximum connected region and setting the pixels of this region and the region itself to 0. Updating the mask to its maximum connected region ensures that only the largest connected portion of the bronchial tree is retained, removing noise and isolated small regions.

[0114] In one embodiment of this application, after the bronchial mask image is skeletonized, the process further includes:

[0115] Based on the skeletonized bronchial mask image, the boundary points are extracted;

[0116] The bronchial mask image after skeletonization is traversed, and the bronchial mask image is divided into bronchial segments based on the type of the traversed boundary points.

[0117] Construct a voxel graph, with each bronchial segment as a node in the voxel graph.

[0118] See Figure 3 A bronchial mask image can be skeletonized using a preset skeletonization algorithm to obtain a centerline image, such as... Figure 3 As shown in Figure (b), for each voxel on the centerline image, a preset number of foreground voxels N in 26 neighboring neighborhoods can be defined, such as endpoint N=1, edge points N=2, and segmentation points N>=3. Then, as shown in Figure (b),... Figure 3 As shown in (c), the branches are divided into segments by the endpoints, edge points, and split points mentioned above, as follows: Figure 3As shown in (d), an empty segment list can be constructed to store the pixels of each segment. The points on the centerline can include endpoints, edge points, and segmentation points. Each pixel in the skeletonized image is classified as follows: endpoint: only one foreground pixel in its 8-neighborhood; edge point: two foreground pixels in its 8-neighborhood; segmentation point: three or more foreground pixels in its 8-neighborhood. Starting from the first endpoint, the process traverses along the centerline until another endpoint or segmentation point is reached. Each time an endpoint or segmentation point is reached, a new segment is generated and added to the segment list. For each segmentation point, a recursive traversal method can be used to traverse each branch and generate more segments. Each bronchial segment can be understood as corresponding to a node in the voxel graph, and the edges of the graph are defined by the connectivity between two line segments. Two line segments can be separated by a segmentation point.

[0119] In one embodiment of this application, the bronchial mask image is input into a convolutional neural network for prediction to obtain high-dimensional convolutional features, including:

[0120] The three-dimensional coordinates of each bronchial segment are extracted from the bronchial mask image after skeletonization to obtain the point cloud features of each bronchial segment. Each feature point in the point cloud features is used to represent a key point in the corresponding bronchial segment.

[0121] For each bronchial segment, select a preset number of key points;

[0122] For each key point, determine the voxel closest to that key point;

[0123] By indexing the key points, the corresponding convolutional features are extracted from the skeletonized bronchial mask image.

[0124] The convolutional features corresponding to each key point are combined to obtain the high-dimensional convolutional features.

[0125] Specifically, the category of each bronchial branch can be predicted using the UNet network, and a 3D feature map can be generated from the penultimate layer of the UNet network. For each bronchial fragment, the nearest voxels of K keypoints can be located, and K convolutional features can be extracted from the feature map based on the voxel exponents. The high-dimensional convolutional features of the bronchial fragment can then be defined as a K-feature combination with a length of C·K, where C is the channel number of the feature map (default C = 24). Therefore, the size of the voxel convolutional features is 240.

[0126] The method for extracting k-convolutional features from the feature map can be as follows: Point cloud features can be generated from the skeletonized bronchial mask, where each point represents a key point in a bronchial segment. For each bronchial segment, K key points can be identified. For example, this can be achieved by uniformly sampling points within each bronchial segment and calculating the position index of each key point in the feature map to find the nearest voxel. Then, the corresponding convolutional features are extracted from the feature map using the key point index. It should be noted that the feature map is a high-dimensional feature map generated by the UNet network.

[0127] In one embodiment of this application, the iterative training of the convolutional neural network and the dot voxel neural network based on the first bronchus segmentation result and the second bronchus segmentation result includes:

[0128] Based on the first bronchus segmentation result and the true label, calculate the first loss value;

[0129] Based on the second bronchial segmentation result and the true label, the second loss value is calculated;

[0130] Calculate the total loss value based on the first loss value and the second loss value;

[0131] The convolutional neural network and the dot voxel neural network are iteratively trained based on the total loss value until the total loss value is less than a preset loss threshold, at which point the training is complete.

[0132] Specifically, for point-voxel neural networks, since adjacent branches in the bronchial tree may belong to the same category in the prior topology, a new Neighborhood Consistency Regularization (NCR) is set to penalize local spatial variations. This makes neighboring nodes of the same category closer in the latent space, which can satisfy the prior information that neighboring graph nodes tend to belong to the same category. For example, Y = y1, y2, ..., y N Z is a set of heat vectors containing the fundamental truth values ​​of each branch, where Z = z1, z2, ..., z N Let be a set of heat vectors predicted by the model for each branch. The second loss value can then be calculated using the NCR loss formula, which is:

[0133]

[0134] Where V i Let z be the set of neighbors of the node at level i, where j represents the j-th node in the set. iLet I(·) represent the logit vector of the i-th node output by the last fully connected layer. I(·) is an indicator function that returns 1 if a condition is met, otherwise it returns 0. M and N are the number of edges and nodes in the graph, respectively. Let λ be a scalar to balance the weights of regularization and CE loss (λ can be set to 1 empirically).

[0135] Therefore, the second loss value can include the aforementioned NCR and a cross-entropy loss, specifically expressed as:

[0136] L pvgnn =L CE +λL NCR

[0137] For convolutional neural networks, a loss value L can be calculated using the cross-entropy loss function, regularization loss function, etc. unet .

[0138] Training is performed end-to-end using both the UNet and PV-GNN networks, as the gradients of the GNN branches can be fed back to the UNet branches via the convolutional voxel feature extraction operation. Therefore, the final total loss L... cls It can be composed of two parts, specifically represented as:

[0139] L cls =L unet +L pvgnn

[0140] Among them, L unet L is the first loss value. pvgnn This is the second loss value.

[0141] It should be noted that a deep graph neural network (GNN) was constructed to better integrate information from the point cloud. Since GNNs are prone to gradient vanishing with increasing depth, a GN was introduced to move and scale the feature values, which makes the graph neural network converge faster. Except for the first block, each block is followed by an element-wise addition as a residual connection. Let H... k Let H represent the output of the k-th block, σ represent the ReLU operation, and MSC represent the Mean sage-convolution (MSC) layer of the GNN. k It can be calculated using the following formula:

[0142] H k =σ(GN(MSC(H) k-1 ))+H k-1 .

[0143] In this embodiment, extracting the relative position information represented by point cloud features helps model the inherent bronchial topology during classification. Simultaneously, high-dimensional voxel features can implicitly capture other types of tracheal attributes such as bronchial diameter, providing a powerful supplement to bronchial classification. Utilizing a point-voxel graph neural network to guide the learning process of a convolutional neural network through convolutional features can improve the classification performance of the convolutional neural network. Bronchial classification is performed at the pixel level using a convolutional neural network. The convolutional neural network can generate coarse bronchial predictions and voxel-level features for further segment-level classification. Furthermore, the graph neural network uses both point cloud features and high-dimensional convolutional features as node features; compared to using only point cloud features or high-dimensional convolutional features, the combined use of features significantly improves accuracy. Multi-task learning is performed jointly by the graph neural network and the first convolutional neural network, thereby increasing the structural information of the shared feature extractor and image features between the two branches. Moreover, during inference, only the convolutional neural network is run for inference, without incurring additional computational costs.

[0144] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0145] In one embodiment, a graph-based bronchial classification device is provided, which corresponds one-to-one with the graph-based bronchial classification method described in the above embodiments. For example... Figure 4 As shown, the graph-based bronchial classification device includes a bronchial mask image acquisition unit 10, a first prediction unit 20, a point-by-point coordinate feature acquisition unit 30, a second prediction unit 40, and a training unit 50. Detailed descriptions of each functional module are as follows:

[0146] The bronchial mask image acquisition unit 10 is used to acquire lung image sample data and extract the binarized segmentation mask of the bronchus from the lung image sample data to obtain a bronchial mask image.

[0147] The first prediction unit 20 is used to input the bronchus mask image into a convolutional neural network for prediction, and obtain high-dimensional convolutional features and the first bronchus segmentation result.

[0148] The point-by-point coordinate feature acquisition unit 30 is used to perform skeletonization processing on the bronchial mask image and obtain point-by-point coordinate features based on the skeletonized bronchial mask image.

[0149] The second prediction unit 40 is used to combine the point-by-point coordinate features and the high-dimensional convolutional features to obtain combined features, and input the combined features into the point voxel map neural network for prediction to obtain the second bronchus segmentation result.

[0150] The training unit 50 is used to iteratively train the convolutional neural network and the dot voxel neural network based on the first bronchus segmentation result and the second bronchus segmentation result, and to perform segmentation processing on the actual lung image to be segmented through the trained convolutional neural network.

[0151] In one embodiment of this application, the point-by-point coordinate feature acquisition unit 30 is further configured to:

[0152] The bronchial mask image is refined into a single-pixel-wide centerline using a preset skeletonization algorithm to obtain a centerline image.

[0153] The three-dimensional coordinates of the centerline voxels are extracted from the centerline image to obtain point cloud features;

[0154] Based on the point cloud features, each three-dimensional coordinate is normalized and the boundary points are extracted to obtain the point-by-point coordinate features.

[0155] In one embodiment of this application, the point-by-point coordinate feature acquisition unit 30 is further configured to:

[0156] The bounding box of each bronchial segment is cropped from the skeletonized bronchial mask image, wherein each bronchial segment is composed of centerline voxels within its corresponding bounding box.

[0157] The three-dimensional coordinates of each voxel in the centerline image are normalized relative to the shape of the bounding box of the corresponding bronchial segment.

[0158] The three-dimensional coordinates of several key points are extracted from each bronchial segment after normalization to obtain the point-by-point coordinate features.

[0159] In one embodiment of this application, the point-by-point coordinate feature acquisition unit 30 is further configured to:

[0160] Step a: Normalize the bronchial mask image;

[0161] Step b: Determine whether each pixel in the normalized bronchial mask image meets the deletion criteria;

[0162] Step c: If so, delete the pixels that meet the deletion conditions to obtain a new bronchial mask image;

[0163] Step d: Repeat steps bc above until no pixels can be deleted, thus obtaining the centerline image.

[0164] In one embodiment of this application, the second prediction unit 40 is further configured to:

[0165] The point-by-point coordinate features and the high-dimensional convolutional features are combined to obtain the combined features;

[0166] The combined features are used as node features of the voxel graph and input into the voxel graph neural network for prediction.

[0167] In one embodiment of this application, the apparatus further includes: a point voxel map construction unit, used for:

[0168] Based on the skeletonized bronchial mask image, the boundary points are extracted;

[0169] The bronchial mask image after skeletonization is traversed, and the bronchial mask image is divided into bronchial segments based on the type of the traversed boundary points.

[0170] Construct a voxel graph, with each bronchial segment as a node in the voxel graph.

[0171] In one embodiment of this application, the device further includes: a bronchial mask image updating unit, used for:

[0172] Connectivity analysis was performed on the bronchial mask image to identify all independent connected regions;

[0173] Calculate the number of pixels in each connected region and determine the largest connected region;

[0174] The bronchial mask image is updated based on the largest connected region, and skeletonization processing is performed based on the updated bronchial mask image.

[0175] In one embodiment of this application, the first prediction unit 20 is further configured to:

[0176] The three-dimensional coordinates of each bronchial segment are extracted from the bronchial mask image after skeletonization to obtain the point cloud features of each bronchial segment. Each feature point in the point cloud features is used to represent a key point in the corresponding bronchial segment.

[0177] For each bronchial segment, select a preset number of key points;

[0178] For each key point, determine the voxel closest to that key point;

[0179] By indexing the key points, the corresponding convolutional features are extracted from the skeletonized bronchial mask image.

[0180] The convolutional features corresponding to each key point are combined to obtain the high-dimensional convolutional features.

[0181] In one embodiment of this application, the training unit 50 is further configured to:

[0182] Based on the first bronchus segmentation result and the true label, calculate the first loss value;

[0183] Based on the second bronchial segmentation result and the true label, the second loss value is calculated;

[0184] Calculate the total loss value based on the first loss value and the second loss value;

[0185] The convolutional neural network and the dot voxel neural network are iteratively trained based on the total loss value until the total loss value is less than a preset loss threshold, at which point the training is complete.

[0186] In this embodiment, extracting the relative position information represented by point cloud features helps model the inherent bronchial topology during classification. Simultaneously, high-dimensional voxel features can implicitly capture other types of tracheal attributes such as bronchial diameter, providing a powerful supplement to bronchial classification. Utilizing a point-voxel graph neural network to guide the learning process of a convolutional neural network through convolutional features can improve the classification performance of the convolutional neural network. Bronchial classification is performed at the pixel level using a convolutional neural network. The convolutional neural network can generate coarse bronchial predictions and voxel-level features for further segment-level classification. Furthermore, the graph neural network uses both point cloud features and high-dimensional convolutional features as node features; compared to using only point cloud features or high-dimensional convolutional features, the combined use of features significantly improves accuracy. Multi-task learning is performed jointly by the graph neural network and the first convolutional neural network, thereby increasing the structural information of the shared feature extractor and image features between the two branches. Moreover, during inference, only the convolutional neural network is run for inference, without incurring additional computational costs.

[0187] Specific limitations regarding the graph-based bronchial classification device can be found in the limitations of the graph-based bronchial classification method described above, and will not be repeated here. Each module in the aforementioned graph-based bronchial classification device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0188] In one embodiment, a computer device is provided, which may be a terminal device, and its internal structure diagram may be as follows: Figure 5As shown, the computer device includes a processor, memory, and network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes a readable storage medium storing computer-readable instructions. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer-readable instructions implement a graph-based bronchial classification method. The readable storage medium provided in this embodiment includes both non-volatile and volatile readable storage media.

[0189] In this application embodiment, a computer device is provided, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor. When the processor executes the computer-readable instructions, it implements the steps of the graph-based bronchial classification method described above.

[0190] In one embodiment of the application, a readable storage medium is provided, which stores computer-readable instructions that, when executed by a processor, implement the steps of the graph-based bronchial classification method described above.

[0191] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware with computer-readable instructions. These computer-readable instructions can be stored in a non-volatile readable storage medium or a volatile readable storage medium. When executed, these computer-readable instructions can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0192] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0193] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A bronchial classification method based on graph structure, characterized in that, The method includes: Acquire lung image sample data, and extract the binarized segmentation mask of the bronchus from the lung image sample data to obtain the bronchus mask image; The bronchial mask image is input into a convolutional neural network for prediction to obtain high-dimensional convolutional features and a first bronchial segmentation result. The high-dimensional convolutional features are obtained as follows: Three-dimensional coordinates of each bronchial segment are extracted from the skeletonized bronchial mask image to obtain point cloud features for each bronchial segment. Each feature point in the point cloud features represents a key point in the corresponding bronchial segment. For each bronchial segment, a preset number of key points are selected. For each key point, the nearest voxel to the key point is determined. Corresponding convolutional features are extracted from the skeletonized bronchial mask image using the index of the key points. The convolutional features corresponding to each key point are combined to obtain the high-dimensional convolutional features. The bronchial mask image is skeletonized, and point-by-point coordinate features are obtained based on the skeletonized bronchial mask image. The point-by-point coordinate features and the high-dimensional convolutional features are combined to obtain combined features. The combined features are then input into a point voxel image neural network for prediction to obtain the second bronchus segmentation result. Based on the first bronchus segmentation result and the second bronchus segmentation result, the convolutional neural network and the dot voxel neural network are iteratively trained, and the trained convolutional neural network is used to segment the actual lung image to be segmented.

2. The bronchial classification method based on graph structure as described in claim 1, characterized in that, The process of skeletonizing the bronchial mask image and obtaining point-by-point coordinate features based on the skeletonized bronchial mask image includes: The bronchial mask image is refined into a single-pixel-wide centerline using a preset skeletonization algorithm to obtain a centerline image. The three-dimensional coordinates of the centerline voxels are extracted from the centerline image to obtain point cloud features; Based on the point cloud features, each three-dimensional coordinate is normalized and the boundary points are extracted to obtain the point-by-point coordinate features.

3. The bronchial classification method based on graph structure as described in claim 2, characterized in that, The process of normalizing each three-dimensional coordinate and extracting boundary points to obtain the point-by-point coordinate features includes: The bounding box of each bronchial segment is cropped from the skeletal mask image after skeletonization, wherein each bronchial segment is composed of the centerline voxel within its corresponding bounding box; The three-dimensional coordinates of each voxel in the centerline image are normalized relative to the shape of the bounding box of the corresponding bronchial segment. The three-dimensional coordinates of several key points are extracted from each bronchial segment after normalization to obtain the point-by-point coordinate features.

4. The bronchial classification method based on graph structure as described in claim 1, characterized in that, The skeletonization process for the bronchial mask image includes: Step a: Normalize the bronchial mask image; Step b: Determine whether each pixel in the normalized bronchial mask image meets the deletion criteria; Step c: If so, delete the pixels that meet the deletion conditions to obtain a new bronchial mask image; Step d: Repeat steps bc above until no pixels can be deleted, then refine the bronchial mask image into a center line that is one pixel wide to obtain the center line image.

5. The bronchial classification method based on graph structure as described in claim 1, characterized in that, The step of combining the point-by-point coordinate features and the high-dimensional convolutional features to obtain combined features, and inputting the combined features into a point voxel image neural network for prediction, includes: The point-by-point coordinate features and the high-dimensional convolutional features are combined to obtain the combined features; The combined features are used as node features of the voxel graph and input into the voxel graph neural network for prediction.

6. The bronchial classification method based on graph structure as described in claim 1, characterized in that, After performing skeletonization processing on the bronchial mask image, the process further includes: Based on the skeletonized bronchial mask image, the boundary points are extracted; The bronchial mask image after skeletonization is traversed, and the bronchial mask image is divided into bronchial segments based on the type of the traversed boundary points. Construct a voxel graph, with each bronchial segment as a node in the voxel graph.

7. The bronchial classification method based on graph structure as described in claim 1, characterized in that, Before performing skeletonization processing on the bronchial mask image, the following steps are included: Connectivity analysis was performed on the bronchial mask image to identify all independent connected regions; Calculate the number of pixels in each connected region and determine the largest connected region; The bronchial mask image is updated based on the largest connected region, and skeletonization processing is performed based on the updated bronchial mask image.

8. The bronchial classification method based on graph structure as described in claim 1, characterized in that, The iterative training of the convolutional neural network and the dot-voxel neural network based on the first bronchus segmentation result and the second bronchus segmentation result includes: Based on the first bronchus segmentation result and the true label, calculate the first loss value; Based on the second bronchial segmentation result and the true label, the second loss value is calculated; Calculate the total loss value based on the first loss value and the second loss value; The convolutional neural network and the dot voxel neural network are iteratively trained based on the total loss value until the total loss value is less than a preset loss threshold, at which point the training is complete.

9. A bronchial classification device based on a graph structure, characterized in that, The device includes: A bronchial mask image acquisition unit is used to acquire lung image sample data and extract the binarized segmentation mask of the bronchus from the lung image sample data to obtain a bronchial mask image. The first prediction unit is used to input the bronchial mask image into a convolutional neural network for prediction, obtaining high-dimensional convolutional features and a first bronchial segmentation result. The high-dimensional convolutional features are obtained as follows: Three-dimensional coordinates of each bronchial segment are extracted from the skeletonized bronchial mask image to obtain point cloud features for each bronchial segment. Each feature point in the point cloud features represents a key point in the corresponding bronchial segment. For each bronchial segment, a preset number of key points are selected. For each key point, the nearest voxel to the key point is determined. Corresponding convolutional features are extracted from the skeletonized bronchial mask image using the index of the key points. The convolutional features corresponding to each key point are combined to obtain the high-dimensional convolutional features. The point-by-point coordinate feature acquisition unit is used to perform skeletonization processing on the bronchial mask image and obtain point-by-point coordinate features based on the skeletonized bronchial mask image. The second prediction unit is used to combine the point-by-point coordinate features and the high-dimensional convolutional features to obtain combined features, and input the combined features into the point voxel image neural network for prediction to obtain the second bronchus segmentation result. The training unit is used to iteratively train the convolutional neural network and the dot voxel neural network based on the first bronchus segmentation result and the second bronchus segmentation result, and to perform segmentation processing on the actual lung image to be segmented through the trained convolutional neural network.

Images