Bronchial classification method and device based on graph structure
Through a graph-based method, combining convolutional neural network and point voxel neural network, the features of bronchial mask images are extracted for iterative training, which solves the problem of insufficient individual differences in bronchial tree classification methods in the existing technology, and improves the Classification accuracy and efficiency.
Patent Information
- Application Number
- CN202411107331.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-12
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-08-12
AI Technical Summary
The existing bronchial tree classification methods rely on manual feature production or additional labels to deal with insufficient individual differences, resulting in limited improvement in classification accuracy and efficiency.
Using a graph structure-based method, the point-by-point coordinate features and high-dimensional convolutional features of bronchial mask images are extracted through the combination of convolutional neural network and point voxel graph neural network, and iterative training is carried out to improve classification performance.
It improves the accuracy and efficiency of bronchial classification, can better handle individual differences in bronchial structure, and enhances the classification performance of convolutional neural networks.
Smart Images

Figure CN119027729A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of neural network technology, and in particular to a bronchial classification method and device based on a graph structure. Background Art
[0002] Existing bronchial tree classification methods mainly include traditional key point detection, rule-based methods and deep learning methods. Among them, traditional key point detection mainly uses key point detection to achieve lobe-level bronchial classification, but it requires manual features. Rule-based methods, such as bottom-up bronchial labeling methods, have poor handling of individual differences. Deep learning methods, such as linear programming post-processing or two-stage convolutional neural networks, have improved, but still have deficiencies in handling bronchial topology. It can be seen that the current bronchial tree classification methods either rely on manual features or require additional labels, and are not efficient in dealing with individual differences in bronchial structure, resulting in limited improvements in classification accuracy and efficiency. Summary of the invention
[0003] Based on this, it is necessary to provide a graph-structured bronchial classification method, apparatus, computer equipment and storage medium to address the above technical issues, so as to solve at least one problem existing in the above-mentioned prior art.
[0004] In a first aspect, the embodiment of the present application is implemented by providing a bronchial classification method based on a graph structure, comprising:
[0005] Acquire lung image sample data, and extract a binary segmentation mask of the bronchus from the lung image sample data to obtain a bronchus mask image;
[0006] Inputting the bronchial mask image into a convolutional neural network for prediction to obtain high-dimensional convolution features and a first bronchial segmentation result;
[0007] Skeletonize the bronchial mask image, and obtain point-by-point coordinate features based on the skeletonized bronchial mask image;
[0008] Combining the point-by-point coordinate features and the high-dimensional convolution features to obtain a combined feature, and inputting the combined feature into a point voxel graph neural network for prediction to obtain a second bronchus segmentation result;
[0009] Based on the first bronchial segmentation result and the second bronchial segmentation result, the convolutional neural network and the point voxel neural network are iteratively trained, and the actual lung image to be segmented is segmented by the trained convolutional neural network.
[0010] In one embodiment, the skeletonizing the bronchial mask image and obtaining point-by-point coordinate features based on the skeletonized bronchial mask image include:
[0011] Refining the bronchial mask image into a centerline with a single pixel width by using a preset skeletonization algorithm to obtain a centerline image;
[0012] Extracting three-dimensional coordinates of centerline voxels from the centerline image to obtain point cloud features;
[0013] Based on the point cloud features, each three-dimensional coordinate is normalized and a demarcation point is extracted to obtain the point-by-point coordinate features.
[0014] In one embodiment, normalizing each three-dimensional coordinate and extracting the demarcation point to obtain the point-by-point coordinate feature includes:
[0015] cropping a bounding box of each bronchial segment from the skeletonized bronchial mask image, wherein each bronchial segment is composed of centerline voxels within its corresponding bounding box;
[0016] normalizing the three-dimensional coordinates of each voxel in the centerline image relative to the shape of a bounding box of the corresponding bronchial segment;
[0017] The three-dimensional coordinates of several key points are extracted from each bronchial segment after normalization to obtain the point-by-point coordinate features.
[0018] In one embodiment, the skeletonization of the bronchial mask image includes:
[0019] Step a: normalizing the bronchial mask image;
[0020] Step b: determining whether each pixel in the normalized bronchial mask image meets the deletion condition;
[0021] Step c: If yes, then the pixels satisfying the deletion condition are deleted to obtain a new bronchial mask image;
[0022] Step d: Repeat the above steps bc until no pixels can be deleted, and obtain the centerline image.
[0023] In one embodiment, the step of combining the point-by-point coordinate features and the high-dimensional convolutional features to obtain a combined feature, and inputting the combined feature into a point voxel graph neural network for prediction includes:
[0024] Combining the point-by-point coordinate features and the high-dimensional convolution features to obtain a combined feature;
[0025] The combined features are used as node features of the point voxel graph and are input into the point voxel graph neural network for prediction.
[0026] In one embodiment of the present application, after skeletonizing the bronchial mask image, the method further includes:
[0027] Extract the demarcation point based on the skeletonized bronchial mask image;
[0028] Traversing the skeletonized bronchial mask image, and dividing the bronchial mask image into bronchial segments based on the types of the traversed demarcation points;
[0029] A point voxel map is constructed, and each of the bronchial segments is used as a node of the point voxel map.
[0030] In one embodiment of the present application, before skeletonizing the bronchial mask image, the process includes:
[0031] Performing connected domain analysis on the bronchial mask image to determine all independent connected regions;
[0032] Calculate the number of pixels in each connected area and determine the largest connected area;
[0033] The bronchial mask image is updated according to the maximum connected area, so as to perform skeletonization processing based on the updated bronchial mask image.
[0034] In one embodiment, after extracting the binary segmentation mask of the bronchus from the lung image sample data to obtain the bronchus mask image, the method includes:
[0035] Performing connected domain analysis on the bronchial mask image to determine all independent connected regions;
[0036] Calculate the number of pixels in each connected area and determine the largest connected area;
[0037] The mask of the bronchial mask image is updated according to the maximum connected area.
[0038] In one embodiment, the bronchial mask image is input into a convolutional neural network for prediction to obtain high-dimensional convolution features, including:
[0039] Extracting the three-dimensional coordinates of each bronchial segment from the skeletonized bronchial mask image to obtain a point cloud feature of each bronchial segment, wherein each feature point in the point cloud feature is used to represent a key point in the corresponding bronchial segment;
[0040] For each bronchial segment, select preset key points;
[0041] For each keypoint, determine the voxel closest to the keypoint;
[0042] Extracting corresponding convolution features from the skeletonized bronchial mask image through the index of the key points;
[0043] The convolution features corresponding to each key point are combined to obtain the high-dimensional convolution features.
[0044] In one embodiment, the iterative training of the convolutional neural network and the point voxel neural network based on the first bronchial segmentation result and the second bronchial segmentation result includes:
[0045] Calculating a first loss value based on the first bronchus segmentation result and the true label;
[0046] Calculating a second loss value based on the second bronchus segmentation result and the true label;
[0047] Calculating a total loss value based on the first loss value and the second loss value;
[0048] The convolutional neural network and the point voxel neural network are iteratively trained based on the total loss value until the total loss value is less than a preset loss threshold and the training is completed.
[0049] In a second aspect, a bronchial classification device based on a graph structure is provided, comprising:
[0050] A bronchial mask image acquisition unit, used for acquiring lung image sample data, and extracting a binary segmentation mask of the bronchus from the lung image sample data to obtain a bronchial mask image;
[0051] A first prediction unit, used for inputting the bronchial mask image into a convolutional neural network for prediction, to obtain a high-dimensional convolution feature and a first bronchial segmentation result;
[0052] A point-by-point coordinate feature acquisition unit, used for skeletonizing the bronchial mask image, and obtaining point-by-point coordinate features based on the skeletonized bronchial mask image;
[0053] a second prediction unit, configured to combine the point-by-point coordinate features and the high-dimensional convolution features to obtain a combined feature, and input the combined feature into a point voxel graph neural network for prediction to obtain a second bronchus segmentation result;
[0054] A training unit is used to iteratively train the convolutional neural network and the point voxel neural network based on the first bronchial segmentation result and the second bronchial segmentation result, and perform segmentation processing on the actual lung image to be segmented through the trained convolutional neural network.
[0055] In a third aspect, a computer device is provided, comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein the processor implements the above-mentioned graph-based bronchial classification method when executing the computer-readable instructions.
[0056] In a fourth aspect, a computer-readable storage medium is provided, wherein when the computer-readable instructions are executed by one or more processors, the one or more processors execute the above-mentioned graph-structure-based bronchial classification method.
[0057] The above-mentioned graph-structure-based bronchial classification method, device, computer equipment and storage medium, the method of which is implemented, includes: obtaining lung image sample data, extracting the binary segmentation mask of the bronchus from the lung image sample data to obtain a bronchial mask image; inputting the bronchial mask image into a convolutional neural network for prediction to obtain a high-dimensional convolution feature and a first bronchial segmentation result; skeletonizing the bronchial mask image, and obtaining point-by-point coordinate features based on the skeletonized bronchial mask image; combining the point-by-point coordinate features and the high-dimensional convolution features to obtain combined features, and inputting the combined features into a point voxel graph neural network for prediction to obtain a second bronchial segmentation result; based on the first bronchial segmentation result and the second bronchial segmentation result, iteratively training the convolutional neural network and the point voxel neural network, and segmenting the actual lung image to be segmented through the trained convolutional neural network. In an embodiment of the present application, extracting the relative position information represented by the point cloud features can help model the inherent bronchial topology during the classification process. At the same time, high-dimensional voxel features can implicitly capture other types of tracheal properties such as bronchial diameter, which is a powerful supplement for bronchial classification. Using point voxel graph neural networks to guide the learning process of convolutional neural networks through convolutional features can improve the classification performance of convolutional neural networks. The convolutional neural network can be used to classify bronchi at the pixel level. The convolutional neural network can generate coarse bronchial predictions and voxel-level features for further segment-level classification. BRIEF DESCRIPTION OF THE DRAWINGS
[0058] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.
[0059] Figure 1 is a schematic diagram of an application environment of a bronchial classification method based on a graph structure in one embodiment of the present invention;
[0060] Figure 2 is a schematic diagram of a process of a bronchial classification method based on a graph structure in one embodiment of the present invention;
[0061] Figure 3 is a schematic diagram of a process of generating a point voxel map in one embodiment of the present invention;
[0062] Figure 4 is a structural schematic diagram of a bronchial classification device based on a graph structure in one embodiment of the present invention;
[0063] Figure 5 is a schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION
[0064] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0065] The bronchial classification method based on graph structure provided in this embodiment can be applied in Figure 1 In the bronchus classification network (BCNet), BCNet includes two network branches: one is a convolutional neural network for pixel-level bronchial labeling, such as UNet, which can also use residual connection neural network, dense connection neural network, or U-type neural network based on self-attention mechanism, and the other is a point voxel graph neural network for segment-level bronchial labeling, such as point voxel neural network PV-GNN, which can also use graph self-attention network GAT, graph sampling aggregation network GraphSAGE and other neural networks. Taking UNet and PV-GNN as examples, the structure-oriented representation learning in BCNet means that UNet can benefit from the gradient propagated by the PV-GNN branch, because PV-GNN uses the convolutional features of the UNet branch to construct the graph nodes.
[0066] Among them, the UNet network includes multi-layer convolution modules. The UNet network can obtain pixel-level bronchial segmentation results and calculate high-dimensional voxel features for the subsequent segment-level bronchial labeling. That is, the penultimate layer of the UNet network can generate a three-dimensional feature map as a high-dimensional convolution feature.
[0067] For lung images, point cloud feature extraction can be performed first, and point-by-point coordinate features can be generated. The point-by-point coordinate features and the high-dimensional convolution features output by the UNet network are combined as graph node features of the point voxel graph, and input into the PV-GNN network for prediction processing. The PV-GNN network can specifically include multiple network modules, and each network model can include a Mean sage-convolution (MSC) layer, a Graph Normalization (GN) layer, and a linear rectification activation function, such as a ReLU activation function.
[0068] The relative position information represented by the extracted point cloud features can help model the intrinsic bronchial topology during the classification process. At the same time, the high-dimensional voxel features can implicitly capture other types of bronchial properties such as bronchial diameter, which is a powerful supplement for bronchial classification. Using GNN to guide the learning process of UNet through convolutional features can improve the classification performance of UNet. By using UNet to classify bronchi at the pixel level, such as Figure 1 As shown in the upper part of . UNet can produce coarse bronchial predictions and voxel-level features for further segment-level classification.
[0069] In one embodiment, if Figure 2 As shown, a bronchial classification method based on a graph structure is provided, and the method is applied in Figure 1 The server in the example is used as an example to illustrate the following steps:
[0070] In step S110, lung image sample data is acquired, and a binary segmentation mask of the bronchus is extracted from the lung image sample data to obtain a bronchus mask image;
[0071] In an embodiment of the present application, a CT scan image of the lungs can be read from a medical image database, and a sample data set, a validation data set, and a test data set can be constructed. The sample data may include a large amount of lung image sample data, and lung image sample data is arbitrarily selected from the sample data set, and a bronchial binary segmentation mask is manually annotated, and a category label of each bronchial segment is annotated based on the binary bronchial mask, and then input into the bronchial classification network BCNet for prediction, and iterative training is performed based on the prediction results and the remaining lung image sample data.
[0072] The binary bronchial mask is a binary representation of the bronchial region extracted from the lung image sample data. The pixel values of the lung image sample data may be normalized, for example, from 0 to 1, so as to reduce the influence of illumination changes and noise, and a pre-trained segmentation model, such as a UNet network, may be used to perform preliminary bronchial segmentation.
[0073] In step S120, the bronchial mask image is input into a convolutional neural network for prediction to obtain a high-dimensional convolution feature and a first bronchial segmentation result;
[0074] In an embodiment of the present application, the convolutional neural network may be a UNet network, which is used to predict the category of each bronchial branch, and generate a three-dimensional feature map of each bronchial branch from the penultimate layer of UNet as the high-dimensional convolution feature. Specifically, for each bronchial fragment, the K voxels closest to the K key points can be located, and K convolution features can be extracted from the feature map according to the index of the K voxels. The voxel convolution feature of the bronchial fragment can be defined as a combination of K features, whose length is C·K, where C is the channel number of the feature map (the default is C=24). Therefore, the size of the voxel convolution feature can be 240.
[0075] In the embodiment of the present application, the specific process of extracting K convolution features from the feature map is as follows: extracting the point cloud features of the bronchial segment from the bronchial mask image, and the point cloud in each point cloud feature represents a key point in the bronchial segment. For each bronchial segment, K key points can be selected, for example, the points on the center line can be uniformly sampled, and the nearest voxel of each key point is found by calculating the position index of the key point in the bronchial mask image, and then the corresponding convolution feature is extracted from the feature map using the index of the key point, and the convolution features of the K key points are combined to obtain the feature vector of the bronchial segment, that is, the high-dimensional convolution feature.
[0076] Among them, the feature map is a high-dimensional feature map generated by the UNet network.
[0077] In an embodiment of the present application, the UNet network is a commonly used convolutional neural network structure for image segmentation tasks. It may specifically include an encoder (downsampling path) and a decoder (upsampling path), which are connected by skip connections in the middle. The encoder part consists of multiple convolutional layers and pooling layers, which gradually reduce the spatial resolution of the image while increasing the depth of the feature map. The jump connection terminology splices the feature map in the encoder in the decoder to combine the low-level and high-level feature decoder parts, where the decoder part may be composed of multiple upsampling layers and convolutional layers to gradually restore the spatial resolution of the image. The output layer may be a 1x1 convolutional layer for mapping the feature map to the required number of categories to obtain the first segmentation result. The prediction process may specifically be to first normalize the input bronchial mask image, scale the pixel values to a standard range, such as 0-1, and then perform prediction processing through the encoder and decoder to obtain the category probability of each pixel. The labeled bronchial mask image can be used as a supervisory signal, and a loss function, such as a cross entropy loss function, is used to calculate the loss between the class probability and the true label, so as to perform training based on the loss. The network weights can be adjusted by a back propagation algorithm to minimize the error between the predicted result and the true label.
[0078] In step S130, skeletonization is performed on the bronchial mask image, and point-by-point coordinate features are obtained based on the skeletonized bronchial mask image;
[0079] In the embodiment of the present application, skeletonization is a technique for converting an object in a binary image into a single-pixel-wide skeleton while retaining its topological structure. By using a preset skeletonization algorithm, such as the Zhang-Suen algorithm and the Guo-Hall algorithm, the boundary pixels of the bronchial mask image are gradually removed until only the centerline image remains, so as to achieve skeletonization of the bronchial mask image.
[0080] In an embodiment of the present application, for the skeletonized bronchial mask image, the three-dimensional coordinates of the voxels of the bronchial segment, that is, the three-dimensional coordinates of the volume pixels, can be first extracted to generate point cloud features. The bronchial mask image is a three-dimensional image, and the foreground pixel value can be 1, and the background pixel value can be 0. The extracted point cloud features, that is, the three-dimensional coordinates of the voxels are normalized and segmented, and k key points are sampled for each segment. For each bronchial segment, its point-by-point coordinate feature can be defined as a list of three-dimensional coordinates of these K points. For example, K is set to 10. Therefore, the size of the point-by-point coordinate feature is 30.
[0081] It should be noted that the point cloud feature is a set of three-dimensional points composed of voxels, i.e., volume pixels, of the bronchial segment, which can contain more spatial distribution information and density information. The point-by-point coordinate feature is usually the three-dimensional coordinates of several key points extracted from the center line of each bronchial segment, reflecting the geometric shape and structure of the bronchial segment. The point-by-point coordinate feature provides the geometric shape and structural information of the bronchial segment. Through these coordinates, the geometric characteristics of the bronchial segment, such as posture, curvature, and direction, can be described.
[0082] In step S140, the point-by-point coordinate features and the high-dimensional convolution features are combined to obtain a combined feature, and the combined feature is input into a point voxel graph neural network for prediction to obtain a second bronchus segmentation result;
[0083] In an embodiment of the present application, for the original lung sample data, a binary bronchial mask image can be extracted, and the three-dimensional coordinates of the key points can be extracted from the bronchial mask image as the point-by-point coordinate feature. The high-dimensional convolution feature is predicted by a convolutional neural network. The bronchial mask image can be skeletonized, the bronchial centerline can be extracted, and the point cloud feature can be extracted from the bronchial centerline, and a point voxel map can be constructed based on the point cloud feature. Each node of the point voxel map identifies a key point of a bronchial segment, and the edge of the point voxel map connects adjacent key points to form a graph structure. The point-by-point coordinate features and the high-dimensional convolution features are combined by feature splicing and the like to obtain a combined feature, and the combined feature is used as a node of the point voxel map, and is predicted by the point voxel map neural network to obtain a second bronchial segmentation result, which is a line segment level prediction result.
[0084] In an embodiment of the present application, the point voxel graph neural network can be specifically composed of a convolution norm block and a fully connected layer. A network module consists of a Mean sage-convolution (MSC) layer, a Graph Normalization (GN) layer, and a ReLU activation function. The MSC layer can use an average aggregation function to merge information from adjacent nodes to overcome the inductive bias. First, the combined features after the combination can be received through the input layer, and then the category of each node is predicted through the output layer after output through the network module. It should be noted that a residual connection is added after each convolution block in the network module to alleviate the problem of gradient disappearance. After each prediction, the loss between the predicted result and the true category can be calculated by a preset loss function, for example, the cross entropy loss CE and the neighborhood consistency regularization (NCR) can be used as the loss function, and the cross entropy loss CE can be used to calculate the error between the predicted result and the true category, and the neighborhood consistency regularization (NCR) is used to punish local spatial changes and encourage neighboring nodes of the same category to be closer in the potential space.
[0085] It should be noted that during the training process of the point voxel graph neural network, the pre-labeled bronchial segment categories can be used as supervisory signals, and the above-mentioned loss function can be used for iterative training. At the same time, the network weights can be adjusted through the back-propagation algorithm.
[0086] In step S150, based on the first bronchial segmentation result and the second bronchial segmentation result, the convolutional neural network and the point voxel neural network are iteratively trained, and the actual lung image to be segmented is segmented by the trained convolutional neural network.
[0087] In an embodiment of the present application, based on the first bronchial segmentation result, the loss value between the first bronchial segmentation result and the true label can be calculated by a preset loss function, such as cross entropy loss. Based on the second bronchial segmentation result, the loss value between the second bronchial segmentation result and the true label can also be calculated by a preset loss function, such as cross entropy loss and neighborhood consistency regularization. The two loss values are added to obtain a total loss value, and the convolutional neural network and the point voxel neural network are iteratively trained based on the total loss value until the preset convergence conditions are met, such as when the total loss value is greater than a preset threshold, or the number of iterations reaches a preset number, the trained convolutional neural network and the point voxel neural network can be obtained. The trained convolutional neural network can perform segmentation processing on the actual lung image to be segmented.
[0088] In an embodiment of the present application, a bronchial classification method based on a graph structure is provided, including: obtaining lung image sample data, extracting a binary segmentation mask of the bronchus from the lung image sample data to obtain a bronchial mask image; inputting the bronchial mask image into a convolutional neural network for prediction to obtain a high-dimensional convolution feature and a first bronchial segmentation result; skeletonizing the bronchial mask image, and obtaining a point-by-point coordinate feature based on the skeletonized bronchial mask image; combining the point-by-point coordinate feature and the high-dimensional convolution feature to obtain a combined feature, and inputting the combined feature into a point voxel graph neural network for prediction to obtain a second bronchial segmentation result; based on the first bronchial segmentation result and the second bronchial segmentation result, iteratively training the convolutional neural network and the point voxel neural network, and segmenting the actual lung image to be segmented through the trained convolutional neural network. In an embodiment of the present application, extracting the relative position information represented by the point cloud feature can help model the inherent bronchial topology during the classification process. At the same time, high-dimensional voxel features can implicitly capture other types of tracheal properties such as bronchial diameter, which is a powerful supplement for bronchial classification. Using point voxel graph neural networks to guide the learning process of convolutional neural networks through convolutional features can improve the classification performance of convolutional neural networks. The convolutional neural network can be used to classify bronchi at the pixel level. The convolutional neural network can generate coarse bronchial predictions and voxel-level features for further segment-level classification.
[0089] In one embodiment of the present application, skeletonizing the bronchial mask image and obtaining point-by-point coordinate features based on the skeletonized bronchial mask image include:
[0090] Refining the bronchial mask image into a centerline with a single pixel width by using a preset skeletonization algorithm to obtain a centerline image;
[0091] Extracting three-dimensional coordinates of centerline voxels from the centerline image to obtain point cloud features;
[0092] Based on the point cloud features, each three-dimensional coordinate is normalized and a demarcation point is extracted to obtain the point-by-point coordinate features.
[0093] Specifically, a binarized bronchial mask image is obtained, which is three-dimensional, in which the object is represented by white, that is, the pixel value is 1, and the background is represented by black, that is, the pixel value is 0. It can be understood that the bronchial mask image is a three-dimensional array, that is, a matrix. The boundary pixels of the object are gradually removed by a preset skeletonization algorithm, such as the Zhang-Suen algorithm and the Guo-Hall algorithm, until the centerline image remains, that is, the bronchial mask image is refined into a centerline image with a single pixel width by the skeletonization algorithm. The three-dimensional coordinates of the centerline voxels are extracted from the centerline image, and point cloud features are generated. The three-dimensional coordinates of the centerline voxels are normalized and the demarcation points are extracted to obtain point-by-point coordinate features.
[0094] In an embodiment of the present application, normalizing each three-dimensional coordinate and extracting the demarcation point to obtain the point-by-point coordinate feature includes:
[0095] A bounding box of each bronchial segment is cropped from the skeletonized bronchial mask image, wherein each bronchial segment is composed of centerline voxels within its corresponding bounding box.
[0096] normalizing the three-dimensional coordinates of each voxel in the centerline image relative to the shape of a bounding box of the corresponding bronchial segment;
[0097] The three-dimensional coordinates of several key points are extracted from each bronchial segment after normalization to obtain the point-by-point coordinate features.
[0098] Specifically, based on the skeletonized bronchial mask image, the bounding box of each bronchial segment can be cut out from the bronchial tree. Each segment consists of the centerline voxels within the bounding box. Then, the coordinates (X, Y, Z) of each voxel can be normalized to [0, 1] relative to the shape of the bounding box. The normalization process helps to describe the posture and angle of a line segment. Assuming that the length of the centerline is L, the centerline can be divided into K-1 intervals of , and collect K points from the start and end points of these K-1 sections. For each bronchial segment, its point-by-point coordinate feature can be defined as a list of 3D coordinates of these K points. K is set to 10. Therefore, the size of the point-by-point coordinate feature is 30.
[0099] In one embodiment of the present application, the skeletonization of the bronchial mask image includes:
[0100] Step a: normalizing the bronchial mask image;
[0101] Step b: determining whether each pixel in the normalized bronchial mask image meets the deletion condition;
[0102] Step c: If yes, then the pixels satisfying the deletion condition are deleted to obtain a new bronchial mask image;
[0103] Step d: Repeat the above steps bc until no pixels can be deleted, and obtain the centerline image.
[0104] The preset skeleton algorithm may include the Zhang-Suen algorithm and the Guo-Hall algorithm. Taking the Zhang-Suen algorithm as an example, the steps of skeletonizing the bronchial mask image are as follows: a marking array may be constructed to mark the pixels to be deleted, and then each pixel in the bronchial mask image is traversed. For each pixel (X, Y), it is determined whether it meets the deletion condition. The deletion condition may include that the pixel (x, y) is a foreground pixel (value is 1); there are 2 to 6 foreground pixels in its 8-neighborhood; the number of transitions from 0 to 1 is 1; at least one neighbor pixel is a background pixel and satisfies one or any combination of the specific patterns. If the above conditions are met, the pixel (X, Y) is marked for deletion, and then all pixels marked for deletion are set to 0, that is, as background pixels. Then, the above steps may be repeated again, but a different deletion condition from the last time is used to delete pixels again until no pixels can be deleted, and a single-pixel-wide centerline image may be obtained.
[0105] In one embodiment of the present application, the point-by-point coordinate features and the high-dimensional convolution features are combined to obtain combined features, and the combined features are input into a point voxel graph neural network for prediction, including:
[0106] Combining the point-by-point coordinate features and the high-dimensional convolution features to obtain a combined feature;
[0107] The combined features are used as node features of the point voxel graph and are input into the point voxel graph neural network for prediction.
[0108] Specifically, the point-by-point coordinate features and high-dimensional convolution features can be spliced and combined through feature splicing and other methods to obtain richer combined features, which are used as node features of the point voxel map and input into the point voxel map neural network for bronchial segment classification.
[0109] In one embodiment of the present application, before skeletonizing the bronchial mask image, the process includes:
[0110] Performing connected domain analysis on the bronchial mask image to determine all independent connected regions;
[0111] Calculate the number of pixels in each connected area and determine the largest connected area;
[0112] The bronchial mask image is updated according to the maximum connected area, so as to perform skeletonization processing based on the updated bronchial mask image.
[0113] Specifically, for the binary bronchial mask image, Figure 3 As shown in Figure (a), the mask is updated to its maximum connected area and defined as a point voxel map, and mixed voxels and point-level features can be used as node embedding. The binary mask is subjected to a connected domain analysis using a connected domain labeling algorithm (such as Flood Fill or Connected Components Labeling) to mark all independent connected regions. The number of pixels in each connected region is calculated, the size of each connected region is recorded, and the region with the largest pixel is found from all connected regions. A new binary bronchial mask is created, and only the pixels of the maximum connected region are retained, and the pixels of the region are set to 0. Updating the mask to its maximum connected area ensures that only the largest connected part of the bronchial tree is retained, removing noise and isolated small areas.
[0114] In one embodiment of the present application, after the skeletonization of the bronchial mask image, the method further includes:
[0115] Extract the demarcation point based on the skeletonized bronchial mask image;
[0116] Traversing the skeletonized bronchial mask image, and dividing the bronchial mask image into bronchial segments based on the types of the traversed demarcation points;
[0117] A point voxel map is constructed, and each of the bronchial segments is used as a node of the point voxel map.
[0118] See also Figure 3 , the bronchial mask image can be skeletonized by a preset skeletonization algorithm to obtain a centerline image, such as Figure 3 As shown in Figure (b), then for each voxel on the centerline image, for example, the number of foreground voxels in 26 adjacent neighborhoods, N can be defined as endpoint N = 1, edge point N = 2, and segmentation point N> = 3, then Figure 3 As shown in (c), the above endpoints, edge points and split points divide the branch into segments, such as Figure 3As shown in (d), an empty segment list can be constructed to store the pixels of each segment. The points of the center line can include endpoints, edge points, and segmentation points. Each pixel in the skeletonized image is classified. Endpoints: There is only one foreground pixel in the 8-neighborhood, edge points: There are two foreground pixels in the 8-neighborhood, and segmentation points: There are three or more foreground pixels in the 8-neighborhood. Starting from the first endpoint, traverse along the center line until another endpoint or segmentation point. Each time an endpoint or segmentation point is traversed, a new segment is generated and added to the segment list. For each segmentation point, a recursive traversal method can be used to traverse each branch to generate more segments. It can be understood that each bronchial segment can correspond to a node in the point voxel map, and the edge of the map is defined by the connectivity between two line segments, and the two line segments can be divided by a segmentation point.
[0119] In one embodiment of the present application, the bronchial mask image is input into a convolutional neural network for prediction to obtain high-dimensional convolution features, including:
[0120] Extracting the three-dimensional coordinates of each bronchial segment from the skeletonized bronchial mask image to obtain a point cloud feature of each bronchial segment, wherein each feature point in the point cloud feature is used to represent a key point in the corresponding bronchial segment;
[0121] For each bronchial segment, select preset key points;
[0122] For each keypoint, determine the voxel closest to the keypoint;
[0123] Extracting corresponding convolution features from the skeletonized bronchial mask image through the index of the key points;
[0124] The convolution features corresponding to each key point are combined to obtain the high-dimensional convolution features.
[0125] Specifically, the category of each bronchial branch can be predicted by the UNet network, and a three-dimensional feature map can be generated from the penultimate layer of the UNet network. For each bronchial fragment, the voxels closest to the K key points can be located, and K convolutional features can be extracted from the feature map according to the index of the voxel. Then the high-dimensional convolutional feature of the bronchial fragment can be defined as a K feature combination, whose length can be C·K, where C is the channel number of the feature map (the default is C=24). Therefore, the size of the voxel convolution feature is 240.
[0126] The method of extracting k convolutional features from the feature map can be specifically to generate a point cloud feature from the skeletonized bronchial mask, and each point in the point cloud feature can represent a key point in the bronchial segment. Then, K key points can be determined for each bronchial segment, for example, by uniformly sampling the points in each bronchial segment, and the nearest voxel of each key point can be found by calculating the position index of the key point in the feature map. Then, the corresponding convolutional feature is extracted from the feature map using the key point index. It should be noted that the feature map is a high-dimensional feature map generated by the UNet network.
[0127] In an embodiment of the present application, the iterative training of the convolutional neural network and the point voxel neural network based on the first bronchial segmentation result and the second bronchial segmentation result includes:
[0128] Calculating a first loss value based on the first bronchus segmentation result and the true label;
[0129] Calculating a second loss value based on the second bronchus segmentation result and the true label;
[0130] Calculating a total loss value based on the first loss value and the second loss value;
[0131] The convolutional neural network and the point voxel neural network are iteratively trained based on the total loss value until the total loss value is less than a preset loss threshold and the training is completed.
[0132] Specifically, for the point voxel neural network, since adjacent branches in the bronchial tree may belong to the same category of prior topology, a new neighborhood consistency regularization (NCR) is set to penalize local spatial changes so that adjacent nodes of the same category are closer in the latent space to satisfy the prior information that adjacent graph nodes often belong to the same category. For example, Y = y 1 ,y 2 ,…,y N is the set of one-hot vectors containing the ground truth of each branch, Z = z 1 ,z 2 ,…,z N is a set of one-hot vectors predicted by the model of each branch. Then the second loss value can be calculated by the NCR loss formula, which can be:
[0133]
[0134] Where V i is the set of neighbors of the i-th layer node, j represents the j-th node in the set, z irepresents the logit vector of the i-th node output by the last fully connected layer, and I(·) is an indicator function that returns 1 when the condition is met and 0 otherwise. M and N are the number of edges and nodes in the graph, respectively. Let λ be a scalar to balance the weights of regularization and CE loss (λ can be set to 1 based on experience).
[0135] Then, the second loss value may include the above NCR and a cross entropy loss, which can be specifically expressed as:
[0136] L pvgnn =L CE +λL NCR
[0137] For convolutional neural networks, a loss value L can be calculated through cross entropy loss function, regularization loss function, etc. unet .
[0138] The training is performed in an end-to-end manner using the UNet network and the PV-GNN network, because the gradient of the GNN branch can be sent back to the UNet branch through the convolutional voxel feature extraction operation. Therefore, the final total loss L cls It can be composed of two parts, which can be expressed as:
[0139] L cls =L unet +L pvgnn
[0140] Among them, L unet is the first loss value, L pvgnn is the second loss value.
[0141] It should be noted that in order to better integrate the information of the point cloud, a deep GNN is constructed. Since the gradient of GNN is at risk of vanishing as the depth increases, GN is introduced to move and scale the feature values, which makes the graph neural network converge faster. Except for the first block, each block is followed by an element addition as a residual connection. Let H k represents the output of the kth block, σ represents the ReLU operation, and MSC represents the Mean sage-convolution (MSC) layer of the GNN. Then H k It can be calculated by the following formula:
[0142] H k =σ(GN(MSC(H k-1 ))+H k-1 .
[0143] In an embodiment of the present application, in an embodiment of the present application, the relative position information represented by the extracted point cloud features can help model the inherent bronchial topology during the classification process. At the same time, high-dimensional voxel features can implicitly capture other types of tracheal properties such as bronchial diameter, which is a powerful supplement to bronchial classification. The classification performance of the convolutional neural network can be improved by using a point voxel graph neural network to guide the learning process of the convolutional neural network through convolutional features. The bronchi are classified at the pixel level using a convolutional neural network. The convolutional neural network can generate coarse bronchial predictions and voxel-level features for further segment-level classification. Moreover, the graph neural network uses both point cloud features and high-dimensional convolutional features as node features. Compared with using only point cloud features or high-dimensional convolutional features, the joint use of features can greatly improve accuracy. Multi-task learning is performed together with the first convolutional neural network, thereby increasing the structural information of the feature extractor and image features shared by the two branches, and in the reasoning process, only the convolutional neural network is run for reasoning, which does not bring additional computational complexity.
[0144] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.
[0145] In one embodiment, a bronchial classification device based on a graph structure is provided, and the bronchial classification device based on a graph structure corresponds one-to-one to the bronchial classification method based on a graph structure in the above embodiment. Figure 4 As shown, the bronchial classification device based on the graph structure includes a bronchial mask image acquisition unit 10, a first prediction unit 20, a point-by-point coordinate feature acquisition unit 30, a second prediction unit 40 and a training unit 50. Each functional module is described in detail as follows:
[0146] A bronchial mask image acquisition unit 10 is used to acquire lung image sample data, and extract a binary segmentation mask of the bronchus from the lung image sample data to obtain a bronchial mask image;
[0147] A first prediction unit 20, configured to input the bronchial mask image into a convolutional neural network for prediction, and obtain a high-dimensional convolution feature and a first bronchial segmentation result;
[0148] A point-by-point coordinate feature acquisition unit 30, configured to perform skeletonization processing on the bronchial mask image, and obtain point-by-point coordinate features based on the skeletonized bronchial mask image;
[0149] A second prediction unit 40 is used to combine the point-by-point coordinate features and the high-dimensional convolution features to obtain a combined feature, and input the combined feature into a point voxel graph neural network for prediction to obtain a second bronchus segmentation result;
[0150] The training unit 50 is used to iteratively train the convolutional neural network and the point voxel neural network based on the first bronchial segmentation result and the second bronchial segmentation result, and perform segmentation processing on the actual lung image to be segmented through the trained convolutional neural network.
[0151] In one embodiment of the present application, the point-by-point coordinate feature acquisition unit 30 is further used to:
[0152] Refining the bronchial mask image into a centerline with a single pixel width by using a preset skeletonization algorithm to obtain a centerline image;
[0153] Extracting three-dimensional coordinates of centerline voxels from the centerline image to obtain point cloud features;
[0154] Based on the point cloud features, each three-dimensional coordinate is normalized and a demarcation point is extracted to obtain the point-by-point coordinate features.
[0155] In one embodiment of the present application, the point-by-point coordinate feature acquisition unit 30 is further used to:
[0156] A bounding box of each bronchial segment is cropped from the skeletonized bronchial mask image, wherein each bronchial segment is composed of centerline voxels within its corresponding bounding box.
[0157] normalizing the three-dimensional coordinates of each voxel in the centerline image relative to the shape of a bounding box of the corresponding bronchial segment;
[0158] The three-dimensional coordinates of several key points are extracted from each bronchial segment after normalization to obtain the point-by-point coordinate features.
[0159] In one embodiment of the present application, the point-by-point coordinate feature acquisition unit 30 is further used to:
[0160] Step a: normalizing the bronchial mask image;
[0161] Step b: determining whether each pixel in the normalized bronchial mask image meets the deletion condition;
[0162] Step c: If yes, then the pixels satisfying the deletion condition are deleted to obtain a new bronchial mask image;
[0163] Step d: Repeat the above steps bc until no pixels can be deleted, and obtain the centerline image.
[0164] In an embodiment of the present application, the second prediction unit 40 is further configured to:
[0165] Combining the point-by-point coordinate features and the high-dimensional convolution features to obtain a combined feature;
[0166] The combined features are used as node features of the point voxel graph and are input into the point voxel graph neural network for prediction.
[0167] In one embodiment of the present application, the device further includes: a point voxel map construction unit, which is used to:
[0168] Extract the demarcation point based on the skeletonized bronchial mask image;
[0169] Traversing the skeletonized bronchial mask image, and dividing the bronchial mask image into bronchial segments based on the types of the traversed demarcation points;
[0170] A point voxel map is constructed, and each bronchial segment is regarded as a node of the point voxel map.
[0171] In one embodiment of the present application, the device further includes: a bronchial mask image updating unit, configured to:
[0172] Performing connected domain analysis on the bronchial mask image to determine all independent connected regions;
[0173] Calculate the number of pixels in each connected area and determine the largest connected area;
[0174] The bronchial mask image is updated according to the maximum connected area, so as to perform skeletonization processing based on the updated bronchial mask image.
[0175] In an embodiment of the present application, the first prediction unit 20 is further configured to:
[0176] Extracting the three-dimensional coordinates of each bronchial segment from the skeletonized bronchial mask image to obtain a point cloud feature of each bronchial segment, wherein each feature point in the point cloud feature is used to represent a key point in the corresponding bronchial segment;
[0177] For each bronchial segment, select preset key points;
[0178] For each keypoint, determine the voxel closest to the keypoint;
[0179] Extracting corresponding convolution features from the skeletonized bronchial mask image through the index of the key points;
[0180] The convolution features corresponding to each key point are combined to obtain the high-dimensional convolution features.
[0181] In one embodiment of the present application, the training unit 50 is further configured to:
[0182] Calculating a first loss value based on the first bronchus segmentation result and the true label;
[0183] Calculating a second loss value based on the second bronchus segmentation result and the true label;
[0184] Calculating a total loss value based on the first loss value and the second loss value;
[0185] The convolutional neural network and the point voxel neural network are iteratively trained based on the total loss value until the total loss value is less than a preset loss threshold and the training is completed.
[0186] In an embodiment of the present application, in an embodiment of the present application, the relative position information represented by the extracted point cloud features can help model the inherent bronchial topology during the classification process. At the same time, high-dimensional voxel features can implicitly capture other types of tracheal properties such as bronchial diameter, which is a powerful supplement to bronchial classification. The classification performance of the convolutional neural network can be improved by using a point voxel graph neural network to guide the learning process of the convolutional neural network through convolutional features. The bronchi are classified at the pixel level using a convolutional neural network. The convolutional neural network can generate coarse bronchial predictions and voxel-level features for further segment-level classification. Moreover, the graph neural network uses both point cloud features and high-dimensional convolutional features as node features. Compared with using only point cloud features or high-dimensional convolutional features, the joint use of features can greatly improve accuracy. Multi-task learning is performed together with the first convolutional neural network, thereby increasing the structural information of the feature extractor and image features shared by the two branches, and in the reasoning process, only the convolutional neural network is run for reasoning, which does not bring additional computational complexity.
[0187] For the specific definition of the bronchial classification device based on the graph structure, please refer to the definition of the bronchial classification method based on the graph structure above, which will not be repeated here. Each module in the above-mentioned bronchial classification device based on the graph structure can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0188] In one embodiment, a computer device is provided. The computer device may be a terminal device, and its internal structure diagram may be as follows: Figure 5As shown. The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a readable storage medium. The readable storage medium stores computer-readable instructions. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer-readable instructions are executed by the processor, a bronchial classification method based on a graph structure is implemented. The readable storage medium provided in this embodiment includes a non-volatile readable storage medium and a volatile readable storage medium.
[0189] In an embodiment of the present application, a computer device is provided, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor. When the processor executes the computer-readable instructions, the steps of the graph-based bronchial classification method as described above are implemented.
[0190] In an embodiment of the application, a readable storage medium is provided, wherein the readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the graph-based bronchial classification method as described above are implemented.
[0191] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through computer-readable instructions, and the computer-readable instructions can be stored in a non-volatile readable storage medium or a volatile readable storage medium. When the computer-readable instructions are executed, they may include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0192] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0193] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.
Claims
1. A bronchial classification method based on graph structure, characterized in that: The method comprises: Acquire lung image sample data, and extract a binary segmentation mask of the bronchus from the lung image sample data to obtain a bronchus mask image; Inputting the bronchial mask image into a convolutional neural network for prediction to obtain high-dimensional convolution features and a first bronchial segmentation result; Skeletonize the bronchial mask image, and obtain point-by-point coordinate features based on the skeletonized bronchial mask image; Combining the point-by-point coordinate features and the high-dimensional convolution features to obtain a combined feature, and inputting the combined feature into a point voxel graph neural network for prediction to obtain a second bronchus segmentation result; Based on the first bronchial segmentation result and the second bronchial segmentation result, the convolutional neural network and the point voxel neural network are iteratively trained, and the actual lung image to be segmented is segmented by the trained convolutional neural network.
2. The bronchial classification method based on graph structure according to claim 1, characterized in that: The skeletonization process is performed on the bronchial mask image, and point-by-point coordinate features are obtained based on the skeletonized bronchial mask image, including: Refining the bronchial mask image into a centerline with a single pixel width by using a preset skeletonization algorithm to obtain a centerline image; Extracting three-dimensional coordinates of centerline voxels from the centerline image to obtain point cloud features; Based on the point cloud features, each three-dimensional coordinate is normalized and a demarcation point is extracted to obtain the point-by-point coordinate features.
3. The bronchial classification method based on graph structure according to claim 2, characterized in that: The step of normalizing each three-dimensional coordinate and extracting the demarcation points to obtain the point-by-point coordinate features includes: A bounding box of each bronchial segment is cropped from the skeletonized bronchial mask image, wherein each bronchial segment is composed of centerline voxels within its corresponding bounding box. normalizing the three-dimensional coordinates of each voxel in the centerline image relative to the shape of a bounding box of the corresponding bronchial segment; The three-dimensional coordinates of several key points are extracted from each bronchial segment after normalization to obtain the point-by-point coordinate features.
4. The bronchial classification method based on graph structure according to claim 1, characterized in that: The skeletonization process of the bronchial mask image includes: Step a: normalizing the bronchial mask image; Step b: determining whether each pixel in the normalized bronchial mask image meets the deletion condition; Step c: If yes, then the pixels satisfying the deletion condition are deleted to obtain a new bronchial mask image; Step d: Repeat the above steps bc until there are no pixels to be deleted, and then refine the bronchial mask image into a center line with a single pixel width to obtain the center line image.
5. The bronchial classification method based on graph structure according to claim 1, characterized in that: The step of combining the point-by-point coordinate features and the high-dimensional convolution features to obtain a combined feature, and inputting the combined feature into a point voxel graph neural network for prediction includes: Combining the point-by-point coordinate feature and the high-dimensional convolution feature to obtain a combined feature; The combined features are used as node features of the point voxel graph and are input into the point voxel graph neural network for prediction.
6. The bronchial classification method based on graph structure according to claim 1, characterized in that: After the skeletonization process is performed on the bronchial mask image, the method further includes: Extract the demarcation point based on the skeletonized bronchial mask image; Traversing the skeletonized bronchial mask image, and dividing the bronchial mask image into bronchial segments based on the types of the traversed demarcation points; A point voxel map is constructed, and each of the bronchial segments is used as a node of the point voxel map.
7. The bronchial classification method based on graph structure according to claim 1, characterized in that: Before skeletonizing the bronchial mask image, the method includes: Performing connected domain analysis on the bronchial mask image to determine all independent connected regions; Calculate the number of pixels in each connected area and determine the largest connected area; The bronchial mask image is updated according to the maximum connected area, so as to perform skeletonization processing based on the updated bronchial mask image.
8. The bronchial classification method based on graph structure according to claim 1, characterized in that: The bronchial mask image is input into a convolutional neural network for prediction to obtain high-dimensional convolution features, including: Extracting the three-dimensional coordinates of each bronchial segment from the skeletonized bronchial mask image to obtain a point cloud feature of each bronchial segment, wherein each feature point in the point cloud feature is used to represent a key point in the corresponding bronchial segment; For each bronchial segment, select preset key points; For each keypoint, determine the voxel closest to the keypoint; Extracting corresponding convolution features from the skeletonized bronchial mask image through the index of the key points; The convolution features corresponding to each key point are combined to obtain the high-dimensional convolution features.
9. The bronchial classification method based on graph structure according to claim 1, characterized in that: The iterative training of the convolutional neural network and the point voxel neural network based on the first bronchial segmentation result and the second bronchial segmentation result includes: Calculating a first loss value based on the first bronchus segmentation result and the true label; Calculating a second loss value based on the second bronchus segmentation result and the true label; Calculating a total loss value based on the first loss value and the second loss value; The convolutional neural network and the point voxel neural network are iteratively trained based on the total loss value until the total loss value is less than a preset loss threshold and the training is completed.
10. A bronchial classification device based on a graph structure, characterized in that: The device comprises: A bronchial mask image acquisition unit, used for acquiring lung image sample data, and extracting a binary segmentation mask of the bronchus from the lung image sample data to obtain a bronchial mask image; A first prediction unit, used for inputting the bronchial mask image into a convolutional neural network for prediction, to obtain a high-dimensional convolution feature and a first bronchial segmentation result; A point-by-point coordinate feature acquisition unit, used for skeletonizing the bronchial mask image, and obtaining point-by-point coordinate features based on the bronchial mask image after the skeletonization; a second prediction unit, configured to combine the point-by-point coordinate features and the high-dimensional convolution features to obtain a combined feature, and input the combined feature into a point voxel graph neural network for prediction to obtain a second bronchus segmentation result; A training unit is used to iteratively train the convolutional neural network and the point voxel neural network based on the first bronchial segmentation result and the second bronchial segmentation result, and perform segmentation processing on the actual lung image to be segmented through the trained convolutional neural network.
Citation Information
Patent Citations
Pulmonary nodule recognition and segmentation method and system based on deep learning
CN112581436A
Deep learning network for the analysis of body tissue composition on body-torso-wide CT images
US20230129957A1
Cited By
Method for training blood vessel segmentation model and related product
CN121074543A
