Method for automatically segmenting pulmonary artery tree and tracheal tree on CTPA image
By constructing a three-dimensional Gaussian distribution thermal map and network structure optimization on CTPA images, the problem of difficult to segment the tiny pulmonary vascular and tracheal structure in the prior art is solved, and higher segmentation accuracy and topological integrity are achieved.
Patent Information
- Application Number
- CN202411832610.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-13
- Publication Date
- 2025-01-14
AI Technical Summary
The prior art is difficult to effectively segment the tiny pulmonary vascular and tracheal structures on CTPA images, especially the terminal structures, and the segmentation method that depends on the center line lacks flexibility and is difficult to ensure the integrity of the topological structure.
By preprocessing the CTPA image data, a three-dimensional Gaussian distribution heat map is constructed, and combined with the initial network and segmentation network, the floating point method and attention module are used for training, and the segmentation of the centerline and tubular structure is optimized to ensure the accuracy of the segmentation results. and topological integrity.
The segmentation accuracy and recognition accuracy of pulmonary artery tree and tracheal tree are improved, the segmentation ability of small branches is enhanced, the missed and missed divisions are reduced, and the topological integrity of the segmentation results is ensured.
Smart Images
Figure CN119313910A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to a method for automatically segmenting a pulmonary artery tree and a tracheal tree on a CTPA image. Background Art
[0002] With the rapid development of deep learning, medical image processing based on deep convolutional neural networks has become a research hotspot. The network structure adopts the UNet network structure or its variants, such as changing the downsampling pooling operation into a convolution operation, changing the number of layers or receptive fields, changing the convolution kernel size, etc. Most of the traditional general segmentation algorithms used for trachea or vascular tree segmentation are designed for tubular structure features, or the difference between arteries and veins, and targeted feature filters are designed separately, and uniformity and robustness are improved on this basis.
[0003] However, the pulmonary vascular / tracheal segmentation must take into account the small structure of the pulmonary vascular / tracheal structure, especially the terminal pulmonary vascular / tracheal structure, which is difficult to extract and requires the network to have a strong ability to analyze small features. At the same time, the topological integrity of the entire vascular / tracheal tree needs to be considered, which can be solved by introducing the centerline. The current method of directly expanding the centerline to obtain the segmentation result is completely dependent on the accuracy of the centerline itself and lacks flexibility. Therefore, it is very necessary to design a method for automatically segmenting the pulmonary artery tree and tracheal tree on CTPA images. Summary of the invention
[0004] The purpose of the present invention is to provide a method for automatically segmenting the pulmonary artery tree and the tracheal tree on CTPA images, and to optimize the vascular segmentation technology through the Gaussian distribution heat map of the center line to improve the recognition accuracy and segmentation precision of the model.
[0005] To achieve the above object, the present invention provides the following solutions: A method for automatically segmenting a pulmonary artery tree and a tracheal tree on a CTPA image comprises the following steps: Performing data preprocessing operation on the pulmonary artery angiography image to obtain a preprocessed image; The preprocessed image is refined to obtain the centerline and radius of the tubular structure; A three-dimensional Gaussian distribution is constructed according to the center line and the radius of the tubular structure, and a Gaussian distribution heat map is obtained according to the three-dimensional Gaussian distribution; Inputting the preprocessed image into the first initial network to obtain an initial prediction result, and training the first initial network through the initial prediction result and the Gaussian distribution heat map to obtain a first segmentation network; Inputting the preprocessed image into the first segmentation network to obtain a first segmentation result; The first segmentation result and the preprocessed image are input into the second segmentation network to obtain the final segmentation result.
[0006] Optionally, the preprocessing operations include: manual labeling, tubular structure enhancement and segment selection.
[0007] Optionally, a three-dimensional Gaussian distribution is constructed according to the center line and the radius of the tubular structure, and a Gaussian distribution heat map is obtained according to the three-dimensional Gaussian distribution, including: The probability density value is obtained according to the distance from different points in the preprocessed image to the center line; Add the probability density values to get the Gaussian distribution thermal value; The Gaussian distribution thermodynamic map is obtained according to the maximum value of the Gaussian distribution thermodynamic value.
[0008] Optionally, the encoder part of the first initial network consists of 2 connected residual modules and 1 convolutional downsampling; the decoder part of the first initial network adopts a convolutional neural network structure, and the connection method of the convolutional neural network structure is inter-layer skip connection and intra-layer residual connection.
[0009] Optionally, the preprocessed image is input into the first initial network to obtain an initial prediction result, and the first initial network is trained by the initial prediction result and the Gaussian distribution heat map to obtain a first segmentation network, specifically: based on the floating point method, the prediction loss is calculated by the initial prediction result and the Gaussian distribution heat map, and the first initial network is trained by the prediction loss to obtain the first segmentation network; the calculation formula of the floating point method is: ; Among them, FD is the prediction loss, C is the prediction result, and D is the Gaussian distribution heat map. is the number of elements in the first set, is the number of elements in the second set.
[0010] Optionally, the second segmentation network has a built-in dual sliding window attention structure consisting of a first attention module and a second attention module; the query and value of the first attention module are both feature maps of the second segmentation network, and the key of the first attention module is the first segmentation result; the second attention module is a self-attention module.
[0011] The present invention discloses the following technical effects: the method provided by the present invention for automatically segmenting the pulmonary artery tree and the tracheal tree on a CTPA image comprises: performing a data preprocessing operation on a pulmonary artery angiography image to obtain a preprocessed image; refining the preprocessed image to obtain a center line and a tubular structure radius; constructing a three-dimensional Gaussian distribution according to the center line and the tubular structure radius, and obtaining a Gaussian distribution heat map according to the three-dimensional Gaussian distribution; inputting the preprocessed image into a first initial network to obtain an initial prediction result, and training the first initial network through the initial prediction result and the Gaussian distribution heat map to obtain a first segmentation network; inputting the preprocessed image into the first segmentation network to obtain a first segmentation result; inputting the first segmentation result and the preprocessed image into a second segmentation network to obtain a final segmentation result. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0013] Figure 1 A flow chart of a method for segmenting a pulmonary artery tree and a tracheal tree according to an embodiment of the present invention; Figure 2 The figure is a flowchart of the overall network structure of an embodiment of the present invention. DETAILED DESCRIPTION
[0014] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.
[0015] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without making any creative work shall fall within the scope of protection of the present invention.
[0016] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and understandable, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.
[0017] like Figure 1 As shown, an embodiment of the present invention provides a method for automatically segmenting a pulmonary artery tree and a tracheal tree on a CTPA image, comprising the following steps: Step 100: performing data preprocessing on the CTPA image to obtain a preprocessed image; Specifically, the preprocessing operations include: manual labeling, tubular structure enhancement and segment selection.
[0018] More specifically, manual annotation involves the doctor manually segmenting the pulmonary artery and trachea in the pulmonary angiography image to obtain vascular labels. The 3DSlicer plug-in Vesselness Filtering is used to perform Frangi filtering on the pulmonary angiography image to enhance the tubular structure. A segment of size 96*96*96 is randomly selected from the pulmonary angiography image, and the proportion of foreground voxels in the selected segment is ensured to be greater than 0.015 to achieve segment selection.
[0019] Step 200: Refine the preprocessed image to obtain the center line and the radius of the tubular structure; Specifically, the preprocessed image is processed using a thinning method to obtain the centerline and the radius of the tubular structure.
[0020] Step 300: constructing a three-dimensional Gaussian distribution according to the center line and the radius of the tubular structure, and obtaining a Gaussian distribution heat map according to the three-dimensional Gaussian distribution; Specifically, for each point on the center line, any point is selected as the target point, with the target point as the center point and 3 times the radius of the tubular structure as the standard deviation to construct a three-dimensional Gaussian distribution. For each point in the vascular label, a probability density value can be obtained on the Gaussian distribution centered on each point on the center line according to the different distances of each point from each point on the center line. These probability density values are added together to obtain the Gaussian distribution thermal value of the point on the vascular label centered on the center line. After the above calculations are performed on each point on the vascular label, the maximum value of the Gaussian distribution thermal value is obtained, and the Gaussian distribution thermal map is obtained according to the decrease in the value of the maximum value of the Gaussian distribution thermal value in all directions away from the center line, and the Gaussian distribution thermal map is normalized to the range of 0-1.
[0021] Step 400: inputting the preprocessed image into the first initial network to obtain an initial prediction result, and training the first initial network through the initial prediction result and the Gaussian distribution heat map to obtain a first segmentation network; Specific as Figure 2 As shown in the figure, the first initial network is an encoder-decoder structure network. The encoder part of the network consists of 2 connected residual modules and 1 convolution downsampling, where the convolution downsampling includes a 3*3 convolution kernel and a downsampling operation. The decoder part adopts a convolutional neural network (CNN) structure, and the internal connection mode of the convolutional neural network structure is inter-layer skip connection and intra-layer residual connection.
[0022] Specifically, the preprocessed image is input into the first initial network to obtain the initial prediction result, and the first initial network is trained by the initial prediction result and the Gaussian distribution heat map. The specific method for obtaining the first segmentation network is: based on the floating point method, the prediction loss is calculated by the initial prediction result and the Gaussian distribution heat map, and the first initial network is trained by the prediction loss to obtain the first segmentation network. The first segmentation network is named the centerline segmentation network.
[0023] More specifically, during the training of the first initial network, a Gaussian distribution heat map is used for supervision, and the traditional dice loss function is replaced by the floating point method of this embodiment. The calculation formula of the floating point method is: ; Among them, FD is the prediction loss, C is the prediction result, and D is the Gaussian distribution heat map. is the number of elements in the first set, is the number of elements in the second set.
[0024] It should be noted that the traditional dice loss function expression is: ; Where A and B are two sets, |A∩B| represents the number of elements in the intersection of sets A and B, and |A| and |B| represent the number of elements in sets A and B respectively. In the process of image segmentation, set A represents the segmentation result predicted by the model, and set B represents the segmentation result of the blood vessel label.
[0025] It is understandable that because the segmentation target is the center line of a single pixel, if the traditional dice loss function is still used to calculate the prediction loss, a small segmentation deviation will lead to a large prediction loss result, making it difficult for the network model to converge. In this embodiment, in order to reduce the large loss caused by a small deviation near the center line when segmenting the center line, the traditional dice loss function is replaced by a floating point method, and the prediction result and the Gaussian distribution heat map (floating point label) are directly multiplied to replace |A∩B| in the traditional dice loss function, thereby reducing the large loss caused by a small segmentation error to the network, and solving the segmentation problem of the center line of a single pixel.
[0026] Step 500: input the preprocessed image into a first segmentation network to obtain a first segmentation result; Step 600: Input the first segmentation result and the preprocessed image into the second segmentation network to obtain the final segmentation result.
[0027] Specifically, the second initial network is also an encoder-decoder structure network, and the network is named as a tubular structure segmentation network. The encoder part of the network adopts a parallel dual sliding window attention structure. The query and value of the first attention module of the attention structure are the feature maps of each layer of the second segmentation network itself, and the key of the first attention module is the feature map of the same level and size in the centerline segmentation network. Among them, the input of the first layer of the network comes from the output of the centerline segmentation network and the preprocessed image. The second attention module is a self-attention module.
[0028] It should be noted that the dual sliding window attention structure selectively transfers the centerline topological connection information from the centerline segmentation network to the tubular structure segmentation network, making the segmentation results of the tubular structure of the pulmonary artery angiography image more coherent. At the same time, the centerline segmentation network pays equal attention to different branches of blood vessels, and can transfer better attention to small blood vessel branches to the tubular structure segmentation network, thereby improving the segmentation accuracy of small branches, reducing the cases of misclassification and omission, and ensuring the topological structure integrity of the segmentation results.
[0029] Furthermore, in this embodiment, trainable weights are assigned to the inter-layer skip connection structure, the intra-layer residual connection structure, and the dual sliding window attention structure, and weighted addition is performed to achieve self-learning of the overall network structure, thereby increasing the flexibility of the overall network structure and improving the segmentation effect.
[0030] Furthermore, this embodiment uses a training strategy of error correction while training: after 500 rounds of training, every 100 rounds of training are repeated, a prediction is made on the training data, the prediction result and the centerline segmentation result are intersected, and an under-segmentation feature map is generated and saved. The segmentation feature map is loaded using a data loader as a weight and training is restarted. This training strategy makes full use of the known centerline information, dynamically finds the undiscovered tubular structure segmentation missing information, and corrects and supplements it in the subsequent training process.
[0031] The beneficial effects of the present invention are as follows: 1) The deep learning method is used as the main method for blood vessel segmentation, and the method of segmenting the centerline structure is used as an auxiliary method to ensure the integrity of the blood vessel topology structure with the strong connectivity of the centerline; 2) The single-pixel centerline of the centerline segmentation label is replaced with a Gaussian distribution heat map, which improves the quality of centerline segmentation; 3) The floating point method is used for model training, which improves the constraint effect of the center line on the model.
[0032] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0033] The present invention uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only used to help understand the method and core ideas of the present invention. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.
Claims
1. A method for automatically segmenting the pulmonary artery tree and tracheal tree on CTPA images, characterized in that: The steps include: Performing data preprocessing operation on the pulmonary artery angiography image to obtain a preprocessed image; Refining the preprocessed image to obtain a center line and a radius of the tubular structure; constructing a three-dimensional Gaussian distribution according to the center line and the radius of the tubular structure, and obtaining a Gaussian distribution heat map according to the three-dimensional Gaussian distribution; Inputting the preprocessed image into a first initial network to obtain an initial prediction result, and training the first initial network by using the initial prediction result and the Gaussian distribution heat map to obtain a first segmentation network; Inputting the preprocessed image into the first segmentation network to obtain a first segmentation result; The first segmentation result and the preprocessed image are input into a second segmentation network to obtain a final segmentation result.
2. The method for automatically segmenting the pulmonary artery tree and the tracheal tree on CTPA images according to claim 1, characterized in that: The preprocessing operation includes: manual labeling, tubular structure enhancement and segment selection.
3. The method for automatically segmenting the pulmonary artery tree and the tracheal tree on CTPA images according to claim 1, characterized in that: Constructing a three-dimensional Gaussian distribution according to the center line and the radius of the tubular structure, and obtaining a Gaussian distribution heat map according to the three-dimensional Gaussian distribution, including: Obtaining probability density values according to the distances from different points in the preprocessed image to the center line; Adding the probability density values to obtain a Gaussian distribution thermal value; The Gaussian distribution thermodynamic map is obtained according to the maximum value of the Gaussian distribution thermodynamic value.
4. The method for automatically segmenting the pulmonary artery tree and the tracheal tree on CTPA images according to claim 1, characterized in that: The encoder part of the first initial network consists of two connected residual modules and one convolutional downsampling; the decoder part of the first initial network adopts a convolutional neural network structure, and the connection method of the convolutional neural network structure is inter-layer skip connection and intra-layer residual connection.
5. The method for automatically segmenting the pulmonary artery tree and the tracheal tree on CTPA images according to claim 1, characterized in that: The preprocessed image is input into the first initial network to obtain an initial prediction result, and the first initial network is trained by the initial prediction result and the Gaussian distribution heat map to obtain a first segmentation network, specifically: based on the floating point method, the prediction loss is calculated by the initial prediction result and the Gaussian distribution heat map, and the first initial network is trained by the prediction loss to obtain the first segmentation network; the calculation formula of the floating point method is: ; Wherein, FD is the prediction loss, C is the prediction result, and D is the Gaussian distribution heat map, is the number of elements in the first set, is the number of elements in the second set.
6. The method for automatically segmenting the pulmonary artery tree and the tracheal tree on CTPA images according to claim 1, characterized in that: The second segmentation network has a built-in dual sliding window attention structure consisting of a first attention module and a second attention module; the query and value of the first attention module are both feature maps of the second segmentation network, and the key of the first attention module is the first segmentation result; the second attention module is a self-attention module.
Citation Information
Cited By
Pulmonary artery hemodynamic parameter prediction method and system
CN120374592A
Tumor segmentation method and device, computer equipment, readable storage medium and program product
CN120876868A
Tumor segmentation methods, devices, computer equipment, readable storage media, and program products
CN120876868B
Image recognition-based pulmonary embolism focus segmentation method and system, and storage medium
CN121330728A