Lung blood vessel segmentation method fusing multi-layer coding features
By introducing a multi-layer feature fusion with a cross-attention mechanism in lung vessel segmentation, the problems of unstable vessel segmentation and insufficient identification of small vessels in existing methods are solved, and a more efficient lung vessel segmentation effect is achieved.
Patent Information
- Application Number
- CN202511415563.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-30
- Publication Date
- 2026-01-13
AI Technical Summary
Existing deep learning methods suffer from problems in lung vessel segmentation, such as poor preservation of overall topology, easy omission of small vessels, and spurious connections in the segmentation results. They also lack robustness and generalization ability, especially under interference from lung lesions.
A deep segmentation network that integrates multi-layer coding features is adopted. By combining the U-Net structure and introducing a cross-attention mechanism, low-level detail features are dynamically selected and fused through multi-layer feature fusion, thereby improving the completeness and accuracy of feature representation.
It significantly improves the accuracy and stability of pulmonary vascular segmentation, especially the ability to identify small blood vessels, reduces missegmentation and missed detection, and enhances the consistency and accuracy of segmentation results.
Smart Images
Figure CN121330286A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical image analysis technology, and more specifically, to a method for lung vessel segmentation that integrates multi-layer coding features. Background Technology
[0002] Clear segmentation of pulmonary vessels is crucial for various clinical tasks, such as the auxiliary diagnosis of pulmonary hypertension, preoperative vascular assessment, lung perfusion analysis, and 3D reconstruction navigation. However, due to the complex branching structure, small diameter range, and similarity in grayscale and morphology to surrounding tissues, traditional image processing methods often face significant challenges in pulmonary vessel segmentation, particularly in the identification of small vessels and the accuracy of their boundaries, exhibiting certain limitations.
[0003] In recent years, with the rapid development of deep learning technology in the field of medical image analysis, segmentation methods based on convolutional neural networks (CNNs) have gradually become mainstream. These methods can automatically learn multi-scale features in images and effectively identify the structural manifestations of pulmonary vessels at different scales and morphologies, thus achieving a significant improvement in segmentation accuracy compared to traditional methods. Especially in 3D CT images, deep networks can fully utilize spatial context information to enhance sensitivity to small blood vessels and blurred boundaries.
[0004] Although some existing deep learning methods have been applied to lung vessel segmentation, several problems still exist in practical clinical use. For example, they may not preserve the overall topological structure of the vessels well, small vessels may be easily missed, or spurious connections may appear in the segmentation results. In addition, image interference from lung lesions (such as nodules and inflammatory areas) can also affect the robustness and generalization ability of the model.
[0005] Therefore, designing a more stable, accurate, and adaptable lung vessel segmentation method for various clinical scenarios has become a key technical issue in the field of medical image analysis. Summary of the Invention
[0006] The purpose of this invention is to provide a method for lung vessel segmentation that integrates multi-layer coding features. This method utilizes a deep segmentation network that integrates multi-layer coding features. This network not only uses feature information from the current layer but also incorporates coding features extracted from previous stages, thereby achieving multi-layered and multi-scale semantic representation. To achieve more effective feature fusion, a cross-attention mechanism is introduced, enabling high-order semantic features to dynamically guide the selection and integration of low-order detail features. Through this fusion approach, the network can simultaneously consider both local details and global structure of the image, improving the completeness and accuracy of feature representation, thus enhancing the efficiency and quality of lung vessel segmentation, especially for small vessels.
[0007] The above-mentioned technical objective of this invention is achieved through the following technical method: a lung vessel segmentation method that integrates multi-layer coding features, which establishes a segmentation model based on a U-Net structure, introduces a multi-layer feature fusion method based on a cross-attention mechanism in the coding stage, wherein the coding layer not only extracts its own features, but also actively participates in the reconstruction and fusion of the coding features of previous layers; through the cross-attention mechanism, the segmentation model can dynamically filter and fuse key details in low-order features according to the current high-order semantic information, thereby avoiding the influence of invalid or interfering information in shallow features on the segmentation results;
[0008] Includes the following steps:
[0009] S1 data input reads 3D image data and labels, performs data augmentation, converts it into 2D image format, and inputs it into the segmentation model for data analysis;
[0010] The S2 segmentation model analyzes the data, takes a 2D image as input, encodes and decodes it using the segmentation model, calculates the loss, and updates the model parameters.
[0011] S3 model selection is based on the validation set, selecting the model with the highest segmentation index from the trained models as the final model;
[0012] The S4 model predicts the results of new test data using the model selected in S3.
[0013] The present invention is further configured such that step S1 specifically includes the following steps:
[0014] S11 Input data, input image data x of training set sample i. (i) and tag y (i) ;where x (i) and y (i) All of them are three-dimensional data;
[0015] S12 data augmentation, for x (i) Perform data augmentation, such as rotation, translation, elastic transformation, or noise addition; for the label y (i) The same spatial transformation is also performed accordingly;
[0016] Image sampling for S13 requires the following input parameters due to the 2D segmentation model: channel * length * width, within x. (i) Randomly select a series of consecutive 2D images, for example, c images, as 2D input, i.e., c × w × d, c channels; the corresponding y (i) Do the same treatment;
[0017] S14 dimensional transformation, for x (i) and y (i) Perform a size transformation to make its dimensions consistent as c×w×d;
[0018] S15 will sample x (i) As input to the segmentation model, y (i) Loss is calculated using labels.
[0019] The present invention is further configured such that step S2 specifically includes the following steps:
[0020] S21 first uses a convolutional network to extract features from the input image, and then uses L coding networks to calculate the features;
[0021] The S22 encoding network consists of L encoding networks. Taking the j-th encoding network as an example, its input comes from the output of the 1st, 2nd, ..., j-1st encoding networks. Let j = 3. First, this encoding network uses a downsampling network and a convolutional network to calculate the output feature F2 from the 2nd encoding network. The size of F2 is c2×w2×d2, resulting in feature F3 with a size of c3×w3×d3. Then, based on F3, the cross-attention between F3 and the output features F1 of the 1st encoding network and F2 of the 2nd encoding network is calculated. The size of F1 is c1×w1×d1, and finally, the features are fused.
[0022] The S23 intermediate layer encodes the features sequentially according to the encoding network. After obtaining the feature output of the Lth encoding network, it is input into the intermediate layer. The intermediate layer consists of a convolutional network and self-attention. That is, the input features are first convolved, then the self-attention module is used for calculation, and finally the convolutional network is used to calculate to obtain the output of the intermediate layer. The intermediate layer only changes the number of channels and does not change the size of the features.
[0023] The S24 decoding network concatenates the output features of the intermediate layers and the output of the Lth encoding network along the channel dimension, and then inputs them into the Lth decoding network. The decoding network is composed of convolutional networks. Taking the jth decoding network as an example, its input contains all high-order and low-order features. Since its input contains the output features of the jth encoding network, this feature integrates the features of the 1st, 2nd, ..., j-1st encoding networks. The second part of the input comes from the features of the j+1th decoding network, and this feature contains the features of the j+1th, j+2th, ..., Lth encoding networks. Therefore, each decoding network contains all high-order and low-order features.
[0024] S25 loss calculation: After obtaining the model prediction results, cross-entropy loss is used to calculate the difference between the prediction and the label, and the model parameters are updated accordingly.
[0025]
[0026] The present invention is further configured such that step S22 specifically includes the following steps:
[0027] With S221 channels being consistent, taking the third encoding network in S22 as an example, feature F1 is first calculated using a 1x1 convolutional network to make its number of channels consistent with feature F3, resulting in a feature of size c3×w1×d1; similarly, feature F2 is calculated using a 1x1 convolutional network to make its number of channels consistent with feature F3, resulting in a feature of size c3×w2×d2.
[0028] S222 calculates K and V using a convolutional network based on the features obtained in S221. The features of size c3×w1×d1 are input into two convolutional networks respectively to obtain the outputs K and V of the two convolutional networks, with their sizes remaining unchanged. Based on this process, K1, V1, K2 and V2 are obtained.
[0029] S223 Dimension Transformation: Transform the dimensions of K1, V1, K2, and V2. The dimensions of K1 and V1 are both N1×c3, and the dimensions of K2 and V2 are both N2×c3.
[0030] S224 calculates cross-attention. After transforming F3, we obtain Q with a size of N3×c3. Then, we calculate the relationship between Q and K1 based on matrix multiplication, that is, the relationship between all feature points in Q and all feature points in K1. After normalization, we obtain the attention matrix A1 with a size of N3×N1. The k-th row of matrix A1 represents the relationship between the k-th feature point in Q and all feature points in K1. Similarly, we can obtain A2. Finally, we use matrix multiplication to multiply A1 and V1, and to multiply A2 and V2 to obtain the final outputs O1 and O2, both with a size of N3×c3.
[0031] S225 feature fusion transforms the size of O1 and O2, resulting in feature sizes of c3×w3×d3. It then concatenates these features with F3 and uses a 1x1 convolutional network to calculate the final output features of the encoding network, which have a size of c3×w3×d3.
[0032] In summary, this invention offers the following advantages: The method introduces a cross-attention mechanism to fuse features from multiple levels during the encoding stage. The encoded features of the current layer not only serve as the basis for semantic expression but also guide the reconstruction and integration of encoded features from previous stages. Through this mechanism, the model can effectively combine low-order detail information with high-order semantic information, thereby simultaneously optimizing global structure perception and local detail representation during the encoding process. This feature fusion method significantly improves the model's segmentation ability and stability for pulmonary blood vessels, especially small and complex vessels. Attached Figure Description
[0033] Figure 1 This is an overall framework diagram of the method in the embodiments of the present invention;
[0034] Figure 2 This is a schematic diagram illustrating the principle of introducing a multi-layer feature fusion method based on cross-attention mechanism in the encoding stage in this embodiment of the invention. Detailed Implementation
[0035] The following is in conjunction with the appendix Figure 1-2 The present invention will be described in further detail below.
[0036] Example: A method for lung vessel segmentation incorporating multi-layer coding features, such as... Figure 1 , Figure 2 As shown, the data contains N samples. Each sample consists of a 3D image x. (i) and tag y (i) The data was divided into training, validation, and test sets in an 8:1:1 ratio. The training set was used to train the model, the validation set was used to select the best-performing model from the trained models, and the test set was used to evaluate the model's performance.
[0037] This segmentation model is based on the U-Net architecture. Its core innovation lies in the introduction of a multi-layer feature fusion method based on a cross-attention mechanism during the encoding stage. Specifically, the current encoding layer not only extracts its own features but also actively participates in the reconstruction and fusion of features from previous layers. Through the cross-attention mechanism, the model can dynamically filter and fuse key details from low-order features based on current high-order semantic information, thereby avoiding the influence of invalid or interfering information in shallow features on the segmentation results.
[0038] This fusion approach enhances feature representation capabilities: by fusing multiple layers of features during the encoding stage, the model can simultaneously retain shallow texture details and deep semantic structures, achieving effective complementarity between global and local information and improving the discriminative power of the overall features. Therefore, this approach can improve blood vessel segmentation performance: especially in lung blood vessel segmentation tasks, this fusion strategy significantly improves the ability to identify small blood vessels, enhances the model's perception of complex branching structures, reduces missegmentation and missed detections, and results in more coherent and accurate overall segmentation results.
[0039] The method specifically includes the following steps:
[0040] The S1 data input reads the 3D image data and labels, performs data augmentation, converts it into a 2D image format, and then inputs it into the segmentation model for data analysis.
[0041] S11 Input data, input image data x of training set sample i. (i) and tag y (i) ;where x (i) and y (i) All of them are three-dimensional data;
[0042] S12 data augmentation, for x (i) Perform data augmentation, such as rotation, translation, elastic transformation, or noise addition; for the label y (i) The same spatial transformation is also performed accordingly;
[0043] Image sampling for S13 requires the following input parameters due to the 2D segmentation model: channel * length * width, within x. (i) Randomly select a series of consecutive 2D images, for example, c images, as 2D input, i.e., c × w × d, c channels; the corresponding y (i) Do the same treatment;
[0044] S14 dimensional transformation, for x (i) and y (i) Perform a size transformation to make its dimensions consistent as c×w×d;
[0045] S15 will sample x (i) As input to the segmentation model, y (i) Loss is calculated using labels.
[0046] The S2 segmentation model analyzes the data, takes a 2D image as input, encodes and decodes it using the segmentation model, calculates the loss, and updates the model parameters.
[0047] S21 first uses a convolutional network to extract features from the input image, and then uses L coding networks to calculate the features;
[0048] The S22 encoding network consists of L encoding networks. Taking the j-th encoding network as an example, its input comes from the output of the 1st, 2nd, ..., j-1st encoding networks. Let j = 3. First, this encoding network uses a downsampling network and a convolutional network to calculate the output feature F2 from the 2nd encoding network. The size of F2 is c2×w2×d2, resulting in feature F3 with a size of c3×w3×d3. Then, based on F3, the cross-attention between F3 and the output features F1 of the 1st encoding network and F2 of the 2nd encoding network is calculated. The size of F1 is c1×w1×d1, and finally, the features are fused.
[0049] With S221 channels being consistent, taking the third encoding network in S22 as an example, feature F1 is first calculated using a 1x1 convolutional network to make its number of channels consistent with feature F3, resulting in a feature of size c3×w1×d1; similarly, feature F2 is calculated using a 1x1 convolutional network to make its number of channels consistent with feature F3, resulting in a feature of size c3×w2×d2.
[0050] S222 calculates K and V using a convolutional network based on the features obtained in S221. The features of size c3×w1×d1 are input into two convolutional networks respectively to obtain the outputs K and V of the two convolutional networks, with their sizes remaining unchanged. Based on this process, K1, V1, K2 and V2 are obtained.
[0051] S223 Dimension Transformation: Transform the dimensions of K1, V1, K2, and V2. The dimensions of K1 and V1 are both N1×c3, and the dimensions of K2 and V2 are both N2×c3.
[0052] S224 calculates cross-attention. After transforming F3, we obtain Q with a size of N3×c3. Then, we calculate the relationship between Q and K1 based on matrix multiplication, that is, the relationship between all feature points in Q and all feature points in K1. After normalization, we obtain the attention matrix A1 with a size of N3×N1. The k-th row of matrix A1 represents the relationship between the k-th feature point in Q and all feature points in K1. Similarly, we can obtain A2. Finally, we use matrix multiplication to multiply A1 and V1, and to multiply A2 and V2 to obtain the final outputs O1 and O2, both with a size of N3×c3.
[0053] S225 feature fusion transforms the size of O1 and O2, resulting in feature sizes of c3×w3×d3. It then concatenates these features with F3 and uses a 1x1 convolutional network to calculate the final output features of the encoding network, which have a size of c3×w3×d3.
[0054] The S23 intermediate layer encodes the features sequentially according to the encoding network. After obtaining the feature output of the Lth encoding network, it is input into the intermediate layer. The intermediate layer consists of a convolutional network and self-attention. That is, the input features are first convolved, then the self-attention module is used for calculation, and finally the convolutional network is used to calculate to obtain the output of the intermediate layer. The intermediate layer only changes the number of channels and does not change the size of the features.
[0055] The S24 decoding network concatenates the output features of the intermediate layers and the output of the Lth encoding network along the channel dimension, and then inputs them into the Lth decoding network. The decoding network is composed of convolutional networks. Taking the jth decoding network as an example, its input contains all high-order and low-order features. Since its input contains the output features of the jth encoding network, this feature integrates the features of the 1st, 2nd, ..., j-1st encoding networks. The second part of the input comes from the features of the j+1th decoding network, and this feature contains the features of the j+1th, j+2th, ..., Lth encoding networks. Therefore, each decoding network contains all high-order and low-order features.
[0056] S25 loss calculation: After obtaining the model prediction results, cross-entropy loss is used to calculate the difference between the prediction and the label, and the model parameters are updated accordingly.
[0057]
[0058] The S3 model selection method, based on the validation set, selects the model with the highest segmentation metric from the trained models as the final model.
[0059] The S4 model predicts the results of new test data using the model selected in S3.
[0060] This specific embodiment is merely an explanation of the present invention and is not intended to limit the invention. After reading this specification, those skilled in the art can make modifications to this embodiment without contributing any inventive step, but such modifications are protected by patent law as long as they are within the scope of the claims of the present invention.
Claims
1. A method for lung vessel segmentation that integrates multi-layer coding features, characterized in that: A segmentation model is built based on the U-Net structure. In the encoding stage, a multi-layer feature fusion method based on cross-attention mechanism is introduced. The encoding layer not only extracts its own features, but also actively participates in the reconstruction and fusion of the encoded features of the previous layers. Through the cross-attention mechanism, the segmentation model can dynamically filter and fuse key details in low-order features according to the current high-order semantic information, thereby avoiding the influence of invalid or interfering information in shallow features on the segmentation results. Includes the following steps: S1 data input reads 3D image data and labels, performs data augmentation, converts it into 2D image format, and inputs it into the segmentation model for data analysis; The S2 segmentation model analyzes the data, takes a 2D image as input, encodes and decodes it using the segmentation model, calculates the loss, and updates the model parameters. S3 model selection is based on the validation set, selecting the model with the highest segmentation index from the trained models as the final model; The S4 model predicts the results of new test data using the model selected in S3.
2. The lung vessel segmentation method incorporating multi-layer coding features according to claim 1, characterized in that: S1 specifically includes the following steps: S11 Input data, input image data x of training set sample i. (i) and tag y (i) ; where x (i) and y (i) All of them are three-dimensional data; S12 data augmentation, for x (i) Perform data augmentation, such as rotation, translation, elastic transformation, or noise addition; for the label y (i) The same spatial transformation is also performed accordingly; Image sampling for S13 requires the following input parameters due to the 2D segmentation model: channel * length * width, within x. (i) Randomly select a series of consecutive 2D images, for example, c images, as 2D input, i.e., c × w × d, c channels; the corresponding y (i) Do the same treatment; S14 dimensional transformation, for x (i) and y (i) Perform a size transformation to make its dimensions consistent as c×w×d; S15 will sample x (i) As input to the segmentation model, y (i) Loss is calculated using labels.
3. The lung vessel segmentation method incorporating multi-layer coding features according to claim 2, characterized in that: S2 specifically includes the following steps: S21 first uses a convolutional network to extract features from the input image, and then uses L coding networks to calculate the features; The S22 encoding network consists of L encoding networks. Taking the j-th encoding network as an example, its input comes from the output of the 1st, 2nd, ..., j-1st encoding networks. Let j = 3. First, this encoding network uses a downsampling network and a convolutional network to calculate the output feature F2 from the 2nd encoding network. The size of F2 is c2×w2×d2, resulting in feature F3 with a size of c3×w3×d3. Then, based on F3, the cross-attention between F3 and the output features F1 of the 1st encoding network and F2 of the 2nd encoding network is calculated. The size of F1 is c1×w1×d1, and finally, the features are fused. The S23 intermediate layer encodes the features sequentially according to the encoding network. After obtaining the feature output of the Lth encoding network, it is input into the intermediate layer. The intermediate layer consists of a convolutional network and self-attention. That is, the input features are first convolved, then the self-attention module is used for calculation, and finally the convolutional network is used to calculate to obtain the output of the intermediate layer. The intermediate layer only changes the number of channels and does not change the size of the features. The S24 decoding network concatenates the output features of the intermediate layers and the output of the Lth encoding network along the channel dimension, and then inputs them into the Lth decoding network. The decoding network is composed of convolutional networks. Taking the jth decoding network as an example, its input contains all high-order and low-order features. Since its input contains the output features of the jth encoding network, this feature integrates the features of the 1st, 2nd, ..., j-1st encoding networks. The second part of the input comes from the features of the j+1th decoding network, and this feature contains the features of the j+1th, j+2th, ..., Lth encoding networks. Therefore, each decoding network contains all high-order and low-order features. S25 loss calculation: After obtaining the model prediction results, cross-entropy loss is used to calculate the difference between the prediction and the label, and the model parameters are updated accordingly.
4. The lung vessel segmentation method incorporating multi-layer coding features according to claim 3, characterized in that: S22 specifically includes the following steps: With S221 channels being consistent, taking the third encoding network in S22 as an example, feature F1 is first calculated using a 1x1 convolutional network to make its number of channels consistent with feature F3, resulting in a feature of size c3×w1×d1; similarly, feature F2 is calculated using a 1x1 convolutional network to make its number of channels consistent with feature F3, resulting in a feature of size c3×w2×d2. S222 calculates K and V using a convolutional network based on the features obtained in S221. The features of size c3×w1×d1 are input into two convolutional networks respectively to obtain the outputs K and V of the two convolutional networks, with their sizes remaining unchanged. Based on this process, K1, V1, K2 and V2 are obtained. S223 Dimension Transformation: Transform the dimensions of K1, V1, K2, and V2. The dimensions of K1 and V1 are both N1×c3, and the dimensions of K2 and V2 are both N2×c3. S224 calculates cross-attention. After transforming F3, we obtain Q with a size of N3×c3. Then, we calculate the relationship between Q and K1 based on matrix multiplication, that is, the relationship between all feature points in Q and all feature points in K1. After normalization, we obtain the attention matrix A1 with a size of N3×N1. The k-th row of matrix A1 represents the relationship between the k-th feature point in Q and all feature points in K1. Similarly, we can obtain A2. Finally, we use matrix multiplication to multiply A1 and V1, and to multiply A2 and V2 to obtain the final outputs O1 and O2, both with a size of N3×c3. S225 feature fusion transforms the size of O1 and O2, resulting in feature sizes of c3×w3×d3. It then concatenates these features with F3 and uses a 1x1 convolutional network to calculate the final output features of the encoding network, which have a size of c3×w3×d3.