Carotid artery calcified plaque CTA image segmentation method based on deep learning
Through improved deep learning methods, combined with UNet network and Transformer module, the automation and robustness of carotid calcified plaque segmentation in traditional methods is solved, and high-precision carotid calcified plaque segmentation is achieved, providing a fast and accurate quantitative analysis tool.
Patent Information
- Application Number
- CN202510632916.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-16
- Publication Date
- 2025-08-15
AI Technical Summary
Traditional methods cannot achieve fully automatic segmentation of carotid calcified plaques, and small segmentation target recognition ability is poor, strong dependence on artificiality, and poor robustness.
A deep learning-based method is adopted to segment carotid calcified plaques through the improved UNet network and Transformer module. Combined with the residual connection structure and deep supervision mechanism, carotid vascular lumen segmentation is performed first, and then expansion and corrosion operations are performed as vascular masks to guide calcified plaque segmentation.
High-precision automatic segmentation of carotid calcified plaques is achieved, segmentation accuracy is improved, and rapid and accurate quantitative analysis tools are provided, reducing the impact of uneven distribution of target and background areas.
Smart Images
Figure CN120495322A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of medical image segmentation, and specifically relates to a CTA image segmentation method for carotid artery calcified plaques based on deep learning. Background Art
[0002] Stroke is a serious threat to human health, and carotid atherosclerosis is a major cause of ischemic stroke. Carotid atherosclerosis presents no clinical symptoms in its early stages, perhaps only mild dizziness or headache. Some patients may not experience any obvious symptoms until the disease is diagnosed. However, early detection of atherosclerosis and timely intervention can effectively slow the progression of the disease. Therefore, early identification of carotid atherosclerosis in stroke patients is crucial for preventing cardiovascular and cerebrovascular events and informing treatment decisions. Numerous imaging techniques have been developed for diagnosing carotid atherosclerosis. Computed tomography (CTA) is widely used clinically due to its fast scanning speed, high resolution, and non-invasive nature. Localizing and segmenting the carotid artery lumen and plaques in stroke patients based on CTA images, leading to early identification, is crucial for preventing cardiovascular and cerebrovascular events and informing treatment decisions. Traditional segmentation methods rely on manual labor and lack robustness, making fully automated segmentation impossible. Furthermore, the carotid artery is difficult to identify due to its small size. Therefore, there is an urgent need for a fully automatic segmentation method for carotid artery calcified plaques that can realize small target recognition. Summary of the Invention
[0003] To solve the above technical problems, the present invention provides a deep learning-based CTA image segmentation method for carotid artery calcified plaques, which solves the problems of poor small target recognition ability, strong dependence on manual work, poor robustness and inability to achieve fully automatic segmentation in the existing carotid artery calcified plaque segmentation technology.
[0004] To achieve the above object, the present invention adopts the following technical solutions:
[0005] A deep learning-based carotid artery calcified plaque CTA image segmentation method includes the following steps:
[0006] Step 1: Acquire a CTA image containing carotid artery calcified plaques;
[0007] Step 2: annotating the CTA image to obtain a CTA dataset, wherein the CTA dataset includes a carotid artery lumen dataset and a carotid artery calcified plaque dataset;
[0008] Step 3: Perform data preprocessing and data enhancement on the CTA dataset;
[0009] Step 4: Input the data-enhanced carotid artery lumen dataset into the improved UNet network segmentation model for training and testing to obtain the carotid artery lumen segmentation model and carotid artery lumen segmentation results;
[0010] Step 5: Using the carotid artery lumen segmentation result as a vascular mask to segment the CTA image into regions of interest, and inputting the segmented image into an improved UNet network segmentation model for training and testing to obtain a carotid artery calcified plaque segmentation model;
[0011] Step 6: Use the carotid artery lumen segmentation model and the calcified plaque segmentation model to perform carotid artery calcified plaque segmentation on the CTA image containing the carotid artery calcified plaque to obtain a carotid artery calcified plaque segmentation result.
[0012] Furthermore, in step 1, a dual-energy dual-source CT scanner is used in a spiral scanning mode to acquire a CTA image containing carotid artery calcified plaques, with the scanning range being from the aortic arch to the skull base, and from the skull base to the skull top.
[0013] Furthermore, in step 2, the data annotation includes selecting the bifurcation of the aortic arch and the left and right common carotid arteries as the starting point, extending upward to the carotid sinus, and after the bifurcation of the internal and external carotid arteries, selecting the blood vessels from the internal carotid artery to the point where the internal carotid artery enters the cranial cavity as the research object, and selecting the plaque part with a calcified tissue CT value ≥150HU in the research object part as the calcified plaque research object for data annotation.
[0014] Furthermore, the data preprocessing in step 3 includes CT value normalization and image segmentation, and the data enhancement includes mirroring and rotating the segmented image to perform data expansion.
[0015] Furthermore, in steps 4 and 5, the improved UNet network segmentation model includes an encoder, a decoder, and a deep supervision mechanism, and the loss function adopts a weighted DiceCE loss function.
[0016] Furthermore, the encoder includes an input module and four downsampling modules, the decoder includes five upsampling modules and an output module, and the encoder is connected to the decoder through a bottleneck layer and a Transformer module connected in sequence.
[0017] Furthermore, in step 5, using the carotid artery lumen segmentation result as a vascular mask to divide the CTA image into regions of interest includes performing image dilation and erosion processing on the carotid artery lumen segmentation result, and dividing the CTA image into regions of interest based on the vascular mask.
[0018] Furthermore, the weighted DiceCE function is calculated by adding the Dice loss function and the cross entropy loss function according to different weights. At the same time, the final loss function calculation needs to be weighted in combination with the output of the deep supervision head.
[0019] Furthermore, a residual connection structure is used in the input module, downsampling module, and bottleneck layer to alleviate gradient vanishing and achieve feature reuse. Each downsampling module is connected to the corresponding upsampling module at the same level through a skip connection. The last downsampling module is connected to the bottleneck layer, and the Transformer module is connected to the first upsampling module. After the first and second upsampling modules, there are auxiliary output branches for deep supervision, which are superimposed on the output module of the decoder.
[0020] Furthermore, when performing carotid artery calcification plaque segmentation, the bottleneck layer is connected to the Transformer module. Each image block output by the bottleneck layer is converted into a vector through linear projection. The vector is added with learnable position embedding information and then input into the Transformer layer. The Transformer layer contains a multi-head attention mechanism. The query matrix Q and the key matrix K are matrix multiplied to obtain the degree of attention between each sequence. After softmax normalization, the matrix is multiplied with the value matrix V to obtain a head of the self-attention layer. The outputs of multiple heads are spliced and linearly transformed to obtain the output of the multi-head attention structure. The output is sent to the feedforward neural network to fuse the different aspects of information extracted by the multi-head attention mechanism, and the fusion result is feature mapped.
[0021] The beneficial effects of the present invention are:
[0022] The present invention first segments the carotid artery lumen, then dilates and corrodes the carotid artery lumen and uses it as a vascular mask to guide the segmentation of carotid calcified plaques. This achieves small target segmentation of carotid calcified plaques and reduces the impact of uneven distribution between target segmentation areas and background areas. By improving the UNet network and adding a residual connection structure, the model gradient vanishing problem is alleviated and feature reuse is achieved. By adding a deep supervision mechanism, the efficiency of model parameter updating during backpropagation is optimized. By adding a Transformer module, the global context information of the image is better captured. Combined with the DiceCE loss function optimization, the segmentation accuracy is significantly improved, providing a fast and accurate tool for quantitative analysis of carotid calcified plaques in clinical practice. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Figure 1 This is a flow chart of a method for segmenting carotid artery calcified plaque CTA images based on deep learning in the present invention;
[0024] Figure 2 It is the overall network model diagram of the improved UNet network of the present invention;
[0025] Figure 3 This is a block diagram of the residual connection structure of the present invention;
[0026] Figure 4 This is a structural block diagram of the Transformer module of the present invention. DETAILED DESCRIPTION
[0027] The present invention will be further described below with reference to the accompanying drawings and examples.
[0028] See also Figure 1 The present invention provides a carotid artery calcified plaque CTA image segmentation method based on deep learning, comprising the following steps:
[0029] Step 1: Acquire a CTA image containing carotid artery calcification plaque. This CTA image is acquired using a dual-energy, dual-source CT scanner in spiral scanning mode, scanning from the aortic arch to the skull base and vice versa. To visualize the carotid artery lumen, inject 30-50 mL of non-ionic contrast agent intravenously via the dorsal hand vein or cubital forearm vein using a double-barreled high-pressure syringe. Scanning is initiated using intelligent triggering software. Immediately after the injection, 50 mL of normal saline is injected via the high-pressure syringe. Table 1 lists the parameters for performing the CTA scan:
[0030] Table 1
[0031] Step 2: Data annotation is performed on the obtained CTA images to obtain a CTA dataset, which includes a carotid artery lumen dataset and a carotid artery calcified plaque dataset. When selecting the carotid artery calcified plaque as the dataset for annotation, the starting point is selected from the bifurcation of the aortic arch and the left and right common carotid arteries, extending upward to the carotid sinus. After the bifurcation of the internal and external carotid arteries, the blood vessels from the internal carotid artery to the point where the internal carotid artery enters the cranial cavity are selected as the research object. In the research object part, the plaque part with a calcified tissue CT value ≥150HU is selected as the calcified plaque research object for data annotation.
[0032] Step 3: Perform data preprocessing and data enhancement on the acquired CTA dataset, specifically preprocessing and data enhancement on the carotid artery lumen dataset and the carotid artery calcified plaque dataset. Data preprocessing includes CT value normalization and image segmentation. Each CTA image used has a different contrast agent dose and scan time, resulting in different CT values during imaging and storage. To ensure consistency in intensity information, this study employed the Z-score normalization method, converting CT values to a standard normal distribution with a mean of 0 and a standard deviation of 1. The formula is as follows:
[0033] ,
[0034] Where z is the normalized intensity, x is the original intensity, μ is the mean CT value in the dataset, and σ is the standard deviation of the CT values in the dataset. Using normalization can effectively eliminate brightness differences between different images.
[0035] During data preprocessing, the image size is reduced by dividing the image into blocks, and the network training is performed using the divided data. Set the block size to , the data is divided into blocks according to the step size of 96±random(-8, 8). During the image block cutting process, the insufficient size parts are filled.
[0036] The strategies of mirroring and random angle rotation are implemented on the block-sampled images to achieve data expansion and enhancement in quantity and style.
[0037] Step 4: Input the data-augmented carotid artery lumen dataset into the improved UNet network segmentation model for training and testing, obtaining a carotid artery lumen segmentation model and segmented carotid artery lumen results. In the improved UNet network model, the encoder includes an input module and four downsampling modules, and the decoder includes five upsampling modules and an output module. The encoder is connected to the decoder via a bottleneck layer and a Transformer module connected in sequence.
[0038] Figure 2 This is the overall network model diagram of the improved UNet network of the present invention. The input size of the input module in the encoder is [B, 1, 128, 128, 128], where B represents the input batch, i.e. the number of batches, and the remaining four dimensions are the number of channels, depth, height, and width. The input of the downsampling module is the feature map output by the previous downsampling module. The residual connection structure (i.e. Figure 2The design of the residual connection structure (represented by the residual connection structure) is used to alleviate the vanishing gradient and realize feature reuse. The step size of the downsampling module is 2. After downsampling, the size of each dimension of the feature map is halved and the number of channels is doubled. The bottleneck layer keeps the number of channels unchanged, and the spatial size is adjusted to [4,4,8] through the asymmetric step size [2,2,1]. The bottleneck layer is connected to the Transformer module to perform global information modeling on the three-dimensional feature map extracted by the network, enhancing the network's ability to recognize small targets and complex boundaries. The input of the upsampling module is the feature map output from the previous layer and the feature map transmitted by the downsampling module of the same layer through the jump connection. The upsampling module splices the two feature maps in the channel dimension, and performs three-dimensional convolution, normalization, and activation on the spliced feature map. Then, a continuous convolution structure is used to expand the size of the spliced feature map to twice the original size, while reducing the number of channels to half the original size. There are auxiliary output branches of deep supervision after the first upsampling module and the second upsampling module, which are interpolated to the input size through upsampling. , ensure that it is consistent with the main output size collected by the last upsampling module, and then use softmax normalization and superimpose it with the main output result. The final output result is calculated through the loss function to reversely update the network model parameters. After iterative training, the carotid artery lumen segmentation model is obtained. Here, the upsampling module connected to the Transformer module is recorded as the first upsampling module, and the subsequent sequence is analogous, that is, the upsampling module close to the output module is the last upsampling module. In the implementation of the present invention, after the spliced features are subjected to the three-dimensional convolution layer, the normalization process adopts the instance normalization method. Assume that the input three-dimensional data is (i.e., the features after splicing), the calculation formula for normalizing the 3D data instance is as follows:
[0039] ,
[0040] Where n is the sample index, c is the channel index, d, h, and w are the indices of the depth, height, and width dimensions, respectively. represents the mean of the cth channel of the nth sample, is the variance of the cth channel of the nth sample. is a small constant used to avoid the denominator being zero; and are respectively the learnable scaling factor and offset factor, which are used to restore the expressive power of the data. First, the mean of each channel of each sample is calculated and variance Then the input 3D data Normalize by subtracting the mean and dividing by the standard deviation. Finally, pass the learnable parameters and Scale and offset the normalized data. and variance The calculation formulas are as follows:
[0041] ,
[0042] ,
[0043] In the implementation of the present invention, the Leaky ReLU activation function is used for nonlinear processing. The calculation formula of Leaky ReLU is:
[0044] ,
[0045] The output of Leaky ReLU when x < 0 is ,in is a small constant that preserves tiny gradients and prevents neurons from completely failing.
[0046] Figure 3 This is a schematic diagram of the residual connection structure of this method. The residual connection design is used in the input module, downsampling module and bottleneck layer. In the residual connection structure design, there are two calculation paths: the main path and the residual path. The input of the main path and the residual path are both feature maps output by the previous layer. In the figure, residual means that the residual path bypasses the residual output of the middle layer and maintains the original input features. The residual path contains a The convolutional layer (i.e., the path on the left in the diagram) ensures that the output size of the residual path matches the output size of the main path. "main" in the diagram represents the output of the main path after multiple convolutions. The final output "out" is the sum of the main path output "main" and the residual path output "residual." Finally, "out" is activated by a Leaky ReLU function and output to the next layer. During deep network training, gradients are easily attenuated through multiple layers of backpropagation. The residual connection structure provides a direct gradient path, ensuring stable gradient propagation and mitigating gradient vanishing.
[0047] In an embodiment of the present invention, Figure 4This is a structural block diagram of the Transformer module in the implementation of the invention. When performing carotid artery calcification plaque segmentation, the bottleneck layer is connected to the Transformer module. Each image block output by the bottleneck layer can be converted into a vector by linear projection. The vector is added with learnable position embedding information and then input into the Transformer layer. The Transformer layer contains a multi-head attention mechanism. The query matrix Q and the key matrix K are matrix multiplied to obtain the degree of attention between each sequence. After softmax normalization, the matrix is multiplied with the value matrix V to obtain a head of the self-attention layer. The outputs of multiple heads are spliced and linearly transformed to obtain the output of the multi-head attention structure. The output is sent to the feedforward neural network. The feedforward neural network contains two linear transformations and a GELU activation function, which is used to fuse different aspects of information extracted by the multi-head attention mechanism and enhance feature expression capabilities. Finally, the result is feature mapped. The feature mapping includes two processes: data reshaping and convolution, which are used to map the feature map to the same size as the input.
[0048] Step 5: Use the carotid artery lumen area obtained by segmenting the carotid artery as a vascular mask to divide the CTA image into regions of interest, limit the segmented area to the area around the carotid artery, and input the segmented image into the improved UNet network segmentation model for training and testing to obtain a carotid artery calcification plaque segmentation model.
[0049] In an embodiment of the present invention, the carotid artery lumen area in step 5 is used as a vascular mask and needs to be processed by image dilation and corrosion. First, seven dilation operations are performed to expand the carotid artery lumen area so that the carotid artery lumen area covers as much of the carotid artery calcification area as possible. Then, a small-scale corrosion operation is performed to reduce noise to obtain a vascular mask. Based on the processed vascular mask, the image is guided to divide the region of interest, and the segmented area is limited to a specific range around the blood vessel, further reducing the volume of the background area, eliminating noise signals far away from the blood vessel wall, and improving the accuracy of plaque segmentation.
[0050] In the embodiment of the present invention, the loss function DiceCE function is calculated by adding the Dice loss function and the cross entropy loss function according to different weights. At the same time, the final loss function calculation needs to be combined with the output of the deep supervision head according to different weights. The Dice indicator formula for binary classification is as follows:
[0051] ,
[0052] Where M represents the number of voxels, T represents the true label, i.e., the gold standard, the positive sample (i.e., the carotid artery calcification voxel point) is labeled 1, the negative sample (background point) is labeled 0, and P represents the model prediction result. To smooth the factor and avoid the situation where the denominator is 0, the Dice loss function needs to be further calculated. The result of subtracting the Dice indicator from 1 is the Dice loss function, that is, .
[0053] The cross entropy loss function formula for the two-class scenario is as follows:
[0054] ,
[0055] in, represents the sigmoid function, represents the true label (0 or 1), Indicates an inactive prediction value.
[0056] The DiceCE loss function is obtained by adding the Dice loss and the cross entropy loss according to the weights. The formula is as follows:
[0057] ,
[0058] in, , represents the weight coefficient, which controls the relative importance of the two losses. Since the network structure contains deep supervision output, it is necessary to perform weighted calculation of the loss function for different outputs. The calculation formula is as follows:
[0059] ,
[0060] Assume that there are A layers in the deep supervision network, and L represents the final loss function result. Represents the loss function result calculated at the a-th level, Indicates the weight of level a.
[0061] Step 6. The CTA image containing carotid artery calcification plaque is first subjected to the carotid artery lumen segmentation model to obtain the carotid artery lumen segmentation result, and then the segmentation result is used as the blood vessel mask to divide the segmented area into regions of interest by mask multiplication, and the segmented area is limited to a specific range around the blood vessel. Then, the carotid artery calcification plaque segmentation model is used to output the carotid artery calcification plaque segmentation result within the limited range.
[0062] In the embodiment of the present invention, the Dice index and the mean intersection over union (mIoU) of the real image and the predicted image are used as evaluation indicators for the segmentation results. The formula is as follows:
[0063] ,
[0064] Where Q represents the total number of categories, including background, represents the IoU of the qth category, TP represents true positives, i.e., the number of pixels predicted to be positive and actually positive, FP represents false positives, i.e., the number of pixels predicted to be positive but actually negative, and FN represents false positives, i.e., the number of pixels predicted to be negative but actually positive.
[0065] The deep learning-based carotid artery calcified plaque CTA image segmentation method proposed in this paper can achieve an average Dice index of 0.74 and an mIoU index of 0.70 on the training set.
[0066] In summary, the present invention first segments the carotid artery lumen, and then uses the carotid artery lumen as a vascular mask to guide the segmentation of carotid calcified plaques, thereby achieving small target segmentation of carotid calcified plaques and reducing the impact of uneven distribution of target segmentation areas and background areas; by improving the UNet network and adding a residual connection structure, the model gradient vanishing problem is alleviated and feature reuse is achieved; by adding a deep supervision mechanism, the efficiency of model parameter updating during back propagation is optimized; by adding a Transformer module, the global context information of the image is better captured, forming a set of carotid artery calcified plaque CTA image segmentation methods based on deep learning.
[0067] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A deep learning-based CTA image segmentation method for carotid artery calcified plaques, characterized in that: The following steps are involved: Step 1: Acquire a CTA image containing carotid artery calcified plaques; Step 2: annotating the CTA image to obtain a CTA dataset, wherein the CTA dataset includes a carotid artery lumen dataset and a carotid artery calcified plaque dataset; Step 3: Perform data preprocessing and data enhancement on the CTA dataset; Step 4: Input the data-enhanced carotid artery lumen dataset into the improved UNet network segmentation model for training and testing to obtain the carotid artery lumen segmentation model and carotid artery lumen segmentation results; Step 5: Using the carotid artery lumen segmentation result as a vascular mask to segment the CTA image into regions of interest, and inputting the segmented image into an improved UNet network segmentation model for training and testing to obtain a carotid artery calcified plaque segmentation model; Step 6: Use the carotid artery lumen segmentation model and the calcified plaque segmentation model to perform carotid artery calcified plaque segmentation on the CTA image containing the carotid artery calcified plaque to obtain a carotid artery calcified plaque segmentation result.
2. The method for segmenting carotid artery calcified plaque CTA images based on deep learning according to claim 1, characterized in that: In step 1, a dual-energy dual-source CT scanner is used in a spiral scanning mode to acquire a CTA image containing carotid artery calcified plaques, with the scanning range extending from the aortic arch to the skull base, and from the skull base to the skull top.
3. The method for segmenting carotid artery calcified plaque CTA images based on deep learning according to claim 1, characterized in that: In step 2, the data annotation includes selecting the bifurcation of the aortic arch and the left and right common carotid arteries as the starting point, extending upward to the carotid sinus, and after the bifurcation of the internal and external carotid arteries, selecting the blood vessels from the internal carotid artery to the point where the internal carotid artery enters the cranial cavity as the research object, and selecting the plaque part with a calcified tissue CT value ≥150HU in the research object part as the calcified plaque research object for data annotation.
4. The method for segmenting carotid artery calcified plaque CTA images based on deep learning according to claim 1, characterized in that: The data preprocessing in step 3 includes CT value normalization and image segmentation, and the data enhancement includes mirroring and rotating the segmented image to perform data expansion.
5. The method for segmenting carotid artery calcified plaque CTA images based on deep learning according to claim 1, characterized in that: In steps 4 and 5, the improved UNet network segmentation model includes an encoder, a decoder, and a deep supervision mechanism, and the loss function adopts a weighted DiceCE loss function.
6. The method for segmenting carotid artery calcified plaque CTA images based on deep learning according to claim 5, characterized in that: The encoder includes an input module and four downsampling modules, the decoder includes five upsampling modules and an output module, and the encoder is connected to the decoder through a bottleneck layer and a Transformer module connected in sequence.
7. The method for segmenting carotid artery calcified plaque CTA images based on deep learning according to claim 1, characterized in that: In step 5, using the carotid artery lumen segmentation result as a vascular mask to divide the CTA image into regions of interest includes performing image dilation and erosion processing on the carotid artery lumen segmentation result, and dividing the CTA image into regions of interest based on the vascular mask.
8. The method for segmenting carotid artery calcified plaque CTA images based on deep learning according to claim 5, characterized in that: The weighted DiceCE function is calculated by adding the Dice loss function and the cross entropy loss function according to different weights. At the same time, the final loss function calculation needs to be weighted in combination with the output of the deep supervision head.
9. The method for segmenting carotid artery calcified plaque CTA images based on deep learning according to claim 6, characterized in that: A residual connection structure design is adopted in the input module, downsampling module and bottleneck layer to alleviate the vanishing gradient and realize feature reuse; each downsampling module is connected to the upsampling module at the same level through a jump connection, among which the last downsampling module is connected to the bottleneck layer, and the Transformer module is connected to the first upsampling module. There are deep-supervised auxiliary output branches after the first upsampling module and the second upsampling module, which are superimposed on the output module of the decoder.
10. The method for segmenting carotid artery calcified plaque CTA images based on deep learning according to claim 6, characterized in that: When performing carotid artery calcification plaque segmentation, the bottleneck layer is connected to the Transformer module. Each image block output by the bottleneck layer is converted into a vector through linear projection. After adding learnable position embedding information to the vector, it is input into the Transformer layer. The Transformer layer contains a multi-head attention mechanism. The query matrix Q and the key matrix K are matrix multiplied to obtain the degree of attention between each sequence. After softmax normalization, the matrix is multiplied with the value matrix V to obtain a head of the self-attention layer. The outputs of multiple heads are spliced and linearly transformed to obtain the output of the multi-head attention structure. The output is sent to the feedforward neural network to fuse the different aspects of information extracted by the multi-head attention mechanism, and the fusion result is feature mapped.