A carotid artery extraction method based on convolutional neural network
Through the carotid artery extraction method based on convolutional neural network, the pre-trained model iteratively tracks and segments the central path of the carotid artery, solving the problems of limited blood vessel extraction effect and difficulty in segmentation in the existing technology, and achieving high-accurate carotid artery Structure extraction.
Patent Information
- Application Number
- CN202210145347.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-02-17
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2042-02-17
AI Technical Summary
The effect of the prior art vascular extraction method in carotid artery CTA images is greatly affected by the shape and image quality of the blood vessels, requiring strong constraint criteria and prior knowledge, and cannot directly segment the blood vessel profile, and traditional methods cannot meet the growing clinical needs.
Using a carotid artery extraction method based on convolutional neural network, by establishing a carotid artery central path prediction model and segmentation model, the pre-trained model iteratively tracks the carotid artery central path and performs segmentation, combining non-offset and offset patch block extraction , realize the accurate segmentation of the carotid artery.
The accuracy of carotid artery central path tracking is improved, the extraction difficulties caused by the proximity of carotid artery and bone tissue is overcome, and the carotid artery segmentation is achieved while tracking the central path, meeting clinical needs.
Smart Images

Figure CN114519722B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of medical image processing, and in particular to a carotid artery extraction method based on a convolutional neural network. Background Art
[0002] Cerebrovascular diseases pose a serious threat to human health, with high morbidity, high disability and high mortality. Enhanced CT scanning has become the primary means of examining vascular diseases in clinical practice due to its rapid, non-invasive, clear image and economic characteristics. The extraction of blood vessels in carotid CTA (Computed Tomography Angiography) images mainly includes the extraction of the central path of the carotid artery and the segmentation of the carotid artery. This is a key step in accurately displaying and quantifying the carotid artery from complex data sets. It is also a prerequisite for the diagnosis and surgical planning of cerebrovascular diseases such as stenosis, plaques and aneurysms, and is crucial for the evaluation of clinical diagnostic results.
[0003] At present, there are two main types of vascular extraction methods based on CTA images: those based on traditional automatic and semi-automatic vascular extraction algorithms and those based on deep learning technology. Traditional automatic and semi-automatic vascular extraction methods mainly rely on rules designed by experts to separate vascular structures from the background, including: 1) filter-based methods, such as using filters based on the Hessian matrix to enhance vascular structures and then segmenting blood vessels on the enhanced image. 2) model-based methods, such as using active contour models to extract vascular structures. 3) centerline tracking methods, such as using the minimum path propagation process to find the centerline and then extracting the complete vascular cavity. The vascular extraction method based on deep learning technology is mainly manifested in the design of various complex deep neural network models, which are applied to the extraction of blood vessels of various organs. For example, deep learning networks such as the deep neural network DeepVesselNet, the lightweight deep learning model VesselNet, and the semantic segmentation network VSSC with an added supervision mechanism are used to segment vascular CTA.
[0004] Although many traditional automatic and semi-automatic blood vessel extraction methods have been proposed in the past few years, there are still many deficiencies that cannot meet the growing clinical needs. For example, the filter-based method is greatly affected by the blood vessel shape and image quality; the model-based method requires strong constraints and relies heavily on prior knowledge; the centerline method cannot directly segment the blood vessel contour and needs to combine the blood vessel appearance and geometric information. With the development of deep learning technology, this technology has shown its excellence in medical image analysis. Due to its strong feature extraction ability, the deep neural network model has a successful performance in the extraction of blood vessel center path and blood vessel segmentation. The current blood vessel center path extraction and blood vessel segmentation are two interdependent and independent tasks, most of which are based on the construction of blood vessels based on the found center path or based on the segmentation results to extract the center path. The present invention is to use deep learning technology to achieve tracking of the carotid artery center path while performing carotid artery segmentation. Summary of the invention
[0005] In view of the deficiencies in the above-mentioned prior art, the present invention provides a carotid artery extraction method based on a convolutional neural network, which aims to achieve accurate and rapid iterative tracking of the central path of the carotid artery from carotid artery CTA images while achieving accurate segmentation of the carotid artery structure.
[0006] To achieve the above object, the present invention provides the following technical solutions:
[0007] A method for extracting carotid arteries based on a convolutional neural network, characterized in that the method comprises the following steps:
[0008] Step 1: Extract patches from each 3D carotid CTA image and its corresponding segmentation label in the original data set, construct a segmentation training data set from the extracted patches, and construct a center path prediction training data set from the patches extracted from each 3D carotid CTA image and the carotid center path prediction label;
[0009] Step 2: Establish a carotid artery central path prediction model, and use the central path prediction training data set to train the carotid artery central path prediction model;
[0010] Step 3: Determine and use the segmentation training data set to train a carotid artery segmentation model;
[0011] Step 4: Input the 3D carotid CTA image of the carotid artery to be predicted and the given seed point;
[0012] Step 5: Based on the seed point, extract the patch block centered on the seed point;
[0013] Step 6: Load the pre-trained carotid artery central path prediction model and the pre-trained carotid artery segmentation model, iteratively track the carotid artery central path based on the patch block centered on the seed point and the pre-trained carotid artery central path prediction model, and use the pre-trained carotid artery segmentation model to complete the carotid artery segmentation during the carotid artery central path tracking process.
[0014] Furthermore, according to the carotid artery extraction method based on convolutional neural network, the patch blocks include two types: non-offset patch blocks and offset patch blocks; the non-offset patch block extraction method is to extract the patch blocks of required size in the original three-dimensional carotid artery CTA image and the segmentation label with the point on the central path of the carotid artery as the center; the offset patch block extraction method is: obtain a random value that obeys the three-dimensional normal distribution as a random offset value, add the random offset value to the point on the central path of the carotid artery, obtain the point of the offset central path, and then extract the patch blocks of required size in the original three-dimensional carotid artery CTA image and the segmentation label with the point on the offset central path obtained as the center.
[0015] Furthermore, according to the carotid artery extraction method based on convolutional neural network, the carotid artery center path prediction model is composed of a first three-dimensional convolution operation, a first attention mechanism, several convolution blocks, a second three-dimensional convolution operation, a second attention mechanism, and an average pooling operation in sequence.
[0016] Further, according to the carotid artery extraction method based on convolutional neural network, the training of the carotid artery central path prediction model using the central path prediction training data set includes the following steps:
[0017] Step 2.2.1: Define the patch blocks in the center path prediction training dataset as a three-dimensional feature array of d×w×h according to their size, expressed as: X(d×w×h), where the three dimensions d, w, and h represent the feature map size of X, which are depth, height, and width respectively;
[0018] Step 2.2.2: Perform feature extraction on X to obtain feature map M1[(d-2)×(w-2)×(h-2)];
[0019] The process of feature extraction for X is as follows: first, the input X undergoes the first three-dimensional convolution operation with a convolution kernel size of 3×3×3, data normalization operation, and activation operation to obtain a feature map of size [(d-2)×(w-2)×(h-2)] Then After the first attention mechanism, the feature map M1[(d-2)×(w-2)×(h-2)] is obtained;
[0020] Step 2.2.2: Extract features from the feature map M1, change the size of M1 through several convolutional block operations, and finally obtain a feature map M5 of size [(d-16)×(w-16)×(h-16)];
[0021] Step 2.2.3: Perform feature extraction on M5 through the second three-dimensional convolution operation;
[0022] The process of extracting features from M5 through the second three-dimensional convolution operation is as follows: first, the input M5 undergoes a second three-dimensional convolution operation with a convolution kernel size of 1×1×1, a data normalization operation, and an activation operation to obtain a feature map. Then After the channel and spatial attention mechanism, the feature map M6 is obtained;
[0023] Step 2.2.4: Perform an average pooling operation on M6 to obtain the output Y. First, predefine a set of direction vectors N as the possible directions of the carotid artery at each point on the central path of the carotid artery. Classify the directions of the carotid artery at each point on the central path of the carotid artery according to Y, and regress the carotid artery radius at each point on the central path of the carotid artery. Use the direction vector in N that is closest to Y as the direction of the carotid artery at each point on the central path of the carotid artery.
[0024] Furthermore, according to the carotid artery extraction method based on convolutional neural network, there are 4 convolution blocks B1, B2, B3 and B4; each convolution block is composed of 1 dilated convolution layer, 1 3×3×3 convolution layer, spatial and channel attention mechanism and a residual layer with dilated convolution, wherein the dilation rates in the second and third convolution blocks B2 and B3 are 2 and 4 respectively; M1 is sequentially subjected to the 4 convolution block operations to obtain M2[(d-4)×(w-4)×(h-4)], M3[(d-8)×(w-8)×(h-8)], M4[(d-16)×(w-16)×(h-16)], and finally the feature map M5[(d-16)×(w-16)×(h-16)].
[0025] Further, according to the carotid artery extraction method based on convolutional neural network, the specific process of obtaining M5 by passing M1 through the four convolution modules includes:
[0026] (a) In the convolution block B1, M1 is serially subjected to a 3×3×3 dilated convolution operation with a dilated convolution rate of 1 and a 3×3×3 convolution operation to change the size of the feature map to obtain feature map P1. P1 is subjected to the channel and spatial attention mechanism to obtain feature map S1. M1 is subjected to a residual layer with dilated convolution to obtain feature map S1. Finally Add to S1 to get M2;
[0027] (b) In convolution block B2, M2 is serially processed through a 3×3×3 dilated convolution operation with a dilated convolution rate of 2 and a 3×3×3 convolution operation to change the size of the feature map to obtain feature map P2. P2 is processed through the channel and spatial attention mechanism to obtain feature map S2. M2 is processed through a residual layer with dilated convolution to obtain feature map Finally Add to S2 to get M3;
[0028] (c) In convolution block B3, M3 is serially processed through a 3×3×3 dilated convolution operation with a dilated convolution rate of 4 and a 3×3×3 convolution operation to change the size of the feature map, and feature map P3 is obtained. P3 is processed through the channel and spatial attention mechanism to obtain feature map S3, and M3 is processed through a residual layer with dilated convolution to obtain feature map Finally Adding it to S3 gives M4;
[0029] (d) In convolution block B4, M4 is serially processed through a 3×3×3 dilated convolution operation with a dilated convolution rate of 1 and a 3×3×3 convolution operation to change the size of the feature map to obtain feature map P4. P4 is processed through the channel and spatial attention mechanism to obtain feature map S4. M4 is processed through a residual layer with dilated convolution to obtain feature map Finally Add it to S4 to get the final feature map M5.
[0030] Furthermore, according to the carotid artery extraction method based on convolutional neural network, a V-Net network based on encoder-decoder is determined as a carotid artery segmentation model, wherein one side of the V-Net network is composed of an encoder and a downsampling layer, and the other side is composed of a decoder and an upsampling layer.
[0031] Furthermore, according to the carotid artery extraction method based on convolutional neural network, the patch blocks in the segmentation training data set are processed, and the carotid artery segmentation model is trained using the processed segmentation training data set; the processing of the patch blocks in the segmentation training data set includes normalizing and upsampling the patch blocks extracted from the original three-dimensional carotid artery CTA image, and corroding and upsampling the patch blocks extracted from the original segmentation label.
[0032] Further, according to the carotid artery extraction method based on convolutional neural network, step 6 includes:
[0033] Load the pre-trained carotid artery center path prediction model to predict the direction and radius of the local area of the carotid artery at the seed point; based on the predicted direction information, move the radius length from the seed point to obtain the next point on the carotid artery center path, then extract the patch block where the point is located, and continue to predict the direction and radius information of the carotid artery at this point, and so on, until the complete carotid artery is iteratively tracked;
[0034] In the process of tracing the center path of the carotid artery, whenever a patch block containing a center point is obtained, the patch block is used to segment the carotid artery, thereby achieving segmentation of the carotid artery in the process of tracing the center path of the carotid artery.
[0035] In general, the above technical scheme conceived by the present invention has the following beneficial effects compared with the prior art: a network structure based on a convolutional neural network is designed to solve the problem of interference in carotid artery extraction caused by the unevenness and complexity of the carotid artery structure and the poor contrast of the carotid artery, thereby improving the accuracy of the network in tracking the central path of the carotid artery; the defect of difficulty in extracting the carotid artery caused by the proximity of the carotid artery to the bone tissue is overcome, and for the first time, the segmentation of the carotid artery is completed while tracking the central path of the carotid artery, thereby realizing the extraction of the carotid artery structure. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 Schematic diagram of the core idea of the carotid artery extraction method based on convolutional neural network in this embodiment;
[0037] Figure 2 Schematic diagram of the process of the carotid artery extraction method based on convolutional neural network in this embodiment;
[0038] Figure 3 (a) is a schematic diagram of the structure of the original carotid artery data set in this embodiment; (b) is a schematic diagram of the carotid artery center path prediction label; (c) is a schematic diagram of the carotid artery segmentation label;
[0039] Figure 4 Schematic diagram of the structure of the carotid artery central path prediction model in this embodiment;
[0040] Figure 5 A schematic diagram of the network structure for implementing carotid artery segmentation in this embodiment;
[0041] Figure 6 The carotid artery center path result diagram obtained in this embodiment;
[0042] Figure 7 FIG. 4 is a diagram showing the carotid artery segmentation result obtained in this embodiment. DETAILED DESCRIPTION
[0043] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The specific embodiments described here are only used to explain the present invention and are not used to limit the present invention.
[0044] Figure 1 Schematic diagram of the core idea of the carotid artery extraction method based on convolutional neural network in this embodiment. Figure 2 is a flow chart of a method for extracting carotid arteries based on a convolutional neural network in this embodiment. The method is used to process a carotid artery CTA image and extract the carotid artery in the image. Figure 1 and Figure 2 As shown, the carotid artery extraction method based on convolutional neural network includes the following steps:
[0045] Step 1: Patch blocks are extracted from each 3D carotid CTA image and its corresponding segmentation label in the original data set, and a segmentation training data set is constructed from the extracted patch blocks, and a center path prediction training data set is constructed from the patch blocks extracted from each 3D carotid CTA image and the carotid center path prediction label;
[0046] In this embodiment, the original data set is as follows Figure 3 (a) shows a total of 135 sets of carotid CTA images collected by ourselves and their corresponding carotid center path prediction labels and segmentation labels. Figure 3 (b) shows a schematic diagram of the predicted label of the carotid artery center path; Figure 3 (c) shows a schematic diagram of the carotid artery segmentation label. Then, the 135 groups of data in the original data set are divided into original data sets for training, verification and testing according to the distribution relationship of 8:1:1. The carotid artery center path prediction label is the coordinate value (x, y, z) and radius value corresponding to the point on the center line corresponding to the four arteries. The segmentation label is the mask image corresponding to the carotid artery.
[0047] In the carotid CTA image, the carotid artery occupies only a small part of the entire image, and most of it belongs to the background area. Therefore, in order to solve this class imbalance problem, a patch block of size ω×ω×ω containing rich carotid artery information is extracted from the three-dimensional carotid CTA image of size T×H×W as training data; the patch block includes a non-offset patch block P no_offset and the offset patch P offset Two. The non-offset patch block P no_offsetThe extraction method is to extract a patch block of size ω×ω×ω in the original three-dimensional carotid CTA image with a point on the carotid central path as the center. The purpose of extracting the offset patch block is to obtain samples that deviate from the carotid central path, so as to prevent the carotid central path prediction model from providing an erroneous prediction when the iteratively tracked carotid central path deviates from the actual central path, thereby causing the iteratively tracked carotid central path to deviate. Therefore, the present invention uses a random deviation value that obeys a three-dimensional normal distribution to further obtain a patch block P of size ω×ω×ω centered on a point on the deviated carotid central path. offset Specifically, a random value that obeys a three-dimensional normal distribution is obtained as a random offset value, and the point on the carotid artery center path is added with the random offset value to obtain the point of the offset center path, and then a patch block P of size ω×ω×ω is extracted with the obtained point of the offset center path as the center. offset .
[0048] Since the training data is a patch block extracted from the original 3D carotid CTA image in the dataset, in order to maintain the consistency between the segmentation label and the training data, it is necessary to extract the patch block from the segmentation label using the same strategy as the original 3D carotid CTA image, specifically including extracting the non-shifted patch block M from the segmentation label using the same method as the non-shifted patch block from the original 3D carotid CTA image. no_offset The offset patch M is extracted from the segmentation label using the same method as that used to extract the offset patch in the original 3D carotid CTA image. offset .
[0049] After extracting patches according to the above method for the original data sets used for training, validation and testing, the extracted patches constitute the training data set, validation data set and test data set for segmentation respectively, and the patches extracted from each three-dimensional carotid CTA image and the carotid central path prediction labels constitute the training data set, validation data set and test data set for central path prediction.
[0050] Step 2: Establish a carotid artery central path prediction model, and use the central path prediction training data set to train the carotid artery central path prediction model.
[0051] The core idea of this step is to define the patch block as a three-dimensional feature array of d×w×h according to its size, expressed as: X(d×w×h). In the preferred embodiment, the size of the patch block is defined as a three-dimensional feature array of 19×19×19, expressed as X(19×19×19), and the three dimensions represent the size of the feature map of X, namely depth, height and width; establish a carotid artery center path prediction model, Figure 4 is a structural diagram of the carotid artery central path prediction model, such as Figure 4As shown, in this embodiment, the carotid artery center path prediction model is composed of a first three-dimensional convolution operation, an attention mechanism, a plurality of convolution blocks, a second three-dimensional convolution operation, an attention mechanism, and an average pooling operation in sequence; X is input into the established model to perform feature extraction on X, and the feature extraction operation is composed of a serial convolution operation and a plurality of convolution block operations, and a set of model training parameters are finally obtained by using the training data set and the validation data set to save and obtain a trained carotid artery center path prediction model. This step specifically includes the following steps:
[0052] Step 2.1: Establish a carotid artery center path prediction model, which is composed of a first three-dimensional convolution operation, a first attention mechanism, several convolution blocks, a second three-dimensional convolution operation, a second attention mechanism, and an average pooling operation in sequence; the attention mechanism is a spatial and channel attention mechanism, including a channel attention module and a spatial attention module.
[0053] Step 2.2: Use the central path prediction training data set to train a carotid artery central path prediction model to obtain a pre-trained carotid artery central path prediction model.
[0054] Step 2.2.1: Define the patch block in the center path prediction training data set as a three-dimensional feature array of d×w×h according to its size, expressed as: X(d×w×h), where the three dimensions d, w, and h represent the size of the feature map of X, which are depth, height, and width, respectively; in a preferred embodiment, the size of the patch block is defined as a three-dimensional feature array of 19×19×19, expressed as X(19×19×19).
[0055] Step 2.2.2: Perform feature extraction on X to obtain feature map M1[(d-2)×(w-2)×(h-2)];
[0056] In the preferred embodiment, the detailed process of feature extraction for X is as follows: first, the input X is subjected to a first three-dimensional convolution operation (in the preferred embodiment, the convolution kernel size is 3×3×3), a data normalization operation, and an activation operation to obtain a feature map of size 17×17×17. Then After the first attention mechanism, higher weights can be given to the concerned channels and regions, resulting in a 17×17×17 feature map M1 with a receptive field of 3.
[0057] Step 2.2.2: Extract features from the feature map M1 [(d-2) × (w-2) × (h-2)], and change the size of M1 through several convolutional block operations to obtain M2 [(d-4) × (w-4) × (h-4)], M3 [(d-8) × (w-8) × (h-8)], M4 [(d-16) × (w-16) × (h-16)], and finally obtain the feature map M5 [(d-16) × (w-16) × (h-16)];
[0058] In the preferred embodiment, there are 4 convolution blocks B1, B2, B3 and B4 for better feature representation. Each convolution block consists of 1 dilated convolution layer, 1 3×3×3 convolution layer, spatial and channel attention mechanisms, and a residual layer with dilated convolution. The dilation rates in the second and third convolution blocks B2 and B3 are 2 and 4 respectively, which can further retain multi-scale features while expanding the receptive field. The convolution block does not use a large convolution kernel to extract features, so when the training data is very limited, it will lead to overfitting. Each convolution layer is followed by data normalization and activation operations to minimize overfitting. At the same time, channel and spatial attention mechanisms are added to each convolution block, which can give more weight to the features and regions of interest. Finally, the features passed through the residual layer and convolution layer are added and input into the subsequent convolution block.
[0059] In a preferred embodiment, the specific process of obtaining M5 by passing M1 through four convolution modules includes: sequentially performing the following steps (a) to (d) to obtain feature maps M2, M3, M4, and M5 respectively:
[0060] (a) In the convolution block B1, M1 is serially subjected to a 3×3×3 dilated convolution operation with a dilated convolution rate of 1 and a 3×3×3 convolution operation to change the size of the feature map, and the feature map P1 (15×15×15) is obtained. At this time, the receptive field becomes 5. P1 is passed through the channel and spatial attention mechanism to give greater weight to the focused features and regions, and the feature map S1 (15×15×15) is obtained. M1 is passed through a residual layer with dilated convolution (the dilated convolution rate and convolution sum size are always consistent with the dilated convolution operation of this step) to obtain Finally Add it to S1 to get M2 (15×15×15) which is input into convolution block B2;
[0061] (b) In convolution block B2, M2 is serially processed through a 3×3×3 dilated convolution operation with a dilated convolution rate of 2 and a 3×3×3 convolution operation to change the size of the feature map, obtaining P2 (11×11×11). At this time, the receptive field becomes 9. P2 is processed through the channel and spatial attention mechanism to give greater weight to the focused features and regions, obtaining S2 (11×11×11). M2 is processed through a residual layer with dilated convolution (the dilated convolution rate and convolution sum size are always consistent with the dilated convolution operation of this step) to obtain Finally Add it to S2 to get M3 (11×11×11) which is input into convolution block B3;
[0062] (c) In convolution block B3, M3 is serially processed through a 3×3×3 dilated convolution operation with a dilated convolution rate of 4 and a 3×3×3 convolution operation to change the size of the feature map, obtaining P3 (3×3×3). At this time, the receptive field becomes 17. P3 is processed through the channel and spatial attention mechanism to give greater weight to the focused features and regions, obtaining S3 (3×3×3). M3 is processed through a residual layer with dilated convolution (the dilated convolution rate and convolution sum size are always consistent with the dilated convolution operation of this step) to obtain 3×3), and finally Add it to S3 to get M4 (3×3×3) which is input into convolution block B4;
[0063] (d) In convolution block B4, M4 is serially processed through a 3×3×3 dilated convolution operation with a dilated convolution rate of 1 and a 3×3×3 convolution operation to change the size of the feature map, obtaining P4 (3×3×3). At this time, the receptive field becomes 19. P4 is processed through the channel and spatial attention mechanism to give greater weight to the focused features and regions, obtaining S4 (3×3×3). M4 is processed through a residual layer with dilated convolution (the dilated convolution rate and convolution sum size are always consistent with the dilated convolution operation of this step) to obtain Finally Add to S4 to get the final feature map M5 (3×3×3);
[0064] Step 2.2.3: Perform feature extraction on M5 through a second three-dimensional convolution operation.
[0065] In the preferred embodiment, the detailed process of extracting features from M5 through the second three-dimensional convolution operation is as follows: first, the input M5 is subjected to the second three-dimensional convolution operation (in the preferred embodiment, the convolution kernel size in the three-dimensional convolution operation of this step is 1×1×1), data normalization operation and activation operation to obtain a feature map Then Through the channel and spatial attention mechanism, higher weights can be given to the focused channels and regions, and the feature map M6 (3×3×3) is obtained.
[0066] Step 2.2.4: Perform an average pooling operation on M6 to obtain the output Y (1×1×1). First, predefine a set of direction vectors N as the possible directions of the carotid artery at each point on the central path of the carotid artery. According to Y, classify the directions of the carotid artery at each point on the central path of the carotid artery and regress the carotid artery radius at each point on the central path of the carotid artery. Use the direction vector in N that is closest to Y as the direction of the carotid artery at each point on the central path of the carotid artery.
[0067] In a preferred embodiment, 500 direction vectors are predefined to form a direction vector set N, and M6 is subjected to an average pooling operation to obtain 501 1×1×1 vectors, of which the first 500 are used to classify the direction of the carotid artery at each point on the carotid artery center path, and the posterior probability of each direction is calculated. The direction with the largest posterior probability is selected as the direction at the center point of the current patch block, and the last one is used for the regression of the carotid artery radius at the center point of the current patch block.
[0068] Step 3: Determine and use the segmentation training data set to train a carotid artery segmentation model;
[0069] Step 3.1: Determine the carotid artery segmentation model;
[0070] Those skilled in the art can easily obtain a V-Net network or a U-Net network as a carotid artery segmentation model. Figure 5 : is a structural diagram of the carotid artery segmentation model of this embodiment. In this embodiment, the existing encoder-decoder based V-Net network is used to obtain the segmentation result of the carotid artery. One side of the network consists of an encoder and a downsampling layer, which is used to extract features from the patch block and obtain a lower resolution feature map; the other side of the network consists of a decoder and an upsampling layer, which is used to extract features in the patch block and expand the obtained lower resolution feature map. At the same time, a jump connection is used to forward the features extracted from the encoder stage to the decoder stage. In this way, the fine-grained details lost in the downsampling layer of the encoder stage can be collected, and the segmentation result of the carotid artery can be finally obtained.
[0071] Step 3.2: Processing the patch blocks in the segmentation training data set: performing normalization and upsampling operations on the patch blocks extracted from the original three-dimensional carotid artery CTA image, and performing erosion and upsampling operations on the patch blocks extracted from the original segmentation label;
[0072] In the patch block extracted from the three-dimensional carotid artery CTA image, the upper and lower limits of the grayscale value of the carotid artery are set to between 70HU and 1000HU, the image is normalized, and the image is upsampled to increase the size of the patch block. In a preferred embodiment, the patch block is upsampled from 19×19×19 to 38×38×38;
[0073] The patch blocks extracted from the segmentation labels are eroded to optimize the labels. At the same time, in order to keep the size of the segmentation label patches consistent with that of the image patches, the segmentation label patches are also upsampled from 19×19×19 to 38×38×38.
[0074] Step 3.3: Use the processed segmentation training data set to train the carotid artery segmentation model to obtain a pre-trained carotid artery segmentation model.
[0075] Step 4: Input the 3D carotid CTA image of the carotid artery to be predicted and the given seed point;
[0076] The given seed points are three points manually selected on each carotid artery in the three-dimensional carotid CTA image of the carotid artery to be predicted as seed points. In a preferred embodiment, the three points are the starting point, the proximal end point (a point close to the starting point, such as a point 1 / 3 away from the starting point of the carotid artery) and the distal end point (a point far away from the starting point, such as a point 2 / 3 away from the starting point) of the carotid artery.
[0077] Step 5: Based on the seed point, extract the patch block centered on the seed point;
[0078] Step 6: Load the pre-trained carotid artery central path prediction model and the pre-trained carotid artery segmentation model, iteratively track the carotid artery central path based on the patch block centered on the seed point and the pre-trained carotid artery central path prediction model, and use the pre-trained carotid artery segmentation model to complete the carotid artery segmentation during the carotid artery central path tracking process.
[0079] Load the pre-trained carotid artery center path prediction model to predict the direction and radius of the local area of the carotid artery at the seed point. Based on the predicted direction information, move the radius length from the seed point to get the next point on the carotid artery center path, then extract the patch where the point is located, and continue to predict the direction and radius information of the carotid artery at this point, and so on, until the complete carotid artery is iteratively tracked, such as Figure 6In this embodiment, whether to stop tracking is determined by calculating the normalized entropy of the posterior probability distribution of the direction classification. First, a threshold is defined. When the iterative tracking reaches the end of the carotid artery or the iterative tracking is close to a point on the tracked center line, a higher normalized entropy will be generated. When the normalized entropy is greater than the given threshold, the tracking will be stopped.
[0080] In the process of tracing the center path of the carotid artery, whenever a patch block containing the center point is obtained, the patch block is normalized and up-sampled as described above, and then the carotid artery is segmented. Thus, the carotid artery is segmented in the process of tracing the center path of the carotid artery. The segmentation result is shown in FIG. Figure 7 shown.
[0081] The method of the present invention was tested on a carotid artery CTA dataset and compared with the manual extraction results of three experts. The quantitative analysis results showed that the extraction results obtained by the method of the present invention were consistent with the extraction results manually calibrated by three experts, and the error evaluation was also within the error range of manual calibration. It also had a good extraction effect on abnormal diseased blood vessels and could meet the needs of actual clinical applications.
[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments can still be modified, or some or all of the technical features therein can be replaced by equivalents. Therefore, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.
Claims
1. A carotid artery extraction method based on convolutional neural network, It is characterized in that The method comprises the following steps: Step 1: Extract patches from each 3D carotid CTA image and its corresponding segmentation label in the original data set, construct a segmentation training data set from the extracted patches, and construct a center path prediction training data set from the patches extracted from each 3D carotid CTA image and the carotid center path prediction label; Step 2: Establish a carotid artery central path prediction model, and use the central path prediction training data set to train the carotid artery central path prediction model; Step 3: Determine and use the segmentation training data set to train a carotid artery segmentation model; Step 4: Input the 3D carotid CTA image of the carotid artery to be predicted and the given seed point; Step 5: Based on the seed point, extract the patch block centered on the seed point; Step 6: Load the pre-trained carotid artery central path prediction model and the pre-trained carotid artery segmentation model, iteratively track the carotid artery central path based on the patch block centered on the seed point and the pre-trained carotid artery central path prediction model, and use the pre-trained carotid artery segmentation model to complete the carotid artery segmentation in the process of carotid artery central path tracking; The patch blocks include two types: non-offset patch blocks and offset patch blocks; the non-offset patch block extraction method is to extract a patch block of a desired size from the original three-dimensional carotid CTA image and the segmentation label with a point on the carotid central path as the center; the offset patch block extraction method is to obtain a random value that obeys a three-dimensional normal distribution as a random offset value, add the random offset value to the point on the carotid central path to obtain a point on the offset central path, and then extract a patch block of a desired size from the original three-dimensional carotid CTA image and the segmentation label with the obtained point on the offset central path as the center; The method of training the carotid artery central path prediction model using the central path prediction training data set comprises the following steps: Step 2.2.1: Define the patch blocks in the center path prediction training dataset as a three-dimensional feature array of d×w×h according to their size, expressed as: X(d×w×h), where the three dimensions d, w, and h represent the feature map size of X, which are depth, height, and width respectively; Step 2.2.2: Perform feature extraction on X to obtain feature map M 1 [(d-2)×(w-2)×(h-2)]; The process of feature extraction for X is as follows: first, the input X undergoes the first three-dimensional convolution operation with a convolution kernel size of 3×3×3, data normalization operation, and activation operation to obtain a feature map of size [(d-2)×(w-2)×(h-2)] Then After the first attention mechanism, the feature map M is obtained 1 [(d-2)×(w-2)×(h-2)]; Step 2.2.2: For the feature map M 1 Perform feature extraction and convert M 1 After several convolution blocks, M is changed 1 The size of the feature map M is [(d-16)×(w-16)×(h-16)] 5 ; Step 2.2.3: For M 5 Feature extraction is performed through a second three-dimensional convolution operation; To M 5 The process of feature extraction through the second three-dimensional convolution operation is as follows: first, the input M 5 After the second three-dimensional convolution operation with a convolution kernel size of 1×1×1, data normalization operation and activation operation, the feature map is obtained. Then After the channel and spatial attention mechanism, the feature map M is obtained 6 ; Step 2.2.4: M 6 The output Y is obtained through the average pooling operation. First, a set of direction vectors N is predefined as the possible directions of the carotid artery at each point on the central path of the carotid artery. The directions of the carotid artery at each point on the central path of the carotid artery are classified according to Y, and the carotid artery radius at each point on the central path of the carotid artery is regressed. The direction vector in N that is closest to Y is used as the direction of the carotid artery at each point on the central path of the carotid artery.
2. The carotid artery extraction method based on convolutional neural network according to claim 1, It is characterized in that The carotid artery center path prediction model is composed of a first three-dimensional convolution operation, a first attention mechanism, a plurality of convolution blocks, a second three-dimensional convolution operation, a second attention mechanism, and an average pooling operation in sequence.
3. The carotid artery extraction method based on convolutional neural network according to claim 1, It is characterized in that There are 4 convolution blocks B in total 1 , B 2 , B 3 and B 4 ; Each convolutional block consists of a dilated convolutional layer, a 3×3×3 convolutional layer, spatial and channel attention mechanisms, and a residual layer with dilated convolutions. The second and third convolutional blocks B 2 and B 3 The void ratios in are 2 and 4 respectively; 1 After the four convolution blocks are operated in sequence, M 2 [(d-4)×(w-4)×(h-4)], M 3 [(d-8)×(w-8)×(h-8)], M 4 [(d-16)×(w-16)×(h-16)], and finally obtain the feature map M 5 [(d-16)×9w-16)×(h-16)].
4. The method for extracting carotid arteries based on a convolutional neural network according to claim 3, It is characterized in that The M 1 After the four convolution blocks, we get M 5 The specific process includes: (a) In convolution block B 1 In the 1 The feature map P is obtained by serially changing the size of the feature map through a 3×3×3 dilated convolution operation with a dilated convolution rate of 1 and a 3×3×3 convolution operation. 1 , P 1 The feature map S is obtained through the channel and spatial attention mechanism 1 , M 1 The feature map is obtained through the residual layer with dilated convolution Finally and S 1 Add together to get M 2 ; (b) In convolution block B 2 In the 2 The feature map P is obtained by serially changing the size of the feature map through a 3×3×3 dilated convolution operation with a dilated convolution rate of 2 and a 3×3×3 convolution operation. 2 , P 2 The feature map S is obtained through the channel and spatial attention mechanism 2 , M 2 The feature map is obtained through the residual layer with dilated convolution Finally and S 2 Add together to get M 3 ; (c) In convolution block B 3 In the 3 The feature map P is obtained by serially changing the size of the feature map through a 3×3×3 dilated convolution operation with a dilated convolution rate of 4 and a 3×3×3 convolution operation. 3 P 3 The feature map S is obtained through the channel and spatial attention mechanism 3 , M 3 The feature map is obtained through the residual layer with dilated convolution Finally and S 3 Add together to get M 4 ; (d) In convolution block B 4 In the 4 The feature map P is obtained by serially changing the size of the feature map through a 3×3×3 dilated convolution operation with a dilated convolution rate of 1 and a 3×3×3 convolution operation. 4 , P 4 The feature map S is obtained through the channel and spatial attention mechanism 4 , M 4 The feature map is obtained through the residual layer with dilated convolution Finally and S 4 Add together to get the final feature map M 5 .
5. The carotid artery extraction method based on convolutional neural network according to claim 1, It is characterized in that The encoder-decoder based V-Net network is determined as the carotid artery segmentation model, wherein one side of the V-Net network is composed of an encoder and a downsampling layer, and the other side is composed of a decoder and an upsampling layer.
6. The method for extracting carotid arteries based on a convolutional neural network according to claim 1, It is characterized in that The patch blocks in the segmentation training data set are processed, and the processed segmentation training data set is used to train a carotid artery segmentation model; the processing of the patch blocks in the segmentation training data set includes normalizing and upsampling the patch blocks extracted from the original three-dimensional carotid artery CTA image, and corroding and upsampling the patch blocks extracted from the original segmentation label.
7. The method for extracting carotid arteries based on a convolutional neural network according to claim 1, It is characterized in that The step 6 comprises: Load the pre-trained carotid artery center path prediction model to predict the direction and radius of the local area of the carotid artery at the seed point; based on the predicted direction information, move the radius length from the seed point to obtain the next point on the carotid artery center path, then extract the patch block where the point is located, and continue to predict the direction and radius information of the carotid artery at this point, and so on, until the complete carotid artery is iteratively tracked; In the process of tracing the center path of the carotid artery, whenever a patch block containing a center point is obtained, the patch block is used to segment the carotid artery, thereby achieving segmentation of the carotid artery in the process of tracing the center path of the carotid artery.
