Method for coronary plaque segmentation and calcified plaque scoring

Abstract

The embodiment of the invention discloses a method for coronary artery plaque segmentation and calcified plaque scoring, and the method comprises the steps: collecting a coronary artery blood vessel optical coherence tomography (IVOCT) image data set of a coronary heart disease patient, and carrying out the data enhancement; inputting the IVOCT image data set after data enhancement into a segmentation model (PMS-Net) to obtain a plaque segmentation result of each image; calculating the thickness, angle and length of the calcified plaque based on the plaque segmentation result of each image; according to a calcification integration system based on optical coherence tomography, corresponding scores are given to the calculated thickness, angle and length of the calcified plaque. According to the technical scheme of the embodiment of the invention, the problems of difficult segmentation and inaccurate target segmentation in the existing coronary plaque segmentation technology are solved, the end-to-end target and background classification of the IVOCT image is efficiently realized, and guidance is provided in the operation through the calcification integral.

Description

Technical Field

[0001] The present invention relates to a method for calcification scoring of coronary artery calcified plaque segmentation machine Background Art

[0002] In recent decades, cardiovascular disease has had an increasingly significant impact on human health and has shown an increasing trend year by year. More than 2 million people undergo percutaneous coronary intervention (PCI) each year to treat coronary artery disease by implanting stents. However, coronary artery calcification may lead to insufficient stent expansion, poor adhesion of the stent to the vessel wall, and stent failure. Therefore, accurate assessment of coronary artery calcification is crucial for planning PCI strategies.

[0003] Intravascular optical coherence tomography (IVOCT) is a powerful imaging technique used to evaluate atherosclerotic lesions in coronary artery disease and percutaneous coronary intervention (PCI). Compared with coronary angiography, computed tomography, or magnetic resonance imaging, IVOCT has higher resolution and can better assess the vascular lumen boundary and plaque phenotype. Although intravascular ultrasound (IVUS) can also be used to evaluate stent implantation and plaque characteristics, acoustic shadowing occurs in calcified lesions and cannot accurately assess the area and thickness of calcified plaques. In contrast, IVOCT can penetrate plaques and may be the best clinical imaging method for characterizing superficial plaques.

[0004] RicoJimenez et al. applied a linear combination model to evaluate each A-line category through a least squares optimization strategy. In this study, the optical attenuation changes (μt) between lesion plaques observed in the A-line profile were considered as the key features for identifying plaques. The classification and quantitative evaluation of the fiber layer based on the A-line characteristics and the boundary detection of calcified plaques were also discussed. Unlike focusing on the characteristics of each A-line, Ren et al. focused on the statistical information of the correlation between adjacent A-lines in the local area and used the random forest method to create a local multi-layer (3-layer) model to extract the required features. Convolutional Neural Network (CNN) has been applied to improve the detection of vascular lesion plaques due to its excellent performance in image recognition and classification tasks. Abdolmanafi et al. used pre-trained CNN to extract tissue features and conduct automatic classification research on coronary artery layers. Kolluru et al. used CNN and artificial neural network (ANN) to identify the type of each A-line as a basic unit. For the study of classifying and labeling IVOCT datasets at the slice level, Gessert et al. studied the recognition capabilities of two constructed CNN models and applied transfer learning methods to improve these models. Ren et al. designed a neural network with a VGG-like structure to study the effects of texture information and the number of channels on the accuracy of lesion area recognition in the circumferential dimension. Semantic segmentation of vascular lesions, especially semantic segmentation based on deep learning, is considered to be an important technology for analyzing the distribution and type of lesion plaques in the future, which can quickly and reliably help clinicians analyze and label lesion areas in human arteries.

[0005] Many previous studies have generated classification results based on A-lines or images, but doctors need to know the exact area, depth, thickness and volume of calcified plaques, as well as the extent to which calcified plaques affect stent implantation. Therefore, a method that can accurately detect, segment and quantify calcified plaques in all IVOCT images is needed. Summary of the invention

[0006] In view of this, it is necessary to provide a method for segmenting and scoring coronary artery calcified plaques.

[0007] A method for coronary artery plaque segmentation and calcified plaque scoring, the method comprising the following steps: A. collecting coronary artery vascular optical coherence tomography (IVOCT) image data sets of patients with coronary heart disease and then performing data enhancement; B. inputting the data-enhanced IVOCT image data sets into a plaque segmentation model (PMS-Net) to obtain the plaque segmentation results of each image; C. calculating the thickness, angle and length of the calcified plaque based on the plaque segmentation results of each image; D. assigning corresponding scores to the calculated calcified plaque thickness, angle and length according to a calcification scoring system based on optical coherence tomography.

[0008] Wherein, the step A specifically includes:

[0009] For each coronary artery image to be segmented, first determine the center point of the image;

[0010] The acquired coronary artery image to be segmented is cropped based on the center point. Specifically, the original 1024×1024 image is cropped to a size of 640×640 by cropping the image into a smaller area;

[0011] The IVOCT dataset is augmented by rotating 90° and 270°.

[0012] The step B specifically comprises:

[0013] The PMS-Net encoder performs multi-scale fusion through skip connections and uses the local attention mechanism to strengthen the focus on important information of different categories;

[0014] The middle layer of PMS-Net uses hole convolution kernels of different step lengths to expand the receptive field and capture broader contextual information;

[0015] The PMS-Net decoder uses skip connections to encode different features of the same scale and fuses them, and then uses the local attention mechanism to extract feature information of the same scale;

[0016] The PMS-Net network is used to segment medical images and obtain segmentation results.

[0017] The step C specifically comprises:

[0018] For each segmented coronary artery image, the coordinates of all points in the coronary artery are calculated, and the average value of the coordinates of all points is calculated as the centroid of the coronary artery;

[0019] In the segmented coronary artery images, the diameter of the ductal area was calculated as the scale;

[0020] Send out rays outward from the center of mass as the origin, find the maximum value of the calcified plaque pixel, and then multiply it by 0.9 and divide it by the catheter diameter;

[0021] Take the centroid as the fixed point, the two extreme points of the calcified plaque, and the centroid connects the two extreme points of the calcified plaque to form two line segments, and the angle formed by the two line segments is taken as the angle value of the calcified plaque;

[0022] Find the coronary artery images where the calcified plaque appears for the first time and the last time, calculate the number of intermediate continuous frames and multiply it by 0.2, and use the result as the length value of the calcified plaque.

[0023] The step D specifically comprises:

[0024] In the calcification scoring system of optical coherence tomography, 2 points are given if the angle of the calcified plaque is greater than 180°, 1 point is given if the thickness of the calcified plaque is greater than 0.5 mm, and 1 point is given if the length of the calcified plaque is greater than 5 mm;

[0025] The consecutive frames where calcified plaques appeared for the first and last time were regarded as a lesion. The angle, thickness and length of calcified plaques in these consecutive coronary artery images were counted and the corresponding scores were given according to the calcification scoring system of optical coherence tomography.

[0026] The present invention can eliminate human subjectivity and avoid diagnostic errors caused by differences in individual knowledge and experience, thereby improving diagnostic accuracy and work efficiency; when faced with a relatively small scale of training data, a data enhancement method is used to solve the problem of overfitting caused by insufficient training data in the training phase of the convolutional neural network; for traditional methods such as manually designed feature extractors and classifiers, since they cannot handle artifact noise well, they cannot accurately segment IVOCT images. The present application constructs a PMS-Net coronary plaque extraction network to accurately segment the above-mentioned IVOCT images with artifacts, and can more efficiently achieve end-to-end pixel-level classification of targets and backgrounds in coronary IVOCT images. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a flow chart of the coronary artery plaque segmentation and calcified plaque scoring method of the present invention;

[0028] Figure 2 A schematic diagram of an IVOCT image data set provided by an embodiment of the present invention;

[0029] Figure 3 A schematic diagram of the overall structure of the deep learning convolutional neural network PMS-Net provided in an embodiment of the present invention;

[0030] Figure 4A schematic diagram of the AFF module in the deep learning convolutional neural network PMS-Net provided in an embodiment of the present invention;

[0031] Figure 5 A schematic diagram of an EFF module in a deep learning convolutional neural network PMS-Net provided in an embodiment of the present invention;

[0032] Figure 6 A schematic diagram of a DFF module in a deep learning convolutional neural network PMS-Net provided in an embodiment of the present invention;

[0033] Figure 7 A schematic diagram of the ASPP module in the deep learning convolutional neural network PMS-Net provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0034] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0035] See also Figure 1 , which is a flowchart of an embodiment of the coronary artery calcified plaque segmentation and calcified plaque scoring method of the present invention.

[0036] Step S1, collect IVOCT image data set (please also refer to Figure 2 ), and the IVOCT image dataset is divided into a training set and a test set. Specifically:

[0037] This example recruited 277 patients during the experiment;

[0038] The images were acquired using an IVOCT catheter equipped with an SJM Fourier Domain OCT imaging system (Abbott), with a retraction speed of 18 mm / s, covering a distance of 54 mm, an interval of 0.1 mm between each frame, and an acquisition time of 3 seconds. The axial and spatial resolutions of the device were 12-15 μm and 9.9 × 9.9 μm, respectively. An independent electronic control system was used to control the retraction motor. The catheter emitted light to scan the vessel wall, received the reflected signal and converted it into an A line;

[0039] We choose to rotate 90° and 270° to perform data augmentation to expand sample diversity. Since the original image size is 1024×024, it is too large and contains too much background information, which is redundant information for the segmentation task, so we need to adjust the image size. After many experiments, we found that the best prediction results were obtained when the original image was adjusted to 640×640.

[0040] Of the 2517 IVOCT image datasets collected, 70% of the data were used as training sets, 10% of the data were used as validation sets, and 20% of the data were used as test sets.

[0041] Step S2, design the PMS-Net architecture using the extracted features useful for medical image segmentation (please also refer to Figure 3 ), and use the PMS-Net architecture to complete end-to-end pixel-level medical image segmentation. Specifically:

[0042] Step S21, the left half of the PMS-Net architecture is the encoder part, which performs multi-scale fusion through skip connections and uses the local attention mechanism to strengthen the focus on important information of different categories, including:

[0043] Step S211, the encoder network uses a four-stage module for extracting low-dimensional features to process input data;

[0044] Step S212, each stage includes an EFF module (please also refer to Figure 5 ) and an AFF module (see also Figure 4 ), apply different convolutional layers and maximum pooling operations to the original data features and the output features of the previous stage, so that the spatial resolution of the feature map is reduced by 1 / 2, the number of channels is doubled, the two input feature maps are spliced ​​through residual connection, and the local attention mechanism is used for feature fusion;

[0045] Step S22, the middle part of the PMS-Net architecture is the encoder and decoder connection part, and the connection part uses a hole convolution kernel of different step lengths to expand the receptive field, specifically including:

[0046] Step S221, the connection part expands the receptive field by using multiple dilated convolutions with different dilation rates in parallel (see also Figure 7 );

[0047] Step S222, the connection part consists of 5 branches, one of which uses a 3×3 convolution kernel, and the other 4 branches use 3×3 convolution kernels with different expansion rates, which are 6, 12, 18 and 24 respectively;

[0048] Step S223, the output of the dilated convolution is subjected to a 1×1 convolution to change the number of channels, generate a final feature map, and send it to the decoder part;

[0049] Step S23, the right half of the PMS-Net architecture is the decoder part, which uses jump connection to encode different features of the same scale and fuses them, and then uses the local attention mechanism to extract feature information of the same scale, which specifically includes:

[0050] Step S231, the encoder network uses a four-stage feature restoration module to fuse different features of different scales, and then uses a local attention mechanism to extract feature information of the same scale;

[0051] Step S232, each stage includes a DFF module (please also refer to Figure 6 ) and an AFF module (see also Figure 4 ), the inverted residual structure and 7×7 large kernel convolution are applied to the features of the same scale and the output features of the previous stage respectively, so that the spatial resolution of the feature map is increased by 2 times and the number of channels is halved. The two input feature maps are spliced ​​through residual connection, and the local attention mechanism is used for feature fusion.

[0052] Step S24, the custom loss function is:

[0053]

[0054] Among them, g i ∈G represents the real pixel, p i ∈P represents the predicted binary pixel, N represents the total number of pixels, G is the actual value, and P is the predicted value.

[0055] FL=-α(1-P t ) γ log(P t )

[0056] α i =1-r i

[0057] When P t When it is close to 1, the pixel prediction is more accurate, the discrimination is higher, and the weight is smaller, which allows the network to focus on pixels that are difficult to classify. i represents the percentage of pixels in each category, α i Represents the weight of each category, γ is a parameter that controls the degree of weight, and γ=2 in this paper.

[0058] loss = 0.6 × DL + 0.4 × FL

[0059] Among them, L and FL represent Dice loss and Focal loss respectively.

[0060] Step S25, using the encoder part, the middle layer and the decoder part of the PMS-Net architecture to segment the medical image, and the segmentation results obtained specifically include:

[0061] True positives (TP): the number of instances correctly classified as positive examples, that is, the number of instances (number of samples) that are actually positive examples and classified as positive examples by the classifier; False positives (FP): the number of instances incorrectly classified as positive examples, that is, the number of instances that are actually negative examples but classified as positive examples by the classifier; False negatives (FN): the number of instances incorrectly classified as negative examples, that is, the number of instances that are actually positive examples but classified as negative examples by the classifier;

[0062] True negatives (TN): the number of instances that are correctly classified as negative examples, that is, the number of instances that are actually negative examples and are classified as negative examples by the classifier;

[0063] The similarity of the segmented images is evaluated by the following commonly used metrics in medical image segmentation:

[0064] The measurement indicators include: Sensitivity, Specificity, Recall, Precision, F1, MIoU (Mean Intersection over Union) and Dice coefficient.

[0065] Sensitivity refers to the proportion of samples that the classifier can correctly predict as positive examples among all samples that are actually positive examples. It measures the classifier's ability to identify samples that are actually positive examples. A higher sensitivity means that the classifier can better capture the true positive examples.

[0066] Specificity refers to the proportion of samples that are actually negative examples that the classifier can correctly exclude as positive examples. It measures the classifier's ability to exclude samples that are actually negative examples. Higher specificity means that the classifier can better distinguish true negative examples.

[0067] Recall measures the proportion of samples that are actually positive that are correctly predicted as positive examples;

[0068] Precision measures the proportion of samples predicted to be positive that are actually positive;

[0069] The F1 value is an indicator of comprehensive accuracy and recall. The performance of the classifier is comprehensively evaluated by calculating the harmonic mean of accuracy and recall. MIoU calculates the average of the intersection and union area ratios between the predicted results and the true results. It measures the accuracy and coverage of the segmentation results.

[0070] The Dice coefficient is an indicator for evaluating the similarity between the predicted segmentation results and the true label segmentation results. It measures the proportion of samples predicted as positive examples among all samples that are actually positive examples.

[0071] Step S3, using the medical image segmentation result map completed by the PMS-Net architecture to calculate the thickness, angle and length of the calcified plaque. Specifically: Step S31, using the medical image segmentation result map completed by the PMS-Net architecture to calculate the thickness of the calcified plaque, specifically includes:

[0072] Step S311, for each segmented coronary artery image, the coordinates of all points in the coronary artery are obtained, and the average value of the coordinates of all points is obtained, which is used as the centroid of the coronary artery;

[0073] Step S312, in the segmented coronary artery image, calculating the diameter of the catheter area as a scale;

[0074] Step S313, emitting rays outward from the center of mass as the origin, finding the maximum thickness of the calcified plaque, and then multiplying by 0.9 and dividing by the catheter diameter;

[0075] Step S32, using the centroid as a fixed point, the two end points of the calcified plaque, and the centroid respectively connects the two end points of the calcified plaque to form two line segments, and the angle formed by the two line segments is used as the angle value of the calcified plaque;

[0076] Step S33, finding the coronary artery images where the calcified plaque appears for the first time and the last time, calculating the number of intermediate continuous frames and multiplying it by 0.2, and using the result as the length value of the calcified plaque.

[0077] Step S4, assigning corresponding scores to the calculated calcified plaque thickness, angle and length according to the calcification scoring system based on optical coherence tomography. Specifically:

[0078] Step S41, in the calcification scoring system of optical coherence tomography, 2 points are awarded if the angle of the calcified plaque is greater than 180°, 1 point is awarded if the thickness of the calcified plaque is greater than 0.5 mm, and 1 point is awarded if the length of the calcified plaque is greater than 5 mm;

[0079] Step S42, taking the consecutive frames where the calcified plaque appears for the first time and the last time as a lesion, counting the angle, thickness and length of the calcified plaque in these consecutive coronary artery images, and giving corresponding scores according to the calcification scoring system of optical coherence tomography.

Claims

1. A method for coronary artery plaque segmentation and calcified plaque scoring, characterized in that: include: A. Collect the coronary artery vascular optical coherence tomography (IVOCT) image dataset of patients with coronary heart disease, and then perform data enhancement; B. Input the data-enhanced IVOCT image dataset into the segmentation model (PMS-Net) to obtain the plaque segmentation results of each image; C. Based on the plaque segmentation results of each image, calculate the thickness, angle and length of the calcified plaque; D. According to the calcification scoring system based on optical coherence tomography, the calculated calcified plaque thickness, angle and length are assigned corresponding scores.

2. The method according to claim 1, characterized in that The step A specifically comprises: For each coronary artery image to be segmented, first determine the center point of the image; The acquired coronary artery image to be segmented is cropped based on the center point. Specifically, the original 1024×1024 image is cropped to a size of 640×640 by cropping the image into a smaller area; The IVOCT dataset is augmented by rotating 90° and 270°.

3. The method according to claim 1, characterized in that The step B specifically comprises: The PMS-Net encoder module performs multi-scale fusion through skip connections and uses the local attention mechanism to strengthen the focus on important information of different categories; The middle layer of PMS-Net uses hole convolution kernels of different step lengths to expand the receptive field and capture broader contextual information; The PMS-Net decoder module uses skip connections to encode different features of the same scale and fuses them, and then uses the local attention mechanism to extract feature information of the same scale; The PMS-Net network is used to segment medical images and obtain segmentation results.

4. The method according to claim 1, characterized in that The step C specifically comprises: For each segmented coronary artery image, the coordinates of all points in the coronary artery are calculated, and the average value of the coordinates of all points is calculated as the centroid of the coronary artery; In the segmented coronary artery images, the diameter of the ductal area was calculated as the scale; Send out rays outward from the center of mass to find the maximum thickness of the calcified plaque, then multiply by 0.9 and divide by the catheter diameter; Take the centroid as the fixed point, the two endpoints of the calcified plaque, and the centroid respectively connects the two endpoints of the calcified plaque to form two line segments, and the angle formed by the two line segments is taken as the angle value of the calcified plaque; Find the coronary artery images where the calcified plaque appears for the first time and the last time, calculate the number of intermediate continuous frames and multiply it by 0.2, and use the result as the length value of the calcified plaque.

5. The method according to claim 1, characterized in that The step D specifically comprises: In the calcification scoring system of optical coherence tomography, 2 points are given if the angle of the calcified plaque is greater than 180°, 1 point is given if the thickness of the calcified plaque is greater than 0.5 mm, and 1 point is given if the length of the calcified plaque is greater than 5 mm; The consecutive frames where calcified plaques appeared for the first and last time were regarded as a lesion. The angle, thickness and length of calcified plaques in these consecutive coronary artery images were counted and the corresponding scores were given according to the calcification scoring system of optical coherence tomography.

Images