Automatic left auricle filling defect segmentation and classification method based on CCTA image

Abstract

The invention discloses a CCTA image-based left auricle filling defect automatic segmentation and classification method, which realizes automatic identification and segmentation of left auricle thrombus, and comprises the following steps: S1, data acquisition and preprocessing; step S21, segmenting and sketching the image; step S22, training a deep learning neural network model by using the three-dimensional region of interest obtained in the step S21; s3, verifying the deep learning neural network model in the step S22 by using an external verification set; step S41, feature extraction; step S42, constructing a classification model; and S5, verifying the classification effect and explaining the classification effect, through the method, compared with previous manual segmentation sketching, a large amount of time and manpower resources are saved, through the method, image features, radiomics features and deep learning features can be combined, and compared with previous independent radiomics feature modeling, multiple artificial intelligence methods are integrated, so that the modeling efficiency is improved. And a more accurate diagnosis model is constructed.

Description

Technical Field

[0001] The invention belongs to the technical field of coronary artery image processing and relates to a left atrial appendage filling defect automatic segmentation and classification method based on CCTA images. Background Art

[0002] Atrial fibrillation is one of the most common arrhythmias. Studies have shown that the incidence of atrial fibrillation in people over 35 years old in China is as high as 0.71%. The risk of stroke in patients with non-valvular atrial fibrillation is 5-6 times that of other patients, mainly due to the detachment of thrombi in the left atrium / left atrial appendage. For patients with non-valvular atrial fibrillation, 90% of thrombi originate from the left atrial appendage. When atrial fibrillation occurs, the atria lose their normal contractile function, and blood flow in the atria is slow and stagnant; especially the irregular shape of the left atrial appendage is prone to thrombus formation. Left atrial thrombus is a contraindication for cardioversion and catheter ablation in patients with atrial fibrillation or atrial flutter, because cardioversion and ablation can cause thrombi to detach and cause cardiogenic stroke. Therefore, accurate identification and diagnosis of left atrial and left atrial appendage thrombi are of guiding significance for the formulation of subsequent treatment plans for patients.

[0003] Transesophageal echocardiography is an effective means of determining whether patients with non-valvular atrial fibrillation have left atrial appendage thrombus, but the operation is invasive and difficult to use routinely. In recent years, cardiac CT has been increasingly used in the diagnosis of left atrial appendage thrombus. Dual-phase CT performs well in detecting left atrial appendage thrombus, similar to the results of transesophageal echocardiography. However, dual-phase scanning is required, which increases the radiation dose. At the same time, the application of imaging omics and deep learning in the cardiovascular field has gradually increased. Imaging omics has achieved good performance in the identification of left atrial appendage thrombus and circulatory stasis, which is higher than the diagnostic results of traditional imaging. Compared with traditional omics, deep learning can be used for the automatic quantitative evaluation of left atrial appendage filling defects (including thrombus and circulatory stasis), showing higher accuracy. However, these studies also have some limitations, such as manual delineation of left atrial appendage thrombus, which is time-consuming and labor-intensive and difficult to be routinely applied in clinical practice. At present, the research on automatic identification and diagnosis of left atrial appendage thrombus based on deep learning technology is an innovative work. Therefore, an automatic segmentation and classification method for left atrial appendage filling defects based on CCTA images is needed to assist doctors in making rapid and accurate diagnoses, assess disease risks, and assist in clinical decision-making. Summary of the invention

[0004] In view of the above problems, the present invention proposes an automatic segmentation and classification method for left atrial appendage filling defects based on CCTA images, which well solves the problems in the prior art.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] A method for automatic segmentation and classification of left atrial appendage filling defects based on CCTA images comprises the following steps:

[0007] Step S1, data acquisition and preprocessing, obtaining an internal data set and an external data set, wherein the internal data set is coronary CTA images of patients in this hospital, and the external data set is CTA images of patients in other hospitals, and the internal data set is uniformly preprocessed by normalization;

[0008] Step S21, segmenting and outlining the image, two radiologists segment the left atrial appendage filling defect area in the image preprocessed in step S1 to obtain slices, and then manually outline each slice to obtain a two-dimensional region of interest, and fuse each of the segmented slices to obtain a three-dimensional region of interest;

[0009] Step S22, using the three-dimensional region of interest obtained in step S21 to train a deep learning neural network model, so that after a coronary CTA image of a patient is input, the deep learning neural network model can automatically obtain the three-dimensional region of interest of the left atrial appendage filling defect segmented by the patient;

[0010] Step S3, verifying the deep learning neural network model in step S22 using the external verification set to verify its segmentation efficiency;

[0011] Step S41, feature extraction, extracting radiomics features and deep learning features from the three-dimensional region of interest obtained in step S22, further selecting the extracted radiomics features and deep learning features, and removing features with low correlation;

[0012] Step S42: constructing a classification model, and constructing different classification models according to the radiomics features and deep learning features selected in step S41;

[0013] Step S5: verify the classification effect and explain it, use the external validation set to test the performance of the model, and verify it using the Delong method.

[0014] Preferably, in step S21, two different doctors segment half of the image respectively; in step S22, the deep learning neural network model adopts a semantic segmentation model of the DeepLabv3 architecture.

[0015] Preferably, in step S3, verifying the model includes the following steps:

[0016] Step S31, using the DICE similarity coefficient and intersection-over-union comparison to evaluate the segmentation performance of the segmentation model;

[0017] Step S32: Calculate the volume of the automatically segmented filling defect and the volume of the manually segmented filling defect, and compare the consistency between the two.

[0018] Preferably, in step S41, the radiomics features to be extracted include: shape, texture, first order, grayscale co-occurrence matrix, grayscale run length matrix, grayscale size area matrix, neighborhood grayscale tone difference matrix and grayscale dependency matrix.

[0019] Preferably, in step S41, the extraction of deep learning features includes the following steps:

[0020] Step S411, reading the coronary CTA image of the patient in step S22 through a convolutional neural network, converting it into a NumPy array, and performing normalization and standardization processing;

[0021] Step S412: Draw a 10 mm 2 The circular region of interest is used to measure the CT value of the left atrial appendage filling defect and the CT value of the ascending aorta root, and the ratio of the two is calculated to obtain the deep learning features affected by imaging.

[0022] In step S41, further selecting the extracted radiomics features and deep learning features includes the following steps:

[0023] Step S413, sequentially pairing the radiomics features obtained in step S41 with the deep learning features obtained in step S412, and calculating the Pearson or Spearman correlation coefficients between all feature pairs, and removing features with correlation coefficients greater than 0.75;

[0024] Step S414: Select features using 5-fold cross-validation minimum absolute shrinkage and selection operator regression to obtain features that are highly correlated with CT features or with the radiomics features or deep learning features of these pre-fusion models.

[0025] Preferably, in step S42, the classification models include: imaging model, radiomics model, deep learning model, radiomics + deep learning pre-fusion model, imaging + radiomics + deep learning pre-fusion model, and support vector machine is used as a classifier to develop single modality and pre-fusion models.

[0026] Preferably, in step S5, the performance of the model is evaluated using the area under the receiver operating characteristic curve, accuracy, sensitivity, specificity, positive predictive value and negative predictive value of the external validation set.

[0027] Preferably, in step S5, a calibration curve is also used to evaluate the consistency between the observed results and the predicted results of the fusion model; the net benefit is quantified by decision curve analysis to evaluate the clinical practicality of the deep learning neural network model in step S22 for diagnosing left atrial appendage filling defects; and the additive interpretation model is used to interpret the results.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] 1. The present invention realizes automatic identification and segmentation of left atrial appendage filling defects, and proposes a model construction method that can automatically segment left atrial appendage filling defects.

[0030] 2. The present invention establishes a classification model by adopting a feature fusion approach, and proposes a comprehensive classification model construction method that fuses multiple features to improve model accuracy.

[0031] 3. From clinical practice to imaging and then to artificial intelligence, the present invention comprehensively and deeply incorporates multiple data, and has perfect evaluation indicators, and provides a visual analysis and explanation of the model results.

[0032] 4. The present invention realizes the construction of automatic recognition segmentation and classification model to give a specific probability of whether a patient has left atrial appendage thrombosis or blood stasis after coronary CTA examination, realizes the automation of the whole process, and in order to improve the generalization ability of the model, an external validation set is collected. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 Patient inclusion diagram for data collection of the present invention.

[0034] Figure 2 This is the overall flow chart of fully automatic segmentation and classification of left atrial appendage filling defects in the present invention.

[0035] Figure 3 This is a flowchart of automatic segmentation using a deep learning neural network model in the present invention.

[0036] Figure 4 It is a classification flow chart of the classification model in the present invention.

[0037] Figure 5 For the present invention Figure 4 A magnified view of a portion of the area.

[0038] Figure 6 For the present invention Figure 4 A magnified view of a portion of the area.

[0039] Figure 7 For the present invention Figure 4 A magnified view of a portion of the area. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] The following is combined with Figure 1-Figure 7 The specific implementation modes of the present invention are described in further detail.

[0042] The present invention comprises the following steps:

[0043] Step S1, data acquisition and preprocessing, preprocessing the patient's coronary CTA image by normalization;

[0044] It should be noted that data collection needs to be conducted on patients with left atrial appendage thrombosis and blood stasis in two different hospitals, and the following patients need to be excluded: patients with congenital heart disease, patients with previous cardiac surgery, patients with active infection, and patients with atrial mass;

[0045] Step S21, segmenting and outlining the image, two radiologists segmented the left atrial appendage filling defect area in the preprocessed image in step S1 to obtain slices, and two different physicians segmented half of the image respectively, and the segmentation was performed on 3DSlicer (version Slicer5.2-2023), and then each slice was manually outlined to obtain a two-dimensional region of interest, and each segmented slice was fused to obtain a three-dimensional region of interest;

[0046] It should be noted that radiologists who perform manual segmentation operations should have been engaged in radiological imaging for at least two years;

[0047] Step S22, using the three-dimensional region of interest obtained in step S21 to train a deep learning neural network model. In this embodiment, the DeepLabv3 architecture is used. DeepLabv3 optimizes the previous versions 1 and 2, so that the network can better handle the multi-scale problem in segmentation and further improve the accuracy, so that the deep learning neural network model can automatically obtain the three-dimensional region of interest of the left atrial appendage filling defect segmented by a patient after inputting the coronary CTA image of the patient;

[0048] Step S3, verifying the deep learning neural network model in step S22 using an external validation set to verify its segmentation efficiency, wherein the external validation set is coronary CTA images of patients from other hospitals, including the following steps:

[0049] Step S31, use the DICE similarity coefficient and intersection-over-union ratio to evaluate the segmentation performance of the segmentation model; the DICE similarity coefficient is a set similarity measurement function, usually used to calculate the similarity of two samples, and is often used to evaluate the quality of image segmentation algorithms. The intersection-over-union ratio measures the ratio of the intersection to the union between the predicted result and the true label, and is a standard metric in semantic segmentation;

[0050] Step S32, calculating the volume of the automatically segmented filling defect and the volume of the manually segmented filling defect, and comparing the consistency between the two;

[0051] Step S41, feature extraction, extracting radiomics features and deep learning features from the three-dimensional region of interest obtained in step S22;

[0052] Radiomics features, namely Rad features, use the open source PyRadiomics package to extract omics features of the region of interest. These features include: shape, texture, first order, gray-level co-occurrence matrix (GLCM), gray-level run length matrix (GLRLM), gray-level size zone matrix (GLSZM), neighborhood gray-level tone difference matrix (NGTDM) and gray-level dependency matrix (GLDM);

[0053] Deep learning features are DL features. Deep learning is a subfield of machine learning. Its core idea is to simulate the working mode of human brain neurons by building a multi-layer neural network model, so that computers can autonomously learn and extract high-level features from the data. The key feature of deep learning is that it can gradually abstract data into more and more advanced feature representations through layer-by-layer nonlinear transformations, thereby showing excellent performance in complex tasks. The implementation of deep learning relies on a large amount of data to train the neural network model, and optimize the network parameters through the back propagation algorithm to achieve efficient learning and prediction. Deep learning features include its ability to automatically learn high-level features from data, as well as its wide application and high performance in various complex tasks. In this embodiment, 3D-ResNet is used to develop a convolutional neural network for extracting deep learning features;

[0054] The extraction of deep learning features includes the following steps:

[0055] Step S411, reading the coronary CTA image of the patient in step S22 through a convolutional neural network, converting it into a NumPy array, and performing normalization and standardization processing;

[0056] Step S412: Draw a 10 mm 2 The circular region of interest is used to measure the CT value of the left atrial appendage filling defect and the CT value of the ascending aorta root, and the ratio of the two is calculated to obtain the deep learning features affected by imaging.

[0057] After that, the extracted radiomics features and deep learning features are further selected, including the following steps:

[0058] Step S413, sequentially pairing the radiomics features obtained in step S41 with the deep learning features obtained in step S412, and calculating the Pearson or Spearman correlation coefficients between all feature pairs, and removing features with correlation coefficients greater than 0.75;

[0059] Step S414, using 5-fold cross-validated least absolute shrinkage and selection operator (LASSO) regression to select features, and obtain features that are highly correlated with CT features or with Rad features or DL ​​features of these pre-fusion models;

[0060] In this embodiment, using 5-fold cross validation will divide the data set into 5 parts, 4 of which are used to train the model and the remaining 1 part is used for testing. This process is repeated 5 times, and a different part of the data is used for testing each time to evaluate the stability and generalization ability of the model. Applying 5-fold cross validation to LASSO regression can help determine the optimal λ value, that is, the value of the regularization parameter, to balance the complexity and fit of the model and avoid overfitting or underfitting problems. In this way, a more reliable and stable model can be obtained, and the accuracy of the prediction can be improved;

[0061] It should be noted that the least absolute shrinkage and selection operator (LASSO) regression is a regression analysis method that penalizes the coefficients in the model through L1 regularization (i.e., the sum of the absolute values ​​of the coefficients), thereby achieving two main purposes: feature selection and adjustment of model complexity. This method is particularly effective when dealing with data with multicollinearity or high-dimensional data. The core of LASSO regression is to achieve variable selection by compressing the coefficients, thereby improving the prediction accuracy and explanatory power of the model. Specifically, LASSO regression controls the penalty for model complexity by adjusting a regularization parameter λ. The larger λ is, the greater the penalty for the coefficients in the model, so that some coefficients become zero and automatic feature selection is achieved;

[0062] Step S42: Figure 4-Figure 7 As shown, a classification model is constructed, and different classification models are constructed according to the omics features screened in step S41; in this embodiment, the classification models include: imaging model, imaging omics model, deep learning model, imaging omics + deep learning pre-fusion model, imaging + imaging omics + deep learning pre-fusion model, and support vector machine is used as a classifier to develop single modality and pre-fusion model;

[0063] Step S5, verifying the classification effect and explaining it, using the external validation set to test the performance of the model, and verifying it using the Delong method;

[0064] Specifically, the performance of the model was evaluated using the area under the receiver operating characteristic (ROC) curve (AUC), accuracy (ACC), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the external validation set, and the consistency between the observed results and the predicted results of the fusion model was evaluated using a calibration curve; the net benefit was quantified by decision curve (DCA) analysis to evaluate the clinical practicality of the deep learning neural network model in step S22 for diagnosing left atrial appendage filling defects; the additive interpretation model Shap values ​​was used to interpret the results, and Shap values ​​can reflect the influence of the features in each sample, and can reflect the positive and negative influences, so that people can intuitively understand the influence of the features on the prediction results;

[0065] In the process of explaining the results of the additive explanation model Shap values: SHAP value (ShapleyAdditiveExPlanations) is a method for explaining the prediction results of machine learning models. It is based on the concept of Shapley value in game theory and helps understand how the model makes predictions by quantifying the contribution of each feature to the model prediction. The calculation of SHAP value involves the following steps:

[0066] ①: Feature contribution: In the calculation of the SHAP value, the contribution of each feature of the model is measured by their marginal contribution to the model's prediction results.

[0067] ②: Model prediction: The contribution of each feature in the entire machine learning model prediction process is determined by observing the role it plays in model prediction.

[0068] ③: Calculate marginal contribution: To calculate the SHAP value of a feature, you need to compare the predictions of two models: a model that includes the feature and a model that does not include the feature. By comparing the difference between the two predictions, you can derive the marginal contribution of the feature to the model prediction.

[0069] ④: Consider the combination of all features: Since model predictions may be affected by multiple features, it is necessary to consider the combination of all features. This involves calculating all possible feature subsets, making model predictions for each subset, and then taking the average of these predictions to obtain the Shapley value of the feature.

[0070] ⑤: Global explanation and local explanation: SHAP value can provide not only the global explanation of the model (i.e. the impact of the feature on the overall model), but also the local explanation (i.e. the impact of the feature on the prediction results of a single sample). The global explanation helps to understand which features have the greatest impact on the overall performance of the model, while the local explanation provides a detailed explanation of the prediction results of a specific sample.

[0071] Through the above steps, SHAP values ​​provide an intuitive and effective way to understand the prediction results of machine learning models, helping analysts better explain the decision-making process of the model, thereby enhancing the transparency and interpretability of the model.

[0072] After the above steps, the present invention establishes a comprehensive model for identifying, segmenting and diagnosing left atrial appendage filling defects by using deep learning and radiomics methods, assisting clinicians in diagnosis and decision making, and visually analyzing the model results.

[0073] The present invention has a novel structure, ingenious conception, and simple and convenient operation. Through this design, left atrial appendage thrombus can be automatically identified and segmented, which saves a lot of time and human resources compared with the previous manual segmentation and outlining. It combines image features, radiomics features, and deep learning features, and compared with the previous single radiomics feature modeling, it integrates multiple artificial intelligence methods to construct a more accurate diagnostic model.

[0074] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for automatic segmentation and classification of left atrial appendage filling defects based on CCTA images, characterized by: The following steps are involved: Step S1, data acquisition and preprocessing, obtaining an internal data set and an external data set, wherein the internal data set is coronary CTA images of patients in this hospital, and the external data set is CTA images of patients in other hospitals, and the internal data set is uniformly preprocessed by normalization; Step S21, segmenting and outlining the image, two radiologists segment the left atrial appendage filling defect area in the image preprocessed in step S1 to obtain slices, and then manually outline each slice to obtain a two-dimensional region of interest, and fuse each of the segmented slices to obtain a three-dimensional region of interest; Step S22, using the three-dimensional region of interest obtained in step S21 to train a deep learning neural network model, so that after a coronary CTA image of a patient is input, the deep learning neural network model can automatically obtain the three-dimensional region of interest of the left atrial appendage filling defect segmented by the patient; Step S3, verifying the deep learning neural network model in step S22 using the external verification set to verify its segmentation efficiency; Step S41, feature extraction, extracting radiomics features and deep learning features from the three-dimensional region of interest obtained in step S22, further selecting the extracted radiomics features and deep learning features, and removing features with low correlation; Step S42: constructing a classification model, and constructing different classification models according to the radiomics features and deep learning features selected in step S41; Step S5: verify the classification effect and explain it, use the external validation set to test the performance of the model, and verify it using the Delong method.

2. The method for automatic segmentation and classification of left atrial appendage filling defects based on CCTA images according to claim 1, characterized in that: In step S21, two different doctors segment half of the image respectively; in step S22, the deep learning neural network model adopts the semantic segmentation model of DeepLabv3 architecture.

3. The method for automatic segmentation and classification of left atrial appendage filling defects based on CCTA images according to claim 1, characterized in that: In step S3, verifying the model includes the following steps: Step S31, using the DICE similarity coefficient and intersection-over-union comparison to evaluate the segmentation performance of the segmentation model; Step S32: Calculate the volume of the automatically segmented filling defect and the volume of the manually segmented filling defect, and compare the consistency between the two.

4. The method for automatic segmentation and classification of left atrial appendage filling defects based on CCTA images according to claim 1, characterized in that: In step S41, the radiomics features to be extracted include: shape, texture, first order, grayscale co-occurrence matrix, grayscale run length matrix, grayscale size area matrix, neighborhood grayscale tone difference matrix and grayscale dependency matrix.

5. The method for automatic segmentation and classification of left atrial appendage filling defects based on CCTA images according to claim 1, characterized in that: In step S41, the extraction of deep learning features includes the following steps: Step S411, reading the coronary CTA image of the patient in step S22 through a convolutional neural network, converting it into a NumPy array, and performing normalization and standardization processing; Step S412: Draw a 10 mm 2 The circular region of interest is used to measure the CT value of the left atrial appendage filling defect and the CT value of the ascending aorta root, and the ratio of the two is calculated to obtain the deep learning features affected by imaging. In step S41, further selecting the extracted radiomics features and deep learning features includes the following steps: Step S413, sequentially pairing the radiomics features obtained in step S41 with the deep learning features obtained in step S412, and calculating the Pearson or Spearman correlation coefficients between all feature pairs, and removing features with correlation coefficients greater than 0.75; Step S414: Select features using 5-fold cross-validation minimum absolute shrinkage and selection operator regression to obtain features that are highly correlated with CT features or with the radiomics features or deep learning features of these pre-fusion models.

6. The method for automatic segmentation and classification of left atrial appendage filling defects based on CCTA images according to claim 1, characterized in that: In step S42, the classification models include: imaging model, radiomics model, deep learning model, radiomics + deep learning pre-fusion model, imaging + radiomics + deep learning pre-fusion model, and support vector machine is used as a classifier to develop single modality and pre-fusion model.

7. The method for automatic segmentation and classification of left atrial appendage filling defects based on CCTA images according to claim 1, characterized in that: In step S5, the performance of the model is evaluated using the area under the receiver operating characteristic curve, accuracy, sensitivity, specificity, positive predictive value and negative predictive value of the external validation set.

8. The method for automatic segmentation and classification of left atrial appendage filling defects based on CCTA images according to claim 1, characterized in that: In step S5, a calibration curve is also used to evaluate the consistency between the observed results and the predicted results of the fusion model; the net benefit is quantified by decision curve analysis to evaluate the clinical practicality of the deep learning neural network model in step S22 for diagnosing left atrial appendage filling defects; and the additive interpretation model is used to interpret the results.

Images