Coronary artery calcification plaque scoring method and device and electronic equipment
By using end-to-end GLF-Net architecture network and vascular segment division technology, the global and local features in coronary calcification assessment were extracted, and the problems of incomplete feature extraction and inaccurate prediction results in the prior art were solved, achieving efficient and accurate calcification scores.
Patent Information
- Application Number
- CN202411736496.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-29
- Publication Date
- 2025-03-21
AI Technical Summary
The prior art has problems in the assessment of coronary artery calcification, inaccurate prediction results, and lack of external verification, resulting in delayed and inaccurate scoring.
The end-to-end GLF-Net architecture network is used for deep learning, and global and local features in the vascular image data are extracted, combined with vascular segment division and weight calculation, calcification score is calculated, and external verification is performed after prediction to correct the prediction results.
It improves the accuracy and efficiency of coronary calcification scores, reduces the amount of calculations, and ensures the accuracy and reliability of the scoring results.
Smart Images
Figure CN119671976A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of medical data processing, and is a coronary artery calcified plaque scoring method, device and electronic equipment. Background Art
[0002] At present, cardiovascular and cerebrovascular diseases are a major hidden danger to the health of the Chinese population, among which the incidence of coronary heart disease is particularly high, mainly due to atherosclerosis. In 2022, the number of coronary artery intervention (PCI) operations caused by atherosclerosis reached 1.29 million, and it is on an upward trend. The severity of atherosclerosis is often reflected by calcified plaques in blood vessels, and these calcified plaques may lead to insufficient expansion of the implanted stents during PCI surgery, poor adhesion of the stents to the blood vessel wall, and stent failure. Therefore, accurate assessment of the degree of coronary artery calcification is crucial for planning PCI strategies.
[0003] Coronary artery calcification (CAC) scoring by intravascular optical coherence tomography (IVOCT) is a powerful independent predictor that can be used to predict coronary artery disease (CAD) and plan PCI. IVOCT technology provides high-resolution imaging of the coronary artery lumen and subintimal lesions of the vessel wall, with axial and lateral resolutions of up to 10μm and 25μm, respectively, which are close to histological resolution. It can clearly present the information of calcified plaques in the blood vessels and various conditions after stent implantation, which helps to guide and optimize PCI surgery and reduce the incidence of adverse cardiovascular events in patients.
[0004] However, despite the clinical value of IVOCT in assessing coronary artery calcification, there are some limitations in the current clinical practice of CAC assessment. First, after IVOCT image acquisition, the tasks of coronary artery segmentation and calcium burden quantification are usually performed using professional software, certified 3D imaging laboratories, and professional technicians on independent workstations. This clinical workflow model may lead to delays in the reporting of formal CAC scores. In addition, some imaging centers may not be able to provide coronary artery calcification risk assessment due to lack of manpower and resources. Second, although many semi-automatic and a few fully automatic calcium scoring methods based on deep learning networks have been proposed, the following problems exist:
[0005] (1) End-to-end deep learning architectures are rarely used for fully automatic coronary artery calcium scoring, and feature extraction is not comprehensive, without sufficient global and local features for calcification classification prediction.
[0006] (2) No external verification is performed after the classification prediction, which may easily lead to incorrect classification prediction results due to prediction errors, resulting in inaccurate coronary artery calcification scores. Summary of the invention
[0007] The present invention provides a coronary artery calcified plaque scoring method, device and electronic equipment for scoring coronary artery calcified plaques through vascular image data.
[0008] In view of this, the first aspect of the present application provides a coronary artery calcified plaque scoring method, comprising:
[0009] Acquire a whole blood vessel image data set to be predicted, wherein the whole blood vessel image data set includes a plurality of continuous blood vessel image data of the same blood vessel;
[0010] Input each vascular image data into the calcification prediction model to obtain the corresponding calcification classification category prediction value, wherein the calcified plaque prediction model is obtained by deep learning the end-to-end GLF-Net architecture network using a number of sample data, and each group of sample data in the multiple groups of sample data includes vascular image data and a label identifying the actual value of the corresponding calcification classification category, and the calcification classification category includes a calcified plaque category for whether there is a calcified plaque, a calcified plaque angle category for whether the calcified plaque angle is greater than an angle threshold, and a calcified plaque thickness category for whether the calcified plaque thickness is greater than a thickness threshold;
[0011] Combined with the calcification classification category prediction value of each blood vessel image data, the blood vessel segmentation standard is used to divide the entire blood vessel to be predicted into blood vessel segments;
[0012] The weight value of each vascular segment is determined, and the calcification score of each vascular segment is calculated. The calcification score calculation formula is as follows:
[0013] Calcification score = calcified plaque thickness category prediction value × 1 + calcified plaque angle category prediction value × 2 + vessel segment weight.
[0014] Preferably, the method further includes calibrating the calcification classification category prediction value of each vascular image data, including:
[0015] Determine whether the calcified plaque category prediction value, calcified plaque thickness category prediction value, and calcified plaque angle category prediction value of a certain blood vessel image data are the same;
[0016] In response to "no", determining whether the calcified plaque category prediction value is the same as at least one of the calcified plaque thickness category prediction value and the calcified plaque angle category prediction value;
[0017] In response to this, the different category prediction values among the calcified plaque category prediction value, the calcified plaque thickness category prediction value, and the calcified plaque angle category prediction value are corrected to be the same as the other category prediction values;
[0018] In response to no, determining whether the calcified plaque category prediction value is no;
[0019] In response to "yes", the predicted value of the calcified plaque category is corrected to be the same as its category;
[0020] In response to no, the calcification classification category prediction value is calibrated using a recalibration condition, wherein the recalibration condition is set based on the continuity of the blood vessel image data in the entire blood vessel image data set to be predicted.
[0021] Preferably, the recalibration conditions include:
[0022] When the calcification classification category prediction values corresponding to the first N pieces of vascular image data and the last N pieces of vascular image data are both negative, the calcification plaque category prediction value is corrected to negative;
[0023] When the number of calcification classification prediction values of the blood vessel image data that is continuous with the blood vessel image data is less than N, the calcification classification prediction values of the continuous blood vessel image data are all corrected to be yes.
[0024] Preferably, the blood vessel segment division standard includes: dividing the continuous blood vessel image data with the same calcification classification category prediction value in the entire blood vessel image data set into one blood vessel segment.
[0025] Preferably, determining the weight value of each blood vessel segment includes: if the amount of blood vessel image data in the blood vessel segment is greater than a quantity threshold, the weight value of the blood vessel segment is 1, otherwise it is 0.
[0026] Preferably, the deep learning process of the calcification prediction model includes:
[0027] Acquire a number of sample data, and divide them into a training set and a test set in proportion, wherein each sample data includes vascular image data and a label identifying an actual value of a corresponding calcification classification category, and the calcification classification category includes a calcification plaque category indicating whether a calcified plaque exists, a calcification plaque angle category indicating whether a calcification plaque angle is greater than an angle threshold, and a calcification plaque thickness category indicating whether a calcification plaque thickness is greater than a thickness threshold;
[0028] The end-to-end GLF-Net architecture network is trained using the training set. A loss function is introduced during training. When the value of the loss function is stable, the training is terminated to obtain a calcification prediction model. The end-to-end GLF-Net architecture network includes a local feature extraction module, a global feature extraction module, a feature fusion module, and a linear classifier. The local feature extraction module includes a 7×7 large kernel convolution and a channel attention mechanism to extract important information of different local categories. The global feature extraction module is based on the Swin Transformer network through a hierarchical attention mechanism to capture feature information of different levels and scales. The feature fusion module uses a residual connection method to integrate global features and local features. The linear classifier performs classification based on global features, local features, and fused features.
[0029] The trained calcification prediction models were tested using the test sets, the test results were evaluated using model metrics, the model parameters of the calcification prediction model were optimized, and the calcification prediction model that met the evaluation requirements was output.
[0030] Preferably, the loss function is:
[0031]
[0032] Among them, p i is the predicted value of calcification classification; g i is the actual value of the calcification classification category; N is the number of training samples.
[0033] In view of this, the second aspect of the present application provides a coronary artery calcified plaque scoring device, comprising:
[0034] A data acquisition unit, which acquires a whole blood vessel image data set to be predicted, wherein the whole blood vessel image data set includes a plurality of continuous blood vessel image data of the same blood vessel;
[0035] The prediction unit inputs each vascular image data into the calcification prediction model to obtain the corresponding calcification classification category prediction value, wherein the calcified plaque prediction model is obtained by deep learning an end-to-end GLF-Net architecture network using a plurality of sample data, and each group of sample data in the plurality of groups of sample data includes vascular image data and a label identifying the actual value of the corresponding calcification classification category, and the calcification classification category includes a calcified plaque category for whether there is a calcified plaque, a calcified plaque angle category for whether the calcified plaque angle is greater than an angle threshold, and a calcified plaque thickness category for whether the calcified plaque thickness is greater than a thickness threshold;
[0036] A segmentation unit, combining the calcification classification category prediction value of each blood vessel image data and using a blood vessel segmentation standard to perform blood vessel segmentation on the entire blood vessel to be predicted;
[0037] The score calculation unit determines the weight value of each blood vessel segment and calculates the calcification score of each blood vessel segment, wherein the calcification score calculation formula is as follows:
[0038] Calcification score = calcified plaque thickness category prediction value × 1 + calcified plaque angle category prediction value × 2 + vessel segment weight.
[0039] Preferably, a calibration unit is also included, including:
[0040] Determine whether the calcified plaque category prediction value, calcified plaque thickness category prediction value, and calcified plaque angle category prediction value of a certain blood vessel image data are the same;
[0041] In response to "no", determining whether the calcified plaque category prediction value is the same as at least one of the calcified plaque thickness category prediction value and the calcified plaque angle category prediction value;
[0042] In response to this, the different category prediction values among the calcified plaque category prediction value, the calcified plaque thickness category prediction value, and the calcified plaque angle category prediction value are corrected to be the same as the other category prediction values;
[0043] In response to no, determining whether the calcified plaque category prediction value is no;
[0044] In response to "yes", the predicted value of the calcified plaque category is corrected to be the same as its category;
[0045] In response to no, calibrating the calcification classification category prediction value using a recalibration condition, wherein the recalibration condition is set based on the continuity of the blood vessel image data in the entire blood vessel image data set to be predicted;
[0046] Among them, the recalibration conditions include:
[0047] When the calcification classification category prediction values corresponding to the first N pieces of vascular image data and the last N pieces of vascular image data are both negative, the calcification plaque category prediction value is corrected to negative;
[0048] When the number of calcification classification prediction values of the blood vessel image data that is continuous with the blood vessel image data is less than N, the calcification classification prediction values of the continuous blood vessel image data are all corrected to be yes.
[0049] In view of this, the third aspect of the present application provides an electronic device, including a processor, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the program includes instructions for executing steps in a coronary artery calcification plaque scoring method.
[0050] The present invention is based on using a number of sample data to perform deep learning on an end-to-end GLF-Net architecture network to obtain a calcification prediction model. Based on the structural characteristics of the GLF-Net architecture network, the features in the vascular image data are fully extracted to increase the prediction accuracy. The continuous vascular image data is used to increase the data volume, thereby improving the adequacy and completeness of the calcification prediction. After the prediction, the vascular segment is divided into vascular segments in combination with the calcification classification category prediction value of each vascular image data to obtain the calcification score of each vascular segment. The calcification score calculation amount is reduced on the basis of ensuring the accuracy of the calcification score. After the calcification prediction model is used for prediction and the corresponding calcification classification category prediction value is obtained, external calibration is introduced to eliminate the calcification classification category error caused by the model error, thereby ensuring the accuracy of the subsequent calcification score. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] Attached Figure 1 The figure is a flow chart of a coronary artery calcified plaque scoring method according to an embodiment of the present invention.
[0052] Attached Figure 2 The figure is a flow chart of a calibration method according to an embodiment of the present invention.
[0053] Attached Figure 3 The figure is a flowchart of a deep learning method for a calcification prediction model in one embodiment of the present invention.
[0054] Attached Figure 4 The figure is a schematic diagram of the GLF-Net architecture network structure according to an embodiment of the present invention.
[0055] Attached Figure 5 FIG. 1 is a schematic diagram of sample calcification classification according to an embodiment of the present invention.
[0056] Attached Figure 6 The figure is a schematic diagram of the structure of a coronary artery calcified plaque scoring device according to an embodiment of the present invention.
[0057] Attached Figure 7 This is a schematic diagram of the structure of yet another coronary artery calcified plaque scoring device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0058] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0059] The present application provides a coronary artery calcification plaque scoring method, device and electronic device, which are used to solve the technical problem that the existing semi-automatic and a few fully automatic calcification scoring methods based on deep learning networks cannot fully extract features, resulting in inaccurate classification predictions. It further solves the technical problem that the existing technology does not perform external verification after classification prediction, which is prone to errors in classification prediction results due to prediction errors, resulting in inaccurate coronary artery calcification evaluation.
[0060] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0061] The following is a detailed description of the embodiments of the present application.
[0062] Embodiment 1: As attached Figure 1 As shown, this embodiment discloses a coronary artery calcified plaque scoring method, comprising:
[0063] Step S110 , obtaining a whole blood vessel image data set to be predicted, wherein the whole blood vessel image data set includes a plurality of continuous blood vessel image data of the same blood vessel.
[0064] In this embodiment, the vascular image data is vascular image data obtained by intravascular optical coherence tomography (IVOCT), and this embodiment scans the entire blood vessel to be predicted to obtain a set of image data of the entire blood vessel to be predicted, and the vascular image data in the set are sorted in chronological order.
[0065] Step S120, each vascular image data is input into the calcification prediction model to obtain the corresponding calcification classification category prediction value, wherein the calcified plaque prediction model is obtained by deep learning the end-to-end GLF-Net architecture network using a number of sample data, and each group of sample data in the multiple groups of sample data includes vascular image data and a label identifying the actual value of the corresponding calcification classification category, and the calcification classification category includes a calcified plaque category for whether a calcified plaque exists, a calcified plaque angle category for whether the calcified plaque angle is greater than an angle threshold, and a calcified plaque thickness category for whether the calcified plaque thickness is greater than a thickness threshold.
[0066] In this embodiment, the calcification classification categories include the calcified plaque category of whether there is a calcified plaque, the calcified plaque angle category of whether the calcified plaque angle is greater than the angle threshold, and the calcified plaque thickness category of whether the calcified plaque thickness is greater than the thickness threshold. That is, the calcified plaque category, calcified plaque angle category, and calcified plaque thickness category are all "yes" or "no", and the corresponding labels can be set as needed. In this embodiment, it can be but not limited to "1" for "yes" and "0" for "no".
[0067] And since the calcification classification categories include calcified plaque category, calcified plaque angle category and calcified plaque thickness category, when the calcified plaque category, calcified plaque angle category and calcified plaque thickness category are all "yes", the calcification classification category is "yes"; when the calcified plaque category, calcified plaque angle category and calcified plaque thickness category are all "no", the calcification classification category is "no".
[0068] In this embodiment, the angle threshold and thickness threshold are determined based on actual medical judgment. In this embodiment, the angle threshold may be 180°, and the thickness threshold may be 0.5 mm. Thus, the calcification classification categories may include the calcification plaque category of whether there is a calcified plaque, the calcification plaque thickness category of whether the calcification plaque angle is greater than 180°, and the calcification plaque angle category of whether the calcification plaque thickness is greater than 0.5 mm.
[0069] Step S130 , combining the calcification classification category prediction value of each blood vessel image data, and using the blood vessel segmentation standard to divide the entire blood vessel to be predicted into blood vessel segments.
[0070] In this embodiment, since a set of the entire blood vessel image data to be predicted is obtained, there is a phenomenon that multiple blood vessel image data correspond to the same blood vessel segment, so the calcification classification category prediction values of the blood vessel image data corresponding to the same blood vessel segment must be the same. Therefore, in this embodiment, the entire blood vessel is divided into blood vessel segments in combination with the calcification classification category prediction value of each blood vessel image data, thereby reducing the amount of calcification score calculation and improving efficiency without affecting the accuracy of the calcification score.
[0071] Step S140, determining the weight value of each blood vessel segment, and calculating the calcification score of each blood vessel segment, wherein the calcification score calculation formula is as follows:
[0072] Calcification score = calcified plaque thickness category prediction value × 1 + calcified plaque angle category prediction value × 2 + vessel segment weight.
[0073] The present invention is based on using a number of sample data to perform deep learning on an end-to-end GLF-Net architecture network to obtain a calcification prediction model. Based on the structural characteristics of the GLF-Net architecture network, the features in the vascular image data are fully extracted to increase the prediction accuracy. Continuous vascular image data is used to increase the data volume, thereby improving the adequacy and completeness of the calcification prediction. After the prediction, the calcification classification category prediction value of each vascular image data is combined to divide the entire blood vessel to be predicted into vascular segments, and the calcification scores of each vascular segment are obtained, thereby reducing the amount of calcification score calculation on the basis of ensuring the accuracy of the calcification score.
[0074] Example 2: Figure 2 As shown in Table 1, this embodiment discloses a coronary artery calcified plaque scoring method, which also includes calibrating the calcification classification category prediction value of each vascular image data, including:
[0075] Step S210, determining whether the calcified plaque category prediction value, the calcified plaque thickness category prediction value, and the calcified plaque angle category prediction value of a certain blood vessel image data are the same;
[0076] Taking Table 2 as an example, the labels corresponding to the calcified plaque category, calcified plaque angle category, and calcified plaque thickness category are set to "1" for "yes" and "0" for "no". Then this step determines whether the calcified plaque category prediction value, calcified plaque thickness category prediction value, and calcified plaque angle category prediction value are all "1" or "0".
[0077] Step S220: In response to "no", it is determined whether the calcified plaque category prediction value is the same as at least one of the calcified plaque thickness category prediction value and the calcified plaque angle category prediction value;
[0078] Taking Table 2 as an example, if the calcified plaque category prediction value, the calcified plaque thickness category prediction value, and the calcified plaque angle category prediction value are all "1" or "0", such as the 2nd and 9th rows in Table 2, no adjustment is made; if the calcified plaque category prediction value, the calcified plaque thickness category prediction value, and the calcified plaque angle category prediction value are not all "1" or "0", then it is determined whether the calcified plaque category prediction value is the same as at least one of the calcified plaque thickness category prediction value and the calcified plaque angle category prediction value.
[0079] Step S230, in response to "yes", the different category prediction values among the calcified plaque category prediction value, the calcified plaque thickness category prediction value, and the calcified plaque angle category prediction value are corrected to be the same as the other category prediction values;
[0080] Taking Table 2 as an example, if the calcified plaque category prediction value is the same as at least one of the calcified plaque thickness category prediction value and the calcified plaque angle category prediction value, such as in Table 2, rows 3, 4, 7, and 8, row 3 corrects the calcified plaque angle category prediction value to "0", and row 7 corrects the calcified plaque thickness category prediction value to "1".
[0081] Step S240, in response to "no", determining whether the calcified plaque category prediction value is "no";
[0082] Taking Table 2 as an example, this step is to determine whether the predicted value of the calcified plaque category is "0".
[0083] Step S250, in response to yes, the predicted value of the calcified plaque category is corrected to be the same as its category;
[0084] As shown in row 5 of Table 2, the predicted value of the calcified plaque category is corrected to "1".
[0085] Step S260: in response to "no", the calcification classification category prediction value is calibrated using a recalibration condition, wherein the recalibration condition is set based on the continuity of the vascular image data in the entire vascular image data set to be predicted.
[0086] Among them, the recalibration conditions include:
[0087] (1) When the calcification classification category prediction values corresponding to the first N pieces of vascular image data and the last N pieces of vascular image data are both negative, the calcification plaque category prediction value is corrected to negative;
[0088] (2) When the number of calcification classification prediction values of the vascular image data that are consecutive to the vascular image data is less than N, the calcification classification prediction values of the consecutive vascular image data are all corrected to be yes.
[0089] Among them, N is determined according to needs. In this embodiment, N can be but is not limited to 3, and the calcification classification category prediction value is yes or no, that is, the calcification plaque category prediction value, calcification plaque thickness category prediction value, and calcification plaque angle category prediction value in the calcification classification category prediction value are all yes or no.
[0090] Table 1 Example table of calibration of classification category prediction values
[0091]
[0092]
[0093] After the calcification prediction model is used for prediction and the corresponding calcification classification category prediction value is obtained, the embodiment of the present invention introduces external calibration to eliminate the calcification classification category error caused by the model error, thereby ensuring the accuracy of the subsequent calcification score.
[0094] Embodiment 3: This embodiment discloses a coronary artery calcified plaque scoring method, which is an optimization of the above embodiment, including:
[0095] The blood vessel segment division standard includes: dividing the continuous blood vessel image data with the same calcification classification category prediction value in the entire blood vessel image data set into one blood vessel segment.
[0096] For example, if 20 consecutive blood vessel image data have the same calcification classification category prediction value, the 20 consecutive blood vessel image data are divided into one blood vessel segment.
[0097] Determining the weight value of each blood vessel segment includes: if the amount of blood vessel image data in the blood vessel segment is greater than a quantity threshold, the weight value of the blood vessel segment is 1, otherwise it is 0.
[0098] In this embodiment, the quantity threshold is set as needed, and can be but is not limited to 25.
[0099] Example 4: Figure 3 , 4 As shown, this embodiment discloses a coronary artery calcified plaque scoring method, wherein the deep learning process of the calcification prediction model includes:
[0100] Step S310, obtain a number of sample data, and divide them into a training set and a test set in proportion, wherein each sample data includes vascular image data and a label identifying the actual value of the corresponding calcification classification category, and the calcification classification category includes a calcification plaque category indicating whether a calcified plaque exists, a calcification plaque angle category indicating whether the calcification plaque angle is greater than an angle threshold, and a calcification plaque thickness category indicating whether the calcification plaque thickness is greater than a thickness threshold.
[0101] In this embodiment, the method for acquiring sample data can be to use an IVOCT catheter equipped with an SJM Fourier Domain OCT imaging system (Abbott), with a pull-back speed of 18 mm / s, covering a distance of 54 mm, an interval of 0.1 mm per frame, and an acquisition time of 3 seconds. The axial resolution and spatial resolution of the device are 12-15 μm and 9.9×9.9 μm, respectively, and an independent electronic control system is used to control the pull-back motor. The catheter emits light to scan the blood vessel wall, receives the reflected signal and converts it into an A line.
[0102] In this embodiment, the label indicating the actual value of the corresponding calcification classification category may be as follows: Figure 5 As shown, the calcified plaque angle category (the category with the maximum angle greater than 180° is set to 1 ( Figure 5 e), the maximum angle is less than 180° and the category is set to 0 ( Figure 5 b)), calcified plaque thickness category (the maximum thickness greater than 0.5 mm is set to 1 ( Figure 5 f), the maximum thickness is less than 0.5mm and is set to 0 ( Figure 5 c)) and calcified plaque category (the presence of calcified plaque in the image is set to 1 ( Figure 2 d), the category of non-calcified plaque in the image is set to 0 ( Figure 5 a)).
[0103] Furthermore, in this embodiment, after the initial sample data is obtained, the initial sample data may be preprocessed, and the preprocessing includes:
[0104] (1) Perform data enhancement on the initial sample data to expand sample diversity. The data enhancement methods include rotating 90° and 270° to enhance the data.
[0105] (2) Each sample after data enhancement is cropped to a size of 640×640 to avoid being too large and containing too much background information, which will affect the accuracy of the prediction.
[0106] Step S320, using the training set to train the end-to-end GLF-Net architecture network, introducing a loss function during training, and ending the training when the value of the loss function is stable to obtain a calcification prediction model, wherein the end-to-end GLF-Net architecture network includes a local feature extraction module, a global feature extraction module, a feature fusion module and a linear classifier, the local feature extraction module includes a 7×7 large kernel convolution and a channel attention mechanism to extract important information of different local categories, the global feature extraction module is based on the Swin Transformer network through a hierarchical attention mechanism to capture feature information of different levels and scales, the feature fusion module uses a residual connection method to integrate global features and local features, and the linear classifier performs classification based on global features, local features and fused features.
[0107] As attached Figure 4 As shown in the figure, the end-to-end GLF-Net architecture network includes a local feature extraction module, a global feature extraction module, a feature fusion module and a linear classifier, as follows:
[0108] (1) Local feature extraction module, using 7×7 large kernel convolution and channel attention mechanism to strengthen the focus on important information of different local categories, including:
[0109] The local feature extraction module uses a four-stage architecture to extract and process local features of the input data; each stage contains a 7×7 large kernel convolution layer and a channel attention mechanism. These two steps process the input features separately: the 7×7 large kernel convolution is used to enhance local details and suppress irrelevant areas, thereby improving the local expression ability of features; the channel attention mechanism aims to capture the correlation between channels between input features to enhance the feature representation of specific semantics. Finally, the two feature maps obtained through these two processing steps are fused through residual connections, which further enriches the feature representation and enables the model to better capture complex semantic information.
[0110] (2) Global feature extraction module: This module mainly uses the global feature extraction module of Swin Transformer. Through the hierarchical attention mechanism, it can capture feature information at different levels and scales, thereby helping the model to extract global information more comprehensively. Specifically, it includes:
[0111] The global feature extraction module uses a four-stage structure to extract and process global features of the input data; the input image is divided into non-overlapping patches, and then the resolution of the feature map is gradually reduced through multiple stages, while the depth of the feature is increased. This hierarchical representation method enables the model to capture features of different scales, similar to the multi-scale feature fusion in convolutional networks; a sliding window mechanism is introduced to calculate self-attention, which includes self-attention within non-overlapping windows (W-MSA) and self-attention between overlapping windows (SW-MSA); in W-MSA, the self-attention within each window is calculated independently to reduce the amount of calculation; in W-MSA, information exchange between windows is achieved through window displacement, enhancing the model's ability to capture global information.
[0112] (3) The feature fusion module integrates the local features extracted in the four stages, the global features, and the fused features generated in the previous stage by adopting the residual connection method, effectively fusing these three features with significant differences, which helps to alleviate the problems of gradient explosion and network degradation to a certain extent.
[0113] (4) Linear classifier is a commonly used classifier in classification learning. It is a model that assumes that there is a linear relationship between features and classification results. This model helps classify decisions by accumulating the product of the features of each dimension and their respective weights.
[0114] In this embodiment, the loss function is:
[0115]
[0116] Among them, p i is the predicted value of calcification classification; g i is the actual value of the calcification classification category; N is the number of training samples.
[0117] Step S330, using the test set to test the trained calcification prediction model, using the model metric index to evaluate the test results, optimizing the model parameters of the calcification prediction model, and outputting the calcification prediction model that meets the evaluation requirements.
[0118] The model metrics in this embodiment include: Accuracy, Recall, F1 and Precision. Specifically:
[0119] Accuracy measures the proportion of samples that are correctly predicted by the model as a whole to the total samples;
[0120] Recall measures the proportion of samples that are actually positive that are correctly predicted as positive examples;
[0121] The F1 value is an indicator of comprehensive accuracy and recall. It comprehensively evaluates the performance of the classifier by calculating the harmonic mean of accuracy and recall.
[0122] Precision measures the proportion of samples predicted to be positive that are actually positive.
[0123] In order to obtain the above model metrics, the following values are required based on the test results:
[0124] True positives (TP): the number of instances that are correctly classified as positive examples, that is, the number of instances (number of samples) that are actually positive examples and are classified as positive examples by the classifier;
[0125] False positives (FP): the number of instances that are mistakenly classified as positive examples, that is, the number of instances that are actually negative examples but are classified as positive examples by the classifier;
[0126] False negatives (FN): The number of instances that are mistakenly classified as negative examples, that is, the number of instances that are actually positive examples but are classified as negative examples by the classifier;
[0127] True negatives (TN): The number of instances that are correctly classified as negative examples, that is, the number of instances that are actually negative examples and are classified as negative examples by the classifier.
[0128] Embodiment 4: As attached Figure 6 As shown, this embodiment discloses a coronary artery calcified plaque scoring device, comprising:
[0129] A data acquisition unit, which acquires a whole blood vessel image data set to be predicted, wherein the whole blood vessel image data set includes a plurality of continuous blood vessel image data of the same blood vessel;
[0130] The prediction unit inputs each vascular image data into the calcification prediction model to obtain the corresponding calcification classification category prediction value, wherein the calcified plaque prediction model is obtained by deep learning an end-to-end GLF-Net architecture network using a plurality of sample data, and each group of sample data in the plurality of groups of sample data includes vascular image data and a label identifying the actual value of the corresponding calcification classification category, and the calcification classification category includes a calcified plaque category for whether there is a calcified plaque, a calcified plaque angle category for whether the calcified plaque angle is greater than an angle threshold, and a calcified plaque thickness category for whether the calcified plaque thickness is greater than a thickness threshold;
[0131] A segmentation unit, combining the calcification classification category prediction value of each blood vessel image data and using a blood vessel segmentation standard to perform blood vessel segmentation on the entire blood vessel to be predicted;
[0132] The score calculation unit determines the weight value of each blood vessel segment and calculates the calcification score of each blood vessel segment, wherein the calcification score calculation formula is as follows:
[0133] Calcification score = calcified plaque thickness category prediction value × 1 + calcified plaque angle category prediction value × 2 + vessel segment weight.
[0134] The specific implementation steps of each unit in this embodiment are the same as those in the above embodiment and will not be repeated here.
[0135] Embodiment 5: As attached Figure 7 As shown, this embodiment discloses a coronary artery calcified plaque scoring device, comprising:
[0136] A data acquisition unit, which acquires a whole blood vessel image data set to be predicted, wherein the whole blood vessel image data set includes a plurality of continuous blood vessel image data of the same blood vessel;
[0137] A prediction unit inputs each vascular image data into a calcification prediction model to obtain a corresponding calcification classification category prediction value, wherein the calcified plaque prediction model is obtained by deep learning an end-to-end GLF-Net architecture network using a plurality of sample data, and each group of sample data in the plurality of groups of sample data includes vascular image data and a label identifying an actual value of a corresponding calcification classification category, and the calcification classification category includes a calcified plaque category for whether a calcified plaque exists, a calcified plaque angle category for whether a calcified plaque angle is greater than an angle threshold, and a calcified plaque thickness category for whether a calcified plaque thickness is greater than a thickness threshold;
[0138] A segmentation unit, combining the calcification classification category prediction value of each blood vessel image data and using a blood vessel segmentation standard to perform blood vessel segmentation on the entire blood vessel to be predicted;
[0139] The score calculation unit determines the weight value of each blood vessel segment and calculates the calcification score of each blood vessel segment, wherein the calcification score calculation formula is as follows:
[0140] Calcification score = calcified plaque thickness category prediction value × 1 + calcified plaque angle category prediction value × 2 + vessel segment weight;
[0141] Calibration unit, comprising:
[0142] Determine whether the calcified plaque category prediction value, calcified plaque thickness category prediction value, and calcified plaque angle category prediction value of a certain blood vessel image data are the same;
[0143] In response to "no", determining whether the calcified plaque category prediction value is the same as at least one of the calcified plaque thickness category prediction value and the calcified plaque angle category prediction value;
[0144] In response to this, the different category prediction values among the calcified plaque category prediction value, the calcified plaque thickness category prediction value, and the calcified plaque angle category prediction value are corrected to be the same as the other category prediction values;
[0145] In response to no, determining whether the calcified plaque category prediction value is no;
[0146] In response to "yes", the predicted value of the calcified plaque category is corrected to be the same as its category;
[0147] In response to no, calibrating the calcification classification category prediction value using a recalibration condition, wherein the recalibration condition is set based on the continuity of the blood vessel image data in the entire blood vessel image data set to be predicted;
[0148] Among them, the recalibration conditions include:
[0149] When the calcification classification category prediction values corresponding to the first N pieces of vascular image data and the last N pieces of vascular image data are both negative, the calcification plaque category prediction value is corrected to negative;
[0150] When the number of calcification classification prediction values of the blood vessel image data that is continuous with the blood vessel image data is less than N, the calcification classification prediction values of the continuous blood vessel image data are all corrected to be yes.
[0151] Embodiment 7: This embodiment discloses an electronic device, comprising a processor, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include instructions for executing steps in a coronary artery calcification plaque scoring method.
[0152] The processor may be a central processing unit (CPU), a general purpose processor, a digital signal processor (DSP), an ASIC, an FPGA or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. It may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like. The memory may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory, a mobile hard disk, a magnetic disk or an optical disk.
[0153] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.
[0154] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0155] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0156] Although the present invention has been described in detail above with general descriptions and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements may be made thereto based on the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all fall within the scope of protection claimed by the present invention.
Claims
1. A coronary artery calcified plaque scoring method, characterized in that: include: Acquire a whole blood vessel image data set to be predicted, wherein the whole blood vessel image data set includes a plurality of continuous blood vessel image data of the same blood vessel; Input each vascular image data into the calcification prediction model to obtain the corresponding calcification classification category prediction value, wherein the calcified plaque prediction model is obtained by deep learning the end-to-end GLF-Net architecture network using a number of sample data, and each group of sample data in the multiple groups of sample data includes vascular image data and a label identifying the actual value of the corresponding calcification classification category, and the calcification classification category includes a calcified plaque category for whether there is a calcified plaque, a calcified plaque angle category for whether the calcified plaque angle is greater than an angle threshold, and a calcified plaque thickness category for whether the calcified plaque thickness is greater than a thickness threshold; Combined with the calcification classification category prediction value of each blood vessel image data, the blood vessel segmentation standard is used to divide the entire blood vessel to be predicted into blood vessel segments; The weight value of each vascular segment is determined, and the calcification score of each vascular segment is calculated. The calcification score calculation formula is as follows: Calcification score = calcified plaque thickness category prediction value × 1 + calcified plaque angle category prediction value × 2 + vessel segment weight.
2. The coronary artery calcification plaque scoring method according to claim 1, characterized in that: It also includes calibration of the calcification classification category prediction value for each vascular image data, including: Determine whether the calcified plaque category prediction value, calcified plaque thickness category prediction value, and calcified plaque angle category prediction value of a certain blood vessel image data are the same; In response to "no", determining whether the calcified plaque category prediction value is the same as at least one of the calcified plaque thickness category prediction value and the calcified plaque angle category prediction value; In response to this, the different category prediction values among the calcified plaque category prediction value, the calcified plaque thickness category prediction value, and the calcified plaque angle category prediction value are corrected to be the same as the other category prediction values; In response to no, determining whether the calcified plaque category prediction value is no; In response to "yes", the predicted value of the calcified plaque category is corrected to be the same as its category; In response to no, the calcification classification category prediction value is calibrated using a recalibration condition, wherein the recalibration condition is set based on the continuity of the blood vessel image data in the entire blood vessel image data set to be predicted.
3. The coronary artery calcification plaque scoring method according to claim 2, characterized in that: The recalibration conditions include: When the calcification classification category prediction values corresponding to the first N pieces of vascular image data and the last N pieces of vascular image data are both negative, the calcification plaque category prediction value is corrected to negative; When the number of calcification classification prediction values of the blood vessel image data that is continuous with the blood vessel image data is less than N, the calcification classification prediction values of the continuous blood vessel image data are all corrected to be yes.
4. The coronary artery calcification plaque scoring method according to any one of claims 1 to 3, characterized in that: The blood vessel segment division standard includes: dividing the continuous blood vessel image data with the same calcification classification category prediction value in the entire blood vessel image data set into one blood vessel segment.
5. The coronary artery calcification plaque scoring method according to claim 4, characterized in that: The step of determining the weight value of each blood vessel segment includes: if the amount of blood vessel image data in the blood vessel segment is greater than a quantity threshold, the weight value of the blood vessel segment is 1, otherwise it is 0.
6. The coronary artery calcification plaque scoring method according to any one of claims 1 to 5, characterized in that: The deep learning process of the calcification prediction model includes: Acquire a number of sample data, and divide them into a training set and a test set in proportion, wherein each sample data includes vascular image data and a label identifying an actual value of a corresponding calcification classification category, and the calcification classification category includes a calcification plaque category indicating whether a calcified plaque exists, a calcification plaque angle category indicating whether a calcification plaque angle is greater than an angle threshold, and a calcification plaque thickness category indicating whether a calcification plaque thickness is greater than a thickness threshold; The end-to-end GLF-Net architecture network is trained using the training set. A loss function is introduced during training. When the value of the loss function is stable, the training is terminated to obtain a calcification prediction model. The end-to-end GLF-Net architecture network includes a local feature extraction module, a global feature extraction module, a feature fusion module, and a linear classifier. The local feature extraction module includes a 7×7 large kernel convolution and a channel attention mechanism to extract important information of different local categories. The global feature extraction module is based on the SwinTransformer network through a hierarchical attention mechanism to capture feature information of different levels and scales. The feature fusion module uses a residual connection method to integrate global features and local features. The linear classifier performs classification based on global features, local features, and fused features. The trained calcification prediction models were tested using the test sets, the test results were evaluated using model metrics, the model parameters of the calcification prediction model were optimized, and the calcification prediction model that met the evaluation requirements was output.
7. The coronary artery calcification plaque scoring method according to claim 6, characterized in that: The loss function is: Among them, p i is the predicted value of calcification classification; g i is the actual value of the calcification classification category; N is the number of training samples.
8. A coronary artery calcified plaque scoring device using the method according to any one of claims 1 to 7, characterized in that: include: A data acquisition unit, which acquires a whole blood vessel image data set to be predicted, wherein the whole blood vessel image data set includes a plurality of continuous blood vessel image data of the same blood vessel; A prediction unit inputs each vascular image data into a calcification prediction model to obtain a corresponding calcification classification category prediction value, wherein the calcified plaque prediction model is obtained by deep learning an end-to-end GLF-Net architecture network using a plurality of sample data, and each group of sample data in the plurality of groups of sample data includes vascular image data and a label identifying an actual value of a corresponding calcification classification category, and the calcification classification category includes a calcified plaque category for whether a calcified plaque exists, a calcified plaque angle category for whether a calcified plaque angle is greater than an angle threshold, and a calcified plaque thickness category for whether a calcified plaque thickness is greater than a thickness threshold; A segmentation unit, combining the calcification classification category prediction value of each blood vessel image data and using a blood vessel segmentation standard to perform blood vessel segmentation on the entire blood vessel to be predicted; The score calculation unit determines the weight value of each blood vessel segment and calculates the calcification score of each blood vessel segment, wherein the calcification score calculation formula is as follows: Calcification score = calcified plaque thickness category prediction value × 1 + calcified plaque angle category prediction value × 2 + vessel segment weight.
9. The coronary artery calcification plaque scoring device according to claim 8, characterized in that: Also included is a calibration unit, consisting of: Determine whether the calcified plaque category prediction value, calcified plaque thickness category prediction value, and calcified plaque angle category prediction value of a certain blood vessel image data are the same; In response to "no", determining whether the calcified plaque category prediction value is the same as at least one of the calcified plaque thickness category prediction value and the calcified plaque angle category prediction value; In response to this, the different category prediction values among the calcified plaque category prediction value, the calcified plaque thickness category prediction value, and the calcified plaque angle category prediction value are corrected to be the same as the other category prediction values; In response to no, determining whether the calcified plaque category prediction value is no; In response to "yes", the predicted value of the calcified plaque category is corrected to be the same as its category; In response to no, calibrating the calcification classification category prediction value using a recalibration condition, wherein the recalibration condition is set based on the continuity of the blood vessel image data in the entire blood vessel image data set to be predicted; Among them, the recalibration conditions include: When the calcification classification category prediction values corresponding to the first N pieces of vascular image data and the last N pieces of vascular image data are both negative, the calcification plaque category prediction value is corrected to negative; When the number of calcification classification prediction values of the blood vessel image data that is continuous with the blood vessel image data is less than N, the calcification classification prediction values of the continuous blood vessel image data are all corrected to be yes.
10. An electronic device, characterized in that: An electronic device includes a processor, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the processor, and the programs include instructions for executing the steps in a coronary artery calcified plaque scoring method.