A Cardiovascular Calcified Plaque Detection System Based on Residual Double Attention Mechanism

Abstract

The present invention discloses a cardiovascular calcified plaque detection system based on a residual dual attention mechanism. The present invention proposes a segmentation model based on a residual dual attention mechanism, which includes a backbone network module BackBone and a residual dual attention module. The residual dual attention module includes a first attention module, a second attention module, and a fusion module in parallel. The input of the two parallel attention modules is the output feature X of the BackBone module, and the output is the segmentation result. By constructing a residual dual attention mechanism module, the present invention solves the problem of gradient disappearance that occurs as the network deepens, and solves the problem that the information of features is less and less completely retained as the forward propagation progresses layer by layer, so that new features can be more accurate and complete, and the segmented calcified area is more precise.

Description

Technical Field

[0001] The present invention belongs to the field of medical image processing, and relates to a cardiovascular calcified plaque detection system based on a residual double attention mechanism. Background Art

[0002] Approximately 6% of adults in the world suffer from cardiovascular diseases, and coronary atherosclerotic diseases are the leading cause of death for most disease patients. The generation of coronary calcified plaques is the main manifestation of the development of coronary atherosclerosis to a certain extent. It is the result of the body's attempt to contain inflammation and stabilize atherosclerotic plaques, and it is a controllable process. Coronary calcification is one of the main indicators of coronary atherosclerosis, and its content can effectively predict cardiovascular disease events. Therefore, it is crucial for the treatment of cardiovascular diseases to be able to detect coronary atherosclerotic plaques deposited on blood vessels at an early stage. Therefore, it is of certain significance to research and explore methods for detecting cardiovascular calcified plaques.

[0003] In the prevention and treatment of cardiovascular diseases, the accurate detection and identification of calcified plaques have important research significance. And accurately and completely detecting the calcified plaque area in the CTA (CT angiography) images of cardiovascular disease patients belongs to the field of image segmentation. In the past, image segmentation was mostly carried out by using pattern recognition methods, and classification was carried out according to the segmentation characteristics such as thresholds, edges, region growing, fuzzy set theory, and graph theory. Pattern recognition methods proposed for calcified area segmentation include methods such as k-means clustering and random forests. Candidate calcified layers to be classified are described by features such as position, size, shape, and intensity. Toumoulin et al. selected the region with higher intensity between the inner and outer boundaries of the coronary artery as the calcified plaque. Wesarg et al. observed that the lumen of the plaque artery is narrower than that of a normal artery, so calcified plaques can be detected by combining lumen radius and lumen intensity features. Wang and Liatsis located calcified stenosis by assuming that a normal artery has a circular cross-section. Since quantitative methods require the normal cross-section of the end site to calculate the plaque volume, prior knowledge of the plaque position is essential for the accurate identification of the plaque. Kurkure et al. and Brunner et al. proposed a cardiac coordinate system centered on the heart. Similarly, Sánchez et al. described the candidate positions relative to anatomical landmarks. Methods such as multi-atlas registration are used to estimate the position of the coronary artery tree. In addition, in the past few years, Wersag et al. proposed a method for localizing calcifications through vessel segmentation. This automatic detection algorithm combines diameter information and gray values to analyze the calcified region. Saur et al. automatically detected and evaluated coronary artery calcified plaques in CT (Multi-Slice Computed Tomography, MSCT) images using a natural vessel dataset to obtain information about each plaque. Most of these above methods perform image segmentation based on the underlying information of image pixels, require relying on artificially designed complex features, are only applicable to current general problems, and the segmentation effect is difficult to be satisfactory.

[0004] With the continuous development of science and technology and imaging medicine, multi-slice spiral CT applied to clinical practice in recent years can non-invasively detect and quantitatively analyze coronary artery calcified plaques. This non-invasive examination method is expected to become the preferred method for the examination and evaluation of coronary heart disease. The selection of its post-processing reconstruction method is very crucial for the detection of calcified plaques and the calculation of scores. Currently, clinicians mainly diagnose the condition by observing medical images. However, due to the small diameter and numerous branches of the coronary arteries, the blood vessels are tortuous on the epicardial surface of the heart, and the spatial distribution positions of the coronary arteries vary among different patients. CT tomographic images usually only show the cross-sections of numerous coronary artery branch vessels, and it is difficult for doctors to track and observe a certain blood vessel in a large number of tomographic images obtained from thin-layer scans. Moreover, during the scanning process, motion artifacts often occur due to cardiac pulsation, increasing the difficulty for doctors to identify small blood vessels.

[0005] Therefore, the intelligent detection research on the coronary heart disease lesion area for detecting and positioning calcified plaques and calculating the calcification score can judge whether the coronary artery is stenosed through quantitative calculation, accurately locate the position of the stenosis area, observe the entire coronary artery vessel in multiple angles and directions, quantitatively analyze the degree of coronary atherosclerosis for doctors, locate the calcified area, and make it more intuitive and accurate for doctors to understand the blood vessel length, lesion position and diagnosis. Summary of the Invention

[0006] The purpose of the present invention is to overcome the limitations of existing methods and provide a cardiovascular calcified plaque detection system based on a residual dual attention mechanism.

[0007] A cardiovascular calcified plaque detection system based on a residual dual attention mechanism includes:

[0008] A data acquisition and preprocessing module, which is used to acquire DICOM (Digital Imaging and Communications in Medicine) data of the cardiovascular system, then convert it into CTA images and corresponding mask subgraphs; then perform binarization processing on the mask subgraphs;

[0009] The cardiovascular calcified plaque segmentation module is used to extract the features of the cardiovascular calcified region of the above CTA image by using the trained segmentation model based on the residual double attention mechanism;

[0010] For the segmentation model based on the residual double attention mechanism, the input is the CTA image and the corresponding mask sub-image, and the output is the segmentation result;

[0011] The segmentation model based on the residual double attention mechanism includes a backbone network module BackBone and a residual double attention module;

[0012] The BackBone module includes a convolutional layer, a max-pooling layer, four stacked layers, and a pooling layer connected in cascade in sequence;

[0013] The four stacked layers respectively include Blocks with the numbers [3, 4, 6, 3], and each Block includes a 3-layer convolution, an activation function, and a max-pooling layer;

[0014] Preferably, the convolution kernels of the 3-layer convolution are 1×1, 3×3, 1×1 respectively, and the activation function is a non-linear activation function;

[0015] The residual double attention module includes a first attention module, a second attention module connected in parallel, and a fusion module; the inputs of the first attention module and the second attention module connected in parallel are the output features X of the BackBone module, and the output is the segmentation result;

[0016] The first attention module includes a convolutional layer and a softmax layer;

[0017] The first attention module calculates and outputs the final feature of the first attention module through the obtained feature map; specifically:

[0018] 1) The original feature A (C×H×W) generates two new feature maps B and F through the convolutional layer, and then deforms the two new feature maps B and F into C×N, where N = H×W is the number of pixels. Then, matrix multiplication is used between the transpose of F and B, and the spatial attention map s ji (N×N):

[0019]

[0020] where s ji represents the influence of the i-th position on the j-th position of the first attention module. The more similar the feature representations of two positions are, the stronger the correlation between them. B i and F jThey are the local feature at the $i$-th position on $B$ and the local feature at the $j$-th position on $F$, respectively.

[0021] 2) The original feature $A$ generates a new feature map $D(C\times H\times W)$ through the convolutional layer and is deformed into $C\times N$. Then, the local features $D$ at each position of $D$ i and the transposed $s$ ji at each position are multiplied using matrix multiplication, and the result is deformed into $C\times H\times W$. Finally, it is multiplied by the scaling parameter $\alpha$ and added to the local feature $A$ at each position of the original feature $A$ j to obtain the final output at the $j$-th position as follows:

[0022]

[0023] where $A$ j represents the local feature of $A$ at the $j$-th position, and $D$ i represents the local feature of $D$ at the $i$-th position. According to the above formula, $j$ is traversed and merged to obtain the final output $E$ 1 ;

[0024] The second attention module directly calculates the original feature to output the final output of the second attention module; specifically:

[0025] The original feature $A$ is deformed into $(C\times N)$. Then, the original feature $A$ and the transpose of the original feature $A$ are multiplied using matrix multiplication, and the channel attention map $x$ ji $(C\times C)$ is obtained through the softmax layer:

[0026]

[0027] where $x$ ji measures the influence of the $i$-th channel of the second attention module on the $j$-th channel. $A$ i and $A$ j are the local features of the original feature $A$ at the $i$-th position and the $j$-th position, respectively. The transpose of $x$ ji and $A$ i are multiplied using matrix multiplication, and the result is deformed into $C\times H\times W$. Then, the result is multiplied by the scaling parameter $\beta$ and added element-wise to $A$ j to obtain the final output at the $j$-th position of the second attention module

[0028]

[0029] According to the above formula, $j$ is traversed and merged to obtain the final output $E$ 2 ;

[0030] The fusion module adds the final output E of the first attention module 1 and the final output E of the second attention module 2 to the original feature A to obtain a new feature, and then upsamples the result E obtained by the first attention module 1 and the result E obtained by the second attention module 2 and this new feature respectively, and then adds the three sets of results to the segmentation result;

[0031] The calcification score calculation module is used to calculate the degree of calcification of the segmented calcified region; specifically:

[0032] 1) Set CT value segmentation thresholds a1, a2, a3, where 0 < a1 < a2 < a3;

[0033] 2) Obtain multiple calcified regions segmented by the segmentation model based on the attention mechanism, and determine the average CT value of each of the above calcified regions; divide the above calcified regions according to the preset CT value segmentation thresholds to obtain 4 calcification partitions with average CT values in (0, a1], (a1, a2], (a2, a3], (a3, +∞];

[0034] 3) Assign weight coefficients to the 4 calcification partitions with average CT values in (0, a1], (a1, a2], (a2, a3], (a3, +∞] respectively;

[0035] 4) Convert the DICOM image pixel value (gray value) to the CT value; then obtain the calcified region area of the calcified plaque segmentation result through the pixel points; specifically:

[0036] First, obtain two DICOM Tag information: rescale intercept and rescale slope;

[0037] Then calculate the CT value through the formula:

[0038] Hu = pixel_val × rescal_slope + rescal_intercept

[0039] where pixel_val is the gray value of the i-th pixel, and Hu is the CT value of the i-th pixel;

[0040] 5) Calculate the Agatston score according to the calcified region area and the weight coefficient assignment;

[0041] AS = Σ (calcification area × weight coefficient)

[0042] The weight coefficient assignment is based on the CT value of the lesion, and the higher the CT value, the greater the weight coefficient.

[0043] Another object of the present invention is to provide a computer-readable storage medium having a computer program stored thereon, which, when executed on a computer, causes the computer to execute the above-described system.

[0044] Yet another object of the present invention is to provide a computing device including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the above-described system is implemented.

[0045] The cardiovascular calcified plaque detection system based on the residual dual attention mechanism has the following characteristics:

[0046] 1. The data processing module can convert the original DICOM data into more intuitive picture data, which can better observe the detection results and provide the original data for the segmentation module.

[0047] 2. By constructing a residual dual attention mechanism module, the problem of gradient disappearance that occurs as the network deepens is solved, so that the problem that the information retained by the features becomes less and less complete as the layers of forward propagation are carried out is solved, so that the new features obtained can be more accurate and complete, and the segmented calcified area is more accurate.

[0048] 3. The integral calculation module takes the image segmented by the segmentation module as input, and calculates the value of the calcification integral through the already segmented calcified area, so as to be able to predict the risk of cardiovascular disease. Description of the Drawings

[0049] Figure 1 is the whole process of calcified plaque detection and its Agatston calculation of calcification integral proposed by the present invention;

[0050] Figure 2 is the main structure of the BackBone in the segmentation module proposed by the present invention;

[0051] Figure 3 is the main structure of the residual dual attention mechanism in the segmentation module proposed by the present invention. Detailed Embodiments

[0052] The present invention will be further described below with reference to the accompanying drawings.

[0053] A cardiovascular calcified plaque detection system based on the residual dual attention mechanism adopts the following method, as Figure 1 The specific steps are as follows:

[0054] Step 1: Process the CTA map data for model input;

[0055] Process the original DICOM data obtained from the hospital, convert the calcified plaque information contained in each document into CTA images and corresponding calcified plaque mask images (one mask image contains the location information of all calcified plaques in the corresponding CTA image), ensure that the CTA images and mask images of all cardiovascular patients input into the training network can be in one-to-one correspondence, and the image size is 512×512. And binarize the mask images;

[0056] Step 2: Build a segmentation model based on the residual double attention mechanism;

[0057] 2-1 Build the BackBone. The BackBone part is composed of a residual network, and its specific structure is as Figure 2 shown. First, there is a convolutional layer with a kernel size of 3 and a fully connected layer, and then four stacked layers layer1-layer4 are connected. The corresponding number of Blocks in the stacked layers are [3, 4, 6, 3] respectively. Each Block consists of three convolutional layers with kernel sizes of 1×1, 3×3, and 1×1 respectively, and the activation function is a non-linear activation function. Finally, the features are input into two attention modules via a max pooling layer;

[0058] 2-2 Build the attention mechanism network. The attention mechanism part is as Figure 3 shown. The output result of the BackBone part is the feature X. X is used as the input and passes through a convolutional layer to output A and send it to two attention modules respectively. For the first attention module, given the local feature A (C×H×W), we first put it into a convolutional layer to generate two new feature maps B and F, and then reshape them into C×N, where N = H×W, which is the number of pixels. Then we use matrix multiplication between the transpose of F and B, and then apply the softmax layer to calculate the spatial attention map s ji (N×N):

[0059]

[0060] where s ji measures the influence of the i-th position on the j-th position. The more similar the feature representations of two positions are, the stronger the correlation between them. B i and F j are the local features at the i-th position on B and the local features at the j-th position on F respectively. At the same time, we put the feature A into a convolutional layer to generate a new feature map D (C×H×W) and reshape it into C×N. Then we use matrix multiplication between D and the transpose of S, and reshape the result into C×H×W. Finally, we multiply it by the scaling parameter α and add it element-wise to A to obtain the final output as follows:

[0061]

[0062] where A j represents the local feature at the j-th position of A, and D i represents the local feature at the i-th position of D; according to the above formula, traverse j and merge to obtain the final output E 1 ;

[0063] The second attention module is different from the first attention module and directly calculates the channel attention map X (C×C) from the original feature A (C×H×W). Reshape A into (C×N), then perform matrix multiplication on A and the transpose of A, and finally use the softmax layer to obtain the channel attention map x ji (C×C):

[0064]

[0065] where x ji measures the influence of the i-th channel on the j-th channel. A i and A j are the local features at the i-th position and the j-th position of the original feature A respectively. Perform matrix multiplication on the transpose of x ji and A and reshape the result into C×H×W. Then multiply the result by the scaling parameter β and add it to A element-wise to obtain the final output

[0066]

[0067] According to the above formula, traverse j and merge to obtain the final output E 2 ;

[0068] where β gradually learns the weight from 0. The above formula indicates that the final feature of each channel is the weighted sum of all channel features and the original feature, which helps to improve the feature discriminability;

[0069] Finally, add the final output of the first attention module and the final output of the second attention module to the original feature to obtain a new feature. Then, upsample the results obtained by the first attention module, the results obtained by the second attention module, and this new feature respectively, and add the three sets of results to the segmentation result;

[0070] Step 3: Train the model based on the residual double attention mechanism;

[0071] 3-1 Construct a training dataset. When using the multi-attention mechanism visual enhancement model, it is necessary to use fixed-size CTA data as training samples, that is, all the image data used for training is converted into the same length through preprocessing, and the corresponding mask map is normalized to the range of 0 to 1. The labels of the training samples are calcification and background;

[0072] Step 4: Detect using the model based on the residual double-attention mechanism;

[0073] 4-1 Input the test data for detection into the trained model for detection;

[0074] Step 5: Calculate the Agatston score of calcification. The calculation process is as Figure 1 shown. The Agatston score is determined by factors such as calcification area, volume, and vascular distribution. The calculation process is as follows:

[0075] 5-1 Measurement and segmentation method of CT value:

[0076] Select a specific calcified plaque as the region of interest and determine its average CT value. And divide each specific calcified plaque into four parts according to the CT value unit, and calculate the ratio of each part after segmentation, that is, 130 - 199 HU, 200 - 299 HU, 300 - 399 HU, and ≥ 400 HU.

[0077] 5-2 Convert the pixel value (gray value) of the DICOM image to the CT value

[0078] First, it is necessary to read two DICOM Tag information: rescale intercept and rescale slope.

[0079] Then calculate the CT value through the formula:

[0080] Hu = pixel_val × rescal_slope + rescal_intercept

[0081] where pixel_val is the gray value of the i-th pixel, and Hu is the CT value of the i-th pixel.

[0082] 5-3 Obtain the calcification area of the calcified plaque segmentation result through pixel points;

[0083] 5-4 Agatston score calculation;

[0084] The Agatston Score (AS) and its correction methods are currently the most commonly used calcification scores and are also the scores that appear in the imaging reports of the vast majority of hospitals. The Agatston Score is determined by factors such as calcification area, volume, and vascular distribution. The principle of its calculation is as follows:

[0085] AS = Σ (calcification area × weight coefficient)

[0086] As Figure 1 shown, first, a 3D connected body is constructed based on the segmented calcification region, and then the weight coefficient is assigned according to the CT value of the lesion. The higher the CT value, the greater the weight coefficient.

[0087] CT value of the lesion Weight coefficient 130 - 199 HU 1 point 200 - 299 HU 2 points 300 - 399 HU 3 points 400 HU and above 4 points

[0088] The calcification score of each layer is equal to the product of the area of the calcification region in that layer and the weight value. The calcification score of each connected body is the sum of the calcification score values of all layers of that connected body. The higher the score, the higher the risk of cardiovascular disease. After traversing all connected bodies, the calcification score values of each connected body are output; otherwise, continue traversing and calculating.

Claims

1. A cardiovascular calcified plaque detection system based on a residual dual attention mechanism, characterized in that Including: A data acquisition and preprocessing module, which is used to acquire DICOM data of the cardiovascular system, and then convert it into CTA images and corresponding mask sub-images; then perform binary processing on the mask sub-images; A cardiovascular calcified plaque segmentation module, which is used to use a trained segmentation model based on the residual double attention mechanism to extract features of the cardiovascular calcified region of the above CTA image; The segmentation model based on the residual double attention mechanism takes the CTA image and the corresponding mask sub-image as inputs and outputs the segmentation result; the segmentation model based on the residual double attention mechanism includes a backbone network module BackBone and a residual double attention module; The backbone network module BackBone includes a convolutional layer, a max pooling layer, four stacked layers, and a pooling layer connected in cascade in sequence; The residual double attention module includes a first attention module, a second attention module, and a fusion module connected in parallel; the inputs of the first attention module and the second attention module are the output features X of the BackBone module, and the output is the segmentation result; The first attention module includes a convolutional layer and a softmax layer; the obtained feature map is used to calculate and output the final feature of the first attention module; Specifically: 1) The original feature A of size C×H×W generates two new feature maps B and F through the convolutional layer. Then, the two new feature maps B and F are reshaped into C×N, where N = H×W is the number of pixels. Next, matrix multiplication is performed between the transpose of F and B, and then the spatial attention map s is calculated through the softmax layer. ji (N×N): where s ji represents the influence of the i-th position of the first attention module on the j-th position, B i and F j are the local feature of the i-th position on B and the local feature of the j-th position on F, respectively; 2) The original feature A generates a new feature map D of size C×H×W through the convolutional layer, and is transformed into C×N. Then, the local features D at each position of D i and the transposed s ji at each position are multiplied using matrix multiplication, and the result is transformed into C×H×W. Finally, it is multiplied by the scaling parameter α and added to the local features A at each position of the original feature A j to obtain the final output at the j-th position with a size of C×H×W Among them, A j represents the local feature at the j-th position of A, and D i represents the local feature at the i-th position of D; Traverse j according to the above formula and merge to obtain the final output E 1 ; The second attention module directly calculates and outputs the final output of the second attention module through the original features; Specifically: The original feature A is deformed into C×N, and then the original feature A and the transpose of the original feature A are multiplied using matrix multiplication, and the channel attention map x with a size of C×C is obtained through the softmax layer ji : where x ji measures the influence of the i-th channel of the second attention module on the j-th channel, A i and A j are the local features at the i-th and j-th positions of the original feature A, respectively; For x ji transpose and A i Use matrix multiplication and reshape the result into C×H×W, then multiply the result by the scaling parameter β and add it element-wise to A j to obtain the final output of the j-th position in the second attention module with size C×H×W Traverse j according to the above formula and merge to obtain the final output E 2 ; The fusion module adds the final output E of the first attention module 1 and the final output E of the second attention module 2 to the original feature A to obtain a new feature, and then up-samples the result E obtained by the first attention module 1 and the result E obtained by the second attention module 2 and this new feature respectively, and then adds the three groups of results to the segmentation result; A calcification score calculation module, which is used to calculate the degree of calcification of the segmented calcified region; specifically: 1) Set CT value segmentation thresholds a1, a2, a3, where 0 < a1 < a2 < a3; 2) Obtain multiple calcified regions segmented by the segmentation model based on the attention mechanism, and determine the average CT value of each of the above calcified regions; divide the above calcified regions according to the preset CT value segmentation thresholds to obtain four calcification partitions with average CT values in (0, a1], (a1, a2], (a2, a3], (a3, +∞]; 3) Assign weight coefficients to the four calcification partitions with average CT values in (0, a1], (a1, a2], (a2, a3], (a3, +∞] respectively; 4) Convert the pixel values of the DICOM image into CT values; then obtain the calcified region area of the calcified plaque segmentation result through pixel points; 5) Calculate the Agatston score according to the calcified region area and the weight coefficient assignment; AS = Σ (calcified area × weight coefficient) Where the weight coefficient assignment is set according to the CT value of the lesion.

2. The system according to claim 1, wherein Each of the four stacked layers in the backbone network module BackBone of the segmentation model based on the residual double attention mechanism includes Blocks with numbers [3, 4, 6, 3] respectively, and each Block includes a 3-layer convolution, an activation function, and a max pooling layer.

3. The system according to claim 2, wherein The convolution kernels of the 3-layer convolution are 1×1, 3×3, and 1×1 respectively.

4. The system according to claim 2, wherein The activation function is a non-linear activation function.

5. The system according to claim 1, wherein Specifically for step 4) of the calcification score calculation module: First, obtain two DICOM Tag information: rescale intercept and rescale slope; Then, the CT value is calculated through a formula: Hu = pixel_val × rescal_slope + rescal_intercept where pixel_val is the gray value of the i-th pixel, and Hu is the CT value of the i-th pixel.

6. A computer-readable storage medium having a computer program stored thereon, which, when executed on a computer, causes the computer to execute the system according to any one of claims 1-5.

7. A computing device comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the system according to any one of claims 1-5 is implemented.

Images