CTA image-oriented intelligent coronary artery stenosis assessment method
By combining the Attention Res-UNet and Res2Net model with three-dimensional topological refinement and multi-angle trilinear interpolation surface reconstruction algorithm, the accuracy and clarity of automatic coronary stenosis assessment in CTA images were solved, and efficient identification of coronary stenosis was achieved. and accurate grading improves the efficiency and accuracy of coronary heart disease diagnosis.
Patent Information
- Application Number
- CN202410955683.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-17
- Publication Date
- 2024-11-15
AI Technical Summary
The automatic evaluation of coronary stenosis in CTA images in prior art is caused by the problem of doctors' dependence on manual reading, insufficient surface reconstruction, and insufficient accuracy in judging of stenosis.
采用基于注意力残差网络的Attention Res-UNet模型进行冠脉血管自动分割,并结合多尺度卷积神经网络Res2Net进行狭窄程度分类,结合三维拓扑细化算法和多角度三线性插值曲面重建算法,提取清晰的冠状动脉中心线和生成高清度的冠状动脉剖面图。
Automatic and efficient identification and accurate grading of coronary artery stenosis is achieved, the efficiency and accuracy of coronary heart disease diagnosis is improved, and accurate geometric parameters are provided and internal conditions of the blood vessels are displayed from multiple angles.
Smart Images

Figure CN118967578A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of medical image processing, and in particular relates to an intelligent coronary artery stenosis assessment method for CTA images. Background Art
[0002] The original CTA image is essentially a transverse tomographic slice of the human body scanned by a CT machine, which contains many organs and tissues of the human body. In actual diagnosis, it is not conducive to doctors to observe the lesions in the specified parts. In the past, the diagnosis of cardiovascular diseases usually relied on the doctor's manual reading experience. With the development of medical imaging technology, it is now possible to reconstruct the original CTA image to obtain the cross-sectional view of the blood vessels, helping doctors to conduct efficient analysis.
[0003] Coronary heart disease is a cardiovascular disease caused by vascular stenosis due to the presence of atherosclerotic plaques in the coronary artery lumen. Deep learning technology can automatically evaluate coronary artery stenosis, helping doctors to efficiently and accurately diagnose whether a patient has coronary heart disease. Summary of the invention
[0004] In order to overcome the shortcomings of the prior art, the present invention provides an intelligent coronary artery stenosis assessment method for CTA images. First, based on the semantic segmentation network U-Net, an Attention Res-UNet coronary artery stenosis segmentation network model based on the attention residual network is proposed for automatic segmentation of coronary vessels. Secondly, the coronary CPR original image is combined with the segmentation image obtained in the previous step for classification prediction, and improvements are made on the Res-Net50 basic model to propose a multi-scale convolutional neural network Res2Net coronary artery stenosis classification network model for judging the degree of stenosis of coronary vessels. The present invention divides the types of coronary artery stenosis into mild stenosis, moderate stenosis and severe stenosis according to the American CASS coronary artery disease vascular diagnostic criteria, thereby realizing automatic assessment of coronary artery stenosis.
[0005] The technical solution adopted by the present invention to solve the technical problem is as follows
[0006] Step 1: Coronary artery centerline extraction based on shortest path branch deletion;
[0007] Step 1-1: Use the deletion points of the twelve directions of the voxel, the twelve directions represent the twelve sides of the cube, and divide them into four groups, each group contains three directions;
[0008] Step 1-2: Convert the 3D skeleton line into a directed graph with directed edges and nodes, where a voxel point with only a single adjacent point is called an endpoint, a voxel point between two adjacent points is called a common link point, and if there are more than two adjacent points, it is regarded as a node; the edge between the common link points constitutes a directed edge; construct an n×n adjacency matrix Z according to the number of nodes N in the directed graph, and its expression is shown in formula (1):
[0009]
[0010] In the process of searching the coronary artery centerline, the path length from the i-th node to the j-th node is represented by zi j ; If the path length between node A and node B is shorter than the path length through node C, it means that node A and node B are adjacent; After searching all nodes, connect them in sequence to obtain the coronary artery centerline;
[0011] Step 2: Multi-angle trilinear interpolation surface reconstruction algorithm;
[0012] Step 2-1: Surface reconstruction algorithm based on Frenet framework;
[0013] The center line of the coronary artery is regarded as the motion trajectory of the center point in the entire three-dimensional space; the Frenet-Serret formula is shown in formula (2)
[0014]
[0015] Where T represents the unit tangent vector; N represents the unit normal vector, which is a vector obtained by differentiating the arc length parameter; B is the binormal vector, which is composed of the outer product of the tangent vector T and the normal vector N; d / ds is the integral used to calculate the arc length, and κ and τ are the curvature and torsion of the described curve respectively;
[0016] When reconstructing the coronary artery, in order to calculate the Frenet coordinates of each center point, the center line is regarded as a discrete center point and discretized; assuming that there are three continuous points M1, M2, and M3 on the center line of the coronary artery, the coordinates of the three points are recorded as [x1, y1, z1], [x2, y2, z2], and [x3, y3, z3] respectively, and the tangent vector of each point is determined by the relationship between the points, so as to obtain the average tangent vector at each point, thereby realizing the straightened surface reconstruction of the coronary artery. Its expression is shown in formula (3):
[0017]
[0018] By calculating the orthogonal relationship between the tangent vector and the normal vector, their product can be recorded as the normal vector of the point, and its expression is shown in formula (4):
[0019]
[0020] The average value of the tangent vector T and the normal vector N is recorded as the binormal vector of the point, and its expression is shown in formula (5):
[0021]
[0022] By orthogonalizing the three vectors, the Frenet coordinates of the center point M2 are calculated, and the gray value of the center point M2 can be obtained based on the coordinates; by traversing all the center points in the center line and using the Frenet formula to calculate their positions and gray values in the TNB coordinate system, and finally splicing these gray values in order, the image of the coronary vascular surface reconstructed can be obtained;
[0023] Step 2-2: trilinear interpolation algorithm;
[0024] Trilinear interpolation requires interpolation of eight adjacent points; point P (x, y, z) is in a three-dimensional space, where the left component point x of point P is rounded up and down to get x L With x H Similarly, the components y and z are rounded to integers, and after obtaining the component coordinates, the y direction is interpolated. V1 is calculated first. The calculation process is shown in formula (5):
[0025] V1=(y H -y)I(x H ,y H , z H ) (5) +(yy L )I(x H ,y H , z H )
[0026] Similarly, V2, V3 and V4 are calculated. According to the calculation results, linear interpolation is performed in the x direction. The calculation process is shown in formula (6):
[0027] V5=(x H -x)V4+(xx L )V3
[0028] V6=(x H -x)V1+(xx L )V2 (6)
[0029] Finally, interpolation calculation is performed in the z direction to obtain the gray value at point P. The calculation process is shown in formula (7):
[0030] V o =(zH -z)V6+(zz L )V5 (7)
[0031] After extracting the grayscale values of all points, the centerline data is uniformly sampled and repeated center points are deleted; ensuring that the minimum interval of the center point data after sampling is a single voxel block;
[0032] Step 2-3: Multi-angle surface reconstruction algorithm;
[0033] The vector of interest of the reconstructed volume data is changed to perform curved surface reconstruction on the coronary artery; five directions are uniformly selected from 0° to 180° along the X-axis direction as mapping directions to perform curved surface reconstruction on the coronary artery;
[0034] Step 3: Construct the Attention Res-UNet coronary artery stenosis segmentation network;
[0035] The Attention Res-UNet coronary artery stenosis segmentation network adds a residual module and a spatial attention mechanism to the U-Net network framework. First, the residual module replaces the convolutional module in the encoding and decoding stages of the basic network. Second, in the decoding stage, the attention mechanism is added before the jump connection. Finally, a 1×1 convolution is used to adjust the channel and output the final prediction result of the model.
[0036] Step 3-1: Residual module in Attention Res-UNet;
[0037] The convolution modules in the original encoding and decoding stages of the U-Net network are changed to residual modules. The residual module in AttentionRes-UNet includes three convolution layers. The first convolution layer is used to reduce the dimension of the feature channel, and then the feature channel is restored in the second 11 convolution layer. Then, the residual block identity mapping method is used for calculation, and finally, they are superimposed as the output of the residual module.
[0038] Step 3-2: Attention mechanism in Attention Res-UNet;
[0039] The Attention Gate module is integrated into the upsampling module of the U-Net network through skip connections to implement a spatial attention mechanism, highlighting the vascular regions that need to be segmented in the CPR image and suppressing background noise, motion artifacts, and other possible vascular regions.
[0040] (1) Spatial Attention
[0041] Use the spatial attention mechanism SAM to allocate attention to the coronary artery area, thereby enhancing the target area and suppressing the background area;
[0042] First, an average pooling operation and a maximum pooling operation are performed on a H×W×C feature F. In this process, two new H×W×1 features are obtained, which are recorded as At the same time, these two new features are concatenated together; a 7×7 convolution operation and Sigmod activation function are performed on them to obtain the weight coefficient Ms; finally, Ms is multiplied by the input feature F, and the result is the effective feature F′ that has more coronary vessel locations than the input feature F; the calculation formula of the SAM weight coefficient Ms is as shown in formula (8), where f 7*7 Represents a 7×7 convolutional layer;
[0043]
[0044] (2) Attention mechanism of Attention Res-UNet;
[0045] The Attention Res-UNet algorithm introduces an attention module in the decoding stage of the U-Net network and adds an attention mechanism before the skip connection;
[0046] Let x l is the feature extracted during the encoding process, g l To extract the features during the decoding process, first convert x l and g l Add them together so that the repeated areas of the two have a larger activation value; secondly, the spatial attention SAM is used to calculate the attention weight coefficient σ1, so as to obtain the vascular feature information to be segmented in the coronary CPR image and suppress useless information such as noise and motion artifacts in the background; the output feature of the attention mechanism is the encoding stage x l with a l The result of element-wise multiplication Finally, the feature is concatenated and fused with the feature obtained in the decoding stage. The calculation formula of the output feature of the attention mechanism is shown in formula (9):
[0047]
[0048] Adopt additive attention to obtain a l , the additive attention formula is shown in formula (10) and formula (11):
[0049] Q l =ψ(σ1(W x x l +W g g l +b g ))+b ψ (10)
[0050] a l =σ2(Q l ) (11)
[0051] Among them, σ1 and σ2 correspond to the ReLu activation function and the Sigmoid activation function respectively, W x With W g is the attention weight, b g With b ψ For the paranoid item in attention;
[0052] Step 4: Coronary artery stenosis classification network model based on multi-scale convolutional neural network Res2Net;
[0053] A multi-channel residual connection is constructed in the Res-Net50 network framework, and a pre-fusion method is used to fuse multiple target-related features. First, the original coronary CPR image is fused with the segmented image so that the network can obtain more features and thus improve the accuracy of classification prediction. Second, the residual module in the Res-Net50 network is reconstructed to increase the receptive field of each layer of the network.
[0054] Step 4-1: Feature fusion in multi-scale convolutional neural network Res2Net;
[0055] The original coronary CPR image and the segmented coronary CPR image are feature-fused to increase the feature dimension input into the network, so that the network can obtain more feature information of the coronary vessels and improve the accuracy of image classification.
[0056] The pre-fusion method is used to perform feature fusion on the coronary artery stenosis classification network: fusion is performed on the input layer, firstly fusing multiple layers of features and then inputting them into the deep learning network;
[0057] Step 4-2: Res2Net module in the multi-scale convolutional neural network Res2Net;
[0058] The residual module Res2Net is used to construct a hierarchical residual connection in the Res-Net network. In the Res2Net module, the 3×3 convolution layer is replaced by a group of small convolutions, and different convolution groups are connected layer by layer in a similar way to the residual connection, forming a new neural network module.
[0059] In the Res2Net module, the feature map x is firstly divided into s subsets after 1×1 convolution, and each subset is represented by x i Represents, where i∈{1, 2, 3, ..., s}; the number of channels of each subset after splitting is 1 / s of the original, and the spatial size is consistent with the original feature map; except for the first feature subset, each feature subset xi There are corresponding 3×3 convolutional layers, with K i () indicates that the output is defined as y i ; In the process, in order to reduce parameters and increase the number of s, x1 does not undergo 3×3 convolution, that is, y1=x1, and the remaining feature subsets x i and K i-1 () are added and sent to K i () is processed; therefore y i It is expressed by formula (12):
[0060]
[0061] Preferably, the Attention Res-UNet coronary stenosis segmentation network consists of an encoding stage, a decoding stage and a prediction stage; in the encoding stage, the original convolution module is replaced by a residual module to form a residual unit for extracting features; the encoding stage consists of five residual units for extracting feature information of the coronary CPR image, and then a 2×2 maximum pooling operation is performed to retain the main features of the image and reduce the image size; in this stage, the network obtains five preliminary effective feature layers, which are then sent to the decoding stage for feature fusion, so the role of the encoding stage is to extract the features of the coronary CPR image and expand the receptive field; in the decoding stage, the preliminary effective feature layer obtained in the encoding stage is used for upsampling, and in this stage, the original convolution module is also replaced by the residual module for deconvolution, and the attention mechanism is added before the jump connection, so that the coronary vascular features transmitted in the process are fused to obtain the final effective feature layer; finally, a 1×1 convolution is used to reduce the dimension of the channel, and the features obtained in the first two stages are used for prediction, and the coronary vessels and background are classified by Softmax, so as to achieve the purpose of segmentation, and the final segmentation result of the coronary CPR image is output.
[0062] Preferably, the coronary CPR original image and the segmented image are fused in an add manner.
[0063] The beneficial effects of the present invention are as follows:
[0064] 1. The present invention proposes a coronary artery centerline extraction method based on a three-dimensional topological thinning algorithm. This method can effectively extract clear vascular centerlines without redundant branches from CT images, providing accurate geometric parameters for subsequent vascular analysis.
[0065] 2. The present invention constructs a multi-angle trilinear interpolation surface reconstruction algorithm, which can display the internal conditions of blood vessels at different angles and improve the accuracy of vascular stenosis assessment. By combining the Frenet framework and trilinear interpolation technology, the algorithm can generate high-definition coronary artery cross-sectional images.
[0066] 3. The present invention has developed two network models based on deep learning: one is Attention Res-UNet, which is used for automatic segmentation of coronary vessels; the other is Res2Net, which is used for automatic classification of the degree of vascular stenosis. These two models can achieve efficient recognition and accurate classification of coronary stenosis by learning image features, thereby assisting doctors in diagnosing coronary heart disease.
[0067] 4. This invention focuses on medical image processing and automatic diagnosis of cardiovascular diseases, and improves the efficiency and accuracy of coronary artery disease diagnosis through algorithm optimization and deep learning technology. BRIEF DESCRIPTION OF THE DRAWINGS
[0068] Figure 1 It is a schematic diagram of the twelve directions of voxels;
[0069] Figure 2 It is the coronary artery centerline directed graph model;
[0070] Figure 3 The Frenet coordinate calculation result of the center point of the coronary artery;
[0071] Figure 4 It is a schematic diagram of the trilinear interpolation principle;
[0072] Figure 5 This is a schematic diagram of multi-angle reconstruction;
[0073] Figure 6 This is the Attention Res-UNet network structure diagram;
[0074] Figure 7 Schematic diagram of the residual module in Attention Res-UNet;
[0075] Figure 8 Schematic diagram of the spatial attention SAM module;
[0076] Fig. 9 Schematic diagram of the convolution structure combined with spatial attention;
[0077] Fig.10 Schematic diagram of the attention mechanism of Attention Res-UNet;
[0078] Fig.11 This is a schematic diagram of the Res2Net module structure;
[0079] Fig.12 Comparison diagram of centerline extraction by traditional algorithm and algorithm of the present invention, (a) centerline extraction effect of traditional algorithm (b) centerline extraction effect of algorithm of the present invention;
[0080] Fig.13 The results of coronary artery centerline extraction are as follows: (a) coronary artery centerline of patient 1, (b) coronary artery centerline of patient 2, (c) coronary artery centerline of patient 3, (d) coronary artery centerline of patient 4, (e) coronary artery centerline of patient 5, and (f) coronary artery centerline of patient 6.
[0081] Fig.14 The coronary artery surface reconstruction effects of different interpolation algorithms, (a) coronary artery straightening CPR image using the traditional zero-order interpolation algorithm, (b) coronary artery straightening CPR image using the algorithm of the present invention;
[0082] Fig.15 The coronary artery segmentation prediction results are as follows: (a) sample 1 CPR image and segmentation image, (b) sample 2 CPR image and segmentation image, (c) sample 3 CPR image and segmentation image, (d) sample 4 CPR image and segmentation image;
[0083] Fig.16 Figures of blood vessels with different degrees of stenosis and their segmentation prediction results, (a) healthy blood vessels and their segmentation prediction map, (b) mildly stenotic blood vessels and their segmentation prediction map, (c) moderately stenotic blood vessels and their segmentation prediction map, (d) severely stenotic blood vessels and their segmentation prediction map;
[0084] Fig.17 The segmentation prediction results of the same coronary artery at different angles;
[0085] Fig.18 The following are the results of the classification and prediction of the degree of coronary artery stenosis: (a) a vessel with mild stenosis, (b) a vessel with moderate stenosis, and (c) a vessel with severe stenosis.
[0086] Fig.19 This is the loss diagram of the classification algorithm of the present invention. DETAILED DESCRIPTION
[0087] The present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0088] An intelligent coronary artery stenosis assessment method for CTA images comprises the following steps:
[0089] 1. Centerline extraction based on shortest path branch deletion;
[0090] In the topology refinement algorithm, simplified points can be effectively eliminated, while the traditional three-dimensional topology refinement algorithm will delete them in the direction pointed by the six cubic faces of the voxel. In this process, the topological structure of the centerline may change, thus affecting the quality and accuracy of the image. At the same time, since the coronary vessels of each patient are not the same regular tubular structure, it is inevitable that redundant small branches will be generated during the three-dimensional topology refinement process, interfering with subsequent coronary surface reconstruction and vascular stenosis assessment research. In response to the above problems, the present invention designs a centerline extraction algorithm based on the shortest path branch deletion. This method can obtain a complete vascular centerline and eliminate the interference caused by redundant branches, further improving the subsequent surface reconstruction effect.
[0091] First, in order to better obtain the centerline of the coronary artery, the present invention uses twelve directions of deletion points, such as Figure 1 As shown in the figure, these twelve directions represent the twelve edges of the cube, which are divided into four groups, each containing three directions, so as to perform the deletion operation of simplified points. The biggest advantage of deleting points in this way is that each group contains six main directions of voxels, which can ensure that the target area is evenly refined in all directions during the refinement process, and deleting simplified points simultaneously in the order of grouping can also improve the efficiency of topological refinement, and can achieve efficient extraction of the centerline of the coronary artery.
[0092] Secondly, the three-dimensional skeleton line is converted into a directed graph with directed edges and nodes, where voxel points with only a single adjacent point are called endpoints, and voxel points between two adjacent points are called ordinary link points. If there are more than two adjacent points, they are regarded as nodes, and the edges between these ordinary link points can form a directed edge. According to the number of nodes N in the directed graph, an n×n adjacency matrix Z is constructed, and its expression is shown in formula (1).
[0093]
[0094] In the process of searching the coronary artery centerline, the path length from the i-th node to the j-th node is represented by z ij , if the path length between node A and node B is shorter than the path length through node C, it means that node A and node B are adjacent. After searching all nodes, connect them one by one to get the complete coronary artery centerline without small branches. Figure 2 As shown in the figure, A and E are the two endpoints of the centerline of the blood vessel, and B, C, and D are the nodes on the centerline. Assume that there are two paths from node B to node C, one of which is path l Bc , and the other is a path that passes through node D and then reaches node D. Since node B and node C are adjacent nodes, the shortest path between node B and node C is l Bc, while nodes B and C are connected to other branch vessels in this process, which do not belong to the shortest path in the centerline, and thus are ignored. By screening each point on the centerline according to this criterion, a complete centerline without redundant small vessels can be obtained.
[0095] 2. Multi-angle trilinear interpolation surface reconstruction algorithm;
[0096] In order to achieve straightened surface reconstruction of coronary artery images, the present invention designs a multi-angle trilinear interpolation surface reconstruction algorithm, combines the coronary centerline with the Frenet framework, and achieves surface reconstruction of coronary artery vessels. At the same time, in view of the problems of motion artifacts and blurred blood vessel edges in traditional surface reconstruction algorithms, the present invention combines the trilinear interpolation algorithm to improve the overall clarity of the surface reconstruction image, and changes the reconstruction angle by changing the vector of interest, solving the limitations of the traditional surface reconstruction algorithm due to the single observation angle, so that the image after surface reconstruction can show the situation inside the coronary artery lumen from multiple angles.
[0097] 2.1 Surface reconstruction algorithm based on Frenet framework;
[0098] The present invention constructs a straightening surface reconstruction algorithm based on the Frenet framework, in which the center line of the coronary artery can be regarded as the motion trajectory of the center point in the entire three-dimensional space, so as to better depict the specific position of the coronary blood vessels in the three-dimensional space and their topological structure. The Frenet-Serret formula is shown in formula (2). Among them, T represents the unit tangent vector, N represents the unit normal vector, which is a vector obtained by differentiating the arc length parameter. B is the binormal vector, which is composed of the outer product of the tangent vector T and the normal vector N. d / ds is used to calculate the integral of the arc length, and κ and τ are the curvature and torsion of the described curve, respectively.
[0099]
[0100] When reconstructing the coronary artery, in order to calculate the Frenet coordinates of each center point, the center line is regarded as a discrete center point and discretized. Assuming that there are three continuous points M1, M2, and M3 on the center line of the coronary artery, the coordinates of the three points are recorded as [x1, y1, z1], [x2, y2, z2], and [x3, y3, z3], respectively, and the tangent vector of each point is determined by the relationship between the points, so as to obtain the average tangent vector at each point, thereby realizing the straightened surface reconstruction of the coronary artery. Its expression is shown in formula (3).
[0101]
[0102] By calculating the orthogonal relationship between the tangent vector and the normal vector, their product can be recorded as the normal vector of the point, and its expression is shown in formula (4).
[0103]
[0104] Since the binormal vector B is the outer product of the tangent vector T and the normal vector N, there is an orthogonal relationship between them. Therefore, their average value can be recorded as the binormal vector of the point, and its expression is shown in formula (5).
[0105]
[0106] By orthogonalizing the three vectors, the Frenet coordinates at the center point M2 can be calculated. Figure 3 As shown. According to the coordinates, the gray value of the center point M2 can be obtained. By traversing all the center points in the center line and using the Frenet formula to calculate their positions and gray values in the TNB coordinate system, and finally splicing these gray values in sequence, the image of the reconstructed coronary vascular surface can be obtained.
[0107] 2.2 Trilinear interpolation algorithm;
[0108] After obtaining the coordinates of the center point in the projection surface in the previous step, grayscale interpolation is required to obtain the grayscale value of each center point. Among the grayscale interpolation algorithms, the zero-order interpolation algorithm has the fastest speed and the least amount of calculation. The output value is determined by the grayscale value of the nearest pixel, but mosaic phenomenon will occur, which can easily cause the edge of the coronary artery to be blurred in the reconstructed image. In order to ensure the speed and accuracy of reconstruction, the present invention designs a trilinear interpolation algorithm to perform grayscale interpolation during the coronary surface reconstruction process. The principle of trilinear interpolation is as follows: Figure 4 shown.
[0109] Trilinear interpolation requires eight adjacent points to be interpolated. As can be seen from the figure, point P (x, y, z) is in a three-dimensional space, where the left component point x of point P is rounded up and down to get x L With x H Similarly, the components y and z are rounded. After obtaining the component coordinates, interpolation is performed in the y direction, and V1 is calculated first. The calculation process is shown in formula (5).
[0110] V1=(y H -y)I(x H ,y H , z H ) (5) +(yy L )I(x H ,y H , z H )
[0111] Similarly, V2, V3 and V4 are calculated. According to the calculation results, linear interpolation is performed in the x direction. The calculation process is shown in formula (6).
[0112] V5=(x H -x)V4+(xx L )V3 (6)
[0114] V6=(x H -x)V1+(xx L )V2
[0115] Finally, interpolation calculation is performed in the z direction to obtain the grayscale value at point P. The calculation process is shown in formula (7).
[0116] V o =(z H -z)V6+(zz L )V5 (7)
[0117] After extracting the grayscale values of all points, in order to ensure the accuracy and authenticity of the blood vessels in the coronary CPR image, the centerline data needs to be uniformly sampled and duplicate center points deleted. In order to ensure that the number of projection points after sampling is equal to the height of the output image, it is necessary to ensure that the minimum interval of the center point data after sampling is a single voxel block, so that an accurate coronary CPR image can be obtained and the original length of the blood vessel can be maintained.
[0118] 2.3 Multi-angle surface reconstruction algorithm;
[0119] Since the area of coronary artery stenosis is located at different positions in the vascular lumen, the degree of stenosis is different for each patient. Reconstructing the coronary artery in a fixed direction alone cannot assist doctors in making accurate diagnoses. To solve this problem, the present invention reconstructs the coronary artery in a curved surface by changing the vector of interest of the reconstructed volume data, such as Figure 5 As shown, five directions are uniformly selected from 0° to 180° along the X-axis direction as mapping directions to perform surface reconstruction on the coronary artery. Through this method, the purpose of observing the entire coronary artery from multiple angles can be achieved.
[0120] 3. Attention Res-UNet coronary artery stenosis segmentation network;
[0121] The present invention proposes an Attention Res-UNet model based on an attention residual network for automatic segmentation of coronary CPR images. The Attention Res-UNet model adds a residual module and a spatial attention mechanism to the U-Net network framework. First, the residual module replaces the convolution module in the encoding and decoding stages of the basic network to improve the performance of network training and enhance the ability of the segmentation network to extract coronary vascular features. Secondly, in the decoding stage, the present invention can effectively suppress the noise generated in the encoding stage and the useless information in the coronary CPR image by adding the attention mechanism before the jump connection, so that the network can focus on the segmentation target, thereby improving the accuracy of the segmentation of the coronary CPR image. Finally, a 1×1 convolution is used to adjust the channel and output the final prediction result of the model.
[0122] Attention Res-UNet model Figure 6 As shown, it is also composed of an encoding stage, a decoding stage, and a prediction stage. The left side of the model is the encoding stage. In the encoding stage, the present invention replaces the original convolution module with a residual module to form a residual unit for extracting features. As can be seen from the figure, the encoding stage is composed of five residual units, which are used to extract the feature information of the coronary CPR image, and then the main features of the image are retained and the image size is reduced after a 2×2 maximum pooling operation. In this stage, the network can obtain five preliminary effective feature layers, which are then sent to the decoding stage for feature fusion. Therefore, the role of the encoding stage is to extract the features of the coronary CPR image and expand the receptive field. The right side of the network is the decoding stage, which uses the preliminary effective feature layer obtained in the encoding stage for upsampling. In this stage, the residual module replaces the original convolution module for deconvolution, and the attention mechanism is added before the jump connection, so that the coronary vascular features transmitted by the process are fused to obtain the final effective feature layer. Finally, a 1×1 convolution is used to reduce the dimension of the channel, and the features obtained in the first two stages are used for prediction. The coronary vessels and background are classified through Softmax to achieve the purpose of segmentation, and the final segmentation result of the coronary CPR image is output.
[0123] Compared with the traditional U-Net basic network, this algorithm has the following advantages:
[0124] (1) The convolution modules in the original encoding and decoding stages of the U-Net network are replaced with residual modules. In this way, the network can retain the information lost in different layers in the original encoding stage, while increasing the network depth to extract more refined features in the image and improve the ability to segment coronary vessels.
[0125] (2) Adding the attention mechanism before the jump connection in the network can effectively suppress the noise generated in the encoding stage and the useless information in the background area, and reduce the loss caused by the loss of part of the image detail information during the upsampling process, further improving the accuracy of coronary artery image segmentation.
[0126] 3.1 Residual module in Attention Res-UNet;
[0127] Therefore, the present invention changes the convolutional modules in the original encoding and decoding stages of the U-Net network into residual modules. This method can not only retain the detailed features that may be lost in each layer, but also increase the network depth to extract more abstract features in the coronary CPR images. Figure 7 shown.
[0128] The residual module in Attention Res-UNet includes three convolutional layers. The first convolutional layer is used to reduce the dimension of the feature channels, and then restore these channels in the second 11 convolutional layer. Then, the residual block identity mapping method is used for calculation, and finally, they are superimposed as the output of the residual module. By adopting this method, the time and parameter amount of feature extraction can be greatly shortened, while effectively utilizing the coronary vascular features extracted by each convolutional layer.
[0129] 3.2Attention mechanism in Res-UNet;
[0130] The present invention integrates the Attention Gate module into the upsampling module of the U-Net network through skip connections, realizes the spatial attention mechanism, highlights the vascular areas that need to be segmented in the CPR image, suppresses noise, motion artifacts and other possible vascular areas in the background, solves the problem that other redundant vascular areas in the background may be segmented in the traditional method, and further improves the accuracy of coronary image segmentation.
[0131] (1) Spatial Attention
[0132] From the perspective of spatial distribution, the vascular area that needs to be segmented in the coronary surface reconstruction image usually only exists in the middle of the image, and other areas are vascular motion artifacts or plaques of various shapes. However, the U-Net network will perform feature extraction for the entire image area, which will cause other tissues in the background area to be regarded as blood vessels that need to be segmented by the network, thereby interfering with the segmentation results, and may segment out redundant blood vessels in the background area or segment out blood vessels with plaques. In order to solve the above problems, the present invention uses a spatial attention mechanism (Spatial Attention Module, SAM) to allocate attention to the coronary vascular area, thereby achieving the purpose of enhancing the target area and suppressing the background area.
[0133] The main function of spatial attention SAM is to pay more attention to the location of the target area in the image space, so that the segmentation network can pay more attention to the location information of the vascular area in the coronary CPR feature map. The implementation process is as follows: First, an H×W×C feature F is averaged and max pooled. In this process, two new H×W×1 features can be obtained, which are respectively denoted as At the same time, these two new features are concatenated together. Then a 7×7 convolution operation and Sigmod activation function are performed to obtain the weight coefficient Ms. Finally, Ms is multiplied by the input feature F, and the result is the effective feature F′ that has more coronary vascular locations than the input feature F. The calculation formula of the SAM weight coefficient Ms is as shown in formula (8). Where, f 7*7 Represents a 7×7 convolutional layer.
[0134]
[0135] The spatial attention SAM module is as follows Figure 8 shown.
[0136] The present invention introduces the spatial attention module SAM into the convolution layer before the skip connection structure of the U-Net network to obtain a more accurate feature map, highlight the blood vessel position information, and suppress useless information in the background, thereby achieving the purpose of faster segmentation of blood vessel parts. The structure is as follows Fig. 9 shown.
[0137] (2) Attention mechanism of Attention Res-UNet
[0138] The Attention Res-UNet algorithm of the present invention solves the problem of detail information loss and edge blurring caused by upsampling in coronary CPR image segmentation of traditional algorithms by introducing an attention module in the decoding stage of the traditional U-Net network. By adding the attention mechanism before the jump connection, the network can enhance the focus on the location information of the coronary vessels and suppress background interference, thereby retaining more detail features in the high-level feature map and improving segmentation accuracy. In addition, the introduction of the attention mechanism is particularly effective in shallow network structures, and the operation during the upsampling process makes the output features more directly affect the classification results, effectively improving the efficiency of the model. The attention mechanism of Attention Res-UNet is as follows: Fig.10 shown.
[0139] Among them, x l is the feature extracted during the encoding process, g l To extract the features during the decoding process, first convert x l and g l Add them together so that the repeated areas of the two have a larger activation value. Secondly, the spatial attention SAM is used to calculate the attention weight coefficient σ1, so as to obtain the vascular feature information to be segmented in the coronary CPR image and suppress useless information such as noise in the background and motion artifacts. The output feature of the attention mechanism is the encoding stage x l with a l The result of element-wise multiplication Finally, the feature is concatenated and fused with the feature obtained in the decoding stage. The calculation formula of the output feature of the attention mechanism is shown in formula (9).
[0140]
[0141] The present invention uses additive attention to obtain a l , so that a larger value can be obtained in the ROI area, and a smaller value can be obtained in irrelevant areas such as the background, thereby achieving the purpose of screening important features in coronary CPR images. The additive attention formula is shown in Equation (10) and Equation (11).
[0142] Q l =ψ(σ1(W x x l +W g g l +b g ))+b ψ (10)
[0143] a l =σ2(Q l ) (11)
[0144] Among them, σ1 and σ2 correspond to the ReLu activation function and the Sigmoid activation function respectively. x With W g is the attention weight, b g With b ψ It is the paranoid item in attention.
[0145] 4. Coronary artery stenosis classification network model based on multi-scale convolutional neural network Res2Net;
[0146] The present invention proposes a multi-scale convolutional neural network Res2Net coronary artery stenosis classification network model, constructs a multi-channel residual connection in the Res-Net50 network framework, and uses a preliminary fusion method to fuse multiple target-related features. First, the coronary CPR original image is fused with the segmented image so that the network can obtain more features to improve the accuracy of classification prediction. Secondly, the residual module in the Res-Net50 network is reconstructed to increase the receptive field of each layer of the network.
[0147] Compared with the traditional algorithm, this algorithm has the following improvements:
[0148] (1) Feature fusion of the original image and segmentation image of the coronary CPR image at the input layer is beneficial for the network to extract features of different attributes, thereby better judging the degree of coronary artery stenosis and improving model performance.
[0149] (2) The residual blocks of the original Res-Net network are improved by creating residual links on each residual block to improve the performance and reliability of the network. In this way, the network can obtain information at more scales, which can better understand the classification target and improve the accuracy of coronary artery stenosis classification.
[0150] 4.1 Feature fusion in multi-scale convolutional neural network Res2Net;
[0151] When classifying the segmented coronary CPR images, sometimes the information of the stenosis is not obvious due to the single information of the segmented image, which leads to misclassification. In view of the above problem, the present invention fuses the features of the original coronary CPR images with the segmented coronary CPR images, increases the feature dimension input to the network, so that the network obtains more feature information of the coronary vessels and improves the accuracy of image classification.
[0152] The purpose of feature fusion is to input features of different attributes into the network, and use the complementarity between features to fuse the advantages between features, thereby improving the performance of the model. The present invention adopts the pre-fusion method to perform feature fusion on the coronary vascular stenosis classification network. This method performs fusion on the input layer, first fuses multiple layers of features, and then inputs them into the deep learning network. Pre-fusion has the following advantages for coronary CPR image classification:
[0153] There may be noise or vascular motion artifacts in each coronary CPR image, and the segmented vascular map is equivalent to marking the vascular part. Fusion of the two and then inputting them into the network helps the network better judge the degree of coronary stenosis and improve the accuracy of classification prediction.
[0154] The early fusion method can effectively extract features and only requires a single feature extraction network, which can save a lot of memory space and greatly improve the efficiency of model training.
[0155] In the process of image fusion, common feature fusion methods include add and concatenate. The add operation can increase the amount of data information in each dimension while keeping the angle constant, while concatenate combines the number of channels of the image to increase the angle of the image feature, but does not change the feature information at the same level. For the coronary stenosis classification network, the increase in the amount of information describing the image in each dimension is more conducive to the classification of coronary vessels with different degrees of stenosis. Therefore, the present invention uses the add method to fuse the original coronary CPR image with its segmentation map. This method performs better in the classification network, and compared with the concatenate method, this method saves more parameters and calculations.
[0156] 4.2 Res2Net module in the multi-scale convolutional neural network Res2Net;
[0157] In order to improve the detail information and receptive field of the coronary vascular area in the feature map during the feature extraction process of the coronary stenosis network, the present invention constructs a hierarchical residual connection in the residual block of ResNet50. This method is beneficial for the network to better extract features at different levels in the coronary CPR original image and segmentation image, learn the differences between blood vessels with different degrees of stenosis, and thus improve the classification accuracy.
[0158] When extracting multi-scale features, the existing feature extraction network usually directly stacks different feature layers to form multi-scale features, but this method can never fundamentally solve the problem of lack of target features. To address this problem, the present invention adopts a residual module Res2Net, which can construct a hierarchical residual connection in the Res-Net network, thereby effectively solving the problem. And as a building block of a convolutional neural network, it can better implement complex model predictions. The structure of Res2Net used in the present invention is as follows: Fig.11 As shown in the figure, compared with the bottleneck structure in ordinary ResNet, in the Res2Net module, the 3×3 convolution layer is replaced by a group of small convolutions, and different convolution groups are connected layer by layer in a similar way to residual connection to form a new neural network module. Through this method, the coronary stenosis classification network can learn more detailed features of the coronary vessels, which is helpful to improve the classification accuracy.
[0159] In the Res2Net module, the feature map x first undergoes a 1×1 convolution and is then divided into s subsets, each with x i Represents, where i∈{1, 2, 3, ..., s}. After splitting, the number of channels in each subset is 1 / s of the original, and the spatial size is consistent with the original feature map. Except for the first feature subset, each feature subset x i There are corresponding 3×3 convolutional layers, with K i () indicates that the output is defined as y i In order to reduce parameters and increase the number of s, x1 is not processed by 3×3 convolution, that is, y1=x1, and the remaining feature subsets x i and K i-1 () are added and sent to K i () is processed. Therefore, y i It can be expressed by formula (12).
[0160]
[0161] Example:
[0162] 1. Experimental results and analysis of centerline extraction;
[0163] (1) Comparison of the traditional algorithm and the algorithm of the present invention
[0164] The comparison between the traditional 3D topology refinement algorithm and the algorithm of the present invention for a single coronary vessel is shown in the figure below. Fig.12As shown in the figure, compared with the traditional three-dimensional topological thinning algorithm, the coronary artery centerline obtained by the algorithm of the present invention has no redundant small branches, and the blood vessel centerline is complete, and the extraction effect is good.
[0165] (2) The effect of the algorithm of the present invention on the extraction of coronary artery centerline of different patients
[0166] Fig.13 The algorithm of the present invention extracts part of the coronary artery centerline from 264 groups of CT images, including the left and right coronary arteries. As can be seen from the figure, the coronary centerline obtained by the algorithm of the present invention is complete and smooth, without redundant vascular branches, and good extraction results can be obtained for different patients, which can provide a data basis for subsequent coronary artery surface reconstruction.
[0167] (3) Evaluation results of the algorithm of the present invention
[0168] The present invention evaluates the accuracy of centerline extraction by using the overall overlap (Overlap, OV) and the overlap until first error (Overlap until First error, OF) in the Rotterdam coronary algorithm evaluation frame, and the calculation formulas are shown in Formulas (13) and (14).
[0169]
[0170] Among them, TPM OV The error radius is a point on the centerline extracted from the standard reference line, and its position may be closer to the centerline than the expected error radius. OV Refers to the point on the actual center line that matches it. The point that exceeds the error radius is marked as FP OV , the point on the actual center line is denoted as FN OF In order to ensure that the algorithm of the present invention has strong robustness, 6 groups of different coronary vessels were selected for evaluation, and the evaluation indexes OV and OF results are shown in Table 1. According to the table, it can be concluded that the average OV and OF of the centerline obtained by the algorithm of the present invention reached 91.4% and 87.3% respectively. The centerline of the coronary vessel can be accurately extracted.
[0171] Table 1 Centerline extraction results of the algorithm of the present invention
[0172]
[0173] 2. Experimental results and analysis of coronary artery curved surface reconstruction;
[0174] Fig.14 The straightened CPR image effect obtained by using the traditional zero-order interpolation algorithm and the grayscale interpolation algorithm of this paper for the same coronary artery centerline is shown. Fig.14(a) is the coronary artery straightening CPR image using the zero-order interpolation algorithm for grayscale interpolation. The edge of the coronary artery appears mosaic and blurred. Fig.14 (b) is a coronary artery straightening CPR image using the grayscale interpolation algorithm of the present invention. Fig.14 (a), the edges of the coronary vessels are clearer and smoother, and the image quality is significantly improved.
[0175] 3. Analysis of coronary artery segmentation experimental results;
[0176] (1) Comparative analysis of coronary vessel segmentation experimental results;
[0177] The model used in the present invention is a semantic segmentation network model. Therefore, the common semantic segmentation network is used to segment the coronary CPR images and compared with the algorithm of the present invention. In order to evaluate the improvement of the model performance of the algorithm constructed by the present invention, the category average pixel accuracy mPA and average intersection over union mIoU evaluation indicators mentioned in the present invention are used to evaluate the model performance. The comparison results of the coronary vessel segmentation experiment are shown in Table 2.
[0178] Table 2 Comparative analysis of experimental performance of different segmentation network models
[0179]
[0180] As can be seen from Table 2, compared with the FCN8s network, SegNet network, DeepLabV3 network, and U-Net network, the segmentation accuracy of the algorithm Attention Res-UNet of the present invention is superior. First of all, the FCN8s network has the lowest accuracy and is not suitable for segmenting medical images. Since the SegNet network lacks the jump connection unique to the U-Net network, some underlying features may be lost during the convolution operation, so the accuracy in the segmentation of coronary CPR images is not high; the segmentation accuracy of the DeepLabV3 network is slightly different from that of the U-Net network, but it is not as good as the U-Net network in terms of model size and computing speed, and its performance on small data sets is not as good as that of the U-Net network. Through the comparison of multiple different network models, it is fully proved that the U-Net network is the most suitable network model for segmenting coronary CPR images. At the same time, the improvement of the U-Net basic network by the algorithm of the present invention has further improved the segmentation accuracy of coronary CPR images.
[0181] (2) Comparative analysis of coronary vessel segmentation and ablation experimental results;
[0182] The algorithm of the present invention is an improved model based on the U-Net network. Therefore, this network is used as the baseline of the experimental results. The same evaluation indicators as the comparative experiment, namely mPA and mIoU evaluation indicators, are used to evaluate the model performance. The comparative results of the coronary vessel segmentation and ablation experiment are shown in Table 3.
[0183] Table 3 Comparative analysis of segmentation model ablation experimental performance
[0184]
[0185] As can be seen from the table, after the convolution module in the original U-Net network is changed to the residual module of Res-Net, the accuracy of coronary vessel segmentation mIoU is increased from 77.62% to 82.91%, which is a relatively large improvement. This is because the addition of the residual module makes it possible to form a deeper network, which is more beneficial for feature extraction and improves the problem of network degradation to a certain extent. After the original U-Net network introduces spatial attention, the accuracy mIoU is increased to 80.98%, indicating that the attention mechanism can effectively screen features and further improve the model performance. This experimental comparison fully illustrates that whether adding a residual module or introducing an attention mechanism has a certain effect on improving the accuracy of coronary vessels. The Attention Res-UNet model based on the attention residual network proposed in the present invention introduces both at the same time, which has a better improvement on the performance of the original network model. The final accuracy of coronary vessel segmentation reaches 86.37%, which is 6.75% higher than the original U-Net basic network. The result proves that the model constructed by the present invention is effective and can obtain a better coronary vessel segmentation effect.
[0186] (3) Comparative analysis with different literature
[0187] A comparison between the algorithm of the present invention and the algorithm proposed by Benjamin in 2018 is shown in Table 4. Benjimin used the U-Net network to segment coronary angiography images with an accuracy of 70.40%. The algorithm of the present invention improved the U-Net network to segment coronary CPR images with an accuracy of 86.37%. In comparison, the algorithm of the present invention has a better accuracy, which is improved by about 16%.
[0188] Table 4 Comparison of different blood vessel segmentation methods
[0189]
[0190] (4) Coronary artery segmentation visualization results
[0191] The coronary artery segmentation map obtained by the algorithm of the present invention is as follows: Fig.15As shown in the figure, these four samples are all coronary vessels with stenosis. From the results, it can be seen that the algorithm of the present invention can effectively segment the blood vessels from the coronary CPR image, and can also achieve complete segmentation for the blood vessel area with large plaques, and the motion artifacts in the background area and the redundant small blood vessel branches can also be ignored, which fully demonstrates the effectiveness of the algorithm of the present invention.
[0192] Fig.16 The segmentation prediction images of four different stenosis degrees of coronary artery, namely healthy blood vessels, mild stenosis, moderate stenosis and severe stenosis, are shown in turn. Fig.16 (a) is a normal person's blood vessel. It can be seen that there is no plaque or stenosis on the blood vessel. Its segmentation prediction map can completely segment the blood vessel and effectively ignore the vascular motion artifacts in the image. Fig.16 In (b), a smaller plaque can be seen on the right side. From the segmentation prediction image, it can be seen that the degree of stenosis caused by the plaque on the blood vessel is very small, which is a mild stenosis. Fig.16 (c) There are two plaques on the coronary artery. The segmentation prediction map effectively separates the plaque area and does not segment it into the blood vessel area. However, the segmentation accuracy of the area with dense plaques needs to be improved. Fig.16 It can be seen from (d) that the plaque area occupies most of the area of the blood vessel lumen, causing severe stenosis. The segmentation effect of the narrowed part of the blood vessel in the segmentation prediction image is good, and there is no breakage phenomenon, which shows that the network pays more attention to the blood vessel area, so the segmentation effect is more accurate.
[0193] Fig.17 The segmentation prediction effect diagrams of the same coronary artery at different angles are shown in sequence, where the red arrows point to the narrowed areas in the original image, and the orange arrows point to the narrowed areas in the coronary artery segmentation prediction result diagram. It can be seen that the shape of the plaques of the same coronary artery at different angles changes slightly, which causes the narrowing of the blood vessels caused by the plaques to change. However, it is not difficult to see from the prediction diagram after segmentation that the degree of narrowing of the blood vessels caused by these plaques of different shapes does not change with the change of the plaques. The grading of the degree of blood vessel narrowing still belongs to moderate or above stenosis. It can still be classified into the category of narrowing greater than 50% based on this set of coronary segmentation images, and can be diagnosed as coronary heart disease.
[0194] 4. Analysis of coronary artery classification experiment results;
[0195] In order to help doctors make a quick diagnosis, the present invention classifies the degree of vascular stenosis according to the coronary stenosis grading criteria specified in the American CASS coronary artery disease vascular diagnostic standard, wherein mild stenosis represents that the vascular lumen area is reduced by less than 50% of its own area; moderate stenosis represents that the vascular lumen area is reduced by 50% to 70% of its own area; severe stenosis represents that the vascular lumen area is reduced by more than 70% of its own area. Usually, whether the degree of vascular stenosis is greater than 50% is used as the standard for determining coronary heart disease in clinical practice. The model of the present invention divides the coronary artery into three levels, wherein the model with a vascular lumen area reduction of less than 50% can be classified as a mild stenosis type, between 50% and 70% can be classified as a moderate stenosis type, and more than 70% can be classified as a severe stenosis type.
[0196] Fig.18 This is the result diagram of the classification prediction of the degree of coronary artery stenosis. Fig.18 (a) is a blood vessel with mild stenosis, which means that the degree of stenosis is less than 50%. Fig.18 (b) belongs to the category of moderately stenotic blood vessels, which means that the degree of stenosis of the blood vessels is between 50% and 70%, and can be diagnosed as coronary heart disease. Fig.18 (c) is a blood vessel with severe stenosis, which means that the degree of stenosis is greater than 70%, and a bypass surgery is required for treatment. As can be seen from the figure, the model of the present invention can accurately classify blood vessels with different degrees of stenosis, which fully proves the effectiveness of the algorithm and can realize the automatic assessment of coronary artery stenosis.
[0197] The algorithm of the present invention is an improved model based on the Res-Net50 network. Therefore, this network is used as the baseline of the experimental results and compared with other mainstream classification networks, including the most common VGG16, MobileNet and DenseNet. During the experiment, four models including the model of the present invention adopt the same training environment and method. In order to evaluate the improvement of the algorithm constructed by the present invention on the model performance, the accuracy and precision evaluation indicators mentioned in the present invention are used to evaluate the model performance. The experimental comparison results are shown in Table 5.
[0198] Table 5 Comparison of classification model improvement performance
[0199]
[0200] As can be seen from Table 5, the basic network Res-Net50 used in the present invention has performed better than other models, and its accuracy and precision are higher than other models. Among them, the VGG16 network has fewer layers, and when extracting features from the original coronary image, it cannot effectively extract its high-level features, so the classification accuracy is not as high as the Res-Net50 network, but it is also about 1% higher than the accuracy and precision of MobileNet. The accuracy of the DenseNet network is similar to that of VGG16, but because the DenseNet network has more layers, it takes more time in the training process. The algorithm of the present invention constructs a multi-channel residual connection in the ResNet50 network, and the network size is moderate, which can not only improve the accuracy, but also does not take too much time during training. The final coronary vessel classification accuracy of the algorithm of the present invention reaches 88.6%, which is 7.08% higher than the original Res-Net50 basic network. This result proves that the model constructed by the present invention has a certain robustness and can assist doctors in diagnosing coronary heart disease.
[0201] The loss function used in the algorithm of the present invention is the cross entropy loss function, which is usually used in classification problems. The implementation formula is shown in formula (15).
[0202]
[0203] Among them, M represents the number of categories, i represents the i-th sample, and y ic represents the true label, and Pic represents the prediction result, that is, the predicted probability that the i-th sample belongs to class c. The loss obtained by the algorithm of the present invention is as follows: Fig.19 It can be seen that the algorithm of the present invention begins to converge to below 0.1 within 10 trainings, and then converges steadily to about 0.05.
[0204] A comparison between the algorithm of the present invention and the algorithm proposed by Benjimin in 2018 is shown in Table 6. Benjimin used the CNN network to classify and predict coronary angiography images and their segmented images, with an accuracy of 82.50%. The present invention improved Res-Net50 and classified and predicted the original coronary CPR images and their segmented images, with an accuracy of 88.6%. In comparison, the accuracy of the algorithm of the present invention is improved by about 6.1%.
[0205] Table 6 Comparison of different methods for classification of vascular stenosis
[0206]
[0207] In summary, the algorithm of the present invention can better classify the types of coronary artery stenosis into mild stenosis, moderate stenosis and severe stenosis, thereby realizing automatic evaluation of the degree of coronary artery stenosis.
Claims
1. An intelligent coronary artery stenosis assessment method for CTA images, characterized in that: The steps include: Step 1: Coronary artery centerline extraction based on shortest path branch deletion; Step 1-1: Use the deletion points of the twelve directions of the voxel, the twelve directions represent the twelve sides of the cube, and divide them into four groups, each group contains three directions; Step 1-2: Convert the 3D skeleton line into a directed graph with directed edges and nodes, where a voxel point with only a single adjacent point is called an endpoint, a voxel point between two adjacent points is called a common link point, and if there are more than two adjacent points, it is regarded as a node; the edge between the common link points constitutes a directed edge; construct an n×n adjacency matrix Z according to the number of nodes N in the directed graph, and its expression is shown in formula (1): In the process of searching the coronary artery centerline, the path length from the i-th node to the j-th node is represented by z ij ; If the path length between node A and node B is shorter than the path length through node C, it means that node A and node B are adjacent; After searching all nodes, connect them in sequence to obtain the coronary artery centerline; Step 2: Multi-angle trilinear interpolation surface reconstruction algorithm; Step 2-1: Surface reconstruction algorithm based on Frenet framework; The center line of the coronary artery is regarded as the motion trajectory of the center point in the entire three-dimensional space; the Frenet-Serret formula is shown in formula (2) Where t represents the unit tangent vector; N represents the unit normal vector, which is a vector obtained by differentiating the arc length parameter; B is the binormal vector, which is composed of the outer product of the tangent vector T and the normal vector N; d / ds is the integral used to calculate the arc length, and κ and τ are the curvature and torsion of the described curve respectively; When reconstructing the coronary artery, in order to calculate the Frenet coordinates of each center point, the center line is regarded as a discrete center point and discretized; assuming that there are three continuous points M1, M2, and M3 on the center line of the coronary artery, the coordinates of the three points are recorded as [x1, y1, z1], [x2, y2, z2], and [x3, y3, z3] respectively, and the tangent vector of each point is determined by the relationship between the points, so as to obtain the average tangent vector at each point, thereby realizing the straightened surface reconstruction of the coronary artery. Its expression is shown in formula (3): By calculating the orthogonal relationship between the tangent vector and the normal vector, their product can be recorded as the normal vector of the point, and its expression is shown in formula (4): The average value of the tangent vector T and the normal vector N is recorded as the binormal vector of the point, and its expression is shown in formula (5): By orthogonalizing the three vectors, the Frenet coordinates of the center point M2 are calculated, and the gray value of the center point M2 can be obtained based on the coordinates; by traversing all the center points in the center line and using the Frenet formula to calculate their positions and gray values in the TNB coordinate system, and finally splicing these gray values in order, the image of the coronary vascular surface reconstructed can be obtained; Step 2-2: trilinear interpolation algorithm; Trilinear interpolation requires interpolation of eight adjacent points; point P (x, y, z) is in a three-dimensional space, where the left component point x of point P is rounded up and down to get x L With x H Similarly, the components y and z are rounded to integers, and after obtaining the component coordinates, the y direction is interpolated. V1 is calculated first. The calculation process is shown in formula (5): V1=(y H -y)I(x H ,y H ,z H ) (5) +(y-yL)I(x H ,y H ,z H ) Similarly, V2, V3 and V4 are calculated, and linear interpolation is performed in the x direction according to the calculation results. The calculation process is shown in formula (6): V5=(x H -x)V4+(x-x L )V3 V6=(x H -x)V1+(x-x L )V2 (6) Finally, interpolation calculation is performed in the z direction to obtain the gray value at point P. The calculation process is shown in formula (7): In o =(with H -z)V6+(zz L )V5 (7) After extracting the grayscale values of all points, the centerline data is uniformly sampled and repeated center points are deleted; ensuring that the minimum interval of the center point data after sampling is a single voxel block; Step 2-3: Multi-angle surface reconstruction algorithm; The vector of interest of the reconstructed volume data is changed to perform curved surface reconstruction on the coronary artery; five directions are uniformly selected from 0° to 180° along the X-axis direction as mapping directions to perform curved surface reconstruction on the coronary artery; Step 3: Construct the Attention Res-UNet coronary artery stenosis segmentation network; The Attention Res-UNet coronary artery stenosis segmentation network adds a residual module and a spatial attention mechanism to the U-Net network framework. First, the residual module replaces the convolutional module in the encoding and decoding stages of the basic network. Second, in the decoding stage, the attention mechanism is added before the jump connection. Finally, a 1×1 convolution is used to adjust the channel and output the final prediction result of the model. Step 3-1: Residual module in Attention Res-UNet; The convolution modules in the original encoding and decoding stages of the U-Net network are changed to residual modules. The residual module in Attention Res-UNet includes three convolution layers. The first convolution layer is used to reduce the dimension of the feature channel, and then the feature channel is restored in the second 11 convolution layer. Then, the residual block identity mapping method is used for calculation, and finally they are superimposed as the output of the residual module. Step 3-2: Attention mechanism in Attention Res-UNet; The Attention Gate module is integrated into the upsampling module of the U-Net network through skip connections to implement a spatial attention mechanism, highlighting the vascular regions that need to be segmented in the CPR image and suppressing background noise, motion artifacts, and other possible vascular regions. (1) Spatial Attention Use the spatial attention mechanism SAM to allocate attention to the coronary artery area, thereby enhancing the target area and suppressing the background area; First, an average pooling operation and a maximum pooling operation are performed on a H×W×C feature F. In this process, two new H×W×1 features are obtained, which are recorded as At the same time, these two new features are concatenated together; a 7×7 convolution operation and Sigmod activation function are performed on them to obtain the weight coefficient Ms; finally, Ms is multiplied by the input feature F, and the result is the effective feature F′ that has more coronary vessel locations than the input feature F; the calculation formula of the SAM weight coefficient Ms is as shown in formula (8), where f 7*7 Represents a 7×7 convolutional layer; (2) Attention mechanism of Attention Res-UNet; The Attention Res-UNet algorithm introduces an attention module in the decoding stage of the U-Net network and adds an attention mechanism before the skip connection; Let x l is the feature extracted during the encoding process, g l To extract the features during the decoding process, first convert x l and g l Add them together so that the repeated areas of the two have a larger activation value; secondly, the spatial attention SAM is used to calculate the attention weight coefficient σ1, so as to obtain the vascular feature information to be segmented in the coronary CPR image and suppress useless information such as noise and motion artifacts in the background; the output feature of the attention mechanism is the encoding stage x l With l The result of element-wise multiplication Finally, the feature is concatenated and fused with the feature obtained in the decoding stage. The calculation formula of the output feature of the attention mechanism is shown in formula (9): Using additive attention to obtain ɑ l , the additive attention formula is shown in formula (10) and formula (11): Q l =ψ(σ1(W x x l +W g g l +b g ))+b ψ (10) a l =σ2(Q l ) (11) Among them, σ1 and σ2 correspond to the ReLu activation function and the Sigmoid activation function respectively, W x With W g is the attention weight, b g With b ψ For the paranoid item in attention; Step 4: Coronary artery stenosis classification network model based on multi-scale convolutional neural network Res2Net; A multi-channel residual connection is constructed in the Res-Net50 network framework, and a pre-fusion method is used to fuse multiple target-related features. First, the original coronary CPR image is fused with the segmented image so that the network can obtain more features and thus improve the accuracy of classification prediction. Second, the residual module in the Res-Net50 network is reconstructed to increase the receptive field of each layer of the network. Step 4-1: Feature fusion in multi-scale convolutional neural network Res2Net; The original coronary CPR image and the segmented coronary CPR image are feature-fused to increase the feature dimension input into the network, so that the network can obtain more feature information of the coronary vessels and improve the accuracy of image classification. The pre-fusion method is used to perform feature fusion on the coronary artery stenosis classification network: fusion is performed on the input layer, firstly fusing multiple layers of features and then inputting them into the deep learning network; Step 4-2: Res2Net module in the multi-scale convolutional neural network Res2Net; The residual module Res2Net is used to construct a hierarchical residual connection in the Res-Net network. In the Res2Net module, the 3×3 convolution layer is replaced by a group of small convolutions, and different convolution groups are connected layer by layer in a similar way to the residual connection, forming a new neural network module. In the Res2Net module, the feature map x is firstly divided into s subsets after 1×1 convolution, and each subset is represented by x i Represents, where i∈{1,2,3,…,s}; the number of channels in each subset after splitting is 1 / s of the original, and the spatial size is consistent with the original feature map; except for the first feature subset, each feature subset x i There are corresponding 3×3 convolutional layers, with K i () indicates that the output is defined as y i ; In the process, in order to reduce parameters and increase the number of s, x1 does not undergo 3×3 convolution, that is, y1=x1, and the remaining feature subsets x i and K i-1 () are added and sent to K i () is processed; therefore y i It is expressed by formula (12):
2. The intelligent coronary artery stenosis assessment method for CTA images according to claim 1, characterized in that: The Attention Res-UNet coronary stenosis segmentation network consists of an encoding stage, a decoding stage, and a prediction stage. In the encoding stage, the original convolution module is replaced by a residual module to form a residual unit for extracting features. The encoding stage consists of five residual units, which are used to extract feature information of the coronary CPR image, and then a 2×2 maximum pooling operation is performed to retain the main features of the image and reduce the image size. In this stage, the network obtains five preliminary effective feature layers, which are then sent to the decoding stage for feature fusion. Therefore, the role of the encoding stage is to extract the features of the coronary CPR image and expand the receptive field. In the decoding stage, the preliminary effective feature layer obtained in the encoding stage is used for upsampling. In this stage, the residual module is also used to replace the original convolution module for deconvolution, and the attention mechanism is added before the jump connection, so that the coronary vascular features transmitted in the process are fused to obtain the final effective feature layer. Finally, a 1×1 convolution is used to reduce the dimension of the channel, and the features obtained in the first two stages are used for prediction. The coronary vessels and background are classified by Softmax, so as to achieve the purpose of segmentation and output the final segmentation result of the coronary CPR image.
3. The intelligent coronary artery stenosis assessment method for CTA images according to claim 1, characterized in that: The coronary CPR original image and the segmented image are fused in an add manner.
Citation Information
Cited By
Scale perception progressive learning method for narrow segmentation of X-ray coronary angiography
CN121353679A
