A coronary artery detection method and system fusing fsrconv and transformer

Abstract

The application provides a kind of coronary artery detection method and system fusing FSRConv and Transformer in the technical field of medical image processing, method includes following steps: step S1, a large number of historical CT angiography images of coronary artery are acquired, and a data set is constructed after each historical CT angiography image is preprocessed;Step S2, a coronary stenosis detection model is created based on a 3D feature space reconstruction module, a self-attention module and an output module;Step S3, the coronary stenosis detection model is trained through the data set;Step S4, the real-time CT angiography image of coronary artery is detected by the trained coronary stenosis detection model.The advantage of the application lies in: greatly improve the accuracy of coronary artery stenosis detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing, and particularly relates to a coronary artery detection method and system fusing FSRConv and a Transformer. BACKGROUND

[0002] Coronary arteries are arteries that supply blood to the heart, originating in the aortic root sinus of the aorta, branching into left and right branches, and running on the surface of the heart. When the coronary arteries become diseased, the branches of the coronary arteries will produce plaques, causing them to become narrow, affecting blood flow speed, and thus causing myocardial infarction. Therefore, there is a need to detect whether the coronary arteries are narrow.

[0003] For the detection of coronary arteries, traditionally, medical images of coronary arteries are obtained through CT imaging, and then the medical images are observed by the human eye to determine whether there is a narrow area in the coronary arteries. However, human eye observation is prone to misjudgment and is low in efficiency. With the rapid development of deep learning, models based on deep learning technology have evolved to detect medical images of coronary arteries. However, existing models ignore the particularity of the topological structure of coronary arteries, that is, coronary arteries are special tubular structures, which only occupy a small part of the entire medical image. This means that there is a large amount of redundant space, resulting in unsatisfactory detection accuracy of existing models.

[0004] Therefore, how to provide a coronary artery detection method and system fusing FSRConv and a Transformer to improve the accuracy of coronary artery stenosis detection has become a technical problem to be solved. SUMMARY

[0005] The technical problem to be solved by the present application is to provide a coronary artery detection method and system fusing FSRConv and a Transformer to improve the accuracy of coronary artery stenosis detection.

[0006] In a first aspect, the present application provides a coronary artery detection method fusing FSRConv and a Transformer, comprising the following steps:

[0007] Step S1, a large number of historical CT angiography images of coronary arteries are obtained, and a data set is constructed after pre-processing each historical CT angiography image;

[0008] Step S2, a coronary artery stenosis detection model is created based on a 3D feature space reconstruction module, a self-attention module and an output module;

[0009] The 3D feature space reconstruction module is constructed based on a gating unit, a reconstruction unit, a re-separation unit and a fusion unit. The 3D feature space reconstruction module is used to extract local semantic features of plaques at different positions from the CT angiography images.

[0010] The self-attention module is constructed based on eight self-attention units, and the input and output of each self-attention unit are connected based on layer normalization and residual connection; each self-attention unit is constructed by a multi-head self-attention network and a feedforward network, and the feedforward network is constructed by two linear layers carrying ReLU activation functions; the self-attention module is used to extract the global semantic features of plaques at different positions from the CT contrast image;

[0011] The output module is used to fuse the local semantic features and the global semantic features to output the coronary artery detection result;

[0012] Step S3, training the coronary stenosis detection model through the data set;

[0013] Step S4, detecting the real-time CT contrast image of the coronary artery through the trained coronary stenosis detection model.

[0014] Further, the step S1 is specifically:

[0015] A large number of historical CT contrast images of coronary arteries are obtained, and the blood vessel center line and the plaque of each historical CT contrast image are labeled, and the straightening operation is performed on each historical CT contrast image based on the blood vessel center line through the straightening surface reconstruction technology to obtain a CPR image, and the data set is constructed after performing the sample expansion operation on each CPR image.

[0016] Further, in the step S2, the calculation process of the gating unit is:

[0017] Calculate the feature map weight value set W γ :

[0018]

[0019] wherein w i represents the feature map weight value of the i-th channel in the CT contrast image; γ i represents the pixel difference coefficient of the i-th channel in the CT contrast image; γ j represents the pixel difference coefficient of the j-th channel in the CT contrast image; C represents the channel dimension of the CT contrast image;

[0020] The feature map weight value set W γ is scaled to the interval (0, 1) through the Sigmoid function, a gating threshold is set, and the scaled feature map weight value set W γ is gated based on the gating threshold to obtain information weight W1 and information weight W2:

[0021] W kGate(Sigmoid(W γ , k = 1 or 2;

[0022] wherein, W k represents an information weight; Gate() represents a gate function, when the feature map weight value is greater than the gate threshold value, W k = W1, when the feature map weight value is less than the gate threshold value, W k = W2; Sigmoid() represents an activation function; GN3D() represents a normalization function; X represents a CT contrast image;

[0023] Based on the information weight W1 and the information weight W2, a large information amount feature and a small information amount feature

[0024]

[0025] The reconstruction unit is used to cross-reconstruct the large information amount feature and the small information amount feature to obtain a spatial refinement feature map X W ;

[0026] The calculation process of the re-separation unit is as follows:

[0027] A segmentation factor a and a squeezing ratio r are set, 0≤a≤1, r = 2;

[0028] Based on the segmentation factor a, the channels of the spatial refinement feature map X W are divided into aC and (1-a)C;

[0029] The 1x1x1 convolution performs a squeezing operation on the spatial refinement feature map X W of the aC channel and the spatial refinement feature map X W of the (1-a)C channel based on the squeezing ratio r, to split the spatial refinement feature map X W into an upper refinement feature map X up and a lower refinement feature map X low ;

[0030] Based on the upper refinement feature map X up and the lower refinement feature map X low , separated feature maps Y1 and Y2 are calculated:

[0031] Y1 = GWC(X up ) + PWC(X up );

[0032] Y2 = PWC(X low)∪X low ;

[0033] wherein, GWC() represents a group convolution operation; PWC() represents a point convolution operation;

[0034] The fusion unit is configured to fuse the separated feature map Y1 and the separated feature map Y2 to obtain a local semantic feature Y:

[0035] Y = β1Y1 + β2Y2;

[0036]

[0037] β1 + β2 = 1;

[0038] wherein, Y represents a local semantic feature; β1 and β2 both represent channel feature importance vectors; S1 represents a channel-wise quantity in the global three-dimensional pooled separated feature map Y1; S2 represents a channel-wise quantity in the global three-dimensional pooled separated feature map Y2; e represents a natural constant.

[0039] Further, the step S3 is specifically:

[0040] The data set is divided into a training set, a validation set and a test set based on a preset ratio, the coronary stenosis detection model is trained through the training set, until the loss function of the coronary stenosis detection model is less than a preset loss threshold, and the hyperparameters of the coronary stenosis detection model are continuously optimized in the training process;

[0041] The trained coronary stenosis detection model is verified through the validation set, to determine whether the detection accuracy is greater than a preset accuracy threshold, if not, the verification fails, and the training set is expanded for continuous training; if yes, the verification passes, and:

[0042] The coronary stenosis detection model that passes the verification is tested through the test set, to determine whether the confidence is greater than a preset confidence threshold, if not, the test fails, and the training set is expanded for continuous training; if yes, the test passes, and the training ends.

[0043] Further, the step S4 is specifically:

[0044] The trained coronary stenosis detection model is deployed on a server, an API interface for calling the coronary stenosis detection model is set, a calling authority of the API interface is set, and the deployed coronary stenosis detection model is accessed after authentication through the API interface, to detect real-time CT angiography images of coronary arteries.

[0045] In a second aspect, the present application provides a coronary artery detection system fusing FSRConv and Transformer, comprising the following modules:

[0046] A data set construction module is configured to obtain a large number of historical CT angiography images of coronary arteries, and construct a data set after preprocessing each historical CT angiography image.

[0047] A coronary artery stenosis detection model creation module is configured to create a coronary artery stenosis detection model based on a 3D feature space reconstruction module, a self-attention module, and an output module.

[0048] The 3D feature space reconstruction module is constructed based on a gating unit, a reconstruction unit, a re-separation unit, and a fusion unit, and is configured to extract local semantic features of plaques at different positions from CT angiography images.

[0049] The self-attention module is constructed based on eight self-attention units, and the input and output of each self-attention unit are connected based on layer normalization and residual connection. Each self-attention unit is constructed by a multi-head self-attention network and a feedforward network, and the feedforward network is constructed by two linear layers with ReLU activation functions. The self-attention module is configured to extract global semantic features of plaques at different positions from CT angiography images.

[0050] The output module is configured to fuse the local semantic features and the global semantic features to output a coronary artery detection result.

[0051] A coronary artery stenosis detection model training module is configured to train the coronary artery stenosis detection model through the data set.

[0052] A coronary artery detection module is configured to detect real-time CT angiography images of coronary arteries through the trained coronary artery stenosis detection model.

[0053] Further, the data set construction module is specifically configured to:

[0054] obtain a large number of historical CT angiography images of coronary arteries, label each historical CT angiography image with a blood vessel centerline and a plaque, perform a straightening operation on each historical CT angiography image based on the blood vessel centerline through a straightening surface reconstruction technique to obtain a CPR image, and construct a data set after performing a sample expansion operation on each CPR image.

[0055] Further, in the coronary artery stenosis detection model creation module, the calculation process of the gating unit is as follows:

[0056] Calculate a feature map weight value set W γ :

[0057]

[0058] wherein w i represents the feature map weight value of the i-th channel in the CT contrast image; γ i represents the pixel difference coefficient of the i-th channel in the CT contrast image; γ j represents the pixel difference coefficient of the j-th channel in the CT contrast image; C represents the channel dimension of the CT contrast image;

[0059] The feature map weight value set W γ is scaled to the interval (0, 1) by a Sigmoid function, a gating threshold is set, and the scaled feature map weight value set W γ is gated based on the gating threshold to obtain information weights W1 and W2:

[0060] W k = Gate (Sigmoid (W γ (GN3D (X))), k = 1 or 2;

[0061] wherein W k represents the information weight; Gate() represents a gating function, when the feature map weight value is greater than the gating threshold, W k =W1, when the feature map weight value is less than the gating threshold, W k =W2; Sigmoid() represents an activation function; GN3D() represents a normalization function; X represents the CT contrast image;

[0062] The large information amount feature and the small information amount feature

[0063]

[0064] The reconstruction unit is configured to cross-reconstruct the large information amount feature and the small information amount feature to obtain a spatial refinement feature map X W .

[0065] The calculation process of the re-separation unit is as follows:

[0066] A segmentation factor α and a squeezing ratio r are set, 0≤α≤1, r = 2;

[0067] Based on the segmentation factor α, the channels of the spatial refinement feature map X W are divided into αC and (1-α)C;

[0068] 1x1x1 convolution is based on the extrusion ratio r, the spatial refinement feature map X of the alpha C channel W and the spatial refinement feature map X of the (1-alpha) C channel W Perform extrusion operation to divide the spatial refinement feature map X W into an upper refinement feature map X up and a lower refinement feature map X low ;

[0069] Based on the upper refinement feature map X up and the lower refinement feature map X low , the separation feature map Y1 and the separation feature map Y2 are calculated:

[0070] Y1=GWC(X up )+PWC(X up );

[0071] Y2=PWC(X low )∪X low ;

[0072] Wherein, GWC() represents a group convolution operation; PWC() represents a point convolution operation;

[0073] The fusion unit is used to fuse the separation feature map Y1 and the separation feature map Y2 to obtain local semantic features:

[0074] Y=β1Y1+β2Y2;

[0075]

[0076] β1+β2=1;

[0077] Wherein, Y represents local semantic features; β1 and β2 both represent channel feature importance vectors; S1 represents channel-wise quantity in the global three-dimensional pooled separation feature map Y1; S2 represents channel-wise quantity in the global three-dimensional pooled separation feature map Y2; e represents a natural constant.

[0078] Further, the coronary stenosis detection model training module is specifically used for:

[0079] Based on the preset proportion, the data set is divided into a training set, a validation set and a test set, the coronary stenosis detection model is trained through the training set, until the loss function of the coronary stenosis detection model is less than the preset loss threshold, and the hyperparameters of the coronary stenosis detection model are continuously optimized in the training process;

[0080] verify the trained coronary stenosis detection model through the verification set to determine whether the detection accuracy is greater than a preset accuracy threshold, if not, the verification fails, the training set is expanded to continue training; if yes, the verification passes, and

[0081] test the coronary stenosis detection model that passes the verification through the test set to determine whether the confidence is greater than a preset confidence threshold, if not, the test fails, the training set is expanded to continue training; if yes, the test passes, and the training ends.

[0082] Further, the coronary detection module is specifically used for:

[0083] deploy the trained coronary stenosis detection model on a server, set an API interface for calling the coronary stenosis detection model, set the calling authority of the API interface, access the deployed coronary stenosis detection model through the API interface after authentication to detect real-time CT angiography images of the coronary artery.

[0084] The present application has the advantages of:

[0085] A large number of historical CT angiography images of the coronary artery are obtained to label the blood vessel center line and the plaque, a straightening operation is performed on each historical CT angiography image based on the blood vessel center line to obtain a CPR image through a straightening surface reconstruction technology, a data set is constructed after a sample expansion operation is performed on each CPR image; then a coronary stenosis detection model is created based on a 3D feature space reconstruction module, a self-attention module and an output module, the coronary stenosis detection model is trained through the data set, and finally the real-time CT angiography images of the coronary artery are detected through the trained coronary stenosis detection model; the 3D feature space reconstruction module of the coronary stenosis detection model is constructed based on a gating unit, a reconstruction unit, a re-separation unit and a fusion unit, and is used to extract local semantic features of plaques at different positions from the CT angiography images; the self-attention module is constructed based on eight self-attention units, and is used to extract global semantic features of the plaques at different positions from the CT angiography images; the output module is used to fuse the local semantic features and the global semantic features to output the coronary artery detection result; that is, a straightening operation is performed on the blood vessels of the CT angiography images, the spatial and channel redundancies are suppressed through the 3D feature space reconstruction module to maximize the extraction of effective feature information (local semantic features) of the coronary artery, the context information between the plaques in the straightened three-dimensional blood vessel segment (CPR image) is fully understood through the self-attention module to robustly capture the global semantic features, and thus the accuracy of the coronary stenosis detection is greatly improved. BRIEF DESCRIPTION OF DRAWINGS

[0086] The present application will be further described below with reference to the accompanying drawings in conjunction with the embodiments.

[0087] Figure 1 is a flow chart of a coronary artery detection method of the present application fusing FSRConv and Transformer.

[0088] Figure 2 is a structural schematic diagram of a coronary artery detection system of the present application fusing FSRConv and Transformer.

[0089] Figure 3 is a flow schematic diagram of the present application.

[0090] Figure 4 is a schematic diagram of a CT contrast image straightening operation of the present application.

[0091] Figure 5 is a schematic diagram of a 3D feature space reconstruction module of the present application.

[0092] Figure 6 is a contrast schematic diagram of coronary artery detection of the present application. DETAILED DESCRIPTION

[0093] The technical solution in the embodiments of the present application has the following general idea: coronary artery stenosis detection is performed by a coronary stenosis detection model created by a 3D feature space reconstruction module, a self-attention module and an output module, before detection, a straightening reconstruction technology is performed on the blood vessels based on the blood vessel center line to perform a straightening operation on the blood vessels, the 3D feature space reconstruction module is used to suppress the redundancy of space and channel, so as to maximize the extraction of local semantic features of the coronary artery, the self-attention module is used to fully understand the context information between plaques in the straightened three-dimensional blood vessel segment, to robustly capture global semantic features, and to improve the accuracy of coronary artery stenosis detection.

[0094] Please refer to Figures 1 to 6 , a preferred embodiment of a coronary artery detection method of the present application fusing FSRConv and Transformer, comprising the following steps:

[0095] Step S1, a large number of historical CT contrast images (CCTA images) of coronary arteries are obtained, and a data set is constructed after pre-processing each historical CT contrast image;

[0096] Step S2, a coronary stenosis detection model (FSR-Former) is created based on a 3D feature space reconstruction module (FSRConv), a self-attention module (Transformer) and an output module;

[0097] The 3D feature space reconstruction module is constructed based on a gating unit, a reconstruction unit, a re-separation unit and a fusion unit; the 3D feature space reconstruction module is used to extract local semantic features of plaques at different positions from CT contrast images; considering the special features of coronary arteries, in order to emphasize the rich features of the tubular part and utilize redundant features, the 3D feature space reconstruction module is introduced;

[0098] The self-attention module is constructed based on eight self-attention units (Transformer Block), and the input and output of each self-attention unit are connected based on layer normalization (LN) and residual connection; each self-attention unit is constructed by a multi-head self-attention network (MSA) and a feedforward network (FFN), and the feedforward network is constructed by two linear layers carrying ReLU activation functions; the self-attention module is used to extract global semantic features of plaques at different positions from CT contrast images, that is, the dependency relationship between plaques at different positions; since there may be multiple plaques in the branches of coronary arteries, there is a strong context relationship between adjacent plaques, and each plaque will affect the blood flow velocity in the coronary artery, so analyzing the potential relationship between plaques at different positions is valuable for clinical diagnosis, and the self-attention module can robustly capture global information, thereby improving the robustness of stenosis detection;

[0099] The output module is used to fuse the local semantic features and the global semantic features to output the coronary artery detection result;

[0100] Step S3, training the coronary artery stenosis detection model through the data set;

[0101] Step S4, detecting the real-time CT contrast image of the coronary artery through the trained coronary artery stenosis detection model.

[0102] The present application proposes a three-dimensional feature space reconstruction convolution method for the special structure of the coronary artery, which can effectively suppress redundancy from space and channel, thereby maximizing the extraction of effective feature information;

[0103] In order to realize more accurate coronary artery stenosis detection, FSRConv is combined with Transformer to effectively integrate local semantic features and global semantic features.

[0104] The step S1 is specifically:

[0105] A large number of historical CT contrast images of coronary arteries are acquired, and the blood vessel center line and the plaque of each historical CT contrast image are labeled, a straightening operation is performed on each historical CT contrast image based on the blood vessel center line through a straightening surface reconstruction technology, a CPR image is obtained, and a data set is constructed after performing a sample expansion operation on each CPR image.

[0106] Since the default CTA view cannot directly observe the cross section of the coronary artery, it hinders the automatic detection of coronary artery stenosis, and the view of the vascular region needs to be converted, so the straightening operation is performed. The format of the historical CT angiography image is.nii.gz or.dicom.

[0107] In step S2, the calculation process of the gating unit is:

[0108] Calculate the feature map weight value set W γ :

[0109]

[0110] where w i represents the feature map weight value of the i-th channel in the CT angiography image; γ i represents the pixel difference coefficient of the i-th channel in the CT angiography image, the larger the value, the richer the spatial information; γ j represents the pixel difference coefficient of the j-th channel in the CT angiography image; C represents the channel dimension of the CT angiography image;

[0111] Scale the feature map weight value set W γ to the interval (0, 1) through the Sigmoid function, set a gating threshold, and gate the scaled feature map weight value set W γ based on the gating threshold to obtain information weight W1 and information weight W2:

[0112] W k = Gate(Sigmoid(W γ (GN3D(X))), k = 1 or 2;

[0113] where W k represents the information weight; Gate() represents the gating function, when the feature map weight value is greater than the gating threshold, W k =W1, when the feature map weight value is less than the gating threshold, W k =W2, and the value of the gating threshold is preferably 0.5; Sigmoid() represents the activation function; GN3D() represents the normalization function; X represents the CT angiography image;

[0114] Based on the information weight W1 and the information weight W2, large information amount features (spatial content with strong expressiveness) and small information amount features (redundant features) are calculated:

[0115]

[0116] The reconstruction unit is used to cross-reconstruct large information amount features and small information amount features to obtain a spatial refined feature map X W .

[0117] In order to reduce spatial redundancy as much as possible, the reconstruction unit is introduced to integrate rich information while saving spatial resources; the reconstruction unit uses cross-reconstruction operation to effectively integrate weighted unique features and enhance information communication between them; the reconstruction unit can not only separate features with large information amount from features with small information amount, but also reconstruct them to enhance representative features while reducing redundancy in spatial dimension.

[0118] The calculation process of the re-separation unit is as follows:

[0119] A split factor a and a squeezing ratio r are set, 0≤a≤1, r=2; the value of a is preferably 0.5; the squeezing ratio r is used to adjust feature channels and balance calculation cost;

[0120] Based on the split factor a, the channels of the spatial refined feature map X W are divided into aC and (1-a)C; C represents channel dimension;

[0121] 1×1×1 convolution based on the squeezing ratio r, performs squeezing operation on the spatial refined feature map X W of the aC channel and the spatial refined feature map X W of the (1-a)C channel, to split the spatial refined feature map X W into an upper refined feature map X up and a lower refined feature map X low .

[0122] Based on the upper refined feature map X up and the lower refined feature map X low , separated feature maps Y1 and Y2 are calculated:

[0123] Y1=GWC(X up )+PWC(X up );

[0124] Y2=PWC(X low )∪X low ;

[0125] Wherein, GWC() represents group-wise convolution operation; PWC() represents point-wise convolution operation;

[0126] By applying a grouped convolution operation and a point convolution operation to replace an expensive k*k*k standard convolution, high-level representative information can be extracted and the calculation cost can be reduced;

[0127] The fusion unit is configured to fuse the separated feature map Y1 and the separated feature map Y2 to obtain a local semantic feature:

[0128] Y = β1Y1 + β2Y2;

[0129]

[0130] β1 + β2 = 1;

[0131] wherein Y represents the local semantic feature; β1 and β2 each represent a channel feature importance vector; S1 represents a channel-wise quantity in the separated feature map Y1 after global three-dimensional pooling; S2 represents a channel-wise quantity in the separated feature map Y2 after global three-dimensional pooling; and e represents a natural constant.

[0132] The formula of the layer normalization is:

[0133]

[0134] wherein B0 represents an output of the layer normalization; y i represents a feature code of the i-th plaque, i = 1, 2,..., L; p i represents a position code of the i-th plaque; L represents a number of plaques in a coronary artery sequence; R represents a real number; C represents a channel dimension; and H represents a side length of the plaque.

[0135] The formula of the multi-head self-attention network is:

[0136]

[0137] wherein B' n represents an output of a multi-head attention mechanism in the n-th Transformer Encoder; MSA() represents the multi-head self-attention network; LN() represents the layer normalization; B n-1 represents an output of the n-1-th Transformer Encoder (attention encoder);

[0138] The formula of the feedforward network is:

[0139]

[0140] wherein B ndenotes the output of the feed-forward network, i.e., the output of the self-attention module; n denotes the number of TransformerEncoders; FFN() denotes the feed-forward network.

[0141] The step S3 is specifically:

[0142] The data set is divided into a training set, a validation set and a test set based on a preset ratio, the coronary stenosis detection model is trained through the training set, until the loss function of the coronary stenosis detection model is less than a preset loss threshold, and the hyperparameters of the coronary stenosis detection model are continuously optimized in the training process;

[0143] The trained coronary stenosis detection model is verified through the validation set, whether the detection accuracy is greater than a preset accuracy threshold is judged, if not, the verification fails, and the training set is expanded for continuous training; if yes, the verification passes, and:

[0144] The coronary stenosis detection model that passes the verification is tested through the test set, whether the confidence is greater than a preset confidence threshold is judged, if not, the test fails, and the training set is expanded for continuous training; if yes, the test passes, and the training is ended.

[0145] The step S4 is specifically:

[0146] The trained coronary stenosis detection model is deployed on a server, an API interface for calling the coronary stenosis detection model is set, the calling authority of the API interface is set, the deployed coronary stenosis detection model is accessed after authentication through the API interface, so as to detect the real-time CT angiography image of the coronary artery.

[0147] The preferred embodiment of the coronary artery detection system fused with FSRConv and Transformer includes the following modules:

[0148] The data set construction module is used for acquiring a large number of historical CT angiography images (CCTA images) of coronary arteries, and constructing a data set after preprocessing each historical CT angiography image.

[0149] The coronary stenosis detection model creation module is used for creating a coronary stenosis detection model (FSR-Former) based on a 3D feature space reconstruction module (FSRConv), a self-attention module (Transformer) and an output module.

[0150] The 3D feature space reconstruction module is constructed based on a gating unit, a reconstruction unit, a re-separation unit and a fusion unit; the 3D feature space reconstruction module is used to extract local semantic features of plaques at different positions from the CT contrast image; considering the special features of the coronary artery itself, in order to emphasize the rich features of the tubular part and utilize redundant features, the 3D feature space reconstruction module is introduced;

[0151] The self-attention module is constructed based on eight self-attention units (Transformer Block), and the input and output of each self-attention unit are connected based on layer normalization (LN) and residual connection; each self-attention unit is constructed by a multi-head self-attention network (MSA) and a feedforward network (FFN), and the feedforward network is constructed by two linear layers carrying ReLU activation functions; the self-attention module is used to extract global semantic features of plaques at different positions from the CT contrast image, that is, the dependency relationship of plaques at different positions; since there may be multiple plaques in the branches of the coronary artery, there is a strong context relationship between adjacent plaques, and each plaque will affect the blood flow velocity in the coronary artery, so analyzing the potential relationship between plaques at different positions is very valuable for clinical diagnosis, and the self-attention module can robustly capture global information, thereby improving the robustness of stenosis detection;

[0152] The output module is used to fuse the local semantic features and the global semantic features to output the coronary artery detection result;

[0153] The coronary artery stenosis detection model training module is used to train the coronary artery stenosis detection model through the data set;

[0154] The coronary artery detection module is used to detect the real-time CT contrast image of the coronary artery through the trained coronary artery stenosis detection model.

[0155] The present application proposes a three-dimensional feature space reconstruction convolution method for the special structure of the coronary artery, which can effectively suppress redundancy from space and channel, thereby maximizing the extraction of effective feature information;

[0156] In order to realize more accurate coronary artery stenosis detection, FSRConv is combined with Transformer to effectively integrate local semantic features and global semantic features.

[0157] The data set construction module is specifically used for:

[0158] A large number of historical coronary CT angiography images are acquired, each of the historical CT angiography images is labeled with a blood vessel centerline and a plaque, a straightening operation is performed on each historical CT angiography image based on the blood vessel centerline through a straightening surface reconstruction technology to obtain a CPR image, and a data set is constructed after a sample expansion operation is performed on each of the CPR images.

[0159] Since the default CTA view cannot directly observe the cross section of the coronary artery, the automatic detection of coronary artery stenosis is hindered, and the view of the blood vessel region needs to be converted, so the straightening operation is performed. The format of the historical CT angiography image is.nii.gz or.dicom.

[0160] In the coronary artery stenosis detection model creation module, the calculation process of the gating unit is:

[0161] Calculate a feature map weight value set W γ :

[0162]

[0163] where w i represents a feature map weight value of the i-th channel in the CT angiography image; γ i represents a pixel difference coefficient of the i-th channel in the CT angiography image, and the larger the value, the richer the spatial information; γ j represents a pixel difference coefficient of the j-th channel in the CT angiography image; C represents the channel dimension of the CT angiography image;

[0164] The feature map weight value set W γ is scaled to the interval (0, 1) through a Sigmoid function, a gating threshold is set, the scaled feature map weight value set W γ is gated based on the gating threshold to obtain information weights W1 and W2:

[0165] W k =Gate(Sigmoid(W γ (GN3D(X))), k=1or2;

[0166] where W k represents an information weight; Gate() represents a gating function, when the feature map weight value is greater than the gating threshold, W k =W1, when the feature map weight value is less than the gating threshold, W k =W2, and the value of the gating threshold is preferably 0.5; Sigmoid() represents an activation function; GN3D() represents a normalization function; X represents a CT angiography image;

[0167] Based on the information weights W1 and W2, the large information content feature is calculated. (Expressive spatial content) and characteristics of low information content (Redundancy feature):

[0168]

[0169] The reconstruction unit is used for features with large amounts of information. and features with small amount of information Cross-reconstruction is performed to obtain the spatially refined feature map X. W ;

[0170] To minimize spatial redundancy, the reconstruction unit is introduced to integrate rich information while conserving space resources; the reconstruction unit employs a cross-reconstruction operation to effectively integrate weighted unique features. It also strengthens information communication between them; the reconstruction unit can not only separate features with large information content from features with small information content, but also reconstruct them to enhance representative features and reduce redundancy in spatial dimensions.

[0171] The calculation process of the re-separation unit is as follows:

[0172] A segmentation factor α and a compression ratio r are set, where 0 ≤ α ≤ 1 and r = 2; the preferred value of α is 0.5; the compression ratio r is used to adjust the feature channels and balance the computational cost.

[0173] Based on the segmentation factor α, the spatially refined feature map X is... W The channels are divided into αC and (1-α)C; C represents the channel dimension;

[0174] A 1×1×1 convolution, based on the compression ratio r, refines the spatial feature map X of the αC channel. W And the spatial refinement feature map X of the (1-α)C channel W Perform a squeezing operation to refine the spatial feature map X. W Decomposed into refined feature map X up and the refined feature map X low ;

[0175] Based on the refined feature map X up and the refined feature map X low The separation feature map Y1 and separation feature map Y2 are calculated as follows:

[0176] Wherein, GWC() represents group-wise convolution; PWC() represents point-wise convolution.

[0177] By applying a group convolution operation and a point convolution operation to replace an expensive k x k x k standard convolution, high-level representative information can be extracted and the calculation cost can be reduced;

[0178] The fusion unit is configured to fuse the separated feature map Y1 and the separated feature map Y2 to obtain a local semantic feature Y:

[0179] Y = β1Y1 + β2Y2;

[0180]

[0181] β1 + β2 = 1;

[0182] wherein Y represents the local semantic feature; β1 and β2 each represent a channel feature importance vector; S1 represents a channel-wise quantity in the separated feature map Y1 after global three-dimensional pooling; S2 represents a channel-wise quantity in the separated feature map Y2 after global three-dimensional pooling; and e represents a natural constant.

[0183] The formula of the layer normalization is:

[0184]

[0185] wherein B0 represents an output of the layer normalization; y i represents a feature code of the i-th plaque, i = 1, 2, …, L; p i represents a position code of the i-th plaque; L represents a number of plaques in a coronary artery sequence; R represents a real number; C represents a channel dimension; and H represents a side length of the plaque.

[0186] The formula of the multi-head self-attention network is:

[0187]

[0188] wherein B' n represents an output of a multi-head attention mechanism in the n-th Transformer Encoder; MSA() represents the multi-head self-attention network; LN() represents the layer normalization; B n-1 represents an output of the n-1-th Transformer Encoder (attention encoder);

[0189] The formula of the feedforward network is:

[0190]

[0191] wherein B ndenotes the output of the feed-forward network, i.e., the output of the self-attention module; n denotes the number of TransformerEncoders; FFN() denotes a feed-forward network.

[0192] The coronary stenosis detection model training module is specifically used for:

[0193] The data set is divided into a training set, a validation set and a test set based on a preset ratio, the coronary stenosis detection model is trained through the training set, until the loss function of the coronary stenosis detection model is less than a preset loss threshold, and the hyperparameters of the coronary stenosis detection model are continuously optimized in the training process;

[0194] The trained coronary stenosis detection model is verified through the validation set, whether the detection accuracy is greater than a preset accuracy threshold is judged, if not, the verification fails, the training set is expanded and the training is continued, and if yes, the verification passes, and:

[0195] The coronary stenosis detection model that passes the verification is tested through the test set, whether the confidence is greater than a preset confidence threshold is judged, if not, the test fails, the training set is expanded and the training is continued, and if yes, the test passes, and the training is ended.

[0196] The coronary stenosis detection model training module is specifically used for:

[0197] The trained coronary stenosis detection model is deployed on a server, an API interface for calling the coronary stenosis detection model is set, the calling authority of the API interface is set, the deployed coronary stenosis detection model is accessed after authentication through the API interface, and the real-time CT angiography image of the coronary artery is detected.

[0198] In summary, the advantages of the present application are:

[0199] The center line of the blood vessel and the plaque are labeled by acquiring a large number of historical CT angiography images of the coronary artery, the straightening operation is performed on each historical CT angiography image based on the blood vessel center line to obtain a CPR image by using the straightening curved surface reconstruction technology, the sample expansion operation is performed on each CPR image, and a data set is constructed; then, a coronary artery stenosis detection model is created based on a 3D feature space reconstruction module, a self-attention module and an output module, the coronary artery stenosis detection model is trained through the data set, and finally, the real-time CT angiography image of the coronary artery is detected through the trained coronary artery stenosis detection model; the 3D feature space reconstruction module of the coronary artery stenosis detection model is constructed based on a gating unit, a reconstruction unit, a re-separation unit and a fusion unit, and is used for extracting local semantic features of plaques at different positions from the CT angiography image; the self-attention module is constructed based on eight self-attention units, and is used for extracting global semantic features of the plaques at different positions from the CT angiography image; and the output module is used for fusing the local semantic features and the global semantic features to output the coronary artery detection result; that is, the straightening operation is performed on the blood vessel of the CT angiography image, the spatial and channel redundancies are suppressed through the 3D feature space reconstruction module, so that the effective feature information (local semantic features) of the coronary artery is extracted to the maximum, the context information between the plaques in the straightened three-dimensional blood vessel segment (CPR image) is fully understood through the self-attention module, so that the global semantic features are robustly captured, and the accuracy of the coronary artery stenosis detection is greatly improved.

[0200] Although the specific embodiments of the present application are described above, those skilled in the art should understand that the specific embodiments described are only illustrative, and are not intended to limit the scope of the present application, and equivalent modifications and changes made by those skilled in the art in accordance with the spirit of the present application should be covered within the scope of the claims of the present application.

Claims

1. A method for coronary artery detection by fusing FSRConv and Transformer, characterized in that: The method comprises the following steps: Step S1, a large number of historical coronary CT angiography images are obtained, each historical CT angiography image is labeled with a blood vessel center line and a plaque, a straightening operation is performed on each historical CT angiography image based on the blood vessel center line through a straightening surface reconstruction technology to obtain a CPR image, and a data set is constructed after a sample expansion operation is performed on each CPR image; Step S2, a coronary stenosis detection model is created based on a 3D feature space reconstruction module, a self-attention module, and an output module; The 3D feature space reconstruction module is constructed based on a gating unit, a reconstruction unit, a re-separation unit, and a fusion unit; the 3D feature space reconstruction module is used to extract local semantic features of plaques at different positions from CT angiography images; The self-attention module is constructed based on eight self-attention units; the input and output of each self-attention unit are connected based on layer normalization and residual connection; each self-attention unit is constructed by a multi-head self-attention network and a feedforward network; the feedforward network is constructed by two linear layers carrying ReLU activation functions; the self-attention module is used to extract global semantic features of plaques at different positions from CT angiography images; The output module is used to fuse the local semantic features and the global semantic features to output a coronary artery detection result; Step S3, the coronary stenosis detection model is trained through the data set; Step S4, a real-time CT angiography image of a coronary artery is detected through the trained coronary stenosis detection model.

2. The method of claim 1, wherein the method is a method of detecting coronary arteries by fusing FSRConv and Transformer, and the method comprises: In step S2, the calculation process of the gating unit is as follows: A set of feature map weight values W γ : wherein w i represents the feature map weight value of the i-th channel in the CT contrast image; γ i represents the pixel difference coefficient of the i-th channel in the CT contrast image; γ j represents the pixel difference coefficient of the j-th channel in the CT contrast image; C represents the channel dimension of the CT contrast image; The feature map weight value set W γ is scaled to the interval (0, 1) by a Sigmoid function, a gating threshold is set, and the scaled feature map weight value set W γ is gated based on the gating threshold to obtain information weights W1 and information weights W2: W k = Gate(Sigmoid(W γ (GN3D(X)))), k = 1 or 2; wherein W k represents an information weight; Gate() represents a gating function, when the feature map weight value is greater than a gating threshold, W k = W1, when the feature map weight value is less than the gating threshold, W k = W2; Sigmoid() represents an activation function; GN3D() represents a normalization function; X represents a CT contrast image; The large information amount feature is calculated based on the information weight W1 and the information weight W2 and the small information amount feature The reconstruction unit is used to cross-reconstruct large information amount features and small information amount features to obtain a spatial refinement feature map X W ; The calculation process of the re-separation unit is as follows: A segmentation factor α and a squeezing ratio r are set, 0≤α≤1, and r=2; refine the spatial feature map X based on the split factor a W into aC and (1 - a)C; 1x1x1 convolution based on the squeeze ratio r, spatial refinement feature map X for the alpha C channel W and spatial refinement feature map X for the (1 - alpha) C channel W performing a squeeze operation to split the spatial refinement feature map X W into an upper refinement feature map X up and a lower refinement feature map X low ; based on the upper refined feature map X up and the lower refined feature map X low The separation feature map Y1 and the separation feature map Y2 are calculated Y1 = GWC(X up ) + PWC(X up ) ; Y2 = PWC(X low )∪X low ; Wherein, GWC() represents a group convolution operation; PWC() represents a point convolution operation; The fusion unit is used to fuse the separated feature map Y1 and the separated feature map Y2 to obtain local semantic features: Y=β1Y1+β2Y2; β1+β2＝1； Wherein, Y represents local semantic features; β1 and β2 both represent channel feature importance vectors; S1 represents a channel-wise quantity in the global three-dimensional pooled separated feature map Y1; S2 represents a channel-wise quantity in the global three-dimensional pooled separated feature map Y2; e represents a natural constant.

3. The method of claim 1, wherein: Step S3 is specifically as follows: The data set is divided into a training set, a validation set, and a test set based on a preset ratio, the coronary stenosis detection model is trained through the training set, until the loss function of the coronary stenosis detection model is less than a preset loss threshold, the hyperparameters of the coronary stenosis detection model are continuously optimized during the training process; The trained coronary stenosis detection model is verified through the validation set, whether the detection accuracy is greater than a preset accuracy threshold is determined, if not, the verification fails, the training set is expanded for continuous training; if yes, the verification passes, and The coronary stenosis detection model that passes the verification is tested by the test set to determine whether the confidence is greater than a preset confidence threshold. If not, the test fails, and the training set is expanded for further training. If yes, the test passes, and the training ends.

4. The method of claim 1, wherein the method is a method of coronary artery detection fusing FSRConv and Transformer, and the method comprises: The step S4 is specifically: The trained coronary stenosis detection model is deployed on a server, an API interface for calling the coronary stenosis detection model is set, the calling authority of the API interface is set, and the deployed coronary stenosis detection model is accessed after authentication through the API interface to detect real-time CT angiography images of coronary arteries.

5. A coronary artery detection system fusing FSRConv and Transformer, characterized in that: Comprise the following modules: A data set construction module is configured to obtain a large number of historical CT angiography images of coronary arteries, label each historical CT angiography image with a blood vessel center line and a plaque, perform a straightening operation on each historical CT angiography image based on the blood vessel center line through a straightening surface reconstruction technology to obtain a CPR image, and construct a data set after performing a sample expansion operation on each CPR image; A coronary stenosis detection model creation module is configured to create a coronary stenosis detection model based on a 3D feature space reconstruction module, a self-attention module, and an output module; The 3D feature space reconstruction module is constructed based on a gating unit, a reconstruction unit, a re-separation unit, and a fusion unit. The 3D feature space reconstruction module is configured to extract local semantic features of plaques at different positions from CT angiography images. The self-attention module is constructed based on eight self-attention units. The input and output of each self-attention unit are connected based on layer normalization and residual connection. Each self-attention unit is constructed by a multi-head self-attention network and a feedforward network. The feedforward network is constructed by two linear layers with ReLU activation functions. The self-attention module is configured to extract global semantic features of plaques at different positions from CT angiography images. The output module is configured to fuse the local semantic features and the global semantic features to output a coronary artery detection result. A coronary stenosis detection model training module is configured to train the coronary stenosis detection model based on the data set. A coronary artery detection module is configured to detect real-time CT angiography images of coronary arteries based on the trained coronary stenosis detection model.

6. The coronary artery detection system fusing FSRConv and Transformer of claim 5, wherein: In the coronary stenosis detection model creation module, the calculation process of the gating unit is as follows: The set of feature map weight values W γ : wherein w i represents the feature map weight value of the i-th channel in the CT contrast image; γ i represents the pixel difference coefficient of the i-th channel in the CT contrast image; γ j represents the pixel difference coefficient of the j-th channel in the CT contrast image; C represents the channel dimension of the CT contrast image; The feature map weight value set W γ is scaled to the interval (0, 1) by a Sigmoid function, a gating threshold is set, and the scaled feature map weight value set W γ is gated based on the gating threshold to obtain information weights W1 and information weights W2: W k = Gate(Sigmoid(W γ (GN3D(X)))), k = 1 or 2; wherein W k represents an information weight; Gate() represents a gating function, when the feature map weight value is greater than a gating threshold, W k = W1, when the feature map weight value is less than the gating threshold, W k = W2; Sigmoid() represents an activation function; GN3D() represents a normalization function; X represents a CT contrast image; The large information amount feature is calculated based on the information weight W1 and the information weight W2 and the small information amount feature The reconstruction unit is used to cross-reconstruct large information amount features and small information amount features to obtain a spatial refinement feature map X W ; The calculation process of the re-separation unit is as follows: A segmentation factor α and a squeezing ratio r are set, 0≤α≤1, and r=2. refine the spatial feature map X based on the split factor a W into aC and (1 - a)C; 1x1x1 convolution based on the squeeze ratio r, spatial refinement feature map X for the alpha C channel W and spatial refinement feature map X for the (1 - alpha) C channel W performing a squeeze operation to spatially refine the spatial refinement feature map X W into an upper refinement feature map X up and a lower refinement feature map X low ; based on the upper refined feature map X up and the lower refined feature map X low The separation feature map Y1 and the separation feature map Y2 are calculated Y1 = GWC(X up ) + PWC(X up ); Y2 = PWC(X low )∪X low ; Wherein, GWC() represents a group convolution operation; PWC() represents a point convolution operation; The fusion unit is configured to fuse the separated feature map Y1 and the separated feature map Y2 to obtain local semantic features: Y=β1Y1+β2Y2; β1+β2＝1； Wherein, Y represents local semantic features; β1 and β2 represent channel feature importance vectors; S1 represents a channel-wise quantity in the global three-dimensional pooled separated feature map Y1; S2 represents a channel-wise quantity in the global three-dimensional pooled separated feature map Y2; e represents a natural constant.

7. The coronary artery detection system fusing FSRConv and Transformer of claim 5, wherein: The coronary stenosis detection model training module is specifically configured to: The data set is divided into a training set, a validation set and a test set based on a preset ratio, the coronary stenosis detection model is trained through the training set, and the training process is continuously optimized until the loss function of the coronary stenosis detection model is less than a preset loss threshold value; The trained coronary stenosis detection model is verified through the validation set, whether the detection accuracy is greater than a preset accuracy threshold value is judged, if not, the verification fails, and the training set is expanded to continue training; if yes, the verification passes, and: The coronary stenosis detection model that passes the verification is tested through the test set, whether the confidence is greater than a preset confidence threshold value is judged, if not, the test fails, the training set is expanded to continue training; if yes, the test passes, and the training ends.

8. The coronary artery detection system fusing FSRConv and Transformer of claim 5, wherein: The coronary artery detection module is specifically used for: The trained coronary stenosis detection model is deployed on a server, an API interface for calling the coronary stenosis detection model is set, the calling authority of the API interface is set, the deployed coronary stenosis detection model is accessed after authentication through the API interface, and real-time CT angiography images of coronary arteries are detected.

Images