Coronary artery detection method and system fusing FSRConv and Transformer

Abstract

The invention provides a coronary artery detection method and system fusing FSRConv and Transform in the technical field of medical image processing, and the method comprises the following steps: S1, obtaining a large number of historical CT angiography images of coronary arteries, carrying out the preprocessing of each historical CT angiography image, and constructing a data set; s2, creating a coronary artery stenosis detection model based on a 3D feature space reconstruction module, a self-attention module and an output module; s3, training a coronary artery stenosis detection model through the data set; and S4, detecting the real-time CT contrast image of the coronary artery through the trained coronary artery stenosis detection model. The coronary artery stenosis detection method has the advantage that the coronary artery stenosis detection precision is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a coronary artery detection method and system integrating FSRConv and Transformer. Background Art

[0002] The coronary artery is an artery that supplies blood to the heart. It originates from the aortic sinus at the root of the aorta, and is divided into two branches, left and right, running along the surface of the heart. When the coronary artery is diseased, plaques will form in the branches of the coronary artery, causing them to become narrow, affecting blood flow and causing myocardial infarction. Therefore, there is a need to detect whether the coronary artery is narrowed.

[0003] For the detection of coronary arteries, medical images of the coronary arteries are traditionally obtained through CT imaging, and then the medical images are observed by the human eye to determine whether there are any narrow areas in the coronary arteries. However, human visual observation is prone to misjudgment and is inefficient. With the rapid development of deep learning, models for detecting medical images of coronary arteries based on deep learning technology have evolved. However, the existing models ignore the particularity of the topological structure of the coronary arteries, that is, the coronary arteries are special tubular structures that only occupy a small part of the entire medical image, which means that there is a large amount of redundant space, resulting in unsatisfactory detection accuracy of the existing models.

[0004] Therefore, how to provide a coronary artery detection method and system that integrates FSRConv and Transformer to improve the accuracy of coronary artery stenosis detection has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a coronary artery detection method and system integrating FSRConv and Transformer, so as to improve the accuracy of coronary artery stenosis detection.

[0006] In a first aspect, the present invention provides a coronary artery detection method integrating FSRConv and Transformer, comprising the following steps:

[0007] Step S1, obtaining a large number of historical CT angiography images of coronary arteries, and constructing a data set after preprocessing each of the historical CT angiography images;

[0008] Step S2, creating a coronary artery stenosis detection model based on the 3D feature space reconstruction module, the self-attention module and the output module;

[0009] The 3D feature space reconstruction module is constructed based on a gating unit, a reconstruction unit, a re-separation unit and a fusion unit; the 3D feature space reconstruction module is used to extract local semantic features of plaques at different positions from CT angiography images;

[0010] The self-attention module is constructed based on eight self-attention units, and the input and output of each self-attention unit are connected based on layer normalization and residual connection; each self-attention unit is constructed by a multi-head self-attention network and a feedforward network, and the feedforward network is constructed by two linear layers carrying ReLU activation functions; the self-attention module is used to extract global semantic features of plaques at different locations from CT angiography images;

[0011] The output module is used to fuse local semantic features and global semantic features to output coronary artery detection results;

[0012] Step S3, training a coronary artery stenosis detection model using the data set;

[0013] Step S4: detecting the real-time CT angiography image of the coronary artery using the trained coronary artery stenosis detection model.

[0014] Furthermore, the step S1 is specifically as follows:

[0015] A large number of historical CT angiography images of the coronary arteries are obtained, the vascular centerline and plaques of each of the historical CT angiography images are annotated, and a straightening operation is performed on each of the historical CT angiography images based on the vascular centerline using a straightening surface reconstruction technique to obtain a CPR image, and a sample expansion operation is performed on each of the CPR images to construct a data set.

[0016] Furthermore, in step S2, the calculation process of the gate control unit is:

[0017] Calculate the feature map weight value set W γ :

[0018]

[0019] Among them, w i Indicates the feature map weight value of the i-th channel in the CT angiography image; γ i Represents the pixel difference coefficient of the i-th channel in the CT angiography image; γ j represents the pixel difference coefficient of the jth channel in the CT contrast image; C represents the channel dimension of the CT contrast image;

[0020] The feature map weight value set W is converted into γ Scale to the interval (0,1), set a gating threshold, and adjust the scaled feature map weight value set W based on the gating threshold γ Perform gating to obtain information weight W1 and information weight W2:

[0021] W k= Gate(Sigmoid(W γ (GN3D(X)))), k = 1 or 2;

[0022] Among them, W k Represents information weight; Gate() represents the gating function. When the feature map weight value is greater than the gating threshold, W k =W1, when the feature map weight value is less than the gate threshold, W k =W2; Sigmoid() represents the activation function; GN3D() represents the normalization function; X represents the CT contrast image;

[0023] The high-information feature is calculated based on the information weight W1 and the information weight W2. And small information features

[0024]

[0025] The reconstruction unit is used to reconstruct features with large information content. And small information features Perform cross reconstruction to obtain the spatial refinement feature map X W ;

[0026] The calculation process of the re-separation unit is:

[0027] Set a segmentation factor α and a compression ratio r, 0≤α≤1, r=2;

[0028] Based on the segmentation factor α, the spatial feature map X is refined W The channels are divided into αC and (1-α)C;

[0029] The 1×1×1 convolution is based on the squeezing ratio r to refine the spatial feature map X of the αC channel. W And the spatial refinement feature map X of the (1-α)C channel W Perform a squeezing operation to refine the spatial feature map X W Split into the upper refined feature map X up And the refined feature map X low ;

[0030] Based on the refined feature map X up And the refined feature map X low The separation feature map Y1 and the separation feature map Y2 are calculated:

[0031] Y1=GWC(X up )+PWC(X up );

[0032] Y2=PWC(X low)∪X low ;

[0033] Among them, GWC() represents the group convolution operation; PWC() represents the point convolution operation;

[0034] The fusion unit is used to fuse the separation feature map Y1 and the separation feature map Y2 to obtain the local semantic feature:

[0035] Y=β1Y1+β2Y2;

[0036]

[0037] β1+β2=1;

[0038] Among them, Y represents the local semantic feature; β1 and β2 both represent the channel feature importance vector; S1 represents the channel-wise amount in the separation feature map Y1 after global three-dimensional pooling; S2 represents the channel-wise amount in the separation feature map Y2 after global three-dimensional pooling; e represents a natural constant.

[0039] Furthermore, the step S3 is specifically as follows:

[0040] Dividing the data set into a training set, a validation set, and a test set based on a preset ratio, training the coronary artery stenosis detection model through the training set until the loss function of the coronary artery stenosis detection model is less than a preset loss threshold, and continuously optimizing the hyperparameters of the coronary artery stenosis detection model during the training process;

[0041] The trained coronary artery stenosis detection model is verified by the verification set to determine whether the detection accuracy is greater than a preset accuracy threshold. If not, the verification fails and the training set is expanded to continue training. If yes, the verification passes, and:

[0042] The verified coronary artery stenosis detection model is tested using the test set to determine whether the confidence is greater than a preset confidence threshold. If not, the test fails and the training set is expanded to continue training; if so, the test passes and training ends.

[0043] Furthermore, the step S4 is specifically as follows:

[0044] The trained coronary artery stenosis detection model is deployed on a server, an API interface for calling the coronary artery stenosis detection model is set, the calling authority of the API interface is set, and the deployed coronary artery stenosis detection model is accessed after authentication through the API interface to detect real-time CT angiography images of the coronary arteries.

[0045] In a second aspect, the present invention provides a coronary artery detection system integrating FSRConv and Transformer, comprising the following modules:

[0046] A data set construction module, used for acquiring a large number of historical CT angiography images of the coronary arteries, and constructing a data set after preprocessing each of the historical CT angiography images;

[0047] A coronary artery stenosis detection model creation module, used to create a coronary artery stenosis detection model based on a 3D feature space reconstruction module, a self-attention module and an output module;

[0048] The 3D feature space reconstruction module is constructed based on a gating unit, a reconstruction unit, a re-separation unit and a fusion unit; the 3D feature space reconstruction module is used to extract local semantic features of plaques at different positions from CT angiography images;

[0049] The self-attention module is constructed based on eight self-attention units, and the input and output of each self-attention unit are connected based on layer normalization and residual connection; each self-attention unit is constructed by a multi-head self-attention network and a feedforward network, and the feedforward network is constructed by two linear layers carrying ReLU activation functions; the self-attention module is used to extract global semantic features of plaques at different locations from CT angiography images;

[0050] The output module is used to fuse local semantic features and global semantic features to output coronary artery detection results;

[0051] A coronary artery stenosis detection model training module, used for training the coronary artery stenosis detection model using the data set;

[0052] The coronary artery detection module is used to detect the real-time CT angiography image of the coronary artery through the trained coronary artery stenosis detection model.

[0053] Furthermore, the data set construction module is specifically used for:

[0054] A large number of historical CT angiography images of the coronary arteries are obtained, the vascular centerline and plaques of each of the historical CT angiography images are annotated, and a straightening operation is performed on each of the historical CT angiography images based on the vascular centerline using a straightening surface reconstruction technique to obtain a CPR image, and a sample expansion operation is performed on each of the CPR images to construct a data set.

[0055] Furthermore, in the coronary artery stenosis detection model creation module, the calculation process of the gating unit is:

[0056] Calculate the feature map weight value set W γ :

[0057]

[0058] Among them, w i Indicates the feature map weight value of the i-th channel in the CT angiography image; γ i Represents the pixel difference coefficient of the i-th channel in the CT angiography image; γ j represents the pixel difference coefficient of the jth channel in the CT contrast image; C represents the channel dimension of the CT contrast image;

[0059] The feature map weight value set W is converted into γ Scale to the interval (0,1), set a gating threshold, and adjust the scaled feature map weight value set W based on the gating threshold γ Perform gating to obtain information weight W1 and information weight W2:

[0060] W k = Gate(Sigmoid(W γ (GN3D(X)))), k = 1 or 2;

[0061] Among them, W k Represents information weight; Gate() represents the gating function. When the feature map weight value is greater than the gating threshold, W k =W1, when the feature map weight value is less than the gate threshold, W k =W2; Sigmoid() represents the activation function; GN3D() represents the normalization function; X represents the CT contrast image;

[0062] The high-information feature is calculated based on the information weight W1 and the information weight W2. And small information features

[0063]

[0064] The reconstruction unit is used to reconstruct features with large information content. And small information features Perform cross reconstruction to obtain the spatial refinement feature map X W ;

[0065] The calculation process of the re-separation unit is:

[0066] Set a segmentation factor α and a compression ratio r, 0≤α≤1, r=2;

[0067] Based on the segmentation factor α, the spatial feature map X is refined W The channels are divided into αC and (1-α)C;

[0068] The 1×1×1 convolution is based on the squeezing ratio r to refine the spatial feature map X of the αC channel. W And the spatial refinement feature map X of the (1-α)C channel W Perform a squeezing operation to refine the spatial feature map X W Split into the upper refined feature map X up And the refined feature map X low ;

[0069] Based on the refined feature map X up And the refined feature map X low The separation feature map Y1 and the separation feature map Y2 are calculated:

[0070] Y1=GWC(X up )+PWC(X up );

[0071] Y2=PWC(X low )∪X low ;

[0072] Among them, GWC() represents the group convolution operation; PWC() represents the point convolution operation;

[0073] The fusion unit is used to fuse the separation feature map Y1 and the separation feature map Y2 to obtain the local semantic feature:

[0074] Y=β1Y1+β2Y2;

[0075]

[0076] β1+β2=1;

[0077] Among them, Y represents the local semantic feature; β1 and β2 both represent the channel feature importance vector; S1 represents the channel-wise amount in the separation feature map Y1 after global three-dimensional pooling; S2 represents the channel-wise amount in the separation feature map Y2 after global three-dimensional pooling; e represents a natural constant.

[0078] Furthermore, the coronary artery stenosis detection model training module is specifically used for:

[0079] Dividing the data set into a training set, a validation set, and a test set based on a preset ratio, training the coronary artery stenosis detection model through the training set until the loss function of the coronary artery stenosis detection model is less than a preset loss threshold, and continuously optimizing the hyperparameters of the coronary artery stenosis detection model during the training process;

[0080] The trained coronary artery stenosis detection model is verified by the verification set to determine whether the detection accuracy is greater than a preset accuracy threshold. If not, the verification fails and the training set is expanded to continue training. If yes, the verification passes, and:

[0081] The verified coronary artery stenosis detection model is tested using the test set to determine whether the confidence is greater than a preset confidence threshold. If not, the test fails and the training set is expanded to continue training; if so, the test passes and training ends.

[0082] Furthermore, the coronary artery detection module is specifically used for:

[0083] The trained coronary artery stenosis detection model is deployed on a server, an API interface for calling the coronary artery stenosis detection model is set, the calling authority of the API interface is set, and the deployed coronary artery stenosis detection model is accessed after authentication through the API interface to detect real-time CT angiography images of the coronary arteries.

[0084] The advantages of the present invention are:

[0085] A large number of historical CT angiography images of the coronary arteries are obtained to mark the vascular centerline and plaques. The CPR image is obtained by straightening the historical CT angiography images based on the vascular centerline through the straightening surface reconstruction technology. The data set is constructed after the sample expansion operation is performed on each CPR image. Then, a coronary stenosis detection model is created based on the 3D feature space reconstruction module, the self-attention module and the output module. The coronary stenosis detection model is trained with the data set. Finally, the real-time CT angiography images of the coronary arteries are detected with the trained coronary stenosis detection model. The 3D feature space reconstruction module of the coronary stenosis detection model is constructed based on the gating unit, the reconstruction unit, the re-separation unit and the fusion unit, and is used to obtain the CPR image from the CT angiography. The local semantic features of plaques at different positions in the image are extracted; the self-attention module is built based on eight self-attention units and is used to extract the global semantic features of plaques at different positions from the CT angiography image; the output module is used to fuse local semantic features and global semantic features to output the coronary artery detection results; that is, the blood vessels in the CT angiography image are straightened, and the redundancy of space and channels is suppressed through the 3D feature space reconstruction module, so as to maximize the extraction of effective feature information (local semantic features) of the coronary artery, and the self-attention module is used to fully understand the contextual information between plaques in the straightened three-dimensional blood vessel segment (CPR image) to robustly capture the global semantic features, thereby greatly improving the accuracy of coronary artery stenosis detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] The present invention will be further described below in conjunction with embodiments with reference to the accompanying drawings.

[0087] Figure 1 It is a flow chart of a coronary artery detection method integrating FSRConv and Transformer of the present invention.

[0088] Figure 2 It is a structural schematic diagram of a coronary artery detection system integrating FSRConv and Transformer according to the present invention.

[0089] Figure 3 It is a schematic diagram of the process of the present invention.

[0090] Figure 4 It is a schematic diagram of the CT angiography image straightening operation of the present invention.

[0091] Figure 5 It is a schematic diagram of the 3D feature space reconstruction module of the present invention.

[0092] Figure 6 1 is a comparative schematic diagram of the coronary artery detection of the present invention. DETAILED DESCRIPTION

[0093] The technical solution in the embodiments of the present application has the following overall idea: coronary artery stenosis detection is performed through a coronary artery stenosis detection model created by a 3D feature space reconstruction module, a self-attention module and an output module; before detection, the blood vessel is straightened based on the blood vessel centerline using a straightening surface reconstruction technique; the 3D feature space reconstruction module is used to suppress spatial and channel redundancy, thereby maximally extracting local semantic features of the coronary artery; the self-attention module is used to fully understand the contextual information between plaques in the straightened three-dimensional blood vessel segment, so as to robustly capture global semantic features, thereby improving the accuracy of coronary artery stenosis detection.

[0094] Please refer to Figures 1 to 6 As shown, a preferred embodiment of the coronary artery detection method integrating FSRConv and Transformer of the present invention comprises the following steps:

[0095] Step S1, obtaining a large number of historical CT angiography images (CCTA images) of the coronary arteries, and constructing a data set after preprocessing each of the historical CT angiography images;

[0096] Step S2, creating a coronary artery stenosis detection model (FSR-Former) based on the 3D feature space reconstruction module (FSRConv), the self-attention module (Transformer) and the output module;

[0097] The 3D feature space reconstruction module is constructed based on a gating unit, a reconstruction unit, a re-separation unit and a fusion unit; the 3D feature space reconstruction module is used to extract local semantic features of plaques at different positions from CT angiography images; considering the special features of the coronary artery itself, in order to emphasize the rich features of the tubular part and utilize redundant features, the 3D feature space reconstruction module is introduced;

[0098] The self-attention module is constructed based on eight self-attention units (Transformer Block), and the input and output of each self-attention unit are connected based on layer normalization (LN) and residual connection; each self-attention unit is constructed by a multi-head self-attention network (MSA) and a feedforward network (FFN), and the feedforward network is constructed by two linear layers carrying ReLU activation functions; the self-attention module is used to extract the global semantic features of plaques at different positions from CT angiography images, that is, the dependency relationship of plaques at different positions; since a branch of a coronary artery may have multiple plaques, there is a strong contextual connection between adjacent plaques, and each plaque affects the blood flow velocity in the coronary artery, so analyzing the potential relationship between plaques at different positions is very valuable for clinical diagnosis, and the self-attention module can robustly capture global information, thereby improving the robustness of stenosis detection;

[0099] The output module is used to fuse local semantic features and global semantic features to output coronary artery detection results;

[0100] Step S3, training a coronary artery stenosis detection model using the data set;

[0101] Step S4: detecting the real-time CT angiography image of the coronary artery using the trained coronary artery stenosis detection model.

[0102] Aiming at the special structure of coronary arteries, the present invention proposes a three-dimensional feature space reconstruction convolution method, which can effectively suppress redundancy from both spatial and channel aspects, thereby maximally extracting effective feature information;

[0103] In order to achieve more accurate coronary artery stenosis detection, FSRConv is combined with Transformer to effectively integrate local semantic features and global semantic features.

[0104] The step S1 is specifically as follows:

[0105] A large number of historical CT angiography images of the coronary arteries are obtained, the vascular centerline and plaques of each of the historical CT angiography images are annotated, and a straightening operation is performed on each of the historical CT angiography images based on the vascular centerline using a straightening surface reconstruction technique to obtain a CPR image, and a sample expansion operation is performed on each of the CPR images to construct a data set.

[0106] Since the default CTA viewing angle cannot directly observe the cross section of the coronary artery, which hinders the automatic detection of coronary artery stenosis, the viewing angle of the vascular region needs to be converted, so the straightening operation is performed. The format of the historical CT angiography image is .nii.gz or .dicom.

[0107] In step S2, the calculation process of the gate control unit is:

[0108] Calculate the feature map weight value set W γ :

[0109]

[0110] Among them, w i Indicates the feature map weight value of the i-th channel in the CT angiography image; γ i It represents the pixel difference coefficient of the ith channel in the CT angiography image. The larger the value, the richer the spatial information. j represents the pixel difference coefficient of the jth channel in the CT contrast image; C represents the channel dimension of the CT contrast image;

[0111] The feature map weight value set W is converted into γ Scale to the interval (0,1), set a gating threshold, and adjust the scaled feature map weight value set W based on the gating threshold γ Perform gating to obtain information weight W1 and information weight W2:

[0112] W k = Gate(Sigmoid(W γ (GN3D(X)))), k = 1 or 2;

[0113] Among them, W k Represents information weight; Gate() represents the gating function. When the feature map weight value is greater than the gating threshold, W k =W1, when the feature map weight value is less than the gate threshold, W k =W2, the value of the gate threshold is preferably 0.5; Sigmoid() represents an activation function; GN3D() represents a normalization function; X represents a CT angiography image;

[0114] The high-information feature is calculated based on the information weight W1 and the information weight W2. (expressive spatial content) and small information content (Redundant features):

[0115]

[0116] The reconstruction unit is used to reconstruct features with large information content. And small information features Perform cross reconstruction to obtain the spatial refinement feature map X W ;

[0117] In order to minimize spatial redundancy, the reconstruction unit is introduced to integrate rich information while saving space resources; the reconstruction unit uses a cross-reconstruction operation to effectively integrate weighted unique features. And strengthen the information communication between them; the reconstruction unit can not only separate the features with large information content from the features with small information content, but also reconstruct them to enhance the representative features while reducing the redundancy in the spatial dimension.

[0118] The calculation process of the re-separation unit is:

[0119] A segmentation factor α and a squeeze ratio r are set, 0≤α≤1, r=2; the value of α is preferably 0.5; the squeeze ratio r is used to adjust the feature channel and balance the calculation cost;

[0120] Based on the segmentation factor α, the spatial feature map X is refined W The channels are divided into αC and (1-α)C; C represents the channel dimension;

[0121] The 1×1×1 convolution is based on the squeezing ratio r to refine the spatial feature map X of the αC channel. W And the spatial refinement feature map X of the (1-α)C channel W Perform a squeezing operation to refine the spatial feature map X W Split into the upper refined feature map X up And the refined feature map X low ;

[0122] Based on the refined feature map X up And the refined feature map X low The separation feature map Y1 and the separation feature map Y2 are calculated:

[0123] Y1=GWC(X up )+PWC(X up );

[0124] Y2=PWC(X low )∪X low ;

[0125] Among them, GWC() represents the group-wise convolution operation; PWC() represents the point-wise convolution operation;

[0126] By applying grouped convolution operations and point convolution operations to replace expensive k×k×k standard convolutions, high-level representative information can be extracted and computational cost can be reduced;

[0127] The fusion unit is used to fuse the separation feature map Y1 and the separation feature map Y2 to obtain the local semantic feature:

[0128] Y=β1Y1+β2Y2;

[0129]

[0130] β1+β2=1;

[0131] Among them, Y represents the local semantic feature; β1 and β2 both represent the channel feature importance vector; S1 represents the channel-wise amount in the separation feature map Y1 after global three-dimensional pooling; S2 represents the channel-wise amount in the separation feature map Y2 after global three-dimensional pooling; e represents a natural constant.

[0132] The formula for layer normalization is:

[0133]

[0134] Among them, B0 represents the normalized output of the layer; y i represents the feature code of the i-th patch, i = 1, 2, ..., L; p i represents the position code of the i-th plaque; L represents the number of plaques in the coronary artery sequence; R represents a real number; C represents the channel dimension; H represents the side length of the plaque;

[0135] The formula of the multi-head self-attention network is:

[0136]

[0137] Among them, B' n represents the output of the multi-head attention mechanism in the nth Transformer Encoder; MSA() represents the multi-head self-attention network; LN() represents layer normalization; B n-1 Represents the output of the n-1th Transformer Encoder (attention encoder);

[0138] The formula of the feedforward network is:

[0139]

[0140] Among them, B nRepresents the output of the feedforward network, that is, the output of the self-attention module; n represents the number of TransformerEncoders; FFN() represents the feedforward network.

[0141] The step S3 is specifically as follows:

[0142] Dividing the data set into a training set, a validation set, and a test set based on a preset ratio, training the coronary artery stenosis detection model through the training set until the loss function of the coronary artery stenosis detection model is less than a preset loss threshold, and continuously optimizing the hyperparameters of the coronary artery stenosis detection model during the training process;

[0143] The trained coronary artery stenosis detection model is verified by the verification set to determine whether the detection accuracy is greater than a preset accuracy threshold. If not, the verification fails and the training set is expanded to continue training. If yes, the verification passes, and:

[0144] The verified coronary artery stenosis detection model is tested using the test set to determine whether the confidence is greater than a preset confidence threshold. If not, the test fails and the training set is expanded to continue training; if so, the test passes and training ends.

[0145] The step S4 is specifically as follows:

[0146] The trained coronary artery stenosis detection model is deployed on a server, an API interface for calling the coronary artery stenosis detection model is set, the calling authority of the API interface is set, and the deployed coronary artery stenosis detection model is accessed after authentication through the API interface to detect real-time CT angiography images of the coronary arteries.

[0147] A preferred embodiment of the coronary artery detection system integrating FSRConv and Transformer of the present invention includes the following modules:

[0148] A data set construction module is used to obtain a large number of historical CT angiography images (CCTA images) of the coronary arteries, and construct a data set after preprocessing each of the historical CT angiography images;

[0149] A coronary artery stenosis detection model creation module, used to create a coronary artery stenosis detection model (FSR-Former) based on a 3D feature space reconstruction module (FSRConv), a self-attention module (Transformer) and an output module;

[0150] The 3D feature space reconstruction module is constructed based on a gating unit, a reconstruction unit, a re-separation unit and a fusion unit; the 3D feature space reconstruction module is used to extract local semantic features of plaques at different positions from CT angiography images; considering the special features of the coronary artery itself, in order to emphasize the rich features of the tubular part and utilize redundant features, the 3D feature space reconstruction module is introduced;

[0151] The self-attention module is constructed based on eight self-attention units (Transformer Block), and the input and output of each self-attention unit are connected based on layer normalization (LN) and residual connection; each self-attention unit is constructed by a multi-head self-attention network (MSA) and a feedforward network (FFN), and the feedforward network is constructed by two linear layers carrying ReLU activation functions; the self-attention module is used to extract the global semantic features of plaques at different positions from CT angiography images, that is, the dependency relationship of plaques at different positions; since a branch of a coronary artery may have multiple plaques, there is a strong contextual connection between adjacent plaques, and each plaque affects the blood flow velocity in the coronary artery, so analyzing the potential relationship between plaques at different positions is very valuable for clinical diagnosis, and the self-attention module can robustly capture global information, thereby improving the robustness of stenosis detection;

[0152] The output module is used to fuse local semantic features and global semantic features to output coronary artery detection results;

[0153] A coronary artery stenosis detection model training module, used for training the coronary artery stenosis detection model using the data set;

[0154] The coronary artery detection module is used to detect the real-time CT angiography image of the coronary artery through the trained coronary artery stenosis detection model.

[0155] Aiming at the special structure of coronary arteries, the present invention proposes a three-dimensional feature space reconstruction convolution method, which can effectively suppress redundancy from both spatial and channel aspects, thereby maximally extracting effective feature information;

[0156] In order to achieve more accurate coronary artery stenosis detection, FSRConv is combined with Transformer to effectively integrate local semantic features and global semantic features.

[0157] The data set construction module is specifically used for:

[0158] A large number of historical CT angiography images of the coronary arteries are obtained, the vascular centerline and plaques of each of the historical CT angiography images are annotated, and a straightening operation is performed on each of the historical CT angiography images based on the vascular centerline using a straightening surface reconstruction technique to obtain a CPR image, and a sample expansion operation is performed on each of the CPR images to construct a data set.

[0159] Since the default CTA viewing angle cannot directly observe the cross section of the coronary artery, which hinders the automatic detection of coronary artery stenosis, the viewing angle of the vascular region needs to be converted, so the straightening operation is performed. The format of the historical CT angiography image is .nii.gz or .dicom.

[0160] In the coronary artery stenosis detection model creation module, the calculation process of the gating unit is:

[0161] Calculate the feature map weight value set W γ :

[0162]

[0163] Among them, w i Indicates the feature map weight value of the i-th channel in the CT angiography image; γ i It represents the pixel difference coefficient of the ith channel in the CT angiography image. The larger the value, the richer the spatial information. j represents the pixel difference coefficient of the jth channel in the CT contrast image; C represents the channel dimension of the CT contrast image;

[0164] The feature map weight value set W is converted into γ Scale to the interval (0,1), set a gating threshold, and adjust the scaled feature map weight value set W based on the gating threshold γ Perform gating to obtain information weight W1 and information weight W2:

[0165] W k = Gate(Sigmoid(W γ (GN3D(X)))), k = 1 or 2;

[0166] Among them, W k Represents information weight; Gate() represents the gating function. When the feature map weight value is greater than the gating threshold, W k =W1, when the feature map weight value is less than the gate threshold, W k =W2, the value of the gate threshold is preferably 0.5; Sigmoid() represents an activation function; GN3D() represents a normalization function; X represents a CT angiography image;

[0167] The high-information feature is calculated based on the information weight W1 and the information weight W2. (expressive spatial content) and small information content (Redundant features):

[0168]

[0169] The reconstruction unit is used to reconstruct features with large information content. And small information features Perform cross reconstruction to obtain the spatial refinement feature map X W ;

[0170] In order to minimize spatial redundancy, the reconstruction unit is introduced to integrate rich information while saving space resources; the reconstruction unit uses a cross-reconstruction operation to effectively integrate weighted unique features. And strengthen the information communication between them; the reconstruction unit can not only separate the features with large information content from the features with small information content, but also reconstruct them to enhance the representative features while reducing the redundancy in the spatial dimension.

[0171] The calculation process of the re-separation unit is:

[0172] A segmentation factor α and a squeeze ratio r are set, 0≤α≤1, r=2; the value of α is preferably 0.5; the squeeze ratio r is used to adjust the feature channel and balance the calculation cost;

[0173] Based on the segmentation factor α, the spatial feature map X is refined W The channels are divided into αC and (1-α)C; C represents the channel dimension;

[0174] The 1×1×1 convolution is based on the squeezing ratio r to refine the spatial feature map X of the αC channel. W And the spatial refinement feature map X of the (1-α)C channel W Perform a squeezing operation to refine the spatial feature map X W Split into the upper refined feature map X up And the refined feature map X low ;

[0175] Based on the refined feature map X up And the refined feature map X low The separation feature map Y1 and the separation feature map Y2 are calculated:

[0176] Among them, GWC() represents the group-wise convolution operation; PWC() represents the point-wise convolution operation;

[0177] By applying grouped convolution operations and point convolution operations to replace expensive k×k×k standard convolutions, high-level representative information can be extracted and computational cost can be reduced;

[0178] The fusion unit is used to fuse the separation feature map Y1 and the separation feature map Y2 to obtain the local semantic feature:

[0179] Y=β1Y1+β2Y2;

[0180]

[0181] β1+β2=1;

[0182] Among them, Y represents the local semantic feature; β1 and β2 both represent the channel feature importance vector; S1 represents the channel-wise amount in the separation feature map Y1 after global three-dimensional pooling; S2 represents the channel-wise amount in the separation feature map Y2 after global three-dimensional pooling; e represents a natural constant.

[0183] The formula for layer normalization is:

[0184]

[0185] Among them, B0 represents the normalized output of the layer; y i represents the feature code of the i-th patch, i = 1, 2, ..., L; p i represents the position code of the i-th plaque; L represents the number of plaques in the coronary artery sequence; R represents a real number; C represents the channel dimension; H represents the side length of the plaque;

[0186] The formula of the multi-head self-attention network is:

[0187]

[0188] Among them, B' n represents the output of the multi-head attention mechanism in the nth Transformer Encoder; MSA() represents the multi-head self-attention network; LN() represents layer normalization; B n-1 Represents the output of the n-1th Transformer Encoder (attention encoder);

[0189] The formula of the feedforward network is:

[0190]

[0191] Among them, B nRepresents the output of the feedforward network, that is, the output of the self-attention module; n represents the number of TransformerEncoders; FFN() represents the feedforward network.

[0192] The coronary artery stenosis detection model training module is specifically used for:

[0193] Dividing the data set into a training set, a validation set, and a test set based on a preset ratio, training the coronary artery stenosis detection model through the training set until the loss function of the coronary artery stenosis detection model is less than a preset loss threshold, and continuously optimizing the hyperparameters of the coronary artery stenosis detection model during the training process;

[0194] The trained coronary artery stenosis detection model is verified by the verification set to determine whether the detection accuracy is greater than a preset accuracy threshold. If not, the verification fails and the training set is expanded to continue training. If yes, the verification passes, and:

[0195] The verified coronary artery stenosis detection model is tested using the test set to determine whether the confidence is greater than a preset confidence threshold. If not, the test fails and the training set is expanded to continue training; if so, the test passes and training ends.

[0196] The coronary artery detection module is specifically used for:

[0197] The trained coronary artery stenosis detection model is deployed on a server, an API interface for calling the coronary artery stenosis detection model is set, the calling authority of the API interface is set, and the deployed coronary artery stenosis detection model is accessed after authentication through the API interface to detect real-time CT angiography images of the coronary arteries.

[0198] In summary, the advantages of the present invention are:

[0199] A large number of historical CT angiography images of the coronary arteries are obtained to mark the vascular centerline and plaques. The CPR image is obtained by straightening the historical CT angiography images based on the vascular centerline through the straightening surface reconstruction technology. The data set is constructed after the sample expansion operation is performed on each CPR image. Then, a coronary stenosis detection model is created based on the 3D feature space reconstruction module, the self-attention module and the output module. The coronary stenosis detection model is trained with the data set. Finally, the real-time CT angiography images of the coronary arteries are detected with the trained coronary stenosis detection model. The 3D feature space reconstruction module of the coronary stenosis detection model is constructed based on the gating unit, the reconstruction unit, the re-separation unit and the fusion unit, and is used to obtain the CPR image from the CT angiography. The local semantic features of plaques at different positions in the image are extracted; the self-attention module is built based on eight self-attention units and is used to extract the global semantic features of plaques at different positions from the CT angiography image; the output module is used to fuse local semantic features and global semantic features to output the coronary artery detection results; that is, the blood vessels in the CT angiography image are straightened, and the redundancy of space and channels is suppressed through the 3D feature space reconstruction module, so as to maximize the extraction of effective feature information (local semantic features) of the coronary artery, and the self-attention module is used to fully understand the contextual information between plaques in the straightened three-dimensional blood vessel segment (CPR image) to robustly capture the global semantic features, thereby greatly improving the accuracy of coronary artery stenosis detection.

[0200] Although the specific implementation modes of the present invention are described above, those skilled in the art should understand that the specific implementation modes described are only illustrative and are not intended to limit the scope of the present invention. Equivalent modifications and changes made by those skilled in the art in accordance with the spirit of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. A coronary artery detection method integrating FSRConv and Transformer, characterized in that: The steps include: Step S1, obtaining a large number of historical CT angiography images of coronary arteries, and constructing a data set after preprocessing each of the historical CT angiography images; Step S2, creating a coronary artery stenosis detection model based on the 3D feature space reconstruction module, the self-attention module and the output module; The 3D feature space reconstruction module is constructed based on a gating unit, a reconstruction unit, a re-separation unit and a fusion unit; the 3D feature space reconstruction module is used to extract local semantic features of plaques at different positions from CT angiography images; The self-attention module is constructed based on eight self-attention units, and the input and output of each self-attention unit are connected based on layer normalization and residual connection; each self-attention unit is constructed by a multi-head self-attention network and a feedforward network, and the feedforward network is constructed by two linear layers carrying ReLU activation functions; the self-attention module is used to extract global semantic features of plaques at different locations from CT angiography images; The output module is used to fuse local semantic features and global semantic features to output coronary artery detection results; Step S3, training a coronary artery stenosis detection model using the data set; Step S4: detecting the real-time CT angiography image of the coronary artery using the trained coronary artery stenosis detection model.

2. The coronary artery detection method integrating FSRConv and Transformer as claimed in claim 1, characterized in that: The step S1 is specifically as follows: A large number of historical CT angiography images of the coronary arteries are obtained, the vascular centerline and plaques of each of the historical CT angiography images are annotated, and a straightening operation is performed on each of the historical CT angiography images based on the vascular centerline using a straightening surface reconstruction technique to obtain a CPR image, and a sample expansion operation is performed on each of the CPR images to construct a data set.

3. The coronary artery detection method integrating FSRConv and Transformer as claimed in claim 1, characterized in that: In step S2, the calculation process of the gate control unit is: Calculate the feature map weight value set W γ : Among them, w i Indicates the feature map weight value of the i-th channel in the CT angiography image; γ i Represents the pixel difference coefficient of the i-th channel in the CT angiography image; γ j represents the pixel difference coefficient of the jth channel in the CT contrast image; C represents the channel dimension of the CT contrast image; The feature map weight value set W is converted into γ Scale to the interval (0,1), set a gating threshold, and adjust the scaled feature map weight value set W based on the gating threshold γ Perform gating to obtain information weight W1 and information weight W2: W k =Gate(Sigmoid(W γ (GN3D(X)))),k=1or2; Among them, W k Represents information weight; Gate() represents the gating function. When the feature map weight value is greater than the gating threshold, W k =W1, when the feature map weight value is less than the gate threshold, W k =W2; Sigmoid() represents the activation function; GN3D() represents the normalization function; X represents the CT contrast image; The high-information feature is calculated based on the information weight W1 and the information weight W2. And small information features The reconstruction unit is used to reconstruct features with large information content. And small information features Perform cross reconstruction to obtain the spatial refinement feature map X W ; The calculation process of the re-separation unit is: Set a segmentation factor α and a compression ratio r, 0≤α≤1, r=2; Based on the segmentation factor α, the spatial feature map X is refined W The channels are divided into αC and (1-α)C; The 1×1×1 convolution is based on the squeezing ratio r to refine the spatial feature map X of the αC channel. W And the spatial refinement feature map X of the (1-α)C channel W Perform a squeezing operation to refine the spatial feature map X W Split into the upper refined feature map X up And the refined feature map X low ; Based on the refined feature map X up And the refined feature map X low The separation feature map Y1 and the separation feature map Y2 are calculated: Y1=GWC(X up )+PWC(X up ); Y2=PWC(X low )∪X low ; Among them, GWC() represents the group convolution operation; PWC() represents the point convolution operation; The fusion unit is used to fuse the separation feature map Y1 and the separation feature map Y2 to obtain the local semantic feature: Y=β1Y1+β2Y2; β1+β2＝1； Among them, Y represents the local semantic feature; β1 and β2 both represent the channel feature importance vector; S1 represents the channel-wise amount in the separation feature map Y1 after global three-dimensional pooling; S2 represents the channel-wise amount in the separation feature map Y2 after global three-dimensional pooling; e represents a natural constant.

4. The coronary artery detection method integrating FSRConv and Transformer as claimed in claim 1, characterized in that: The step S3 is specifically as follows: Dividing the data set into a training set, a validation set, and a test set based on a preset ratio, training the coronary artery stenosis detection model through the training set until the loss function of the coronary artery stenosis detection model is less than a preset loss threshold, and continuously optimizing the hyperparameters of the coronary artery stenosis detection model during the training process; The trained coronary artery stenosis detection model is verified by the verification set to determine whether the detection accuracy is greater than a preset accuracy threshold. If not, the verification fails and the training set is expanded to continue training. If yes, the verification passes, and: The verified coronary artery stenosis detection model is tested using the test set to determine whether the confidence is greater than a preset confidence threshold. If not, the test fails and the training set is expanded to continue training; if so, the test passes and training ends.

5. The coronary artery detection method integrating FSRConv and Transformer as claimed in claim 1, characterized in that: The step S4 is specifically as follows: The trained coronary artery stenosis detection model is deployed on a server, an API interface for calling the coronary artery stenosis detection model is set, the calling authority of the API interface is set, and the deployed coronary artery stenosis detection model is accessed after authentication through the API interface to detect real-time CT angiography images of the coronary arteries.

6. A coronary artery detection system integrating FSRConv and Transformer, characterized by: Includes the following modules: A data set construction module, used for acquiring a large number of historical CT angiography images of the coronary arteries, and constructing a data set after preprocessing each of the historical CT angiography images; A coronary artery stenosis detection model creation module, used to create a coronary artery stenosis detection model based on a 3D feature space reconstruction module, a self-attention module and an output module; The 3D feature space reconstruction module is constructed based on a gating unit, a reconstruction unit, a re-separation unit and a fusion unit; the 3D feature space reconstruction module is used to extract local semantic features of plaques at different positions from CT angiography images; The self-attention module is constructed based on eight self-attention units, and the input and output of each self-attention unit are connected based on layer normalization and residual connection; each self-attention unit is constructed by a multi-head self-attention network and a feedforward network, and the feedforward network is constructed by two linear layers carrying ReLU activation functions; the self-attention module is used to extract global semantic features of plaques at different locations from CT angiography images; The output module is used to fuse local semantic features and global semantic features to output coronary artery detection results; A coronary artery stenosis detection model training module, used for training the coronary artery stenosis detection model using the data set; The coronary artery detection module is used to detect the real-time CT angiography image of the coronary artery through the trained coronary artery stenosis detection model.

7. The coronary artery detection system integrating FSRConv and Transformer as claimed in claim 6, characterized in that: The data set construction module is specifically used for: A large number of historical CT angiography images of the coronary arteries are obtained, the vascular centerline and plaques of each of the historical CT angiography images are annotated, and a straightening operation is performed on each of the historical CT angiography images based on the vascular centerline using a straightening surface reconstruction technique to obtain a CPR image, and a sample expansion operation is performed on each of the CPR images to construct a data set.

8. The coronary artery detection system integrating FSRConv and Transformer as claimed in claim 6, characterized in that: In the coronary artery stenosis detection model creation module, the calculation process of the gating unit is: Calculate the feature map weight value set W γ : Among them, w i Indicates the feature map weight value of the i-th channel in the CT angiography image; γ i Represents the pixel difference coefficient of the i-th channel in the CT angiography image; γ j represents the pixel difference coefficient of the jth channel in the CT contrast image; C represents the channel dimension of the CT contrast image; The feature map weight value set W is converted into γ Scale to the interval (0,1), set a gating threshold, and adjust the scaled feature map weight value set W based on the gating threshold γ Perform gating to obtain information weight W1 and information weight W2: W k =Gate(Sigmoid(W γ (GN3D(X)))),k=1or2; Among them, W k Represents information weight; Gate() represents the gating function. When the feature map weight value is greater than the gating threshold, W k =W1, when the feature map weight value is less than the gate threshold, W k =W2; Sigmoid() represents the activation function; GN3D() represents the normalization function; X represents the CT contrast image; The high-information feature is calculated based on the information weight W1 and the information weight W2. And small information features The reconstruction unit is used to reconstruct features with large information content. And small information features Perform cross reconstruction to obtain the spatial refinement feature map X W ; The calculation process of the re-separation unit is: Set a segmentation factor α and a compression ratio r, 0≤α≤1, r=2; Based on the segmentation factor α, the spatial feature map X is refined W The channels are divided into αC and (1-α)C; The 1×1×1 convolution is based on the squeezing ratio r to refine the spatial feature map X of the αC channel. W And the spatial refinement feature map X of the (1-α)C channel W Perform a squeezing operation to refine the spatial feature map X W Split into the upper refined feature map X up And the refined feature map X low ; Based on the refined feature map X up And the refined feature map X low The separation feature map Y1 and the separation feature map Y2 are calculated: Y1=GWC(X up )+PWC(X up ); Y2=PWC(X low )∪X low ; Among them, GWC() represents the group convolution operation; PWC() represents the point convolution operation; The fusion unit is used to fuse the separation feature map Y1 and the separation feature map Y2 to obtain the local semantic feature: Y=β1Y1+β2Y2; β1+β2＝1； Among them, Y represents the local semantic feature; β1 and β2 both represent the channel feature importance vector; S1 represents the channel-wise amount in the separation feature map Y1 after global three-dimensional pooling; S2 represents the channel-wise amount in the separation feature map Y2 after global three-dimensional pooling; e represents a natural constant.

9. The coronary artery detection system integrating FSRConv and Transformer as claimed in claim 6, characterized in that: The coronary artery stenosis detection model training module is specifically used for: Dividing the data set into a training set, a validation set, and a test set based on a preset ratio, training the coronary artery stenosis detection model through the training set until the loss function of the coronary artery stenosis detection model is less than a preset loss threshold, and continuously optimizing the hyperparameters of the coronary artery stenosis detection model during the training process; The trained coronary artery stenosis detection model is verified by the verification set to determine whether the detection accuracy is greater than a preset accuracy threshold. If not, the verification fails and the training set is expanded to continue training. If yes, the verification passes, and: The verified coronary artery stenosis detection model is tested using the test set to determine whether the confidence is greater than a preset confidence threshold. If not, the test fails and the training set is expanded to continue training; if so, the test passes and training ends.

10. The coronary artery detection system integrating FSRConv and Transformer as claimed in claim 6, characterized in that: The coronary artery detection module is specifically used for: The trained coronary artery stenosis detection model is deployed on a server, an API interface for calling the coronary artery stenosis detection model is set, the calling authority of the API interface is set, and the deployed coronary artery stenosis detection model is accessed after authentication through the API interface to detect real-time CT angiography images of the coronary arteries.

Images