A coronary stenosis detection method based on coronary angiography and spatiotemporal collaborative learning

Abstract

The application discloses a coronary stenosis detection method based on coronary angiography and space-time collaborative learning, and belongs to the field of intelligent medical treatment. The deep feature extraction network based on the channel mixing structure reparameterization convolution single aggregation module is used, the channel segmentation and multi-branch training are carried out, the feature reuse and information flow are enhanced, the double-flow path is introduced into the feature fusion network to realize adaptive weighted fusion of different scale features, and the representation of spatial features is improved; the space-time collaborative analysis module capable of simultaneously fusing time information and local space information is used, the time sequence connection is constructed, the space-time feature map is fused, the vascular shape deformation and hemodynamic information in the space-time sequence are effectively captured, the effective feature information is enhanced, and therefore the detection accuracy and robustness of coronary stenosis lesions are improved. The time and space information of the coronary angiography sequence image is comprehensively considered, the number of false positive detections is reduced, and the detection capability of the model for coronary stenosis lesions is comprehensively improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of intelligent medical treatment of artificial intelligence assisted diagnosis, and particularly relates to a coronary artery stenosis detection method based on sequence coronary angiography images and spatio-temporal collaborative analysis. BACKGROUND

[0002] At present, artificial intelligence driven by big data has become one of the research frontiers in the field of medical auxiliary diagnosis due to its high accuracy and high stability. Coronary artery disease (CAD) is a disease caused by coronary atherosclerosis, which leads to insufficient blood supply to the heart. It is one of the most common cardiovascular diseases worldwide and is the main cause of myocardial infarction, heart failure and cardiac death. According to the statistics of the World Health Organization (WHO), CAD caused about 8.8 million deaths worldwide in 2019, accounting for 15.8% of all disease causes. The harmfulness of coronary artery stenosis is mainly reflected by the number, location and degree of stenosis. In current hospital practice, the complexity of the detection background (vessel dilation and contraction, occlusion, imaging angle, etc.), as well as the subjectivity caused by the experience and level of the practitioner, restricts the accuracy and objectivity of coronary stenosis detection (number and location) and stenosis degree determination.

[0003] Therefore, it is particularly important to carry out research on computer-aided coronary stenosis diagnosis technology. At present, the methods for detecting coronary artery stenosis based on CAG mainly include coronary artery stenosis detection based on vessel diameter and coronary artery stenosis detection based on vessel features. Coronary artery stenosis detection based on vessel diameter includes steps such as vessel segmentation, artery centerline extraction, diameter calculation and stenosis analysis. Since there are errors in these steps themselves, the errors of multiple steps will reduce the robustness and robustness of stenosis evaluation. Therefore, the stenosis detection method based on vessel features is more effective than the method based on vessel diameter. At present, some methods have adopted the way of directly processing X-ray coronary angiography images to exclude the influence of multi-step vessel segmentation on the accuracy of coronary artery stenosis detection. However, most of the current researches only use the spatial structure information in a single frame image for stenosis detection, without considering the inter-frame context relationship of the image sequence. In some frames, the robustness of single-frame detection stenosis is insufficient due to the instantaneous unevenness of contrast agent, which to some extent reduces the overall detection accuracy. SUMMARY

[0004] The present application provides a coronary stenosis detection method based on coronary angiography and spatio-temporal collaborative learning to improve the detection performance of coronary stenosis.

[0005] The technical scheme adopted by the present application is as follows: a coronary stenosis detection method based on coronary angiography and spatio-temporal collaborative learning, which comprises the following steps:

[0006] Step 1: Obtain raw coronary angiography data and doctor's diagnostic information;

[0007] Step 2: Perform data preprocessing on the raw coronary angiography data and doctor's diagnostic information to obtain sequential coronary angiography images and corresponding stenosis location information labels;

[0008] Step 3: Construct a spatial feature constraint network for coronary angiography images, which includes a deep feature extraction network and a dual-flow path multi-scale feature fusion network;

[0009] Among them, the deep feature extraction network is a network structure based on the channel shuffling structure reparameterized convolution single aggregation module, which is used to extract spatial features at three different scales of each frame of coronary angiography images in the sequence.

[0010] The dual-stream path multi-scale feature fusion network enhances and fuses the multi-scale spatial features extracted by the deep feature extraction network through a bottom-up and top-down dual-stream path to obtain three enhanced spatial features at different scales for each frame of coronary angiography image; wherein the three scales of the enhanced spatial features are consistent with the scales of the spatial features extracted by the deep feature extraction network.

[0011] Step 4: Construct a spatiotemporal information collaborative analysis module. Input the enhanced spatial features of each frame of the extracted coronary angiography sequence at three scales into the spatiotemporal information collaborative analysis module. Using the single-frame spatial features as prior knowledge, construct temporal connections between sequence frames to obtain spatiotemporal features containing information from past frames. This yields the spatiotemporal features of the current frame at three scales containing information from past frames, further enhancing the effective feature information. Then, fuse these spatiotemporal features with the enhanced spatial features at the corresponding scale of the current frame to obtain the final spatiotemporal features of the current frame at three scales. That is, using the single-frame spatial features as prior knowledge, and simultaneously capturing the temporal dependence between sequence frames and the spatial dependence within each frame, to obtain the spatiotemporal features of each frame at three different scales. This achieves the enhancement of the effective stenosis lesion feature information of the current frame by the information from past frames in the integrated sequence, weakens the redundant information in the spatiotemporal feature sequence, and realizes the spatiotemporal collaborative analysis of coronary angiography sequence images.

[0012] The multi-scale spatiotemporal features of each frame are then input into the detection head to perform coronary artery stenosis target detection, and the coronary artery stenosis detection result of the current frame is obtained.

[0013] Furthermore, the loss functions used when training the model parameters of the deep feature extraction network, the dual-stream path multi-scale feature fusion network, and the spatiotemporal information collaborative analysis module include:

[0014] The distribution focal point loss function and the SIoU loss function, wherein the distribution focal point loss function optimizes the probability of the left and right two positions closest to the label in the form of cross entropy, so that the network focuses on the target position and the distribution of the adjacent area faster; the SIoU enables the loss function to consider the position, size and shape information of the boundary box during the optimization process, thereby improving the coronary stenosis lesion detection effect.

[0015] Further, the deep feature extraction network sequentially includes a Focus layer, a convolutional block, a plurality of stacked units composed of the convolutional block and a structural reparameterization convolution single aggregation (RCS-OSA) module, and an SPPF module.

[0016] The convolutional block sequentially includes a convolutional layer, a batch normalization layer and an activation function layer.

[0017] The SPPF module is configured to: input of the SPPF module is first processed by a convolutional block, and then the output of the convolutional block is respectively processed by three parallel maximum pooling layers, and the three-pooling results are spliced by a splicing layer and then input into a convolutional block, and the output of the convolutional block is taken as the output of the SPPF module.

[0018] Further, the number of stacked units composed of the convolutional block and the structural reparameterization convolution single aggregation module is set to 4.

[0019] Further, the structural reparameterization convolution single aggregation module includes two RepVGG modules, two RCS modules and one channel reshaping module; wherein, the input of the RCS-OSA module is first input into the RepVGG module 1 for feature extraction, the output features of the RepVGG module 1 are respectively input into the channel reshaping module and the RCS module 1, the output features of the RCS module 1 are respectively input into the channel reshaping module and the RCS module 2, and the output features of the RCS module 2 are also input into the channel reshaping module; the channel reshaping module performs channel reshaping operation on the three inputs respectively, and then performs channel dimension splicing through a splicing layer, and then inputs into the RepVGG module 2, and the output of the RepVGG module 2 is taken as the output of the RCS-OSA module.

[0020] The RCS module is configured: the input features of the RCS module are segmented in the channel dimension, and are segmented into two sub-features with the same feature plane dimension (width, height) and different channel numbers, wherein the first sub-feature is mapped through an activation function after the outputs of three parallel branches of an identity branch, a convolution layer with a 1x1 convolution kernel and a convolution layer with a 3x3 convolution kernel are added, to obtain the intermediate mapping features of the first sub-feature; the second sub-feature is spliced with the intermediate mapping features of the first sub-feature through a channel division (Channel Split) Channel Split operation layer and a splicing layer, and the spliced features are output through a channel mixing module of the RCS module;

[0021] Further, the multi-scale feature fusion network of the double-flow path adaptively fuses three spatial features of different scales extracted by the deep feature extraction network, specifically including:

[0022] The three spatial features of different scales are defined as large-scale spatial features, medium-scale spatial features and small-scale spatial features, respectively.

[0023] The small-scale spatial features are upsampled to the dimension of the medium-scale spatial features, and then are weighted and spliced with the medium-scale spatial features to obtain first spliced features; the first spliced features are upsampled to the dimension of the large-scale spatial features, and then are weighted and spliced with the large-scale spatial features to obtain second spliced features.

[0024] The second spliced features are downsampled to the medium-scale spatial features, and then are weighted and spliced with the first spliced features to obtain third spliced features, and the third spliced features are downsampled to the dimension of the small-scale spatial features, and then are spliced with the small-scale spatial features to obtain fourth spliced features.

[0025] Based on the second spliced features, the third spliced features and the fourth spliced features, three scale enhanced spatial features are obtained.

[0026] Further, the spatio-temporal information collaborative analysis module is specifically configured as:

[0027] The three scale enhanced spatial features of each frame of the sequence coronary angiography image are extracted by a long short-term memory network (ST-LSTM) unit based on spatio-temporal characteristics.

[0028] If the current frame is the first frame of the sequence, the enhanced spatial features of each scale are input into the corresponding ST-LSTM unit, and the output features of the ST-LSTM unit are added to the input features of the ST-LSTM unit to obtain the final spatio-temporal features of the current frame and the current scale.

[0029] From the second frame of the sequence, the input of the ST-LSTM unit includes the enhanced spatial features of the current frame at the current scale, and also includes the output features of the enhanced spatial features of the previous frame at the same scale through the ST-LSTM unit; then the output features of the ST-LSTM unit corresponding to the current frame at the current scale are added to the enhanced spatial features of the current frame at the current scale, to obtain the final spatio-temporal features of the current frame at the current scale.

[0030] In summary, the technical scheme provided by the application at least brings the following beneficial effects:

[0031] The application extracts effective feature layers at different scales by reasonably utilizing the deep feature extraction network and performs weighted fusion, comprehensively considers the time and space information of the coronary angiography sequence image, and improves the loss function to further distinguish the coronary stenosis target and the background interference, thereby reducing the number of false positive detections and comprehensively improving the detection capability of the model for coronary stenosis lesions. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0033] Figure 1 The flowchart of a coronary stenosis detection method based on coronary angiography and spatio-temporal collaborative learning in the embodiment;

[0034] Figure 2 The structural diagram of the deep feature extraction network;

[0035] Figure 3 The structural diagram of the RCS module;

[0036] Figure 4 The structural diagram of the weighted double-flow path multi-scale feature fusion network;

[0037] Figure 5A schematic diagram of the overall structure of the spatiotemporal collaborative analysis network;

[0038] Figure 6 A schematic diagram of the structure of the spatiotemporal collaborative analysis module;

[0039] Figure 7 A schematic diagram of the calculation of the IoU bounding box loss function;

[0040] Figure 8 A schematic diagram of the calculation of the SIoU bounding box loss function;

[0041] Figure 9 A comparison chart of the coronary stenosis detection results of the method of the embodiment of the application. DETAILED DESCRIPTION

[0042] In order to make the objectives, technical solutions and advantages of the embodiments of the application clearer, the technical solutions in the embodiments of the application will be described in detail and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only some of the embodiments of the application, rather than all the embodiments. Generally, the components of the embodiments of the application described and shown in the drawings can be arranged and designed using different configurations. Therefore, the following detailed description of the embodiments of the application provided in the drawings is not intended to limit the scope of the claimed application, but only represents selected embodiments of the application.

[0043] The embodiment of the application provides a coronary stenosis detection method based on coronary angiography and spatiotemporal collaborative learning, referring to Figure 1 which comprises the following steps:

[0044] S1: obtaining original data information. Including coronary angiography data and corresponding diagnosis information (to obtain the detection label of the stenosis position coordinate). In the embodiment of the application, the coronary angiography original data is obtained from the hospital equipment, and the corresponding diagnosis information is given by a professional doctor;

[0045] S2: data preprocessing. The coronary angiography original data is usually in the dicom format, and it is necessary to convert the coronary angiography data in the dicom format into a sequence image format required by a deep learning model, and to make coronary stenosis diagnosis information into an xml format label, so as to facilitate the deep learning training of the network model parameters;

[0046] Then, the coronary angiography sequence images are randomly shuffled to make a five-fold cross-validation data set, and each subset is divided according to the ratio of training set: test set = 4:1. Among them, the training set is used to train the model parameters, and the test set is used to evaluate the detection effect of the trained model.

[0047] S3: spatial feature extraction. The deep feature extraction network architecture of the embodiment of the application is as shown in Figure 2As shown, it gradually compresses the size of the feature map in the spatial dimension while expanding the number of channels of the feature map to extract the high-level semantic features of the image. Meanwhile, the RCS-OSA (RCS-One-Shot Aggregation) module is introduced to ensure the reuse of features and enhance the flow of information between different layers by stacking RCS.

[0048] In this embodiment, the deep feature extraction network used includes, in sequence, a Focus layer (by rearranging the spatial dimension information to the channel dimension, the purpose of improving the computational efficiency without losing information is achieved, i.e., by slicing operation, the high-resolution feature map is split into multiple low-resolution feature maps, and by splicing and convolution operation, a low-resolution but increased channel number feature map is finally obtained), a convolution block, a stacked network of a plurality of stacked units composed of a convolution block and a structural re-parameterization convolution single aggregation (RCS-OSA) module, and an SPPF module (Spatial Pyramid Pooling Fast); in order to realize the extraction of multi-scale features, this embodiment extracts feature maps of different scales at three different levels of the deep feature extraction network (the second last RCS-OSA module, the third last RCS-OSA module, and the last layer, i.e., the output of the SPPF module) to input the subsequent weighted double-flow path multi-scale feature fusion network, and the feature maps of the three scales correspond to coronary stenosis targets of large, medium and small scales, respectively. The convolution block includes, in sequence, a convolution layer, a batch normalization layer and an activation function layer. In this embodiment, the RCS-OSA module is as shown in Figure 2 As shown, it includes two RepVGG modules, two RCS modules and one channel reshaping module; wherein the input of the RCS-OSA module is first sent to the RepVGG module 1 for feature extraction, the output feature map of the RepVGG module 1 is sent to the channel reshaping module and the RCS module 1 respectively, the output feature map of the RCS module 1 is sent to the channel reshaping module and the RCS module 2 respectively, and the output feature map of the RCS module 2 is also sent to the channel mixing module; after the channel mixing operation of the three inputs respectively, the channel mixing feature map is spliced by the splicing layer to obtain a feature map with the same input dimension, which is then sent to the RepVGG module 2, and the output thereof is obtained based on the output of the RCS-OSA module. Wherein, the channel mixing operation is as follows: first, reshape the input four-dimensional tensor into a five-dimensional tensor, divide the channel number into multiple groups, each group contains an equal number of channels; then, change the order of the channels by transposition operation to enhance the diversity of the features. Finally, adjust the tensor back to its original four-dimensional form. This process effectively mixes the channels, thereby improving the feature expression ability of the model.

[0049] Referring to Figure 2In the embodiment, the SPPF module is specifically configured as follows: the input of the SPPF module is first subjected to a convolution block, and then the output thereof is subjected to pooling processing through three parallel maximum pooling layers respectively, and the three-way pooling results are spliced through a splicing layer and then input into a convolution block, and the output of the convolution block is used to obtain the output of the SPPF module.

[0050] Referring to Figure 3 In the embodiment, the structure of the RCS module is specifically configured as follows: the feature dimension of a given input tensor is CxHxW, and after a channel segmentation operator, the input tensor is divided into two different channel tensors with equal dimensions, C / 2xHxW. For one of the two tensors, an identity branch, a 1x1 convolution and a 3x3 convolution are used. After multi-branch training of one of the two tensors, the other tensor is concatenated in a channel mode, and channel shuffling is applied to enhance the information fusion between the two tensors. Then, the RCS module is repeatedly stacked in the RCS-OSA module to ensure the reuse of features and enhance the information flow between different channels of adjacent layers. In order to reduce the degree of network fragmentation, only three feature concatenations are retained on the one-time aggregation path, which can reduce the network calculation burden and reduce the memory occupation.

[0051] S4: Weighted double-flow path multi-scale feature fusion. The feature fusion network is responsible for fusing and optimizing the three different scale features extracted by the deep feature extraction network. The double-flow multi-scale feature fusion network established in the present embodiment is shown in Figure 4 As shown in the figure, the double-flow multi-scale feature fusion network captures the bottom-level detail features and high-level semantic features of the image through two parallel paths, a bottom-level feature extraction path and a high-level feature extraction path. The bottom-level feature extraction path mainly extracts edge, texture and other detail information through a shallow convolution network, while the high-level feature extraction path extracts semantic information features through a deep convolution network. These two kinds of features are fused together at each layer of the feature pyramid to form a rich and comprehensive feature representation.

[0052] In the embodiment, the double-flow multi-scale feature fusion network is specifically configured as follows:

[0053] The three different scale spatial features output by the deep feature extraction network are defined as large-scale spatial features, medium-scale spatial features and small-scale spatial features, respectively.

[0054] The small-scale spatial features are up-sampled to the dimension of the medium-scale spatial features, and then are spliced with the medium-scale spatial features to obtain first spliced features; the first spliced features are up-sampled to the dimension of the large-scale spatial features, and then are spliced with the large-scale spatial features to obtain second spliced features; then, the second spliced features (P3) are down-sampled to the medium-scale spatial features, and then are spliced with the first spliced features to obtain third spliced features (P4), and the third spliced features are down-sampled to the dimension of the small-scale spatial features, and then are spliced with the small-scale spatial features to obtain fourth spliced features (P5); finally, based on the second spliced features, the third spliced features and the fourth spliced features, enhanced spatial features of three scales are obtained.

[0055] S5: spatio-temporal collaborative analysis. The spatio-temporal collaborative analysis network architecture of the embodiment of the present application is shown in FIG. 5. After passing through the spatial feature constraint network, each frame of coronary angiography image obtains three enhanced spatial features of different scales, and then the enhanced spatial features of different scales are input into the spatio-temporal collaborative analysis module, a time sequence connection is constructed between the sequence frames, a spatio-temporal feature map containing past frame information is obtained, and then the spatio-temporal feature map is fused with the spatial feature map of the current frame to further enhance the effective feature information, and finally a multi-scale spatio-temporal feature is obtained. Figure 5

[0056] Finally, the multi-scale spatio-temporal feature of each frame obtained finally is input into a detection head to perform coronary stenosis target detection, and a coronary stenosis detection result of the current frame is obtained, including object category of the target frame, front and background class information of the target frame and position information of the target frame.

[0057] That is, in the embodiment of the present application, the final spatio-temporal features (spatio-temporal features of three scales) are used for judgment, and the stenosis target bounding box (i.e. positioning detection), the category (whether it is a stenosis lesion, i.e. classification detection) and the corresponding probability (confidence of the prediction frame, i.e. regression detection) are output on different scales. The coronary stenosis lesion detection result is obtained by calculation and screening, so as to improve the accuracy and robustness of coronary stenosis lesion detection.

[0058] In one embodiment, the loss function used when training the model parameters of the deep feature extraction network, the dual-flow path multi-scale feature fusion network and the spatio-temporal information collaborative analysis module constructed in the present application is designed as follows:

[0059] For the features of the coronary angiography sequence image, a new bounding box loss function is set in the embodiment, and the function is composed of two parts:

[0060] ​The first part is to calculate the loss of the distribution probability of the bounding box and the distribution probability of the label by DFL (Debiased Focal Loss), learn the loss of the position around the label, enhance the generalization of the model under complex conditions (the coronary blood vessels move with the heart), and thus optimize each edge;

[0061] The second part is to restore the distribution probability of the bounding box to the predicted box, and calculate the loss of the predicted box and the actual box of the label by SIOU (Simplified IoU) loss, so as to optimize the whole predicted box.

[0062] As a possible implementation, in the dual-flow multi-scale feature fusion network, the two weights involved in any two features (also called feature maps) to be weighted and spliced are learnable parameters, respectively defined as ω a and ω b First, they are randomly initialized, and then the weights are normalized to the interval [0, 1], and the normalization method is as follows:

[0063]

[0064] Where ε is a preset learning rate to avoid unstable weights. In this embodiment, it is set to 0.0001,

[0065] The two feature maps F a and F b to be weighted and spliced are weighted, and the spliced feature map F out after fusion and splicing is expressed as follows:

[0066] F out =Concat(ω a F a ,ω b F b )

[0067] As a possible implementation, in the example of the application, the space-time collaborative analysis module adopts an ST-LSTM unit that can simultaneously fuse time information and local spatial information, and the unfolded structure is as shown in Figure 6 The specific calculation formula is as follows:

[0068]

[0069] Where σ() is a sigmoid activation function, * and ⊙ represent convolution operators and Hadamard products, respectively, i t , f t , g t , o trespectively represent the input gate, the forget gate, the input modulation gate and the output gate. The input gate determines how much new information will be written into the cell state; the forget gate determines how much past information will be retained in the cell state; and the output gate determines how much information of the cell state will be output to the hidden state of the current time step. xg hg xi hi xf hf are convolution kernel parameters, and b g , b i , b f are bias parameters, which are constantly updated by network learning. t , i' t , f' t are auxiliary parameters involved in the calculation, is the standard time unit, which is passed from the previous node at t-1 to the current time step in each LSTM unit. is the current part of the spatiotemporal memory, which is vertically passed from the l-1 layer to the current node at the same time step. Finally, the final hidden state of the node is the output of the current time step, which depends on the fused spatiotemporal memory. These memories from different directions are connected together, and then a 1x1 convolution layer is applied for dimension reduction to make the hidden state have the same dimension as the storage unit. Unlike simple memory splicing, using a shared output gate for the two types of memory enables seamless memory fusion, which can effectively capture the shape deformation of blood vessels and hemodynamic information in the spatiotemporal sequence, thus enabling spatiotemporal collaborative analysis.

[0070] In this embodiment, referring to Figure 5 ​​​​​The enhanced spatial features of three scales (large, medium and small) of each frame of the sequence are subjected to spatio-temporal collaborative analysis by the ST-LSTM unit. From the second frame of the sequence, the input data of the ST-LSTM unit includes the enhanced spatial features of the current frame, and also includes the enhanced spatial features of the same scale of the previous frame subjected to the output features of the ST-LSTM unit. Finally, the output features of the ST-LSTM unit corresponding to the current scale of the current frame are added to the enhanced spatial features of the current scale of the current frame to obtain the final spatio-temporal features of the current scale of the current frame. For example, for the small-scale enhanced spatial features of the second frame, the small-scale enhanced spatial features of the second frame are first sent to the ST-LSTM unit, and the input of the ST-LSTM unit also includes the output features of the ST-LSTM unit corresponding to the small scale of the first frame. Then, the output features of the ST-LSTM unit corresponding to the small scale of the second frame are added to the small-scale enhanced spatial features of the second frame to obtain the final spatio-temporal features of the small scale of the second frame, so that the spatio-temporal features of three scales are obtained for each frame. Then, the spatio-temporal features of three scales of each frame are input into the detection head (Dectect) in the application for target detection of coronary stenosis. Figure 5

[0071] In this embodiment, referring to Figure 7 , the IoU intersection over union and the boundary box loss of the prediction box A and the real box B are defined as follows:

[0072]

[0073] L IoU = 1 - IoU

[0074] As a possible implementation manner, the SIoU loss function is used for boundary box regression in the application example. The SIoU loss function is composed of four Cost cost functions: Angle cost (angle cost), Distance cost (distance cost), Shape cost (shape cost) and IoU cost.

[0075] The Angle cost cost calculation formula is as follows:

[0076] Λ = 1 - 2 * sin 2 (arcsin(x) - 4π)

[0077]

[0078] Referring to Figure 8 , c h is the height difference between the center points of the real box and the prediction box, and σ is the distance between the center points of the real box and the prediction box, represent the x coordinates of the real box and the prediction box, respectively, and the y coordinates of the real box and the prediction box, respectively.​

[0079] The values of x and angle a can be brought into the formula to simplify:

[0080]

[0081]

[0082] The distance between the real value bounding box and the bounding box prediction value is calculated as shown in Figure 8 The distance cost is redefined as:

[0083]

[0084] wherein, γ=2-Λ.

[0085] The shape cost is defined as:

[0086]

[0087] wherein, The value of θ defines the cost of the shape, and its value is unique for each data set.

[0088] The final regression loss of SIOU is:

[0089]

[0090] As a possible implementation, the DFL loss in the present embodiment is defined as follows:

[0091] DFL(S i ,S i+1 )=-((y i+1 -y)log(S i )+(y-y i )log(S i+1 ))

[0092] In the formula, i.e. the prediction value of the network output, i.e. the adjacent prediction value of the network output, y represents the real target label, y i and y i+1 represent the right boundary and the left boundary of the target (y8≤y≤y 8+1 ) respectively.

[0093] In order to further analyze the performance of the method of the present embodiment, the performance of the basic detection network and the spatio-temporal information collaborative network (spatio-temporal information collaborative analysis module) constructed in the present embodiment in the detection of sequence coronary angiography images is compared, as shown in Figure 9As shown in the figure, the results of the base detection network are displayed on the left side, and there are two frame misdiagnosis phenomena, which significantly affect the overall detection accuracy. The spatio-temporal information collaborative network on the right side successfully detects the coronary stenosis lesions in all five frames of images, showing good stability and accuracy. This result shows that the spatio-temporal information collaborative network can more effectively integrate the information in the time dimension when processing coronary angiography images, thereby improving the accuracy and reliability of the detection. The method of the embodiment of the present application has the advantage that it can capture the dynamic changes and correlations between different frames in the image sequence, thereby reducing the missed and misdiagnosis cases, and ensuring that important information in each frame can be fully utilized.

[0094] In summary, the spatio-temporal information collaborative network constructed in the embodiment of the present application has obvious advantages in the detection task of coronary angiography images, and can provide more accurate auxiliary diagnosis for clinical practice, which has important application value

[0095] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

[0096] The above only describes some embodiments of the present application. For those skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which are all within the protection scope of the present application.

Claims

1. A method for detecting coronary artery stenosis based on coronary angiography and spatiotemporal collaborative learning, characterized in that, Includes the following steps: Step 1: Obtain raw coronary angiography data and doctor's diagnostic information; Step 2: Perform data preprocessing on the raw coronary angiography data and doctor's diagnostic information to obtain sequential coronary angiography images and corresponding stenosis location information labels; Step 3: Construct a spatial feature constraint network for coronary angiography images, which includes a deep feature extraction network and a dual-flow path multi-scale feature fusion network; Among them, the deep feature extraction network is a network structure based on the channel shuffling structure reparameterized convolution single aggregation module, which is used to extract spatial features at three different scales of each frame of coronary angiography images in the sequence. The dual-stream path multi-scale feature fusion network enhances and fuses the multi-scale spatial features extracted by the deep feature extraction network through a bottom-up and top-down dual-stream path to obtain three enhanced spatial features at different scales for each frame of coronary angiography image; wherein the three scales of the enhanced spatial features are consistent with the scales of the spatial features extracted by the deep feature extraction network. Step 4: Construct a spatiotemporal information collaborative analysis module. Input the enhanced spatial features of each frame of the extracted coronary angiography sequence into the spatiotemporal information collaborative analysis module. Using the single-frame spatial features as prior knowledge, construct temporal connections between sequence frames to obtain spatiotemporal features containing information from past frames. This yields the spatiotemporal features of the current frame at three scales containing information from past frames. Then, fuse these spatiotemporal features with the enhanced spatial features at the corresponding scale of the current frame to obtain the final spatiotemporal features of the current frame. Then, the multi-scale spatiotemporal features of each frame are input into the detection head to perform coronary artery stenosis target detection, and the coronary artery stenosis detection result of the current frame is obtained; The spatiotemporal information collaborative analysis module is specifically configured as follows: Enhanced spatial features at three scales for each frame of sequential coronary angiography images are extracted using a long short-term memory network (ST-LSTM) unit based on spatiotemporal characteristics. If the current frame is the first frame of the sequence, the enhanced spatial features of each scale are input into the corresponding ST-LSTM unit, and the output features of the ST-LSTM unit are added to the input features of the ST-LSTM unit to obtain the final spatiotemporal features of the current frame at the current scale. Starting from the second frame of the sequence, the input of the ST-LSTM unit includes the augmented spatial features of the current frame at the current scale, as well as the output features of the ST-LSTM unit of the augmented spatial features of the previous frame at the same scale. Then, the output features of the ST-LSTM unit corresponding to the current frame at the current scale are added to the augmented spatial features of the current frame at the current scale to obtain the final spatiotemporal features of the current frame at the current scale.

2. The method as described in claim 1, characterized in that, The loss functions used when training the model parameters of the deep feature extraction network, the dual-stream path multi-scale feature fusion network, and the spatiotemporal information collaborative analysis module include: The distributed focus loss function and the SIoU loss function are used. The distributed focus loss function optimizes the probability of the two positions closest to the label, one to the left and one to the right, in the form of cross-entropy.

3. The method as described in claim 1, characterized in that, The deep feature extraction network consists of a Focus layer, a convolutional block, a stacked network consisting of several stacked units composed of convolutional blocks and structure-reparameterized convolutional single-pass aggregation modules, and an SPPF module. The convolutional block consists of a convolutional layer, a batch normalization layer, and an activation function layer, in sequence. The SPPF module is configured as follows: the input of the SPPF module is first passed through a convolutional block, and then its output is processed by three parallel max pooling layers. The three pooling results are then concatenated by a concatenation layer and fed into a convolutional block. The output of the SPPF module is obtained based on the output of this convolutional block.

4. The method as described in claim 3, characterized in that, The number of stacked units consisting of convolutional blocks and structure-reparameterized convolutional single-pass aggregation modules is set to 4.

5. The method as described in claim 3, characterized in that, The structure-reparameterized convolutional single-pass aggregation module includes two RepVGG modules, two RCS modules, and one channel reshaping module. The input to the RCS-OSA module is first fed into RepVGG module 1 for feature extraction. The output features of RepVGG module 1 are then fed into the channel reshaping module and RCS module 1, respectively. The output features of RCS module 1 are fed into the channel reshaping module and RCS module 2, respectively, and the output features of RCS module 2 are also fed into the channel shuffling module. The channel shuffling module performs channel shuffling on the three inputs, then concatenates them along the channel dimensions through a concatenation layer before feeding them into RepVGG module 2. The output of the RCS-OSA module is obtained based on the output of RepVGG module 2. The RCS module is configured as follows: The input features of the RCS module are segmented according to the channel dimension, dividing them into two sub-features with different numbers of channels but the same feature plane dimension. The first sub-feature is passed through three parallel branches: an identity branch, a 1×1 convolutional layer, and a 3×3 convolutional layer. The outputs of the three branches are then added together and mapped by an activation function to obtain the intermediate mapped features of the first sub-feature. The second sub-feature is processed through a channel segmentation operation layer and then concatenated with the intermediate mapped features of the first sub-feature through a concatenation layer. The concatenated features are then passed through a channel shuffling module to obtain the output of the RCS module.

6. The method as described in claim 1, characterized in that, The multi-scale feature fusion network for dual-stream paths adaptively weights and fuses spatial features at three different scales extracted by a deep feature extraction network, specifically including: The spatial features at three different scales are defined as large-scale spatial features, medium-scale spatial features, and small-scale spatial features, respectively. The small-scale spatial features are upsampled to the dimension of the medium-scale spatial features, and then weighted and concatenated with the medium-scale spatial features to obtain the first concatenated feature; the first concatenated feature is then upsampled to the dimension of the large-scale spatial features, and then weighted and concatenated with the large-scale spatial features to obtain the second concatenated feature. The second stitching feature is downsampled to a medium-scale spatial feature and then weighted and stitched with the first stitching feature to obtain the third stitching feature. The third stitching feature is downsampled to the dimension of a small-scale spatial feature and then stitched with the small-scale spatial feature to obtain the fourth stitching feature. Enhanced spatial features at three scales are obtained based on the second, third, and fourth splicing features.

Images