A Coronary Stent Detection Method Based on Improved YOLOv5

Abstract

This invention discloses a coronary artery stent detection method based on an improved YOLOv5 network. The method includes: acquiring and expanding detection images containing coronary artery stents to obtain a coronary artery stent detection training sample library; annotating the detection images with coronary artery stents to obtain an annotated coronary artery stent detection training set and a coronary artery stent detection test set; data augmentation; improving the YOLOv5 network for small object detection; introducing an attention mechanism to obtain an improved YOLOv5 network; and using the improved YOLOv5 network to detect coronary artery stents in the detection images in the coronary artery stent detection test set to obtain coronary artery stent detection results. This invention, based on an improved YOLOv5 network, enables one-step detection from image input to output, offering advantages such as fast detection speed and excellent detection results. It improves the detection accuracy of coronary artery stents in OCT images, bringing convenience to medical detection.

Description

Technical Field

[0001] This invention relates to the field of medical testing, and more particularly to a method for detecting coronary stents based on an improved YOLOv5. Background Technology

[0002] Coronary atherosclerosis is a pathological condition characterized by lipid deposition in the coronary arteries, leading to hardening or varying degrees of narrowing. When the narrowing worsens or exceeds 50%, causing clinical symptoms, it is termed coronary heart disease. Coronary artery stenting has become a crucial treatment for coronary atherosclerotic heart disease. Coronary arteries are characterized by narrowing and thin walls, making detection extremely complex. Intravascular coherence tomography (IVOCT) systems are widely used in coronary artery detection. IVOCT is a frequency-domain OCT system based on a swept-frequency light source, which can separate longitudinal and lateral resolution, significantly improving image resolution. The matching degree between intravascular IVOCT images and stent acquisition is extremely high. Using images generated by the IVOCT system for stent detection helps improve the accuracy of stent detection.

[0003] Currently, there are the following problems with the detection of stents after coronary artery stent implantation: (1) Many images are generated for each patient's detection, and there are many stents in each image, resulting in a large workload for annotation; (2) Small targets are not easy to annotate, and annotation takes a long time; (3) Manual annotation is subject to subjective judgment, which affects the annotation results; (4) Traditional detection algorithms have low accuracy and large errors.

[0004] Current coronary stent detection algorithms include those based on random forest classifiers, automatic stent detection techniques based on fully convolutional networks (FCNs) and U-Net principles, and stent detection implemented using U-Net's attention mechanism. However, these traditional algorithms all suffer from high false negative rates, slow manual labeling speeds, long processing times, and high human resource requirements, making it difficult to complete complex detection tasks quickly. Summary of the Invention

[0005] To address the aforementioned problems in the prior art, this invention provides a coronary stent detection method based on an improved YOLOv5.

[0006] This invention provides a method for detecting coronary stents based on an improved YOLOv5, comprising the following steps:

[0007] Step S1: Collect detection images containing coronary artery stents, expand the detection images, and form a coronary artery stent detection training sample library with the expanded images;

[0008] Step S2: Annotate the detection images in the coronary stent detection training sample library with coronary stents. After annotation, save the annotation information of the detection images to obtain the annotated coronary stent detection training set and coronary stent detection test set.

[0009] Step S3: Perform data augmentation on the labeled coronary stent detection training set;

[0010] Step S4: Improve small target detection in the YOLOv5 network;

[0011] Step S5: Introduce an attention mechanism into the YOLOv5 network improved by small object detection to obtain the improved YOLOv5 network;

[0012] Step S6: Use the improved YOLOv5 network to perform coronary artery stent detection on the detection images in the coronary artery stent detection test set to obtain the coronary artery stent detection results.

[0013] In one embodiment of the present invention, the detection image is an OCT detection image.

[0014] In one embodiment of the present invention, step S1 includes the following steps:

[0015] Step S1.1: Detect the object with implanted coronary artery stent using an OCT scanner, and select OCT images containing coronary artery stents from the obtained OCT images;

[0016] Step S1.2: Rotate, denoise, and mirror the OCT image containing the coronary artery stent obtained in step S1.1 to augment the OCT image;

[0017] Step S1.3: Combine the OCT images containing coronary artery stents obtained in step S1.1 with the expanded OCT images obtained in step S1.2 to form a coronary artery stent detection training sample library.

[0018] In one embodiment of the present invention, step S2 includes the following steps:

[0019] Step S2.1: Annotate the coronary artery stents in the detection images in the coronary artery stent detection training sample library;

[0020] Step S2.2: Generate a corresponding annotation information file based on the annotated detection image information. The annotation information file contains one or more of the following information: the minimum horizontal coordinate of the annotation box Xmin, the maximum vertical coordinate of the annotation box Ymax, the width of the annotation box w, and the height of the annotation box h, four annotation box position information values, and one annotation category information value.

[0021] Step S2.3: Divide the coronary stent detection training sample library into a coronary stent detection training set and a coronary stent detection test set according to a preset ratio.

[0022] In one embodiment of the present invention, in step S3, four detection images are randomly selected from the coronary stent detection training set in each batch. These four detection images are randomly processed, and the randomly processed images are combined into a new detection image. After multiple batches of iteration, a data-enhanced coronary stent detection training set is obtained.

[0023] In one embodiment of the present invention, step S4 includes the following steps:

[0024] Step S4.1: Add two anchor modules to the anchors module in the original yml file of the yolov5 network to generate 15 anchor boxes of different sizes.

[0025] Step S4.2: Add a C3 and CBS layer after Focus in the Backbone module of the YOLOv5 network;

[0026] Step S4.3: Based on step S4.2, add four connection layers, four C3 layers, four CBS layers, four SPP layers, and two upsampling layers to the Neck layer, and add two convolutional layers and two small target detectors to the Head layer.

[0027] In one embodiment of the present invention, step S5 includes the following steps:

[0028] Step S5.1: Replace the last C3 layer in the Backbone module of the YOLOv5 network improved with small object detection with the SE attention mechanism module;

[0029] Step S5.2: Based on step S5.1, a CBAM attention mechanism module is added between the C3 and CBS layers of the Neck layer of the YOLOv5 network to obtain the improved YOLOv5 network.

[0030] In one embodiment of the present invention, the SE attention mechanism module includes a compression submodule and an expansion submodule, wherein the compression submodule consists of a global pooling layer; and the expansion submodule consists of two fully connected layers, a ReLU activation function, and a Sigmoid activation function.

[0031] In one embodiment of the present invention, the CBAM attention mechanism module includes a spatial attention module and a channel attention module. The spatial attention module includes a cascaded global average pooling and global max pooling layer, two convolutional layers, and two activation function layers. The channel attention module includes a channel pooling layer, a 7×7 convolutional layer, a BN normalization layer, and an activation function layer.

[0032] Compared with the prior art, the present invention has the following technical effects:

[0033] This invention is based on an improved YOLOv5 for the detection of coronary artery stents. It can achieve one-step image input to output, and has the advantages of fast detection speed and excellent detection results. It improves the detection accuracy of coronary artery stents in OCT images and brings convenience to medical testing.

[0034] It should be noted that this invention was supported by the National Natural Science Foundation of China (No. 11404240), the Tianjin Municipal Natural Science Foundation (General Project No. 20JCYBJC00300), and the Enterprise Commissioned Project No. 52H22035. Attached Figure Description

[0035] Figure 1 A flowchart illustrating a coronary stent detection method based on an improved YOLOv5 according to an embodiment of the present invention;

[0036] Figure 2 This is a schematic diagram of a Mosaic image enhancement process according to an embodiment of the present invention;

[0037] Figure 3 The diagram shows the YOLOv5 model before the improvement.

[0038] Figure 4 This is a schematic diagram of the improved YOLOv5 Backbone network layer according to an embodiment of the present invention;

[0039] Figure 5 This is a schematic diagram of the improved yolov5Neck network layer according to an embodiment of the present invention;

[0040] Figure 6 A diagram of the improved YOLOv5 model according to an embodiment of the present invention;

[0041] Figure 7 This is a schematic diagram of an SE attention module according to an embodiment of the present invention;

[0042] Figure 8 This is a schematic diagram of a CBAM attention module according to an embodiment of the present invention;

[0043] Figure 9This is a diagram showing the experimental test results according to an embodiment of the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and the accompanying drawings. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0045] Figure 1 This is a flowchart of a coronary stent detection method based on an improved YOLOv5 according to an embodiment of the present invention, as shown below. Figure 1 As shown, the coronary stent detection method based on the improved YOLOv5 includes the following steps:

[0046] Step S1: Collect detection images containing coronary artery stents, expand the detection images, and form a coronary artery stent detection training sample library with the expanded images;

[0047] In one embodiment of the present invention, the detection image may be, for example, an OCT detection image.

[0048] Further, in this embodiment, step S1 may include the following steps:

[0049] Step S1.1: Detect the object with implanted coronary artery stent using an OCT scanner, and select OCT images containing coronary artery stents from the obtained OCT images;

[0050] Step S1.2: To improve the detection performance of the algorithm, the OCT detection image containing the coronary artery stent obtained in step S1.1 is rotated, denoised, and mirrored to expand the OCT detection image;

[0051] Step S1.3: Combine the OCT images containing coronary artery stents obtained in step S1.1 with the expanded OCT images obtained in step S1.2 to form a coronary artery stent detection training sample library; for example, in one embodiment of the present invention, a coronary artery stent detection training sample library containing 1200 OCT images can be obtained.

[0052] Step S2: Annotate the detection images in the coronary stent detection training sample library with coronary stents. After annotation, save the annotation information of the detection images to obtain the annotated coronary stent detection training set and coronary stent detection test set.

[0053] In one embodiment of the present invention, the Labelimg software can be used to annotate the detection images in the coronary stent detection training sample library with coronary stents.

[0054] Further, in this embodiment, step S2 may include the following steps:

[0055] Step S2.1: Use the Labelimg software to annotate the detection images in the coronary stent detection training sample library with coronary stents. For example, the coronary stent can be annotated as stent.

[0056] Step S2.2: Generate a corresponding annotation information file based on the annotated detection image information. The annotation information file contains one or more of the following information: four annotation box position information values: minimum horizontal coordinate Xmin, maximum vertical coordinate Ymax, annotation box width w, and annotation box height h, and an annotation category information value. In the example above, since there is only one detected target stent, the corresponding annotation category information value is 1.

[0057] Step S2.3: Divide the coronary stent detection training sample library into a coronary stent detection training set and a coronary stent detection test set according to a preset ratio. Taking the example above, assuming the preset ratio is 8:2, the coronary stent detection training sample library containing 1200 detection images can be divided into a coronary stent detection training set containing 960 detection images and a coronary stent detection test set containing 240 detection images.

[0058] Because YOLOv5 requires TXT files for data training, if the obtained annotation information files are in a different format than TXT (such as XML), a script is needed to convert them to TXT format. Taking XML as an example, each converted TXT file corresponds to one detection image. Each line in the TXT file contains information about a single labeled target, including the label category, the x and y coordinates of the target's center position, and the width and height of the bounding box.

[0059] Step S3: Perform data augmentation on the labeled coronary stent detection training set;

[0060] In one embodiment of the present invention, the Mosaic method can be used to perform data augmentation on the labeled coronary stent detection training set. The Mosaic data augmentation process is as follows: Figure 2As shown, each batch randomly selects four detection images from the coronary stent detection training set. These four detection images undergo random processing such as random cropping, random arrangement, and random scaling. Then, the randomly processed images are combined into a new detection image. After multiple batches of iteration, such as repeating the total batch size, the data-augmented coronary stent detection training set can be obtained. Data augmentation enables the detection network to learn to autonomously detect targets smaller than the normal size.

[0061] Step S4: Improve small target detection in the YOLOv5 network;

[0062] In one embodiment of the present invention, the improvement of small object detection in the YOLOv5 network can be achieved by adding two anchors for small object detection to the existing YOLOv5 network anchors, and adding a network layer for small object detection. Figure 3 This is a diagram of the YOLOv5 model before the improvement. Figure 4 This is a schematic diagram of the improved YOLOv5 Backbone network layer according to an embodiment of the present invention. Figure 5 This is a schematic diagram of an improved YOLOv5 Neck network layer according to an embodiment of the present invention. Figure 6 The diagram shows an improved YOLOv5 model according to an embodiment of the present invention, where x in C3_x represents the number of Bottleneck modules, such as... Figures 3-6 As shown, specifically, the improvement of small target detection in the YOLOv5 network may include the following steps:

[0063] Step S4.1: Add two anchor modules to the anchors module in the original YAML file of the YOLOv5 network to generate 15 anchor boxes of different sizes. This allows smaller anchor boxes to be used to train and predict smaller coronary stents in the detection image, thereby improving the final detection accuracy.

[0064] Step S4.2: Add a C3 and CBS layer after the Focus layer in the Backbone module of the YOLOv5 network, as follows: Figure 4 As shown, the C3 layer in the YOLOv5 network plays the role of reducing computation, optimizing the gradient propagation of network information, and preventing gradient explosion. The role of CBS in the YOLOv5 network is to extract feature information from the feature map.

[0065] Step S4.3: Based on step S4.2, add four concat layers, four C3 layers, four CBS layers, four SPP layers, and two upsampling layers to the Neck layer. The added structure is as follows: Figure 5 As shown in the shaded box, you can also refer to... Figure 6 The dashed box in the Neck layer is used to obtain a larger feature map for small object detection, and two convolutional layers and two small object detectors are added to the Head layer.

[0066] Step S5: Introduce an attention mechanism into the YOLOv5 network improved by small object detection to obtain the improved YOLOv5 network;

[0067] In one embodiment of the present invention, the introduction of an attention mechanism into the YOLOv5 network improved for small object detection can be implemented by introducing an SE attention mechanism into the Backbone module of the YOLOv5 network improved for small object detection, and a CBAM attention mechanism into the Head layer, so as to further improve the training effect of the detection model. Specifically, the introduction of an attention mechanism into the YOLOv5 network improved for small object detection may include the following steps:

[0068] Step S5.1: Replace the last C3 layer in the Backbone module of the YOLOv5 network improved for small object detection with an SE (Squeeze-and-Excitation) attention mechanism module, as follows: Figure 4 As shown, the SE attention mechanism module consists of a squeezing submodule and an excitation submodule. The squeezing submodule comprises a global pooling layer; the excitation submodule comprises two fully connected layers, a ReLU activation function (non-linear), and a Sigmoid activation function, as shown. Figure 7 As shown in the diagram, the compression submodule in the SE attention mechanism module uses global average pooling to compress the two-dimensional features (H*W) of each channel into a single real number, reducing the feature map from [h,w,c] to [1,1,c]. The expansion submodule in the SE attention mechanism module assigns a weight value to each feature channel and then uses two fully connected layers to construct the correlation between channels, ensuring that the number of output weight values ​​is the same as the number of channels in the input feature map, thus restoring the feature map from [1,1,c] to [h,w,c]. Finally, the normalized weights obtained in the expansion submodule are applied to the features of each channel.

[0069] Step S5.2: Based on step S5.1, a lightweight attention mechanism module CBAM (Convolutional Block Attention Module) is added between the C3 and CBS layers of the Neck layer in the YOLOv5 network, such as... Figure 5As shown, this yields the improved YOLOv5 network, as follows: Figure 6 As shown. That is, in Figure 6 In the Backbone layer, a CBS layer and a C3 layer are added after the Focus layer, and an Attention layer is added after the SPP layer to replace the previous C3 layer. The Attention layer is the SE attention module layer. The Neck layer adds a CBAM layer between the C3 and CBS layers, and then adds four fully connected layers, four C3 layers, four CBS layers, four SPP layers, and two upsampling layers. The Head layer adds two convolutional layers and two small object detectors.

[0070] The CBAM attention mechanism module includes a spatial attention module and a channel attention module, such as... Figure 8 As shown, this approach aims to improve the accuracy of feature extraction from coronary artery stents in detected images, thereby increasing the target detection rate. Specifically, the spatial attention module of CBAM comprises a cascaded global average pooling and global max pooling layer (GAP+GMP), two convolutional layers (Conv and 1×1Conv), and two activation function layers (ReLU and Sigmoid). The channel attention module of CBAM comprises a channel pooling layer, a 7×7 convolutional layer (7×7Conv), a BN normalization layer (BN), and an activation function layer (Sigmoid).

[0071] Step S6: Use the improved YOLOv5 network to perform coronary artery stent detection on the detection images in the coronary artery stent detection test set to obtain the coronary artery stent detection results.

[0072] The coronary stent detection method based on the improved YOLOv5 provided by this invention was applied to a coronary stent detection training set containing 960 OCT images and a coronary stent detection test set containing 240 OCT images for experiments.

[0073] To ensure the fairness of the experiment, the network models used for comparison and the training process provided in this experiment all used the following equipment configuration and training parameters: computer graphics card was NVIDIA-GeForce-RTX-3060Ti; YOLOv5 algorithm version 5.0; total number of training iterations (Epochs) was 300; batch size was 16; momentum was 0.937; weight decay was 0.0005; initial learning rate was 0.001; and the detection image size was 640×640.

[0074] The network models compared in the experiments included YOLOv5, YOLOv5-5D (introducing two anchor network models), YOLOv5-SE-5D (adding only the SE attention mechanism block to YOLOv5-5D), and YOLOv5-SE CBAM-5D (the improved YOLOv5 network model provided in this invention). The performance of the detection method in this invention was evaluated using precision, recall, average precision, and F-Measure. Precision refers to the ratio of the number of detected positive samples to the total number of predicted samples; recall refers to the ratio of the number of detected features of a certain class to the total number of features of that class in the dataset, measuring the recall rate of the retrieval system; average precision refers to the performance metric of the accuracy of predicting the target location; and F-Measure is a comprehensive evaluation metric for precision and recall.

[0075] Precision=TP / (TP+FP), Recall=TP / (TP+FN),

[0076] F1=2(1 / P+1 / R)=2PR / (P+R).

[0077] Where TP is the number of positive samples predicted as positive by the model, FP is the number of negative samples predicted as positive by the model, FN is the number of positive samples predicted as negative by the model, P is the model detection precision, R is the model recall, AP is the model mean precision, and F1 is the harmonic mean of precision and recall.

[0078] The experimental data for different network models are shown in Table 1.

[0079] Table 1

[0080] Network Model P R AP F1 yolov5 86.32％ 80.61％ 82.38％ 83.32％ yolov5-5D 93.68％ 91.67％ 91.68％ 92.66％ yolov5-5D-SE 94.62％ 92.46％ 92.59％ 93.52％ yolov5-5D-SE-CBAM 94.82％ 93.34％ 93.67％ 94.07％

[0081] As shown in Table 1, using YOLOv5, the precision was 86.32%, the recall was 80.61%, the mean precision was 82.38%, and the F1 score was 83.32%. YOLOv5-5D improved precision by 7.36%, recall by 11.06%, mean precision by 9.3%, and the F1 score by 9.34% compared to YOLOv5. YOLOv5-5D-SE improved precision by 0.94%, recall by 0.79%, mean precision by 0.91%, and the F1 score by 0.86% compared to YOLOv5-5D. YOLOv5-5D-SE-CBAM improved precision by 0.2%, recall by 0.88%, mean precision by 1.08%, and the F1 score by 0.55% compared to YOLOv5-5D-SE. Specific experimental results are as follows: Figure 9 As shown.

[0082] The comparison of the above experimental data further demonstrates that the stability and accuracy of the coronary stent detection method based on the improved YOLOv5 proposed in this invention have been effectively improved.

[0083] The above embodiments are implemented based on the technical solution of the present invention, providing detailed implementation methods and specific operation processes. However, the scope of protection of the present invention is not limited to the above embodiments. Unless otherwise specified, the methods used in the above embodiments are conventional methods.

[0084] It should be understood that the specific embodiments described above are merely illustrative or explanatory of the principles of the invention and do not constitute a limitation thereof. Therefore, any modifications, equivalent substitutions, improvements, etc., made without departing from the spirit and scope of the invention should be included within the protection scope of the invention. Furthermore, the appended claims are intended to cover all variations and modifications falling within the scope and boundaries of the appended claims, or equivalent forms of such scope and boundaries.

Claims

1. A method for detecting coronary stents based on an improved YOLOv5, characterized in that, The method includes the following steps: Step S1: Collect detection images containing coronary artery stents, expand the detection images, and form a coronary artery stent detection training sample library with the expanded images; Step S2: Annotate the detection images in the coronary stent detection training sample library with coronary stents. After annotation, save the annotation information of the detection images to obtain the annotated coronary stent detection training set and coronary stent detection test set. Step S3: Perform data augmentation on the labeled coronary stent detection training set; Step S4: Improve small target detection in the YOLOv5 network; Step S5: Introduce an attention mechanism into the YOLOv5 network improved by small object detection to obtain the improved YOLOv5 network; Step S6: Use the improved YOLOv5 network to detect coronary artery stents in the detection images in the coronary artery stent detection test set to obtain coronary artery stent detection results; Step S4 includes the following steps: Step S4.1: Add two anchor modules to the anchors module in the original yml file of the yolov5 network to generate 15 anchor boxes of different sizes. Step S4.2: Add a C3 and CBS layer after Focus in the Backbone module of the YOLOv5 network. The C3 layer in the YOLOv5 network plays the role of reducing the amount of computation, optimizing the gradient propagation of network information, and preventing gradient explosion. The role of CBS in the YOLOv5 network is to extract feature information from the feature map. Step S4.3: Based on step S4.2, add four connection layers, four C3 layers, four CBS layers, four SPP layers, and two upsampling layers to the Neck layer, and add two convolutional layers and two small target detectors to the Head layer. Step S5 includes the following steps: Step S5.1: Replace the last C3 layer in the Backbone module of the YOLOv5 network improved with small object detection with the SE attention mechanism module; Step S5.2: Based on step S5.1, a CBAM attention mechanism module is added between the C3 and CBS layers of the Neck layer of the YOLOv5 network to obtain the improved YOLOv5 network.

2. The method according to claim 1, characterized in that, The detected image is an OCT detection image.

3. The method according to claim 2, characterized in that, Step S1 includes the following steps: Step S1.1: Detect the object with implanted coronary artery stent using an OCT scanner, and select OCT images containing coronary artery stents from the obtained OCT images; Step S1.2: Rotate, denoise, and mirror the OCT image containing the coronary artery stent obtained in step S1.1 to augment the OCT image; Step S1.3: Combine the OCT images containing coronary artery stents obtained in step S1.1 with the expanded OCT images obtained in step S1.2 to form a coronary artery stent detection training sample library.

4. The method according to any one of claims 1-3, wherein step S2 comprises the following steps: Step S2.1: Annotate the detection images in the coronary stent detection training sample library with coronary stent labels; Step S2.2: Generate a corresponding annotation information file based on the annotated detection image information. The annotation information file contains one or more of the following information: the minimum horizontal coordinate of the annotation box Xmin, the maximum vertical coordinate of the annotation box Ymax, the width of the annotation box w, and the height of the annotation box h, four annotation box position information values, and one annotation category information value. Step S2.3: Divide the coronary stent detection training sample library into a coronary stent detection training set and a coronary stent detection test set according to a preset ratio.

5. The method according to any one of claims 1-3, characterized in that, In step S3, four detection images are randomly selected from the coronary stent detection training set for each batch. These four detection images are then randomly processed to form a new detection image. After multiple iterations, the data-enhanced coronary stent detection training set is obtained.

6. The method according to claim 1, wherein, The SE attention mechanism module includes a compression submodule and an expansion submodule. The compression submodule consists of a global pooling layer, and the expansion submodule consists of two fully connected layers, a ReLU activation function, and a Sigmoid activation function.

7. The method according to any one of claims 1-3 and 6, wherein, The CBAM attention mechanism module includes a spatial attention module and a channel attention module. The spatial attention module includes a cascaded global average pooling and global max pooling layer, two convolutional layers, and two activation function layers. The channel attention module includes a channel pooling layer, a 7×7 convolutional layer, a BN normalization layer, and an activation function layer.

Images