A method for identifying vascular stenosis lesions based on object detection and its application

Abstract

The present invention discloses a method for identifying vascular stenosis lesions based on object detection and its application. The method includes: after preprocessing and vascular segmentation of the vascular DSA image data to be identified, inputting it into a lesion recognition model, and the lesion recognition model outputs the label types corresponding to each position in the vascular DSA image data to be identified; the lesion recognition model is an improved YOLOv7 algorithm model, and the improvement of the improved YOLOv7 algorithm model lies in that the Head in the model improves the assignment strategy using a positive sample assignment strategy, and the AuxHead uses a RepConv layer to replace the convolutional layer and the normalization layer. The recognition method of the present invention is based on the currently state-of-the-art YOLOv7 object detection model and improves the performance on it, further improving the object detection accuracy, and is particularly suitable for detecting vascular stenosis lesions; it can mark the lesion sites of vascular stenosis, which is intuitive and easy to understand.

Description

Technical Field

[0001] The present invention relates to the technical field of pan-vascular intelligent diagnosis and treatment, and relates to a method for identifying vascular stenosis lesions based on object detection and its application. Background Art

[0002] Coronary heart disease is a disease caused by stenosis of the coronary arteries, resulting in insufficient blood supply to the heart. Therefore, in the field of diagnosis and treatment of coronary heart disease, it is necessary to identify vascular stenosis lesions.

[0003] Identifying vascular stenosis lesions can help doctors diagnose coronary heart disease. By detecting the degree and location of stenosis of the coronary arteries, the severity of the lesions can be determined, providing doctors with treatment plans; at the same time, detecting vascular stenosis lesions can provide more accurate treatment plans for patients with coronary heart disease. According to the degree of the lesions, doctors can decide whether surgical treatment or drug treatment is needed, and tracking and evaluating the treatment progress of patients is also an important reference.

[0004] With the development of various imaging technologies, there are multiple medical images that can be used to identify vascular stenosis lesions. Ultrasonography is a non-invasive method that uses sound waves to observe the internal structure of blood vessels and blood flow velocity. CTA imaging can model blood vessels and visually view the stenotic parts of blood vessels. Doppler ultrasound can detect blocked or stenotic arteries, and intravascular ultrasound (IVUS) can more accurately evaluate the characteristics of the blood vessel wall and plaques. Fractional flow reserve can estimate the location of vascular stenosis through changes in values. In addition, coronary angiography is the "gold standard" for clinical diagnosis of coronary heart disease, and it has also become a major research hotspot to process angiography images and identify vascular stenosis through various methods.

[0005] CN 202010354792.8 discloses a method and device for detecting vascular stenosis in coronary artery X-ray sequence angiography, which proposes a stenosis detection method that performs deep sequence feature fusion on the detection results of consecutive multi-frame images; CN 201910142018.8 discloses a method and device for detecting vascular stenosis, which proposes a method for determining the stenotic parts of blood vessels by finding the gradient value of blood vessel diameter. These methods can respectively determine the locations of vascular stenosis lesions, but currently there is less research on using object detection methods to identify the locations of vascular stenosis lesions. The object detection method has the characteristics of good real-time performance, fast speed, and simple training and use, making it particularly suitable for application scenarios such as surgery.

[0006] Therefore, it is of great practical significance to develop a method for identifying the locations of vascular stenosis lesions based on object detection methods. Summary of the Invention

[0007] Due to the above-mentioned defects in the prior art, the present invention provides a method for identifying vascular stenosis lesions based on object detection methods, which overcomes the defects that the real-time performance and speed of existing identification methods are difficult to meet the requirements of surgical application scenarios.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] A method for identifying vascular stenosis lesions based on object detection. After preprocessing and vascular segmentation of the vascular DSA image data to be identified, it is input into the lesion identification model, and the lesion identification model outputs the label types corresponding to each position in the vascular DSA image data to be identified. The label types include normal and lesion.

[0010] The lesion identification model is an improved YOLO v7 algorithm model. The improvement of the improved YOLO v7 algorithm model lies in that the Head in the model improves the soft-label assignment strategy by using a positive sample assignment strategy, that is, a soft label between 0 and 1 is assigned to the prediction box according to the IoU between the prediction box and the ground truth box, instead of a hard label of 0 or 1. This can better reflect the similarity between the prediction box and the ground truth box, and at the same time balance the proportion of positive and negative samples. The Aux Head uses a RepConv layer to replace the convolutional layer and the normalization layer, that is, the convolutional layer and the normalization layer are merged, reducing the number of parameters and the amount of calculation.

[0011] The training process of the improved YOLO v7 algorithm model is a process of continuously adjusting the parameters of the model with the image data in the training dataset as the input and the known label data corresponding to the image data as the theoretical output. The training data in the training dataset includes training data with the label type of normal in the image and training data with the label type of lesion in the image.

[0012] The method for identifying vascular stenosis lesions based on object detection of the present invention uses an improved YOLO v7 algorithm model as the object detection classification model. Based on the current relatively good YOLOv7 model, it is improved (the assignment strategy and the Aux Head are improved) at the same time, optimizing the model performance, further improving the accuracy of object detection, especially suitable for identifying vascular stenosis lesions, with a simple processing sequence, a small amount of data processing and a fast feedback speed, and good application prospects.

[0013] As a preferred technical solution:

[0014] In the method for identifying vascular stenosis lesions based on object detection as described above, the preprocessing includes resizing, normalization, denoising and smoothing to reduce the influence of noise and other interference factors and improve the accuracy of the model.

[0015] A method for identifying vascular stenosis lesions based on object detection as described above, wherein the training data in the training dataset is the data after preprocessing and vascular segmentation.

[0016] A method for identifying vascular stenosis lesions based on object detection as described above, wherein the vascular segmentation is completed by applying semantic segmentation.

[0017] The present invention also provides a computer device, which includes:

[0018] At least one processor; and,

[0019] A memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, it implements the method for identifying vascular stenosis lesions based on object detection as described above.

[0021] In addition, the present invention also provides a computer-readable storage medium, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by a processor, they implement the method for identifying vascular stenosis lesions based on object detection as described above.

[0022] The above technical solutions are only one feasible technical solution of the present invention, and the protection scope of the present invention is not limited thereto. Those skilled in the art can reasonably adjust the specific design according to actual needs.

[0023] The above invention has the following advantages or beneficial effects:

[0024] (1) The method for identifying vascular stenosis lesions based on object detection of the present invention can provide more objective quantitative information and reduce subjective errors and human interference compared with traditional manual film reading and evaluation;

[0025] (2) The method for identifying vascular stenosis lesions based on object detection of the present invention can mark the lesion sites of vascular stenosis on the image, and can provide intuitive and easy-to-understand lesion location and morphological information for doctors or patients;

[0026] (3) The method for identifying vascular stenosis lesions based on object detection of the present invention adopts the currently state-of-the-art YOLOv7 object detection model and improves its performance, further improving object detection, especially in the field of vascular stenosis lesion detection;

[0027] (4) The method for identifying vascular stenosis lesions based on object detection of the present invention. The object detection model is developed based on a deep learning framework, which can be conveniently optimized and deployed, and can also be easily extended and modified to adapt to the application requirements in different scenarios, with good application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] By reading the detailed description of the non-restrictive embodiments with reference to the following drawings, the present invention and its features, appearance, and advantages will become more obvious. The same reference numerals indicate the same parts in all the drawings. The drawings are not drawn to scale, and the emphasis is on showing the gist of the present invention.

[0029] Figure 1 It is a sequence diagram of the method for identifying vascular stenosis lesions based on object detection of the present invention;

[0030] Figure 2 It is a schematic diagram of the improved YOLOv7 model used in the present invention;

[0031] Figure 3 It is a schematic diagram of the sequence set in Example 1;

[0032] Figure 4 It is a schematic diagram of the effect of the present invention;

[0033] Figure 5 It is a schematic diagram of the structure of the computer device in Example 2. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0034] The following further describes the structure in the present invention with reference to the drawings and specific embodiments, but it is not a limitation of the present invention.

[0035] Example 1

[0036] A method for identifying vascular stenosis lesions based on object detection, the sequence is as Figure 1 shown, specifically as follows:

[0037] (1) Preprocess the vascular DSA image data to be identified. The preprocessing includes resizing, normalizing, denoising, and smoothing;

[0038] (2) Perform vascular semantic segmentation on the preprocessed data;

[0039] (3) After inputting the data obtained in step (2) into the lesion recognition model, the lesion recognition model outputs the label types corresponding to each position in the vascular DSA image data to be identified. The label types include normal and lesion;

[0040] The lesion recognition model is an improved YOLO v7 algorithm model. The improvement of the YOLO v7 algorithm model lies in that the Head in the model improves the soft-label assignment strategy by applying a positive sample assignment strategy, and the Aux Head uses a RepConv layer to replace the convolutional layer and the normalization layer;

[0041] The training process of the improved YOLO v7 algorithm model is a process of taking the image data in the training dataset (specifically, the data after preprocessing and vascular segmentation) as the input, taking the known label data corresponding to the image data as the theoretical output, and continuously adjusting the parameters of the model. The training data in the training dataset includes training data with a label type of normal in the image and training data with a label type of lesion in the image.

[0042] The above improved YOLOv7 object detection model (such as Figure 2 shown) is composed and processed as follows:

[0043] Input: Input an image, perform batch processing on its image size, and then perform normalization and data augmentation processing.

[0044] Backbone (a deep network composed of 50 convolutional layers, normalization layers, activation functions, pooling layers, and ELAN modules, used to extract high-level semantic features of the input image): Send the input image into the Backbone network to extract three feature maps C3, C4, and C5 with different scales and numbers of channels.

[0045] Head (a PAFPN structure composed of an SPPCSP module, an ELAN-H module, a RepConv layer, and convolutional layers, used to perform upsampling, downsampling, fusion, and prediction on the feature maps extracted by the Backbone): Send the feature maps C3, C4, and C5 extracted by the Backbone into the Head network for upsampling, downsampling, fusion, and prediction to obtain three prediction results P3’, P4’, and P5’ with different scales and numbers of channels (first perform an SPPCSP operation on C5, then fuse it with C4 and C3 from top to bottom to obtain three feature maps with different scales and numbers of channels, denoted as P3, P4, and P5 respectively, and then fuse them with P4 and P5 from bottom to top to obtain the fused feature maps P3’, P4’, and P5’), which respectively contain the object category, confidence, and bounding box information of each grid cell. Assign a soft label between 0 and 1 to the prediction box according to the IoU between the prediction box and the ground truth box as the positive sample assignment strategy.

[0046] The improvement of the Head lies in that it has improved the soft-label assignment strategy by adopting a positive sample assignment strategy, that is, a soft label between 0 and 1 is assigned to the prediction box according to the IoU between the prediction box and the ground truth box, instead of a hard label of 0 or 1. This can better reflect the similarity between the prediction box and the ground truth box, and at the same time balance the proportion of positive and negative samples.

[0047] During training, the Aux Head (a simple network composed of a convolutional layer, a normalization layer, an activation function, a pooling layer, and a fully connected layer, which is used to provide additional supervision signals during training to improve network performance) is used: the feature map C5 extracted by the Backbone is fed into the Aux Head network, and a binary classification result is output, indicating whether there is an object in the picture, as an additional supervision signal.

[0048] The improvement of the Aux head lies in that it has merged the convolutional layer and the normalization layer, that is, the RepConv layer is used to replace the original convolutional layer and normalization layer, thereby reducing the number of parameters and the amount of computation.

[0049] Output: The three prediction results P3’, P4’, and P5’ output by the Head are post-processed, including: confidence threshold filtering and class selection, to obtain the final detection results, including the class, confidence, and bounding box coordinates of the object.

[0050] Specifically, the steps of constructing the method of the present invention are as follows (as Figure 3 shown):

[0051] 1. Data collection: Obtain a large amount of vascular DSA image data of vascular stenosis lesions. These images can come from hospitals, research institutions, and other medical institutions. The data must include normal and diseased images for model training;

[0052] The purpose of this step is to obtain enough data to train an effective model, because the quality and quantity of the data directly affect the performance of the model. Vascular DSA imaging is a medical imaging technology that uses X-rays and contrast agents to display the structure and function of blood vessels, and it can be used to diagnose diseases such as vascular stenosis.

[0053] To obtain these images, it is necessary to contact the cooperating medical institutions, obtain their authorization and consent, and comply with relevant ethical and legal norms. The data must include normal and diseased images so that the model can distinguish between the two and learn the characteristics and locations of the lesions.

[0054] 2. Data processing: Preprocess the collected images, including resizing, normalization, denoising, and smoothing. This step is to reduce the influence of noise and other interference factors to improve the accuracy of the model;

[0055] The purpose of this step is to make the data more suitable for the input format and requirements of the model, as well as to improve the quality of the data. Since images from different sources may have different attributes such as resolution, size, brightness, and contrast, they need to be unified into a standardized form so that the model can process them better.

[0056] In addition, since there may be adverse factors such as noise, blur, and artifacts in the images, it is necessary to denoise and smooth the images to reduce the interference of these factors on the model's recognition ability.

[0057] 3. Model training: Use the improved YOLOv7 algorithm to train a recognition model for vascular stenosis lesions, which can identify vascular stenosis lesions. A large amount of image data is required during the training process so that the model can identify lesions in different conditions;

[0058] The purpose of this step is to use the data to train a model that can automatically identify vascular stenosis lesions. The improved YOLOv7 algorithm used in the present invention is an algorithm based on deep learning and computer vision. It can detect multiple targets in a single image and give their positions and categories. This algorithm has higher speed and accuracy compared to other algorithms and can adapt to different scenarios and environments. To train this algorithm, a large amount of image data with labeled normal and diseased regions is required so that the algorithm can learn how to distinguish between the two and can adapt to changes in different sizes, shapes, positions, angles, etc.

[0059] 4. Model evaluation: Evaluate the accuracy and performance of the model. This step is to determine the actual effect and potential defects of the model so that improvements can be made when needed;

[0060] The purpose of this step is to test whether the model can achieve the expected goals and whether there are areas for improvement. To evaluate the model, some metrics are needed to measure the performance of the model, including MIoU, AP50, etc. These metrics can reflect the correctness and completeness of the model in identifying vascular stenosis lesions. In addition to these metrics, factors such as the speed, stability, and scalability of the model also need to be considered to ensure the usability and efficiency of the model in practical applications.

[0061] 5. Optimization and improvement: Optimize and improve the model according to the evaluation results. This includes using more data to train the model, adjusting hyperparameters, etc.;

[0062] The purpose of this step is to improve the performance and quality of the model to adapt to more complex and variable actual situations. According to the evaluation results, the advantages and disadvantages of the model, as well as possible problems and deficiencies, can be discovered. Based on this information, the model can be optimized and improved. For example, more data can be used to train the model to increase its generalization ability and robustness; hyperparameters such as the learning rate, batch size, optimizer, etc. can be adjusted to improve the convergence speed and effect of the model.

[0063] 6. Application and Deployment: Deploy the optimized model to medical devices to achieve real-time detection of vascular stenosis lesions. During the deployment process, it is necessary to ensure the stability, security, and reliability of the model and formulate corresponding management measures.

[0064] The purpose of this step is to apply the trained model to the actual scenario to assist doctors and patients in the diagnosis and treatment of vascular stenosis lesions. To deploy the model, the model needs to be converted into a format suitable for running on medical devices and connected and interacted with the devices. During the deployment process, it is necessary to ensure the stability, security, and reliability of the model, that is, the model can work properly in different environments and will not have errors or failures. In addition, corresponding management measures need to be formulated, such as monitoring the running status of the model, updating the version of the model, and handling abnormal situations.

[0065] A schematic diagram of using the method of the present invention to identify the DSA image data of blood vessels with vascular stenosis lesions is as Figure 4 shown, Figure 4 In the upper plate, the first column in the upper plate is the original angiogram image, the second column is the blood vessel image after segmentation processing, and the last column is the final detection result of the vascular stenosis. The stenotic part is marked by a red square frame. The lower plate shows more blood vessel detection results. After performing relevant algorithm processing on the original angiogram image, the present invention performs image noise reduction and blood vessel segmentation on the image, and finally locates the vascular stenosis, improving the accuracy of the final target detection through preprocessing.

[0066] Embodiment 2

[0067] A computer device, as Figure 5 shown, includes: at least one processor and a memory communicatively connected to at least one processor;

[0068] Wherein, the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, it implements the method for judging the position of intravascular stenosis based on the steepest descent method as described in Embodiment 1.

[0069] Embodiment 3

[0070] A computer-readable storage medium stores computer-readable instructions thereon. When the computer-readable instructions are executed by a processor, a method for judging the position of intravascular stenosis based on the steepest descent method as described in Embodiment 1 is implemented.

[0071] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0072] Those skilled in the art should understand that those skilled in the art can implement variations in combination with the prior art and the above embodiments, which will not be elaborated here. Such variations do not affect the essence of the present invention and will not be elaborated here.

[0073] The above describes the preferred embodiments of the present invention. It should be understood that the present invention is not limited to the above specific implementation manners. The devices and structures not described in detail should be understood to be implemented in a common manner in the art; any person skilled in the art can make many possible changes and modifications to the technical solution of the present invention, or modify it into an equivalent embodiment with equivalent changes, without departing from the scope of the technical solution of the present invention. This does not affect the essence of the present invention. Therefore, any simple modification, equivalent change, and modification made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A method for identifying vascular stenosis lesions based on object detection, characterized in that: After preprocessing the vascular DSA image data to be recognized and segmenting the blood vessels, the processed data is input into the lesion recognition model, which outputs the label types corresponding to each position in the vascular DSA image data to be recognized. The label types include normal and lesion. The lesion recognition model is an improved YOLO v7 algorithm model, which specifically includes: Input: Input a preprocessed vascular DSA image data to be recognized. Backbone: Consisting of 50 convolutional layers, normalization layers, activation functions, pooling layers, and ELAN modules, the input image is sent into the Backbone network to extract three feature maps C3, C4, and C5 with different scales and numbers of channels. Head: A PAFPN structure composed of an SPPCSP module, an ELAN-H module, a RepConv layer, and convolutional layers. The feature maps C3, C4, and C5 extracted by the Backbone are sent into the Head network for upsampling, downsampling, fusion, and prediction to obtain three prediction results P3’, P4’, and P5’ with different scales and numbers of channels. Specifically, perform an SPPCSP operation on C5, then fuse it with C4 and C3 from top to bottom to obtain three feature maps with different scales and numbers of channels, denoted as P3, P4, and P5 respectively. Then fuse them with P4 and P5 from bottom to top to obtain the fused feature maps P3’, P4’, and P5’, which contain the target category, confidence, and bounding box information of each grid cell. Assign a soft label between 0 and 1 to the prediction box according to the IoU between the prediction box and the ground truth box as the positive sample assignment strategy. During training, use Aux Head. Send the feature map C5 extracted by the Backbone into the Aux Head network to output a binary classification result. Aux Head consists of convolutional layers, normalization layers, activation functions, pooling layers, and fully connected layers. The convolutional layer and the normalization layer are merged, that is, the RepConv layer is used to replace the original convolutional layer and normalization layer. Output: Post-process the three prediction results P3’, P4’, and P5’ output by the Head, including confidence threshold filtering and class selection, to obtain the final detection results, including the category, confidence, and bounding box coordinates of the target. The training process of the improved YOLO v7 algorithm model is a process of continuously adjusting the model parameters with the image data in the training dataset as the input and the known label data corresponding to the image data as the theoretical output. The training data in the training dataset includes training data with the label type of normal in the image and training data with the label type of lesion in the image.

2. The method for identifying vascular stenosis lesions based on object detection according to claim 1, wherein The preprocessing includes resizing, normalization, denoising, and smoothing.

3. The method for identifying vascular stenosis lesions based on object detection according to claim 1, characterized in that, The training data in the training dataset is the data after preprocessing and blood vessel segmentation.

4. A method for identifying vascular stenosis lesions based on object detection according to claim 1, characterized in that, The blood vessel segmentation is completed by applying semantic segmentation.

5. A computer device, characterized in that: The computer device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, it implements the method for identifying vascular stenosis lesions based on object detection according to any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that: Computer-readable instructions are stored on the computer-readable storage medium, and when the computer-readable instructions are executed by the processor, it implements the method for identifying vascular stenosis lesions based on object detection according to any one of claims 1 to 4.

Images