DSA image segmentation method and device based on fine tuning MedSAM model

Abstract

The invention discloses a DSA image segmentation method and device based on a fine tuning MedSAM model and a medium, and the method comprises the steps: obtaining a DSA image data set in a PCI operation, and carrying out the preprocessing of the data set; the data set is subjected to coronary artery region labeling in advance; fine adjustment is carried out on the structure and the training strategy of the model, a convolution front-end module is added in front of a ViT encoder, and data enhancement training, course learning training and / or difficult case mining training are / is carried out according to the features of the DSA image; carrying out optimization design on a loss function and a feature extraction part of the model; and inputting the preprocessed DSA image data set into the fine-tuned and optimized model to realize key region segmentation of the DSA image. According to the method, targeted fine adjustment is carried out on the basis of the MedSAM model, so that the MedSAM model can better adapt to DSA image segmentation requirements in a PCI operation, and efficient, accurate and consistent coronary artery segmentation is realized.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, specifically to a digital subtraction angiography (DSA) image segmentation method based on a fine-tuned MedSAM model, which is applied in robot-assisted percutaneous coronary intervention (PCI). Background Technology

[0002] Percutaneous coronary intervention (PCI) is an important and widely used interventional treatment for patients with coronary artery disease. This technique relies on accurate visualization of the coronary artery anatomy to ensure the catheter can precisely reach the lesion site and deliver effective treatment. Digital subtraction angiography (DSA), a key imaging technique used in PCI to visualize the coronary arteries, removes background interference from bone and soft tissue, allowing the vascular structure to be clearly presented, thus aiding physicians in accurate diagnosis and treatment.

[0003] However, traditional DSA image segmentation typically relies on manual operation. This process is not only time-consuming and labor-intensive, but also susceptible to subjective influences due to its dependence on the operator's experience and skill level, leading to inconsistent segmentation results. This inconsistency can affect surgical accuracy, increase the risk of surgical complications, and adversely affect patient prognosis. Therefore, developing an efficient and automated DSA image segmentation method to improve surgical efficiency and reduce human error has become a crucial problem urgently needing to be solved in the field of medical image processing.

[0004] Against the backdrop of the rapid development of deep learning technology, the application of artificial intelligence (AI) in medical image processing has made significant progress. In recent years, an increasing number of studies have focused on how to utilize AI algorithms to achieve automatic segmentation of medical images, reducing the workload of manual segmentation and improving the consistency and accuracy of segmentation. The Segment Anything Model (SAM), as a general segmentation model, has demonstrated outstanding performance in various computer vision tasks. SAM is designed to provide efficient solutions for a wide range of image segmentation tasks through its general segmentation capabilities. This model can accurately segment target regions in an image by inputting different cue information (such as points, boxes, masks, etc.). Its strong generalization performance and flexible adaptability have led to its widespread attention and application in the field of computer vision.

[0005] Building upon the powerful capabilities of the SAM model, researchers have further developed the MedSAM model for the medical field, aiming to provide a more accurate solution for medical image segmentation tasks. The MedSAM model is derived from the SAM model through fine-tuning training on a large amount of medical image data. By incorporating a vast amount of medical image data, the MedSAM model has demonstrated excellent performance in handling complex medical image segmentation tasks, especially in processing common medical images such as CT (computed tomography) and MRI (magnetic resonance imaging), providing accurate segmentation results and achieving outstanding performance in multiple medical image segmentation tasks.

[0006] However, despite the significant progress made by the MedSAM model in medical image segmentation, its training data primarily focuses on common medical image types such as CT and MRI, and does not yet cover DSA images. DSA images possess unique characteristics, such as high-contrast vascular structures, complex backgrounds, and a low signal-to-noise ratio. These characteristics make DSA images more challenging to segment compared to other types of medical images. The characteristics of DSA images require the model not only to accurately identify and segment vascular structures but also to handle fine structures within complex backgrounds. Directly applying the existing MedSAM model to DSA image segmentation tasks cannot fully utilize its segmentation performance and fails to meet the needs of practical clinical applications.

[0007] Therefore, there is an urgent need for a method that optimizes the characteristics of DSA images and can achieve high-precision automatic segmentation in order to improve the efficiency and safety of PCI surgery. Summary of the Invention

[0008] This application provides a DSA image segmentation method based on a fine-tuned MedSAM model, addressing issues such as inconsistent segmentation results, low efficiency, and poor adaptability to DSA images in existing technologies. By specifically fine-tuning the MedSAM model, it better adapts to the DSA image segmentation requirements of PCI surgery, achieving efficient, accurate, and consistent coronary artery segmentation. This supports real-time decision-making in robot-assisted PCI surgery, improves the practical effectiveness of clinical applications, reduces surgical time, and minimizes surgical risks.

[0009] To address the aforementioned problems, in a first aspect, embodiments of this application provide a DSA image segmentation method based on a fine-tuned MedSAM model, the method comprising the following steps:

[0010] A DSA image dataset for PCI surgery was acquired and preprocessed; the DSA image dataset was pre-annotated with the coronary artery region.

[0011] The structure of the MedSAM model is fine-tuned by adding a convolutional front-end module consisting of several convolutional layers before the ViT encoder. The convolutional front-end module is used to process the input DSA image and extract the local features of the low layer of the DSA image.

[0012] Fine-tune the training strategy of the MedSAM model, and perform data augmentation training, course learning training and / or hard example mining training based on the features of DSA images;

[0013] The loss function of the MedSAM model is optimized by adopting a hybrid loss function that combines Dice loss and cross-entropy loss.

[0014] The feature extraction part of the MedSAM model is optimized by adopting a multi-scale feature fusion strategy to combine features at different levels in order to capture the local details and global structure of blood vessels in DSA images.

[0015] The preprocessed DSA image dataset is input into the fine-tuned and optimized MedSAM model to achieve key region segmentation of the DSA image.

[0016] Preferably, the data augmentation training includes at least one of the following:

[0017] Targeted enhancements: rotation, scaling, flipping, and enhancements at edges, textures, and / or corners tailored to the characteristics of DSA images;

[0018] Randomly adjust contrast / brightness: simulate the imaging effect under different contrast agent concentrations, X-ray doses, and equipment parameters;

[0019] Gaussian noise / salt-and-pepper noise: simulates the electronic and quantum noise inherent in DSA images;

[0020] Elastic deformation: Simulates the non-rigid deformation of blood vessels caused by slight patient movement or breathing.

[0021] Preferably, the course learning and training specifically involves: classifying the training data according to difficulty level, training the model in stages according to the image difficulty, starting from simple samples, gradually transitioning to complex samples, and gradually improving the model's adaptability to complex DSA images.

[0022] Preferably, the hard example mining training specifically involves: in each training batch, calculating the loss of all pixels, but not calculating the average of the losses of all pixels, instead focusing on the pixels with the highest loss values ​​and giving them higher weights, or using only the losses of these pixels for backpropagation, thereby forcing the model to concentrate on distinguishing the most easily confused coronary artery foreground and background.

[0023] Preferably, the expression for the hybrid loss function is as follows:

[0024] L total =α·L BCE +β·L Dice

[0025] Among them, L total Let L be a mixed loss function, where α and β are the cross-entropy losses, respectively. BCE And Dice lost L Dice The weighting coefficients.

[0026] More preferably, regularization is employed to add a penalty term to the mixture loss function. This penalty term is proportional to the sum of the squares of all weights in the model. The expression for the regularized mixture loss function is:

[0027] L reg =L total +λ∑w i 2

[0028] Among them, L reg Let λ be the regularized hybrid loss function, and ∑w be the regularization coefficient. i 2 This is the sum of squares of all weights in the model.

[0029] Preferably, the key region segmentation of the DSA image includes:

[0030] In a single image of coronary artery stenosis, the stenotic region is segmented.

[0031] In multiple images of arterial lesions, all key vascular regions were segmented and multiple lesion sites were marked;

[0032] The coronary artery structure was segmented from the DSA image containing noise interference.

[0033] Secondly, embodiments of this application provide a DSA image segmentation apparatus based on a fine-tuned MedSAM model, characterized in that it includes:

[0034] The data acquisition and preprocessing module is used to acquire the DSA image dataset during PCI surgery and to preprocess the DSA image dataset.

[0035] The structure fine-tuning module is used to fine-tune the structure of the MedSAM model. Before the ViT encoder, a convolutional front-end module consisting of several convolutional layers is added. The convolutional front-end module is used to process the input DSA image and extract the local features of the low layer of the DSA image.

[0036] The training strategy fine-tuning module is used to fine-tune the training strategy of the MedSAM model, and to perform data augmentation training, course learning training and / or hard example mining training based on the features of DSA images.

[0037] The loss function optimization design module is used to optimize the loss function of the MedSAM model, and adopts a hybrid loss function that combines Dice loss and cross-entropy loss.

[0038] The feature extraction optimization design module is used to optimize the feature extraction part of the MedSAM model. It adopts a multi-scale feature fusion strategy to combine features at different levels to capture the local details and global structure of blood vessels in DSA images.

[0039] The DSA image segmentation module is used to input the preprocessed DSA image dataset into the fine-tuned and optimized MedSAM model to achieve key region segmentation of the DSA image.

[0040] Thirdly, embodiments of this application provide a DSA image segmentation module based on a fine-tuned MedSAM model, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of the aforementioned DSA image segmentation method based on a fine-tuned MedSAM model.

[0041] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the DSA image segmentation method based on the fine-tuned MedSAM model described above.

[0042] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0043] 1. This embodiment provides a DSA image segmentation method based on a fine-tuned MedSAM model. The MedSAM model is specially fine-tuned and optimized to adapt to the specific needs of DSA image segmentation. It can better identify the coronary artery structure in DSA images and improve the stability and consistency of segmentation, thereby providing more reliable technical support for coronary artery visualization in PCI surgery.

[0044] 2. This application makes innovative adjustments to the model structure by adding a convolutional front-end module before the ViT encoder, which combines the local detail perception of the convolutional neural network (which is beneficial for capturing blood vessel edges) and the global relation modeling of ViT (which is beneficial for understanding blood vessel connectivity). This is an important enhancement to the original pure Transformer structure.

[0045] 3. This application employs data augmentation training for DSA images, which goes beyond conventional rotation and flipping. Instead, it introduces contrast jitter, noise injection, and elastic deformation to simulate real clinical imaging variations, greatly improving the robustness and generalization ability of the model.

[0046] 4. This application employs a combination of course learning and challenging case mining training. By training in a sequence from easy to difficult, the model is guided to converge stably. By focusing on samples that are difficult to segment, the model's weaknesses are actively addressed. These two strategies work together to systematically improve the model's performance on challenging cases.

[0047] 5. This application employs a hybrid loss function combining Dice loss and cross-entropy loss to specifically address the extreme class imbalance problem between foreground (blood vessels) and background in DSA images, and forces the model to focus on boundary pixels. Regularization techniques are also used to prevent overfitting.

[0048] 6. This application optimizes the enhanced decoder and multi-scale fusion, which fuses the high-resolution detail features of the shallow layer of the encoder with the strong semantic features of the deep layer in the decoder, specifically for the accurate reconstruction of small blood vessel branches and terminals.

[0049] 7. This application integrates an attention mechanism, enabling the model to learn to dynamically and adaptively focus on the blood vessel region, while suppressing the interference of background noise and artifacts, thus improving the segmentation accuracy in complex backgrounds. Attached Figure Description

[0050] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0051] Figure 1 This is a flowchart of the DSA image segmentation method based on the fine-tuned MedSAM model provided in Embodiment 1 of this application;

[0052] Figure 2 This is the original DSA image provided in Embodiment 1 of this application;

[0053] Figure 3 This is a segmentation effect diagram provided in Embodiment 1 of this application;

[0054] Figure 4 This is a structural block diagram of DSA image segmentation based on a fine-tuned MedSAM model provided in Embodiment 2 of this application;

[0055] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0056] This application provides a DSA image segmentation method based on a fine-tuned MedSAM model, addressing issues such as inconsistent segmentation results, low efficiency, and poor adaptability to DSA images in existing technologies. By specifically fine-tuning the MedSAM model, it better adapts to the DSA image segmentation requirements of PCI surgery, achieving efficient, accurate, and consistent coronary artery segmentation. This supports real-time decision-making in robot-assisted PCI surgery, improves the practical effectiveness of clinical applications, reduces surgical time, and minimizes surgical risks.

[0057] To better understand the above technical solutions, exemplary embodiments will be described in detail below, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses consistent with some aspects of this application as detailed in the appended claims.

[0058] Example 1

[0059] Figure 1 This is a flowchart of the DSA image segmentation method based on the fine-tuned MedSAM model provided in Embodiment 1 of this application. The DSA image segmentation method based on the fine-tuned MedSAM model includes the following steps:

[0060] Step S1: Obtain the DSA image dataset during PCI and preprocess the DSA image dataset.

[0061] First, a large dataset of DSA images used in percutaneous coronary intervention (PCI) needs to be collected. The dataset should include diverse images from different patients to cover a wide range of clinical scenarios. This data can be extracted from hospital imaging databases, ensuring the inclusion of patient data of different ages, genders, and lesion types to improve the model's generalization ability.

[0062] Then, after data collection is complete, the data is preprocessed. The preprocessing includes:

[0063] Standardization processing: used to eliminate differences between different imaging devices and ensure that the data input to the model is consistent.

[0064] Contrast enhancement and noise reduction: used to improve the clarity of vascular structures in images, making it easier for the model to learn and recognize them.

[0065] The coronary artery region is labeled: the labeling is done by professional doctors to ensure that the coronary artery region in each image is accurately labeled. These labels will be used as target data for the model during training.

[0066] Finally, the preprocessed dataset is divided into training, validation, and test sets. Typically, the training set comprises 70%-80% of the total dataset, the validation set 10%-15%, and the test set 10%-15%. The training set is used for model training, the validation set for model optimization and tuning, and the test set for final performance evaluation.

[0067] Step S2: Fine-tune the structure of the MedSAM model. Before the ViT encoder, add a convolutional front-end module consisting of several convolutional layers. The convolutional front-end module is used to process the input DSA image and extract the local features of the lower layer of the DSA image.

[0068] Specifically, the MedSAM model structure was fine-tuned to address the characteristics of "high-contrast blood vessels" and "complex background / noise" in DSA images. This was done from two perspectives: "enabling the model to see details clearly" and "enabling the model to eliminate interference."

[0069] The MedSAM model is based on the SAM model, and its core is the Vision Transformer encoder, which can be simplified as the ViT encoder. The standard ViT encoder is strong at handling global contextual relationships, but it is not as good as convolutional neural networks (CNNs) at capturing local, fine textures (such as the edges of small blood vessels). Therefore, adjustments can focus on enhancing the ability to extract local features.

[0070] Before the ViT encoder, a "CNN front-end" consisting of several convolutional layers is added, named the Convolutional Front-End Module. The Convolutional Front-End Module is responsible for processing the input DSA image, extracting low-level local features such as edges, textures, and corners. The Convolutional Front-End Module and the ViT encoder together form a hybrid encoder structure (CNN + Transformer).

[0071] Workflow: The DSA image is first passed through a convolutional front-end module to generate a set of low-resolution feature maps rich in local details. These feature maps are then "flattened" and fed as sequence input into the subsequent ViT encoder.

[0072] Advantages: This method combines the advantages of CNNs in local feature extraction with the advantages of Transformers in global dependency modeling. The CNN part can effectively capture the clear boundaries of blood vessels, while the Transformer part can understand the overall trend and connectivity of blood vessels, thus better distinguishing intersecting blood vessels from background artifacts.

[0073] Step S3: Fine-tune the training strategy of the MedSAM model, and perform data augmentation training, course learning training and / or hard example mining training based on the features of DSA images.

[0074] Specifically,

[0075] (1) Data Augmentation Training

[0076] Data augmentation training, specifically including:

[0077] Targeted enhancement: In addition to conventional rotation, scaling, and flipping, enhancements should be made to target the characteristics of DSA images, such as enhancements at edges, textures, and corners;

[0078] Randomly adjust contrast / brightness: simulate the imaging effect under different contrast agent concentrations, X-ray doses, and equipment parameters;

[0079] Gaussian noise / salt-and-pepper noise: simulates the electronic and quantum noise inherent in DSA images;

[0080] Elastic deformation: Simulates the non-rigid deformation of blood vessels caused by slight patient movement or breathing;

[0081] Data augmentation training expands the training data, enabling the model to learn various possible images. This allows it to maintain good segmentation performance when faced with DSA images from different angles, scales, contrasts / brightnesses, noise levels, and deformations, thus enhancing the model's robustness.

[0082] (2) Course Learning and Training

[0083] This is a training strategy that simulates the human learning process. Training starts with "simple" samples and gradually transitions to "complex" samples. The training data is graded by difficulty; for example, images with high signal-to-noise ratio, large blood vessels, and simple structures are classified as "simple," while images with low signal-to-noise ratio, small blood vessels, and complex branches are classified as "difficult."

[0084] In the initial training phase, the model is trained using only simple samples, allowing it to quickly learn to recognize basic blood vessel morphology. As training progresses, the proportion of difficult samples is gradually increased, forcing the model to learn more challenging features.

[0085] Learning and training through courses helps the model converge stably and avoids being "led astray" by a large amount of noise and complex samples in the early stages, thus learning more robust feature representations.

[0086] (2) Difficult Example Discovery Training

[0087] The background area of ​​a DSA image is much larger than the vascular area, and the background may contain structures that are easily confused with blood vessels, such as the edge of the spine and duct shadows (i.e., "difficult to distinguish negative examples").

[0088] In each training batch, the loss for all pixels is calculated. Instead of averaging the loss of all pixels, the focus is on the pixels with the highest loss values ​​(i.e., the pixels the model is most likely to mispredict), and these are given higher weights, or the loss of only these pixels is used for backpropagation.

[0089] By training with hard examples, the model is forced to focus on distinguishing the most easily confused foreground (blood vessels) from the background, thereby significantly improving the accuracy of segmentation.

[0090] Step S4: Optimize the loss function of the MedSAM model by using a hybrid loss function that combines Dice loss and cross-entropy loss.

[0091] To enhance the model's accuracy in handling coronary artery details, this invention employs a hybrid loss function combining cross-entropy loss and Dice loss. Dice loss helps improve the model's sensitivity to small targets and imbalanced data, while cross-entropy loss optimizes the model's overall classification ability. By combining these two loss functions, the model can better handle the fine structure of blood vessels and complex backgrounds when segmenting coronary arteries.

[0092] The expression for the hybrid loss function is as follows:

[0093] L total =α·L BCE +β·L Dice

[0094] Where α and β are the cross-entropy loss L BCE And Dice lost L Dice The weighting coefficients.

[0095] The specific process is as follows:

[0096] Forward propagation: A DSA image is input into the MedSAM model after fine-tuning the structure and training strategy. The model outputs a probability map of the same size as the original image, where the value of each pixel represents the probability that it belongs to a blood vessel.

[0097] Loss calculation: Using the probabilistic graph and the corresponding true segmentation mask, calculate the cross-entropy loss and Dice loss. Calculate the total loss based on preset α and β values.

[0098] Backpropagation: Calculates the gradient of the total loss with respect to all trainable parameters (weights and biases) of the model.

[0099] Parameter update: Use optimizers (such as Adam, SGD, etc.) to update the model parameters based on the calculated gradients.

[0100] This process is repeated on the entire training set until the model's performance on the validation set no longer improves, at which point the optimization process ends.

[0101] Furthermore, to avoid model overfitting, regularization techniques, such as L2 regularization, are employed. By introducing a weight decay term into the loss function, the model is suppressed from overfitting the training data, thereby improving its generalization ability and enabling it to maintain stable performance in different clinical scenarios.

[0102] The core idea of ​​L2 regularization is to add a penalty term to the loss function, which is proportional to the sum of the squares of all model weights. The expression for the regularized mixed loss function is:

[0103] L reg =L total +λ∑w i 2

[0104] Among them, L reg Let λ be the regularized hybrid loss function, and ∑w be the regularization coefficient. i 2 This is the sum of squares of all weights in the model.

[0105] Intuitively, it aims to prevent any weight in the model from becoming too large. An excessively large weight means the model is overly reliant on a particular input feature, which is a sign of overfitting. By penalizing large weights, L2 regularization forces the model to learn a "simpler" model with a more evenly distributed weight distribution, which typically has better generalization ability.

[0106] When training the fine-tuned MedSAM model, L2 regularization is directly integrated into the loss function. It applies to all trainable parameters of the model, including all weight matrices in the convolutional front-end module, the ViT encoder, and the decoder.

[0107] Step S5: Optimize the feature extraction part of the MedSAM model by adopting a multi-scale feature fusion strategy to combine features at different levels in order to capture the local details and global structure of blood vessels in DSA images.

[0108] To better capture coronary artery details in DSA images, the feature extraction part of the model was optimized. Specifically, a multi-scale feature fusion strategy was adopted to combine features from different levels to capture both local details and global structure of the blood vessels. This multi-scale feature fusion method improves the model's ability to recognize vascular structures of varying sizes, enabling it to accurately segment coronary arteries in complex backgrounds.

[0109] Multi-scale feature fusion is based on the idea of ​​utilizing feature maps from different depths of the network, as they contain information at different scales.

[0110] Shallow features: These come from early layers of the encoder. These feature maps are high-resolution and preserve a wealth of spatial detail, such as edges, texture, and color. For segmentation tasks, they are crucial for accurately locating the boundaries of targets. In a DSA image, this is the edge of a blood vessel.

[0111] Deep features: These come from the final layers of the encoder. These feature maps have low resolution but have undergone multiple non-linear transformations, containing rich semantic information. They help the model understand the context of an image, such as "this area is the main coronary artery" rather than "this is a bright line".

[0112] The DSA vessel segmentation task requires both types of information: accurately depicting the outlines of small branches (requiring shallow features) and identifying the complete vascular tree structure as a whole while eliminating background interference (requiring deep features).

[0113] A multi-scale feature fusion strategy is employed, using skip cut connections within the UNet architecture. This idea is then incorporated into the decoder part of the MedSAM fine-tuning process. The model has an encoder-decoder structure:

[0114] Encoder: A series of downsampling modules (such as convolution + pooling) that progressively extract features and reduce resolution.

[0115] Decoder: A series of upsampling modules (such as deconvolution or interpolation + convolution) gradually restore the resolution and generate a segmentation map.

[0116] Fusion process: The first module of the decoder upsamples the features from the deepest layer of the encoder.

[0117] Fusion Point 1: Before the upsampled feature map is fed into the next module of the decoder, it is concatenated with the feature map from the encoder at the corresponding scale. This concatenation operation is called a "skip connection". The second module of the decoder processes this fused feature map and upsamples it again.

[0118] Fusion point 2: Similarly, the upsampled result is concatenated with the features of the corresponding scale of the encoder.

[0119] This process continues until the decoder outputs the final segmentation map.

[0120] In this way, the decoder can directly obtain the original, high-frequency detail information extracted by the encoder at the same resolution when reconstructing the image at each layer. This greatly alleviates the problem of spatial information loss during downsampling, enabling the final segmentation mask to have very fine boundaries, which is crucial for accurately segmenting the tiny branches and tips of the coronary arteries.

[0121] Furthermore, attention mechanisms can be introduced to enhance the model's focus on important regions (such as blood vessels) and reduce interference from background noise. In this way, the model can process the coronary artery region more effectively, thereby improving the accuracy and consistency of segmentation.

[0122] Attention mechanisms mimic the human visual system, allowing the model to dynamically and adaptively focus on the most important parts of the input information while ignoring irrelevant parts. In DSA image segmentation, this means concentrating the model's "computing power" on analyzing vascular regions, while suppressing the effects of background noise and artifacts.

[0123] While MedSAM's ViT encoder is built on self-attention, we can further optimize it by inserting additional, more explicit attention modules elsewhere in the model. A very popular and effective module is the Convolutional Block Attention Module (CBAM). CBAM consists of two cascaded sub-modules: a channel attention module and a spatial attention module. It can be seamlessly integrated after any convolutional network layer.

[0124] The channel attention module learns the importance of different feature channels. In DSA image processing, some channels may be particularly sensitive to detecting high-brightness contrast agents, while others may be more sensitive to edge information. Channel attention can dynamically increase the weight of "vessel-related" channels and decrease the weight of "background noise-related" channels.

[0125] The spatial attention module learns the importance of different spatial locations on the feature map. It generates a 2D "attention map," where highlighted areas correspond to locations the model should focus on (i.e., the areas where blood vessels are located).

[0126] The CBAM module can be inserted after each convolutional block of the encoder, or after each upsampling block of the decoder. In this way, the model is "recalibrated" at each step of feature propagation, dynamically enhancing the response to vascular features and suppressing background noise, resulting in a cleaner and more accurate final segmentation.

[0127] Step S6: Input the preprocessed DSA image dataset into the fine-tuned and optimized MedSAM model to achieve key region segmentation of the DSA image.

[0128] Specifically, based on the aforementioned fine-tuning and optimization strategies, the preprocessed DSA image dataset is input into the fine-tuned and optimized MedSAM model for fine-tuning training. Fine-tuning training typically employs a low learning rate to prevent over-updating of model weights, which could destroy useful features acquired during pre-training. During training, the model is periodically evaluated on a validation set to monitor its performance, and training parameters (such as learning rate and regularization coefficients) are adjusted based on the evaluation results.

[0129] The goal of training is to enable the model to achieve the best segmentation results on DSA images while maintaining a certain level of generalization ability, so that it can handle DSA images from different patients, different devices, and different operating conditions.

[0130] The DSA image segmentation method based on the fine-tuned MedSAM model provided in this embodiment can be applied to the following different DSA image segmentation cases.

[0131] Example 1: Single artery stenosis

[0132] In a patient case with a single coronary artery stenosis.

[0133] The loss function uses a hybrid loss with α = 0.6 and β = 0.4; a 3-layer convolutional front-end module is added before the ViT encoder, and a CBAM module is inserted after each encoder block; training is carried out in stages, gradually increasing the proportion of noise and complex samples, and the Adam optimizer is used with an L2 coefficient λ = 0.01.

[0134] The model successfully identified and segmented the lesion area. The automatic segmentation results highly matched the doctor's manual segmentation annotations, with a Dice similarity coefficient exceeding 0.95. In a comparative experiment, the method of this invention was compared with traditional methods.

[0135]

[0136] As shown in the table above, the method of this invention can accurately detect and segment narrow regions, and its segmentation performance is significantly superior to other traditional methods. It boasts higher accuracy and efficiency, as well as stronger robustness. This result provides doctors with precise references to assist in developing further treatment plans.

[0137] Example 2: Segmentation of complex arterial lesions

[0138] In a complex case with multiple arterial lesions.

[0139] The loss function uses a hybrid loss with α = 0.7 and β = 0.3. A four-layer convolutional front-end module is added before the ViT encoder. Data augmentation is used to expand the training data. In the early stages of training, only simple samples are used to train the model, and the proportion of difficult samples is gradually increased as training progresses. A multi-scale feature fusion strategy is adopted to combine features from different levels. The L2 coefficient λ = 0.02.

[0140] The model successfully segmented all key vascular regions and labeled multiple lesion sites. Figure 2 The image shown is the original DSA image. Figure 3 The image shown is a segmentation result of the DSA image segmentation method based on the fine-tuned MedSAM model provided in this embodiment. Although there are slight differences between the automatic segmentation result and the manual annotation (mainly at very small blood vessel branches), the overall performance is still satisfactory. With further fine-tuning by the physician, the final segmentation result was confirmed and used to guide surgical procedures.

[0141] Example 3: Segmentation under Noise Interference

[0142] In noisy DSA images, the model successfully suppressed the influence of background noise and accurately segmented the coronary artery structures through its multi-scale feature fusion and attention mechanism. Compared with traditional manual segmentation, the model's results are more consistent and significantly reduce the workload of physicians.

[0143] Example 2

[0144] Based on the same inventive concept as the DSA image segmentation method based on the fine-tuned MedSAM model in Embodiment 1 above, this embodiment also provides a DSA image segmentation apparatus based on the fine-tuned MedSAM model, such as... Figure 4 As shown, the device includes:

[0145] The data acquisition and preprocessing module 10 is used to acquire the DSA image dataset during PCI surgery and to preprocess the DSA image dataset.

[0146] The structure fine-tuning module 20 is used to fine-tune the structure of the MedSAM model. Before the ViT encoder, a convolutional front-end module consisting of several convolutional layers is added. The convolutional front-end module is used to process the input DSA image and extract the local features of the lower layer of the DSA image.

[0147] The training strategy fine-tuning module 30 is used to fine-tune the training strategy of the MedSAM model, and to perform data augmentation training, course learning training and / or hard example mining training based on the features of DSA images.

[0148] The loss function optimization design module 40 is used to optimize the loss function of the MedSAM model, and adopts a hybrid loss function that combines Dice loss and cross-entropy loss.

[0149] The feature extraction optimization design module 50 is used to optimize the feature extraction part of the MedSAM model. It adopts a multi-scale feature fusion strategy to combine features at different levels to capture the local details and global structure of blood vessels in DSA images.

[0150] The DSA image segmentation module 60 is used to input the preprocessed DSA image dataset into the fine-tuned and optimized MedSAM model to achieve key region segmentation of the DSA image.

[0151] The various specific processes and examples of the DSA image segmentation method based on the fine-tuned MedSAM model in the aforementioned Embodiment 1 are also applicable to the DSA image segmentation device based on the fine-tuned MedSAM model in this embodiment. Through the detailed description of Embodiment 1, those skilled in the art can clearly understand the implementation method of the DSA image segmentation device based on the fine-tuned MedSAM model in this embodiment. Therefore, for the sake of brevity, it will not be described in detail here.

[0152] Example 3

[0153] Based on the same inventive concept as the DSA image segmentation method based on the fine-tuned MedSAM model in Embodiment 1 above, this embodiment also provides an automatic segmentation module for DSA image and video sequences, which stores a computer program. When the program is executed by a processor, it implements the steps of the DSA image segmentation method based on the fine-tuned MedSAM model described in Embodiment 1.

[0154] Example 4

[0155] Based on the same inventive concept as the DSA image segmentation method based on the fine-tuned MedSAM model in Embodiment 1 above, this embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the DSA image segmentation method based on the fine-tuned MedSAM model described in Embodiment 1.

[0156] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0157] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0158] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0159] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0160] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0161] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A DSA image segmentation method based on a fine-tuned MedSAM model, the method comprising the following steps: A DSA image dataset for PCI surgery was acquired and preprocessed; the DSA image dataset was pre-annotated with the coronary artery region. The structure of the MedSAM model is fine-tuned by adding a convolutional front-end module consisting of several convolutional layers before the ViT encoder. The convolutional front-end module is used to process the input DSA image and extract the local features of the low layer of the DSA image. Fine-tune the training strategy of the MedSAM model, and perform data augmentation training, course learning training and / or hard example mining training based on the features of DSA images; The loss function of the MedSAM model is optimized by adopting a hybrid loss function that combines Dice loss and cross-entropy loss. The feature extraction part of the MedSAM model is optimized by adopting a multi-scale feature fusion strategy to combine features at different levels in order to capture the local details and global structure of blood vessels in DSA images. The preprocessed DSA image dataset is input into the fine-tuned and optimized MedSAM model to achieve key region segmentation of the DSA image.

2. The DSA image segmentation method based on the fine-tuned MedSAM model as described in claim 1, characterized in that, The data augmentation training includes at least one of the following: Targeted enhancements: rotation, scaling, flipping, and enhancements at edges, textures, and / or corners tailored to the characteristics of DSA images; Randomly adjust contrast / brightness: simulate the imaging effect under different contrast agent concentrations, X-ray doses, and equipment parameters; Gaussian noise / salt-and-pepper noise: simulates the electronic and quantum noise inherent in DSA images; Elastic deformation: Simulates the non-rigid deformation of blood vessels caused by slight patient movement or breathing.

3. The DSA image segmentation method based on the fine-tuned MedSAM model as described in claim 1, characterized in that, The course learning and training specifically involves: classifying the training data according to difficulty level, training the model in stages according to the image difficulty, starting with simple samples, gradually transitioning to complex samples, and gradually improving the model's adaptability to complex DSA images.

4. The DSA image segmentation method based on the fine-tuned MedSAM model as described in claim 1, characterized in that, The hard example mining training specifically involves: in each training batch, calculating the loss of all pixels, but not calculating the average of the losses of all pixels. Instead, focusing on the pixels with the highest loss values ​​and giving them higher weights, or using only the losses of these pixels for backpropagation, thereby forcing the model to concentrate on distinguishing the most easily confused coronary artery foreground and background.

5. The DSA image segmentation method based on the fine-tuned MedSAM model as described in claim 1, characterized in that, The expression for the hybrid loss function is as follows: L total =α·L BCE +β·L Dice Among them, L total Let L be a mixed loss function, where α and β are the cross-entropy losses, respectively. BCE And Dice lost L Dice The weighting coefficients.

6. The DSA image segmentation method based on the fine-tuned MedSAM model as described in claim 5, characterized in that, Regularization is used to add a penalty term to the mixture loss function. This penalty term is proportional to the sum of the squares of all weights in the model. The expression for the regularized mixture loss function is: L reg =L total +λ∑w i 2 Among them, L reg Let λ be the regularized hybrid loss function, and ∑w be the regularization coefficient. i 2 This is the sum of squares of all weights in the model.

7. The DSA image segmentation method based on the fine-tuned MedSAM model as described in claim 1, characterized in that, The key region segmentation of the DSA image includes: In a single image of coronary artery stenosis, the stenotic region is segmented. In multiple images of arterial lesions, all key vascular regions were segmented and multiple lesion sites were marked; The coronary artery structure was segmented from the DSA image containing noise interference.

8. A DSA image segmentation device based on a fine-tuned MedSAM model, characterized in that, include: The data acquisition and preprocessing module is used to acquire the DSA image dataset during PCI surgery and to preprocess the DSA image dataset. The structure fine-tuning module is used to fine-tune the structure of the MedSAM model. Before the ViT encoder, a convolutional front-end module consisting of several convolutional layers is added. The convolutional front-end module is used to process the input DSA image and extract the local features of the low layer of the DSA image. The training strategy fine-tuning module is used to fine-tune the training strategy of the MedSAM model, and to perform data augmentation training, course learning training and / or hard example mining training based on the features of DSA images. The loss function optimization design module is used to optimize the loss function of the MedSAM model, and adopts a hybrid loss function that combines Dice loss and cross-entropy loss. The feature extraction optimization design module is used to optimize the feature extraction part of the MedSAM model. It adopts a multi-scale feature fusion strategy to combine features at different levels to capture the local details and global structure of blood vessels in DSA images. The DSA image segmentation module is used to input the preprocessed DSA image dataset into the fine-tuned and optimized MedSAM model to achieve key region segmentation of the DSA image.

9. A DSA image segmentation module based on a fine-tuned MedSAM model, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the DSA image segmentation method based on the fine-tuned MedSAM model as described in any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the DSA image segmentation method based on the fine-tuned MedSAM model as described in any one of claims 1 to 6.

Images