An intravascular ultrasound stent and lumen collaborative segmentation system based on a CNN-Transformer hybrid model

Abstract

The application discloses a kind of intravascular ultrasound stent and lumen collaborative segmentation systems based on CNN-Transformer hybrid model, belong to medical image processing and artificial intelligence technical field.The system of the present application includes data preprocessing module, double-branch encoder module, multi-scale context enhancement module, cross-modal feature fusion module, decoder module, topological consistency constraint module, clinical index calculation module and front-end interaction module.The present application effectively solves the key technical problems such as poor anatomical adaptability, double-target collaborative difficulty and low clinical evaluation efficiency in IVUS image intelligent segmentation, and provides a fully automatic, high-precision intelligent solution for stent evaluation after coronary intervention.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing and artificial intelligence, specifically to an intravascular ultrasound (IVUS) stent and lumen co-segmentation system based on a CNN-Transformer hybrid model, and a method for quantitative evaluation of stent clinical indicators based on the segmentation results. Background Technology

[0002] Coronary atherosclerotic heart disease is one of the most important public health problems worldwide. The development of percutaneous coronary intervention (PCI) has made stent implantation a key method for treating coronary artery stenosis. However, complications such as insufficient stent expansion, poor apposition, and restenosis significantly impact long-term clinical outcomes. Intravascular ultrasound (IVUS) provides high-resolution cross-sectional images of blood vessels, clearly displaying the vessel wall and internal structures, and has become an important imaging tool for guiding stent implantation and post-operative evaluation. Studies have shown that IVUS-guided PCI significantly reduces adverse events such as cardiac mortality, myocardial infarction, and stent thrombosis compared to angiography-guided PCI alone.

[0003] The most similar papers are:

[0004] Hao Wenyue, Cai Huaiyu, et al. A segmentation method for intravascular ultrasound images combining Transformer branching and topology forcing [J]. Progress in Laser & Optoelectronics, 2024;

[0005] Yang Xin, Li Yang. An automatic segmentation algorithm for IVUS lumen and external elastic membrane based on improved U-Net [J]. Chinese Journal of Biomedical Engineering, 2023;

[0006] LiuH,etal.Cascadedlearninginintravascularultrasound:coronarystentdelineationinmanualpullbacks[J].ComputerMethodsandProgramsinBiomedicine,2022;

[0007] ZhangM,etal.POST-IVUS:Aperceptualorganisation-awareselectivetransformerframeworkforintravascularultrasoundsegmentation[J].IEEETransactionsonMedicalImaging,2023.

[0008] The closest patents are:

[0009] Patent No. CN116206023A: An automatic segmentation system for intravascular ultrasound lumen based on an improved U-Net;

[0010] Patent No. CN117199487A: An IVUS patch intelligent segmentation device based on Transformer.

[0011] Currently, research and technical solutions related to IVUS image intelligent segmentation can be mainly divided into four categories: traditional image processing, pure convolutional neural networks, CNN-Transformer hybrid models, and dedicated scaffold detection algorithms. Representative technologies are as follows:

[0012] IVUS segmentation based on traditional image processing: The lumen contour is extracted by relying on manually designed gradients and texture features through algorithms such as edge detection, active contour model, level set, and graph cut.

[0013] IVUS segmentation and recognition based on pure CNN: U-Net, AttentionU-Net and other networks are used to extract multi-scale local features to achieve segmentation of lumen and adventitia of blood vessels, and skip connections are used to improve the segmentation effect of details.

[0014] CNN-Transformer hybrid architecture segmentation: Models such as TransUNet, Swin-UNet, and TransFuse introduce self-attention mechanisms to model global information for segmentation tasks of polyps, blood vessels, and skin lesions.

[0015] Machine learning-based support structure detection scheme: cascaded convolutional networks and traditional machine learning features are used to complete support beam detection and contour fitting.

[0016] Geometrically constrained medical segmentation models: Networks such as Geo-UNet add geometric constraints to single-connected regions to optimize the continuity of the lumen profile.

[0017] Closest existing technology:

[0018] The closest existing technologies to this invention are the TransFuse parallel CNN-Transformer hybrid segmentation architecture, the CSWinTransformer visual basic model, the IVUS lumen segmentation algorithm with geometric constraints, and the deep learning framework for independent scaffold detection.

[0019] Hybrid architecture segmentation scheme: Only a single-branch multi-scale enhancement module is built, the attention mechanism is not optimized for IVUS ring structure, there are no anatomical topology constraints, and the stent over-boundary segmentation results are prone to occur.

[0020] The general IVUS single-target segmentation system can only segment the lumen or plaque separately, and cannot simultaneously complete the coordinated segmentation of stents and lumens, and lacks the function of quantitative assessment of stent morphology.

[0021] In summary, existing technologies generally suffer from problems such as models not being adapted to the circumferential and radial anatomical structures of IVUS, lack of stent-lumen topological constraints, lack of multi-branch synchronous multi-scale enhancement, and separation between segmentation and clinical quantitative assessment. There is a lack of an integrated system that adapts to IVUS imaging features, incorporates anatomical priors, and automatically completes stent-lumen collaborative segmentation and outputs clinical indicators.

[0022] Traditional image processing methods have significant drawbacks:

[0023] Poor robustness and weak generalization ability: It is highly dependent on manually designed features, and the segmentation boundary is easily broken when faced with IVUS speckle noise, ring artifacts, and calcified acoustic shadows, requiring a lot of manual correction.

[0024] Limited feature extraction capability: It can only capture local gray-level gradients, cannot model long-range continuous vascular structures, and has extremely low accuracy in recognizing small stent beams.

[0025] For pure CNN segmentation schemes, there are obvious performance shortcomings:

[0026] Insufficient global modeling capability: The convolutional receptive field has inherent limitations, making it difficult to capture retracted IVUS sequences, the overall distribution of circular vessels, and large segmentation errors in the overall stent contour.

[0027] Insufficient multi-scale feature extraction: relying solely on skip connections to fuse multi-resolution features, lacking hollow spatial pyramid pooling to expand the receptive field, and resulting in severe loss of micro-scaffold details.

[0028] Existing CNN-Transformer hybrid models suffer from architectural adaptation flaws:

[0029] Attention mechanisms do not match IVUS imaging features: many use general Swing and ViT attention, but do not model the ring structure in both the angular and radial directions of IVUS, resulting in low efficiency in global feature extraction.

[0030] No dual-branch synchronous enhancement design: If the ASPP module is set only at the back end of a single branch, the local details of the CNN and the global information of the Transformer cannot be synchronously enhanced at multiple scales.

[0031] Lack of anatomical prior constraints: The segmentation channels of the lumen and stent are predicted independently, without introducing the physiological rule that "the stent is completely inside the lumen", resulting in a large number of segmentation masks that violate clinical logic.

[0032] Existing stent detection and single-target IVUS segmentation equipment have functional deficiencies:

[0033] The stent and lumen were not integrated into a single segmentation process: the stent was identified or the lumen was segmented separately, but the contours of the two types of targets could not be obtained simultaneously, and the stent could not be directly used for a comprehensive evaluation of stent adhesion and expansion effects.

[0034] There is no automated clinical quantitative calculation process: After the segmentation is completed, key parameters such as stent ellipticity, stent area, and stent-lumen area ratio need to be manually calculated by doctors, which is labor-intensive and has poor measurement consistency.

[0035] Lack of standardized IVUS preprocessing links: Incompatible with proprietary image formats from multiple vendors, lack of standardized wavelet denoising and contrast enhancement processes, resulting in large fluctuations in image segmentation effects from different devices.

[0036] The system lacks a complete, deployed inference system: it only has offline model code and lacks lightweight front-end and back-end interaction, batch image sequence processing, and result export visualization functions, resulting in high deployment costs and high barriers to entry for hospitals.

[0037] In summary, existing technologies generally suffer from problems such as mismatch between model structure and IVUS circular anatomy, lack of topological anatomical constraints, insufficient multi-scale feature extraction, inability to coordinate stent and lumen segmentation, lack of fully automated clinical quantitative assessment, and lack of standardized image preprocessing and lightweight deployment solutions.

[0038] Currently, there is a lack of an integrated IVUS intelligent segmentation system based on the ResNet50-CSWinTransformer dual-branch architecture, dual-branch ASPP multi-scale enhancement, embedded topological consistency loss, capable of simultaneously segmenting stents and lumens and automatically calculating clinical indicators. Summary of the Invention

[0039] The technical objective of this invention is to solve the problems of insufficient global modeling of pure CNN, the incompatibility of general Transformer with IVUS angular / radial annular anatomical structures, low segmentation accuracy of scaffolds and lumens, and poor contour fitting.

[0040] To address the shortcomings of existing hybrid models, such as single-branch multi-scale enhancement, loss of fine stent beams and vessel edge features, and excessively large Hausdorff distance;

[0041] It addresses the problems of traditional models lacking anatomical prior constraints, having stent prediction areas that exceed the lumen, and segmentation results that do not conform to common knowledge of coronary anatomy, making them unusable in clinical practice;

[0042] To address the issues of fragmentation between existing IVUS analysis tools and clinical assessment, as well as the problems of manual calculation of stent morphology and expansion-related indicators, poor repeatability, and low efficiency;

[0043] This addresses the issues of diverse IVUS image formats from multiple vendors, severe noise interference, lack of a standardized wavelet preprocessing pipeline, and weak model generalization ability.

[0044] We provide a complete deployment solution that combines a lightweight Flask backend inference service with a frontend visual interaction, supporting batch IVUS sequence processing and reducing implementation costs for primary hospitals and radiology departments.

[0045] The technical solution adopted in this invention is an intravascular ultrasound stent and lumen collaborative segmentation system based on a CNN-Transformer hybrid model, comprising:

[0046] The data preprocessing module is located at the system input port and is configured to perform standardized preprocessing on the raw intravascular ultrasound images and output the standardized grayscale images generated after preprocessing to the downstream dual-branch encoder module.

[0047] The data preprocessing module includes:

[0048] The medical format parsing unit is used to read intravascular ultrasound images in DICOM standard format, extract metadata such as transducer model, retraction speed, and lesion location, and supports batch conversion of various vendors' proprietary ultrasound image formats into standard PNG or JPG grayscale images, preserving the original grayscale information throughout the process;

[0049] The resolution normalization unit is used to uniformly crop and scale all input images to a standard size.

[0050] The adaptive image noise reduction and enhancement unit uses wavelet transform algorithm to suppress speckle noise, ring artifacts and calcification sound shadow interference, and combines gamma correction and local contrast equalization to eliminate uneven device brightness and central dark area interference, and outputs a standardized grayscale image.

[0051] A dual-branch encoder module, whose input end is connected to the output end of the data preprocessing module, is used to receive the standardized grayscale image. The dual-branch encoder module is configured with two parallel feature extraction backbones, including:

[0052] The ResNet50 convolutional neural network branch is configured to extract short-range local features of local texture of blood vessel wall, fine-grained edges of stent beams and gray-scale details of plaque, and output the extracted first-stage feature map to the subsequent multi-scale context enhancement module.

[0053] The CSWinTransformer branch employs a cross-window self-attention mechanism to calculate global self-attention along the angular and radial dimensions of the intravascular ultrasound image. It models the long-range dependencies of the vascular ring continuity and the overall spatial distribution of the stent, and outputs the extracted first-stage feature map in parallel to the subsequent multi-scale context enhancement module.

[0054] A multi-scale context enhancement module is independently embedded at the end of the ResNet50 convolutional neural network branch and the end of the CSWinTransformer branch, respectively, to receive the first-stage feature maps output by each of the two branches. Each multi-scale context enhancement module is an ASPP module, namely a dilated spatial pyramid pooling module. The ASPP module contains five parallel feature extraction branches, namely a 1×1 basic convolution branch, a first dilated convolution branch with different dilation rates, a second dilated convolution branch, a third dilated convolution branch, and a global average pooling branch. After the outputs of each branch are concatenated and fused in the channel dimension, the number of channels is unified by 1×1 convolution to expand the receptive field without losing the spatial resolution of the image, capture multi-scale vascular and stent features, and output the second-stage feature maps enhanced by each of the two branches to the cross-modal feature fusion module respectively.

[0055] The cross-modal feature fusion module has its input terminals connected to the output terminals of the multi-scale context enhancement module embedded at the end of the ResNet50 convolutional neural network branch and the multi-scale context enhancement module embedded at the end of the CSWinTransformer branch, respectively. It is used to simultaneously receive the second-stage feature maps of the two branches after enhancement. The cross-modal feature fusion module is configured to fuse the spatial attention features output by the ResNet50 convolutional neural network branch and the channel attention features output by the CSWinTransformer branch, and introduce a bilinear interaction unit to complete the deep complementary fusion of local details and global ring structure features. The unified fused feature map generated after fusion is output to the decoder module.

[0056] The decoder module, whose input is connected to the output of the cross-modal feature fusion module, is used to receive the unified fused feature map. The decoder module is configured as a hierarchical progressive upsampling decoding structure, with attention-enhanced skip connections, and is aligned with the cross-resolution semantic features of each level of the dual-branch encoder module. After restoring the spatial resolution through progressive upsampling, it finally outputs a dual-channel segmentation probability mask, where the first channel corresponds to the lumen region and the second channel corresponds to the stent region. The dual-channel segmentation probability mask is then output to the topology consistency constraint module and the clinical indicator calculation module, respectively.

[0057] The topology consistency constraint module, whose input is connected to the output of the decoder module, is used to receive the dual-channel segmentation probability mask. The topology consistency constraint module is embedded in the loss calculation link of the model training stage and is configured to transform the clinical anatomical prior that the stent is completely inside the lumen into a differentiable topology loss function. During the training stage, it penalizes the prediction results of the stent pixels falling outside the lumen and outputs the final segmentation mask optimized by topology constraints to the clinical index calculation module.

[0058] The clinical indicator calculation module has its input terminals connected to the output terminals of the decoder module and the topology consistency constraint module, respectively. It receives the binary segmentation mask for the lumen and the binary segmentation mask for the stent, as well as the final segmentation mask optimized by topology constraints, output by the decoder. It automatically calculates core clinical quantitative indicators after PCI. These clinical indicators include: stent ellipticity quantified by the ratio of the minor axis length to the major axis length through minimum circumscribed ellipse fitting of the stent contour; stent area calculated based on the product of the effective stent pixel count and the area of ​​a single pixel; and stent-lumen area ratio calculated as the ratio of the total stent area to the total lumen area.

[0059] The front-end and back-end interaction module, whose input end is connected to the output end of the clinical indicator calculation module, is used to receive segmentation results and clinical quantitative indicators. The front-end and back-end interaction module includes a lightweight front-end visualization interface and a back-end inference service unit. The front-end visualization interface supports single image upload and batch upload of intravascular ultrasound retraction image sequences, visualization of segmentation contours and original image overlay, real-time preview of clinical indicator reports, and result export. The back-end inference service unit is based on the Flask framework and PyTorch to build an inference REST API, perform image inference frame by frame, and batch output original intravascular ultrasound images, color segmentation overlay effect diagrams, and standardized quantitative indicator tables.

[0060] Furthermore, the cross-shaped window self-attention mechanism in the CSWinTransformer branch is specifically configured as follows: the attention window is divided into multiple cross-shaped strip windows along the horizontal and vertical directions, and the self-attention within the horizontal strip window and the vertical strip window is calculated respectively. By alternately stacking the horizontal cross-shaped window attention and the vertical cross-shaped window attention, global long-range dependency modeling along the angular and radial directions of the intravascular ultrasound image is realized.

[0061] Furthermore, in the ASPP module, the ASPP module of the ResNet50 convolutional neural network branch has 1024 input channels and 512 output channels; the ASPP module of the CSWinTransformer branch has 384 input channels and 512 output channels; in the five parallel branches, the dilation rates of each dilated convolutional branch are 12, 24 and 36, respectively; the output feature maps of the 1×1 convolutional branch, the three sets of dilated convolutional branches and the global average pooling branch are concatenated in the channel dimension to obtain multi-scale aggregated features, which are then fused and reduced in dimension by a 1×1 convolutional layer to output an enhanced feature map with a uniform number of channels.

[0062] Furthermore, the topology consistency constraint module also includes a post-processing topology verification unit during the inference phase. The input of the post-processing topology verification unit is connected to the output of the decoder module and is used to receive the stent segmentation mask and the lumen segmentation mask. The post-processing topology verification unit is configured to: check the stent segmentation mask output by the decoder module pixel by pixel, remove the stent prediction pixels located outside the lumen area defined by the lumen segmentation mask, and output the verified and corrected stent segmentation mask and the lumen segmentation mask together to the clinical indicator calculation module to ensure that the final output stent segmentation result strictly wraps inside the lumen.

[0063] Furthermore, the data preprocessing module also includes a multi-vendor format conversion toolchain unit, which is located before the medical format parsing unit. This toolchain unit is used to connect to the input ports of the original image data from various vendors, and supports batch conversion of proprietary image formats from various intravascular ultrasound equipment manufacturers into standard PNG or JPG formats. It maintains the consistency of the original image grayscale values ​​throughout the entire process. The converted standard format images are sequentially sent to the resolution normalization unit and the adaptive image denoising and enhancement unit. The wavelet transform algorithm of the adaptive image denoising and enhancement unit specifically includes three sub-steps: wavelet decomposition, threshold shrinkage denoising, and wavelet reconstruction. The threshold shrinkage adopts an adaptive threshold strategy.

[0064] A method for collaborative segmentation of intravascular ultrasound stents and lumens based on a CNN-Transformer hybrid model includes the following steps, which are executed sequentially as follows:

[0065] Step S1: Standardization and preprocessing of raw intravascular ultrasound images

[0066] First, the system reads raw intravascular ultrasound images in various formats as input, performs medical format standardization parsing, and extracts metadata including transducer model, retraction speed, and lesion location.

[0067] Next, a batch conversion tool is used to convert the proprietary ultrasound image formats of various manufacturers into standard PNG or JPG grayscale images, preserving the original grayscale information throughout the process;

[0068] Then, crop and scale all images to the standard size.

[0069] Then, wavelet transform algorithm is used for adaptive image denoising to suppress speckle noise, ring artifacts and calcification sound shadow interference;

[0070] Finally, image enhancement is performed by combining gamma correction and local contrast equalization to eliminate uneven device brightness and interference from the central dark area, generating a standardized grayscale image, which is then used as the input for step S2.

[0071] Step S2: Parallel Feature Extraction via Two Branches

[0072] The standardized grayscale image output from step S1 is simultaneously fed into two parallel feature extraction backbones, namely the ResNet50 convolutional neural network branch and the CSWinTransformer branch.

[0073] The ResNet50 convolutional neural network branch extracts short-range local features of local texture of blood vessel wall, fine-grained edge of stent beam and gray-scale details of plaque, generating the first branch first stage feature map;

[0074] Meanwhile, the CSWinTransformer branch employs a cross-window self-attention mechanism to calculate global self-attention along the angular and radial dimensions of the intravascular ultrasound image, modeling the long-range dependency of the vascular ring continuity and the overall spatial distribution of the stent, and generating the first-stage feature map of the second branch.

[0075] The first-stage feature map of the first branch and the first-stage feature map of the second branch are respectively used as the inputs of the two branches in step S3;

[0076] Step S3: Two-branch independent multi-scale context enhancement

[0077] The first-stage feature map of the first branch output of the ResNet50 convolutional neural network in step S2 is fed into the first ASPP module embedded at the end of the branch for multi-scale context enhancement.

[0078] Simultaneously, the first-stage feature map of the second branch output by the CSWinTransformer branch in step S2 is sent to the second ASPP module embedded at the end of the branch for multi-scale context enhancement.

[0079] Each of the ASPP modules extracts multi-scale features through five parallel branches, including a 1×1 basic convolution branch, three sets of dilated convolution branches with dilation rates of 12, 24 and 36, and a global average pooling branch.

[0080] After concatenating and fusing the outputs of each branch along the channel dimension, the number of channels is reduced by 1×1 convolution to unify the dimensionality of the channel, thus obtaining the second-stage enhanced feature map of the first branch and the second-stage enhanced feature map of the second branch respectively.

[0081] The enhanced feature map of the second stage of the first branch and the enhanced feature map of the second stage of the second branch are used together as the input of step S4;

[0082] Step S4: Cross-modal deep feature fusion

[0083] Simultaneously receive the first branch second-stage enhanced feature map and the second branch second-stage enhanced feature map after enhancement by the dual-branch ASPP module in step S3;

[0084] The spatial attention features output by the ResNet50 convolutional neural network branch are fused with the channel attention features output by the CSWinTransformer branch;

[0085] The bilinear interaction unit achieves deep complementary fusion of local details and global ring structure features, outputs a unified fusion feature map, and uses this unified fusion feature map as the input for step S5.

[0086] Step S5: Layered decoding to generate a dual-channel segmentation mask

[0087] The unified fused feature map output from step S4 is input into the hierarchical decoder and aligned with the cross-resolution semantic features of each level of the encoder through attention-enhanced skip connections.

[0088] The spatial resolution of the feature map is restored through a stepwise upsampling operation;

[0089] The final output is a dual-channel segmentation probability mask, where the first channel corresponds to the segmentation result of the lumen region and the second channel corresponds to the segmentation result of the support region.

[0090] The dual-channel segmentation probability mask is used as the input for both steps S6 and S7.

[0091] Step S6: Topology Consistency Constraints

[0092] First, during the model training phase, the clinical anatomical prior that the stent is completely located inside the lumen is transformed into a differentiable topological loss function.

[0093] The model training is supervised by the total loss function, where the segmentation loss is composed of Dice loss and binary cross-entropy loss, and the topology loss is a weighted sum of the segmentation loss. Prior knowledge of anatomy is embedded into the entire feature learning process through backpropagation.

[0094] Then, during the inference phase, the dual-channel segmentation probability mask output in step S5 is received, and the bracket segmentation mask is post-processed for topology verification. The bracket prediction pixels located outside the cavity region defined by the cavity segmentation mask are removed, and a cavity binary segmentation mask and a bracket binary segmentation mask optimized by topology constraints are generated.

[0095] The topology-constrained binary segmentation mask for the lumen and the binary segmentation mask for the support are used as inputs for step S7.

[0096] Step S7: Automatically calculate clinical quantitative assessment indicators

[0097] The binary segmentation mask of the lumen and the binary segmentation mask of the support output in step S6 are received as input;

[0098] The stent profile is fitted with a minimum circumscribed ellipse, and the ellipticity of the stent is calculated by the ratio of the minor axis length to the major axis length, thus quantifying the uniformity of stent expansion.

[0099] The bracket area is calculated by multiplying the effective bracket pixel count by the area of ​​a single pixel.

[0100] The stent-lumen area ratio is calculated as the ratio of the total stent area to the total lumen area, and is used to assess the stent's apposition to the lumen and the risk of residual stenosis.

[0101] The segmentation results and clinical quantitative indicators are visualized through a front-end and back-end interactive module, and batch export of evaluation reports is supported.

[0102] Further, the wavelet transform algorithm for adaptive image denoising in step S1 specifically includes: performing wavelet decomposition on the input image to obtain wavelet coefficients of multiple scales and directions; applying adaptive thresholds to the wavelet coefficients of each layer for threshold shrinkage processing to suppress high-frequency components corresponding to speckle noise and ring artifacts; and obtaining the denoised image through wavelet reconstruction. The gamma correction and local contrast equalization specifically include: adjusting the gamma value of the image grayscale distribution to correct brightness unevenness, and performing contrast equalization processing within local image blocks to enhance the visibility of the lumen boundary and the stent edge.

[0103] Furthermore, in step S3, the global average pooling branch of each ASPP module is specifically executed as follows: global average pooling is performed on the input feature map to compress the spatial information of each channel into a single value to obtain a channel description vector; the channel description vector is transformed by 1×1 convolution and then upsampled to restore the original feature map spatial size to obtain global context features, which are then concatenated and fused with the outputs of the other four branches in the channel dimension.

[0104] Furthermore, the cross-modal deep feature fusion in step S4 specifically includes: generating a spatial attention weight map from the enhanced feature map output by the ResNet50 convolutional neural network branch through a spatial attention mechanism to enhance local texture and edge feature responses; generating a channel attention weight vector from the enhanced feature map output by the CSWinTransformer branch through a channel attention mechanism to enhance the feature channel responses related to the global ring structure; and inputting the spatial attention-weighted CNN features and the channel attention-weighted Transformer features into a bilinear interaction unit, which captures second-order feature interaction information through outer product operations to achieve deep complementary fusion of local details and global ring structure features.

[0105] Furthermore, the layered decoder in step S5 specifically includes multiple upsampling stages, each upsampling stage comprising an upsampling layer, a skip connection layer, and a convolutional layer; the upsampling layer uses bilinear interpolation or transposed convolution to progressively restore the feature map resolution; the skip connection layer concatenates the features of the current decoding layer with the features of the corresponding layer of the encoder, and before concatenation, selectively enhances the encoder features through an attention gating unit to suppress irrelevant background responses and highlight relevant features of the lumen and support regions; the convolutional layer performs nonlinear transformations on the concatenated features to progressively restore spatial detail information.

[0106] Furthermore, after step S7, there are also front-end and back-end interaction steps: the front-end visualization interface receives a single image or a batch of intravascular ultrasound retraction image sequences uploaded by the user; the back-end RESTAPI service built on the Flask framework and PyTorch executes the complete inference process of steps S1 to S7 frame by frame; after the inference is completed, the original intravascular ultrasound images, segmentation contour overlay visualization images, color segmentation mask overlay effect images, and standardized quantitative index tables containing stent ellipticity, stent area, and stent-lumen area ratio are output in batches; the front-end interface previews the segmentation and quantification results in real time and provides a result export function.

[0107] A computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements all the steps of a method for co-segmentation of intravascular ultrasound stents and lumens based on a CNN-Transformer hybrid model.

[0108] An intravascular ultrasound intelligent segmentation and evaluation device includes a memory and a processor. The memory stores an executable computer program. When the processor executes the computer program, it implements all the steps of the intravascular ultrasound stent and lumen collaborative segmentation method based on the CNN-Transformer hybrid model, including: intravascular ultrasound image standardization preprocessing, dual-branch encoder feature extraction, ASPP multi-scale context enhancement, BiFusion cross-modal feature fusion, hierarchical decoding to generate a segmentation mask, topological consistency constraint correction, and automatic calculation of clinical quantitative indicators. Attached Figure Description

[0109] Figure 1 : A schematic diagram of the overall structure of the system of the present invention.

[0110] Figure 2 : Workflow diagram of the system of this invention.

[0111] Figure 3 : The architecture diagram of the StentSegNet model in this invention.

[0112] Figure 4: A structural diagram of the ASPP module in this invention.

[0113] Figure 5 Comparison of segmentation results on different models of this invention.

[0114] Figure 6 Screenshot of the user interface prototype of the system of this invention. Detailed Implementation

[0115] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0116] An intravascular ultrasound stent-lumen co-segmentation system based on a CNN-Transformer hybrid model includes: a data preprocessing module for standardizing raw IVUS images; a dual-branch encoder module comprising a ResNet50CNN branch for local texture extraction and a CSWinTransformer branch for global dependency modeling; a multi-scale context enhancement module, i.e., an ASPP module, embedded at the ends of the CNN and Transformer branches respectively; a cross-modal feature fusion module, i.e., a BiFusion module, fusing spatial attention and channel attention; a decoder module for generating a lumen-stent segmentation mask; a topology consistency constraint module for constraining the predicted stent location within the lumen using a topology loss function; and a clinical indicator calculation module.

[0117] The CSWinTransformer branch uses the Cross-ShapedWindowAttention mechanism to calculate self-attention in both angular and radial directions.

[0118] The ASPP module contains five parallel branches: a 1x1 convolution branch, three dilated convolution branches with dilation rates of 12, 24, and 36, and a global average pooling branch. The outputs are concatenated and then fused through a 1x1 convolution.

[0119] The topological constraint loss function is L_topo=(1 / N)xsum(S_ix(1-L_i)), and the total loss function is L_Total=L_Seg+alphaxL_Topo, where alpha is 0.5.

[0120] The clinical indicators include: stent ellipticity, stent area, and stent-to-lumen area ratio.

[0121] A collaborative segmentation method for intravascular ultrasound stents and lumens based on a CNN-Transformer hybrid model includes the following steps: S1, acquiring and preprocessing raw IVUS images; S2, extracting local texture features through a ResNet50 branch and extracting global dependency features through a CSWinTransformer branch; S3, performing multi-scale context enhancement on the features of the two branches using the ASPP module; S4, fusing spatial attention and channel attention features using the BiFusion module; S5, decoding to generate segmentation masks for the lumen and stent; S6, applying topological constraints to ensure the stent is located inside the lumen; S7, calculating clinical quantitative indicators based on the segmentation results.

[0122] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method.

[0123] Example

[0124] An intravascular ultrasound stent and lumen collaborative segmentation system based on a CNN-Transformer hybrid model is generally divided into a front-end visualization and interaction layer, a back-end inference service layer, and a StentSegNet deep learning model layer, comprising eight core functional modules:

[0125] (1) Data preprocessing module

[0126] Deployed at the backend input entry point, configured as follows:

[0127] ① Medical format standardization and parsing: Read DICOM standard IVUS images and extract metadata such as transducer model, retraction speed, and lesion location; develop batch conversion tools to uniformly convert proprietary ultrasound formats from various manufacturers into PNG / JPG grayscale images, preserving the original grayscale information throughout the process;

[0128] ② Resolution normalization: uniformly crop and scale to the standard size of 224×224 or 512×512 to adapt to model input;

[0129] ③ Adaptive image noise reduction and enhancement: Wavelet transform algorithm is used to suppress speckle noise, ring artifacts, and calcification sound shadows; combined with gamma correction and local contrast equalization, uneven device brightness and central dark area interference are eliminated;

[0130] ④ Output the standardized grayscale image and send it to the dual-branch encoder module.

[0131] (2) Dual-branch encoder module

[0132] To interface with the data preprocessing module output, two parallel feature extraction backbones are set up:

[0133] ①ResNet50CNN branch: responsible for extracting short-range local features such as local texture of blood vessel walls, fine-grained edges of stent beams, and gray-scale details of plaques;

[0134] ②CSWinTransformer branch: Employs a cross-shaped window self-attention mechanism to calculate global self-attention along the angular and radial dimensions of the IVUS image, accurately modeling the long-range dependence of vascular ring continuity and overall stent spatial distribution, and matching the physical characteristics of IVUS ring imaging.

[0135] (3) Multi-scale context enhancement module (ASPP module)

[0136] Embedded independently at the ends of the CNN branch and the CSWinTransformer branch, configured as follows:

[0137] Each ASPP contains 5 parallel feature extraction branches: a 1×1 basic convolution branch, three groups of dilated convolution branches with dilation rates of 12 / 24 / 36, and a global average pooling branch; the outputs of each branch are spliced ​​and fused in the channel dimension, and then the number of channels is unified by 1×1 convolution to reduce the dimensionality; the receptive field is expanded without losing the spatial resolution of the image, and multi-scale features of large blood vessels and small stents are captured.

[0138] Parameter configuration: CNN branch ASPP input channels 1024, output channels 512; Transformer branch ASPP input channels 384, output channels 512.

[0139] (4) Cross-modal feature fusion module (BiFusion module)

[0140] Receive 512-channel feature maps from two branches after ASPP multi-scale enhancement, configured as follows:

[0141] By fusing the spatial attention features from the CNN branch output and the channel attention features from the CSWinTransformer branch output, a bilinear interaction unit is introduced to achieve deep complementary fusion of local details and global ring structure features, and a unified fused feature map is output and sent to the decoder.

[0142] (5) Decoder module

[0143] Connect to the output of the BiFusion module and configure it as follows:

[0144] A hierarchical progressive upsampling architecture, coupled with attention-enhanced skip connections, aligns semantic features of different resolutions at each level of the encoder; the final output is a dual-channel segmentation probability mask, with channel 1 corresponding to the lumen region and channel 2 corresponding to the stent region.

[0145] (6) Topology consistency constraint module (training-specific loss constraint unit)

[0146] The embedded model training loss calculation chain is configured as follows:

[0147] The innovative approach transforms the clinical anatomical prior that "the stent must be completely inside the lumen" into a differentiable topological loss function. During the training phase, it penalizes unreasonable predictions that stent pixels fall outside the lumen, thus embedding anatomical knowledge into the entire feature learning process.

[0148] Formula for calculating topology loss:

[0149] Total loss function of the model:

[0150] Among them, segmentation loss The balance coefficient α defaults to 0.5; Si is the prediction probability of pixel i (support), Li is the prediction probability of pixel i (cavity), and N is the total number of pixels in the image.

[0151] (7) Clinical indicator calculation module

[0152] The receiver decodes the output of the lumen and stent binary segmentation mask and automatically calculates three core post-PCI clinical quantitative indicators:

[0153] ① Stent ellipticity: Fit the minimum circumscribed ellipse of the stent profile. Ellipticity = minor axis length / major axis length, which quantifies the uniformity of stent expansion.

[0154] ② Bracket area: Effective bracket pixel count and single pixel area;

[0155] ③ Stent-Lumen Area Ratio: The ratio of stent lumen area to total stent area is used to assess the degree of stent apposition and the risk of residual stenosis.

[0156] (8) Front-end and back-end interaction module

[0157] The front end features a lightweight visualization interface that supports uploading single / batch IVUS pullback image sequences, visualization of segmentation contour overlay of original images, real-time preview of indicator reports, and result export. The back end is built on the Flask framework and PyTorch to implement an inference REST API, which performs image inference frame by frame and outputs original IVUS images, color segmentation overlay effect images, and standardized quantitative indicator tables in batches.

[0158] Further design optimization:

[0159] CSWinTransformer's cross-shaped window focuses on calculations along both the angular and radial directions, which naturally matches the IVUS annular anatomical structure, resulting in improved global modeling accuracy compared to the general Swin / ViT.

[0160] The system supports two deployment modes: Flask backend offline inference mode and cross-device domain adaptive image enhancement mode, which are compatible with different brands of IVUS imaging equipment;

[0161] A standardized dataset annotation process is provided: two-stage AI pre-annotation + manual verification by cardiovascular intervention experts to ensure the accuracy of training sample annotation.

[0162] The specific implementation method includes the following steps:

[0163] Step S1: IVUS raw image normalization preprocessing

[0164] Read raw IVUS images in multiple formats, perform batch conversion of proprietary formats, resolution normalization, wavelet adaptive denoising, local contrast enhancement, and generate a unified standard grayscale input image.

[0165] Step S2: Parallel Feature Extraction via Two Branches

[0166] The preprocessed image is simultaneously fed into the ResNet50CNN branch to extract local texture edge features and the CSWinTransformer branch to extract angular / radial global circular dependency features.

[0167] Step S3: Two-branch independent multi-scale context enhancement

[0168] The output features of the two branches are fed into a dedicated ASPP module, which captures multi-scale vascular and stent features through multi-dilation rate dilated convolution and global average pooling, and outputs a unified 512-channel enhanced feature map.

[0169] Step S4: BiFusion cross-modal deep feature fusion

[0170] By integrating CNN spatial attention and Transformer channel attention, and leveraging bilinear interaction units, complementary local details and global vascular structure features are achieved.

[0171] Step S5: Layered decoding to generate a dual-channel segmentation mask

[0172] The fused features are upsampled hierarchically via attention-enhanced skip connections to output a dual-channel segmentation probability mask for the lumen and scaffold.

[0173] Step S6: Training the topology consistency loss constraint model

[0174] During the model training phase, a topological loss function is introduced to constrain the stent prediction region to strictly wrap inside the lumen, eliminating segmentation predictions that violate anatomical logic; during the inference phase, post-processing topological verification is added to remove stent pixels that cross the boundary.

[0175] Step S7: Automatically calculate clinical quantitative assessment indicators

[0176] Based on segmentation contour ellipse fitting and pixel area conversion, the system batch outputs scaffold ellipticity, scaffold area, and scaffold-to-lumen area ratio, visually displays segmentation results and quantitative indicators, and supports batch export of evaluation reports.

[0177] A computer-readable storage medium storing a computer program thereon, which, when executed by a processor, enables the above-described method for the coordinated segmentation of the stent and lumen.

[0178] An IVUS intelligent segmentation and evaluation device includes a memory and a processor. The memory stores runnable inference programs, and the processor executes the programs to realize the entire process of image preprocessing, bi-branch model segmentation, topological constraint correction, and automatic calculation of clinical indicators.

[0179] The system of this invention takes a deep learning segmentation model as its core and combines standardized data preprocessing, annotation construction, inference and visualization modules to form a complete technical chain covering data input, feature learning, boundary generation and quantitative analysis.

[0180] Before model inference, all IVUS images undergo a series of preprocessing operations: (1) Format unification: All raw IVUS data are saved in standard medical formats (such as DICOM) and key metadata such as transducer model, frame rate, pullback speed and lesion location are retained; (2) Develop an automated multi-vendor conversion toolchain to batch convert proprietary formats to standard image formats (such as PNG or JPG) while strictly maintaining the original grayscale consistency; (3) All images are uniformly adjusted and cropped to a standard resolution suitable for deep learning (such as 224x224 or 512x512); (4) A wavelet-based automatic denoising strategy is adopted to reduce speckle noise, ring artifacts and high-intensity reflections; (5) Gamma correction and local contrast enhancement are combined to alleviate the problem of device-specific brightness unevenness and central shadow enhancement.

[0181] StentSegNet model architecture

[0182] The StentSegNet model is an enhanced design based on the classic TransFuse framework, and its specific implementation is as follows:

[0183] (1) Dual-branch encoder: ResNet50 is used as the backbone of the CNN branch to extract local vascular texture and fine-grained edge details; CSWinTransformer is used as the backbone of the Transformer branch to model long-range dependencies related to lumen continuity and stent distribution.

[0184] (2) ASPP module: ASPP modules are embedded after each of the two branches. The ASPP of the CNN branch has 1024 input channels and 512 output channels; the ASPP of the Transformer branch has 384 input channels and 512 output channels.

[0185] (3) BiFusion cross-modal fusion: The fusion module integrates the CNN-driven spatial attention mechanism and the Transformer-driven channel attention mechanism, and introduces a bilinear interaction unit to achieve deep complementary fusion of local and global features.

[0186] (4) Decoder: It adopts a progressive upsampling structure and strengthens the semantic alignment between different resolution levels through attention-enhanced skip connections, and finally generates a high-precision lumen-scaffold segmentation mask (two-channel output).

[0187] This invention innovatively introduces a topological consistency constraint loss to explicitly encode the anatomical prior that "the stent must be completely inside the lumen." For each pixel i, let S_i be the predicted stent probability and L_i be the predicted lumen probability. Then, the topological loss is defined as L_topo = (1 / N) x sum(S_ix(1-L_i)). This loss penalizes predictions of the stent exceeding the lumen during training, directly embedding anatomical knowledge into the feature learning process. The total loss function is L_Total = L_Seg + alpha x L_Topo, where alpha is 0.5.

[0188] After the system is segmented, it automatically calculates three clinical quantitative indicators: stent ellipticity, stent area, and stent / lumen ratio.

[0189] Experimental verification

[0190] This invention was tested on a self-built dataset containing 422 IVUS images. The experimental environment consisted of a 64-bit Windows 10 system, Python 3.8.11, PyTorch 2.1.1, and an NVIDIA GeForce RTX 3090 GPU (24GB VRAM). The AdamW optimizer was used, with an initial learning rate of 1e-4, a cosine annealing learning rate strategy, a batch size of 3, a maximum of 500 training epochs, and an early stopping strategy for 50 epochs.

[0191] Comparative experimental results show that the proposed StentSegNet outperforms existing methods such as UNet, Attention-UNet, TCB, TransFuse-S, TransFuse-L, and TransSegNet in both lumen and stent segmentation tasks, achieving a Dice of 0.9609 for lumens and 0.9659 for stents, with an average Hausdorff distance of only 0.135 mm.

[0192] This invention supports two deployment modes: (1) Python backend inference mode: using the Flask framework to provide RESTAPI services; (2) pix2pix-based image conversion enhancement mode: supporting domain adaptation and image quality enhancement between different IVUS imaging devices.

[0193] We propose an enhanced hybrid deep learning model, StentSegNet, based on the TransFuse architecture. This model combines a parallel ResNet50CNN branch and a CSWinTransformer branch, embedding ASPP modules in both branches. We introduce CSWinTransformer to replace the traditional DeiT / ViT as the global branch, utilizing the Cross-ShapedWindowAttention mechanism to compute self-attention along the angular and radial directions. We innovatively design a topological consistency constraint loss function to explicitly encode the anatomical prior that "the stent is completely located inside the lumen." We propose three clinical quantitative indicators for stents (ellipticity, area, and stent-lumen area ratio), forming a complete "segmentation-quantification" evaluation process. We construct a complete IVUS data standardization and annotation workflow, including wavelet-based automatic denoising, multi-vendor format conversion, two-stage AI-assisted annotation, and expert verification.

Claims

1. A vascular ultrasound stent and lumen collaborative segmentation system based on a CNN-Transformer hybrid model, characterized in that, include: The data preprocessing module is located at the system input port and is configured to perform standardized preprocessing on the raw intravascular ultrasound images and output the standardized grayscale images generated after preprocessing to the downstream dual-branch encoder module. The data preprocessing module includes: The medical format parsing unit is used to read intravascular ultrasound images in DICOM standard format, extract metadata such as transducer model, retraction speed, and lesion location, and supports batch conversion of various vendors' proprietary ultrasound image formats into standard PNG or JPG grayscale images, preserving the original grayscale information throughout the process; The resolution normalization unit is used to uniformly crop and scale all input images to a standard size. The adaptive image noise reduction and enhancement unit uses wavelet transform algorithm to suppress speckle noise, ring artifacts and calcification sound shadow interference, and combines gamma correction and local contrast equalization to eliminate uneven device brightness and central dark area interference, and outputs a standardized grayscale image. A dual-branch encoder module, whose input end is connected to the output end of the data preprocessing module, is used to receive the standardized grayscale image. The dual-branch encoder module is configured with two parallel feature extraction backbones, including: The ResNet50 convolutional neural network branch is configured to extract short-range local features of local texture of blood vessel wall, fine-grained edges of stent beams and gray-scale details of plaque, and output the extracted first-stage feature map to the subsequent multi-scale context enhancement module. The CSWinTransformer branch employs a cross-window self-attention mechanism to calculate global self-attention along the angular and radial dimensions of the intravascular ultrasound image. It models the long-range dependencies of the vascular ring continuity and the overall spatial distribution of the stent, and outputs the extracted first-stage feature map in parallel to the subsequent multi-scale context enhancement module. A multi-scale context enhancement module is independently embedded at the end of the ResNet50 convolutional neural network branch and the end of the CSWinTransformer branch, respectively, to receive the first-stage feature maps output by each of the two branches. Each multi-scale context enhancement module is an ASPP module, namely a dilated spatial pyramid pooling module. The ASPP module contains five parallel feature extraction branches, namely a 1×1 basic convolution branch, a first dilated convolution branch with different dilation rates, a second dilated convolution branch, a third dilated convolution branch, and a global average pooling branch. After the outputs of each branch are concatenated and fused in the channel dimension, the number of channels is unified by 1×1 convolution to expand the receptive field without losing the spatial resolution of the image, capture multi-scale vascular and stent features, and output the second-stage feature maps enhanced by each of the two branches to the cross-modal feature fusion module respectively. The cross-modal feature fusion module has its input terminals connected to the output terminals of the multi-scale context enhancement module embedded at the end of the ResNet50 convolutional neural network branch and the multi-scale context enhancement module embedded at the end of the CSWinTransformer branch, respectively. It is used to simultaneously receive the second-stage feature maps of the two branches after enhancement. The cross-modal feature fusion module is configured to fuse the spatial attention features output by the ResNet50 convolutional neural network branch and the channel attention features output by the CSWinTransformer branch, and introduce a bilinear interaction unit to complete the deep complementary fusion of local details and global ring structure features. The unified fused feature map generated after fusion is output to the decoder module. The decoder module, whose input is connected to the output of the cross-modal feature fusion module, is used to receive the unified fused feature map. The decoder module is configured as a hierarchical progressive upsampling decoding structure, with attention-enhanced skip connections, and is aligned with the cross-resolution semantic features of each level of the dual-branch encoder module. After restoring the spatial resolution through progressive upsampling, it finally outputs a dual-channel segmentation probability mask, where the first channel corresponds to the lumen region and the second channel corresponds to the stent region. The dual-channel segmentation probability mask is then output to the topology consistency constraint module and the clinical indicator calculation module, respectively. The topology consistency constraint module, whose input is connected to the output of the decoder module, is used to receive the dual-channel segmentation probability mask. The topology consistency constraint module is embedded in the loss calculation link of the model training stage and is configured to transform the clinical anatomical prior that the stent is completely inside the lumen into a differentiable topology loss function. During the training stage, it penalizes the prediction results of the stent pixels falling outside the lumen and outputs the final segmentation mask optimized by topology constraints to the clinical index calculation module. The clinical indicator calculation module has its input terminals connected to the output terminals of the decoder module and the topology consistency constraint module, respectively. It receives the binary segmentation mask for the lumen and the binary segmentation mask for the stent, as well as the final segmentation mask optimized by topology constraints, output by the decoder. It automatically calculates core clinical quantitative indicators after PCI. These clinical indicators include: stent ellipticity quantified by the ratio of the minor axis length to the major axis length through minimum circumscribed ellipse fitting of the stent contour; stent area calculated based on the product of the effective stent pixel count and the area of ​​a single pixel; and stent-lumen area ratio calculated as the ratio of the total stent area to the total lumen area. The front-end and back-end interaction module, whose input end is connected to the output end of the clinical indicator calculation module, is used to receive segmentation results and clinical quantitative indicators. The front-end and back-end interaction module includes a lightweight front-end visualization interface and a back-end inference service unit. The front-end visualization interface supports single image upload and batch upload of intravascular ultrasound retraction image sequences, visualization of segmentation contours and original image overlay, real-time preview of clinical indicator reports, and result export. The back-end inference service unit is based on the Flask framework and PyTorch to build an inference REST API, perform image inference frame by frame, and batch output original intravascular ultrasound images, color segmentation overlay effect diagrams, and standardized quantitative indicator tables.

2. The system according to claim 1, characterized in that, The cross-shaped window self-attention mechanism in the CSWinTransformer branch is specifically configured as follows: the attention window is divided into multiple cross-shaped strip windows along the horizontal and vertical directions, and the self-attention within the horizontal strip window and the vertical strip window is calculated respectively. By alternately stacking the horizontal cross-shaped window attention and the vertical cross-shaped window attention, global long-range dependency modeling along the angular and radial directions of the intravascular ultrasound image is realized. In the ASPP module, the ASPP module of the ResNet50 convolutional neural network branch has 1024 input channels and 512 output channels; the ASPP module of the CSWinTransformer branch has 384 input channels and 512 output channels; in the five parallel branches, the dilation rates of each dilated convolutional branch are 12, 24 and 36, respectively; the output feature maps of the 1×1 convolutional branch, the three sets of dilated convolutional branches and the global average pooling branch are concatenated in the channel dimension to obtain multi-scale aggregated features, which are then fused and reduced in dimension by a 1×1 convolutional layer to output an enhanced feature map with a uniform number of channels.

3. The system according to claim 1, characterized in that, The topology consistency constraint module also includes a post-processing topology verification unit during the inference phase. The input of the post-processing topology verification unit is connected to the output of the decoder module and is used to receive the stent segmentation mask and the lumen segmentation mask. The post-processing topology verification unit is configured to: check the stent segmentation mask output by the decoder module pixel by pixel, remove the stent prediction pixels located outside the lumen area defined by the lumen segmentation mask, and output the verified and corrected stent segmentation mask and the lumen segmentation mask together to the clinical indicator calculation module to ensure that the final output stent segmentation result strictly wraps inside the lumen. The data preprocessing module also includes a multi-vendor format conversion toolchain unit, which is located before the medical format parsing unit. This toolchain unit is used to connect to the input ports of the original image data from various vendors, and supports batch conversion of proprietary image formats from various intravascular ultrasound equipment manufacturers into standard PNG or JPG formats. It maintains the consistency of the original image grayscale values ​​throughout the entire process. The converted standard format images are sequentially sent to the resolution normalization unit and the adaptive image denoising and enhancement unit. The wavelet transform algorithm of the adaptive image denoising and enhancement unit specifically includes three sub-steps: wavelet decomposition, threshold shrinkage denoising, and wavelet reconstruction. The threshold shrinkage adopts an adaptive threshold strategy.

4. A method for collaborative segmentation of intravascular ultrasound stents and lumens based on a CNN-Transformer hybrid model, implemented using the system described in any one of claims 1 to 3, characterized in that, The steps include the following steps, which are executed sequentially in the following order: Step S1: Standardization and preprocessing of raw intravascular ultrasound images First, the system reads raw intravascular ultrasound images in various formats as input, performs medical format standardization parsing, and extracts metadata including transducer model, retraction speed, and lesion location. Next, a batch conversion tool is used to convert the proprietary ultrasound image formats of various manufacturers into standard PNG or JPG grayscale images, preserving the original grayscale information throughout the process; Then, crop and scale all images to the standard size. Then, wavelet transform algorithm is used for adaptive image denoising to suppress speckle noise, ring artifacts and calcification sound shadow interference; Finally, image enhancement is performed by combining gamma correction and local contrast equalization to eliminate uneven device brightness and interference from the central dark area, generating a standardized grayscale image, which is then used as the input for step S2. Step S2: Parallel Feature Extraction via Two Branches The standardized grayscale image output from step S1 is simultaneously fed into two parallel feature extraction backbones, namely the ResNet50 convolutional neural network branch and the CSWinTransformer branch. The ResNet50 convolutional neural network branch extracts short-range local features of local texture of blood vessel wall, fine-grained edge of stent beam and gray-scale details of plaque, generating the first branch first stage feature map; Meanwhile, the CSWinTransformer branch employs a cross-window self-attention mechanism to calculate global self-attention along the angular and radial dimensions of the intravascular ultrasound image, modeling the long-range dependency of the vascular ring continuity and the overall spatial distribution of the stent, and generating the first-stage feature map of the second branch. The first-stage feature map of the first branch and the first-stage feature map of the second branch are respectively used as the inputs of the two branches in step S3; Step S3: Two-branch independent multi-scale context enhancement The first-stage feature map of the first branch output of the ResNet50 convolutional neural network in step S2 is fed into the first ASPP module embedded at the end of the branch for multi-scale context enhancement. Simultaneously, the first-stage feature map of the second branch output by the CSWinTransformer branch in step S2 is sent to the second ASPP module embedded at the end of the branch for multi-scale context enhancement. Each of the ASPP modules extracts multi-scale features through five parallel branches, including a 1×1 basic convolution branch, three sets of dilated convolution branches with dilation rates of 12, 24 and 36, and a global average pooling branch. After concatenating and fusing the outputs of each branch along the channel dimension, the number of channels is reduced by 1×1 convolution to unify the dimensionality of the channel, thus obtaining the second-stage enhanced feature map of the first branch and the second-stage enhanced feature map of the second branch respectively. The enhanced feature map of the second stage of the first branch and the enhanced feature map of the second stage of the second branch are used together as the input of step S4; Step S4: Cross-modal deep feature fusion Simultaneously receive the first branch second-stage enhanced feature map and the second branch second-stage enhanced feature map after enhancement by the dual-branch ASPP module in step S3; The spatial attention features output by the ResNet50 convolutional neural network branch are fused with the channel attention features output by the CSWinTransformer branch; The bilinear interaction unit achieves deep complementary fusion of local details and global ring structure features, outputs a unified fusion feature map, and uses this unified fusion feature map as the input for step S5. Step S5: Layered decoding to generate a dual-channel segmentation mask The unified fused feature map output from step S4 is input into the hierarchical decoder and aligned with the cross-resolution semantic features of each level of the encoder through attention-enhanced skip connections. The spatial resolution of the feature map is restored through a stepwise upsampling operation; The final output is a dual-channel segmentation probability mask, where the first channel corresponds to the segmentation result of the lumen region and the second channel corresponds to the segmentation result of the support region. The dual-channel segmentation probability mask is used as the input for both steps S6 and S7. Step S6: Topology Consistency Constraints First, during the model training phase, the clinical anatomical prior that the stent is completely located inside the lumen is transformed into a differentiable topological loss function. The model training is supervised by the total loss function, where the segmentation loss is composed of Dice loss and binary cross-entropy loss, and the topology loss is a weighted sum of the segmentation loss. Prior knowledge of anatomy is embedded into the entire feature learning process through backpropagation. Then, during the inference phase, the dual-channel segmentation probability mask output in step S5 is received, and the bracket segmentation mask is post-processed for topology verification. The bracket prediction pixels located outside the cavity region defined by the cavity segmentation mask are removed, and a cavity binary segmentation mask and a bracket binary segmentation mask optimized by topology constraints are generated. The topology-constrained binary segmentation mask for the lumen and the binary segmentation mask for the support are used as inputs for step S7. Step S7: Automatically calculate clinical quantitative assessment indicators The binary segmentation mask of the lumen and the binary segmentation mask of the support output in step S6 are received as input; The stent profile is fitted with a minimum circumscribed ellipse, and the ellipticity of the stent is calculated by the ratio of the minor axis length to the major axis length, thus quantifying the uniformity of stent expansion. The bracket area is calculated by multiplying the effective bracket pixel count by the area of ​​a single pixel. The stent-lumen area ratio is calculated as the ratio of the total stent area to the total lumen area, and is used to assess the stent's apposition to the lumen and the risk of residual stenosis. The segmentation results and clinical quantitative indicators are visualized through a front-end and back-end interactive module, and batch export of evaluation reports is supported.

5. The method according to claim 6, characterized in that, The adaptive image denoising using the wavelet transform algorithm in step S1 specifically includes: performing wavelet decomposition on the input image to obtain wavelet coefficients of multiple scales and directions; applying adaptive thresholding to the wavelet coefficients of each layer to perform threshold shrinkage processing to suppress high-frequency components corresponding to speckle noise and ring artifacts; and obtaining the denoised image through wavelet reconstruction. The gamma correction and local contrast equalization specifically include: adjusting the gamma value of the image grayscale distribution to correct brightness unevenness, and performing contrast equalization processing within local image blocks to enhance the visibility of the lumen boundary and the stent edge. In step S3, the global average pooling branch of each ASPP module is specifically executed as follows: global average pooling is performed on the input feature map to compress the spatial information of each channel into a single value to obtain the channel description vector; the channel description vector is transformed by 1×1 convolution and then upsampled to restore the original feature map spatial size to obtain the global context features, which are then concatenated and fused with the outputs of the other four branches in the channel dimension.

6. The method according to claim 4, characterized in that, The cross-modal deep feature fusion in step S4 specifically includes: generating a spatial attention weight map from the enhanced feature map output by the ResNet50 convolutional neural network branch through a spatial attention mechanism to enhance local texture and edge feature responses; generating a channel attention weight vector from the enhanced feature map output by the CSWinTransformer branch through a channel attention mechanism to enhance the feature channel responses related to the global ring structure; and inputting the spatial attention-weighted CNN features and the channel attention-weighted Transformer features into a bilinear interaction unit, which captures second-order feature interaction information through outer product operations to achieve deep complementary fusion of local details and global ring structure features.

7. The method according to claim 4, characterized in that, The layered decoder in step S5 specifically includes multiple upsampling stages, each of which contains an upsampling layer, a skip connection layer, and a convolutional layer. The upsampling layer uses bilinear interpolation or transposed convolution to restore the feature map resolution step by step. The skip connection layer concatenates the features of the current decoding layer with the features of the corresponding layer of the encoder. Before concatenation, the encoder features are selectively enhanced by an attention gating unit to suppress irrelevant background responses and highlight relevant features of the lumen and support areas. The convolutional layer performs nonlinear transformations on the concatenated features to restore spatial detail information step by step.

8. The method according to claim 4, characterized in that, Following step S7, the process also includes a front-end and back-end interaction step: the front-end visualization interface receives single images or batches of intravascular ultrasound retraction image sequences uploaded by the user; the back-end REST API service, built on the Flask framework and PyTorch, executes the complete inference process from step S1 to step S7 frame by frame; after inference is completed, the system outputs batches of original intravascular ultrasound images, segmented contour overlay visualization images, color segmentation mask overlay effect images, and standardized quantitative index tables containing stent ellipticity, stent area, and stent-lumen area ratio; the front-end interface previews the segmentation and quantification results in real time and provides a result export function.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements all the steps of the intravascular ultrasound stent and lumen co-segmentation method based on the CNN-Transformer hybrid model as described in any one of claims 5 to 8.

10. An intravascular ultrasound intelligent segmentation and assessment device, characterized in that, The method includes a memory and a processor, wherein the memory stores an executable computer program, and the processor executes the computer program to implement all the steps of the intravascular ultrasound stent and lumen collaborative segmentation method based on the CNN-Transformer hybrid model as described in any one of claims 5 to 8, including: intravascular ultrasound image standardization preprocessing, bi-branch encoder feature extraction, ASPP multi-scale context enhancement, BiFusion cross-modal feature fusion, hierarchical decoding to generate a segmentation mask, topology consistency constraint correction, and automatic calculation of clinical quantitative indicators.

Images