Dynamic recognition system and method for vulnerable plaque of intravascular ultrasonic image

Abstract

The invention discloses a system and a method for dynamically identifying vulnerable plaques of an intravascular ultrasonic image. According to the system, plaque evolution information of continuous IVUS image frames is extracted by constructing a feature extraction module based on a space-time convolutional neural network; combining limited OCT annotation data through a transfer learning model; the blood flow wall shear force and the inflammatory marker IL-6 are fused in a plaque rupture risk prediction matrix, and a new mathematical calculation model is provided to predict the plaque risk. The system supports real-time analysis in an interventional operation and automatically marks a high-risk plaque area, and the vulnerable plaque detection sensitivity can reach 94% or above. The hardware architecture comprises an image acquisition unit, a deep learning calculation module and a labeling interaction module; the software architecture comprises a plaque recognition engine, a shear force calculation module, a biological data fusion interface and the like. The system has the advantages of being easy and convenient to operate, high in real-time performance, high in accuracy and the like, and an efficient intraoperative plaque risk assessment tool is provided for interventional therapy.

Description

Technical Field

[0001] The present invention relates to the technical fields of medical impact analysis and artificial intelligence-assisted diagnosis, and in particular to a system and method for dynamically identifying vulnerable plaques using intravascular ultrasound images. Background Art

[0002] Coronary artery disease caused by atherosclerosis is one of the leading causes of cardiovascular events, including myocardial infarction. Vulnerable plaques typically present with a lipid-rich core and active inflammatory cells. Rupture can trigger acute thrombosis, carrying a high risk of fatality. Commonly used intravascular imaging techniques in clinical practice include intravascular optical coherence tomography (OCT) and intravascular ultrasound (IVUS). OCT has extremely high axial resolution (approximately 1218 μm), enabling precise measurement of plaque fibrous cap thickness and identification of thin-capped, lipid-rich plaques. However, OCT images are characterized by large data volumes, high levels of noise, and difficulty in real-time processing. They are also sensitive to vessel tortuosity. IVUS, while simple to use and highly real-time, has a lower resolution (approximately 150-200 μm), making it difficult to distinguish plaque components, particularly the lipid core. Furthermore, a growing body of research indicates that hemodynamic factors and biomarkers significantly influence plaque instability. Blood flow wall shear stress (WSS) regulates endothelial cell function. Low WSS areas are prone to plaque formation, while high WSS may be associated with the formation of vulnerable plaques. Circulatory inflammatory factors, such as interleukin-6 (IL-6), have also been shown to predict the fragility and progression of arterial plaques. However, there is currently a lack of comprehensive recognition systems that combine IVUS sequence image features, WSS, and biomarkers (such as IL-6), making it difficult to achieve real-time dynamic assessment of vulnerable plaques during interventional procedures. Summary of the Invention

[0003] Technical purpose: In response to the shortcomings of the existing technology, the present invention discloses a dynamic identification system and method for vulnerable plaques based on intravascular ultrasound images. Through deep learning and multi-source information fusion, plaque evolution feature extraction, plaque component identification and rupture risk prediction can be achieved in continuous intravascular ultrasound images, thereby improving the accuracy and real-time performance of vulnerable plaque detection.

[0004] Technical solution: To achieve the above technical objectives, the present invention adopts the following technical solution:

[0005] A dynamic identification system for vulnerable plaques using intravascular ultrasound imaging, comprising:

[0006] An image acquisition module, used for acquiring or receiving intravascular ultrasound image sequence data;

[0007] a deep learning computing module, connected to the image acquisition module, for deploying a spatiotemporal convolutional neural network model and other neural network models to process the intravascular ultrasound image sequence;

[0008] An annotation interactive module, used to display intravascular ultrasound images and annotate vulnerable plaque areas in real time based on risk indicators;

[0009] Shear force calculation module, used to calculate the blood flow wall shear force at the plaque;

[0010] Biological data fusion interface, used to receive the serum IL-6 inflammatory marker data of the test subject;

[0011] The risk prediction module is used to input the shear force and the IL-6 value into a preset risk calculation model to calculate and output a plaque rupture risk index.

[0012] Preferably, a plaque recognition engine runs on the deep learning computing model, which includes a spatiotemporal feature extraction submodule and a transfer learning classification submodule. The spatiotemporal feature extraction submodule takes multiple frames of intravascular ultrasound images as input and extracts the spatiotemporal evolution characteristics of the plaque through a spatiotemporal convolutional neural network. The transfer learning classification submodule identifies plaque components based on a neural network model pre-trained on limited optical coherence tomography annotation data.

[0013] Preferably, the spatiotemporal feature extraction submodule uses a three-dimensional convolutional neural network to perform convolution operations in the spatial and temporal directions on consecutive frames of intravascular ultrasound images to extract the changing characteristics of plaque morphology in the time series.

[0014] Preferably, the transfer learning classification submodule utilizes a deep neural network structure pre-trained on an optical coherence tomography plaque dataset and is fine-tuned on limited intravascular ultrasound plaque annotation data.

[0015] Preferably, the specific formula of the risk calculation model in the risk prediction module is:

[0016] ,

[0017] in is the blood flow wall shear force at the plaque site to be detected, is the normal reference shear force, is the interleukin-6 concentration in the serum of the corresponding test subject, is the normal reference serum interleukin-6 concentration, 、 and Dimensionless weight coefficient determined for historical case training.

[0018] Preferably, the risk assessment module and Perform standardization to convert it into a dimensionless value with a mean of 0 and a variance of 1 for input into calculations.

[0019] Preferably, a GPU accelerator is further included, and the deep learning calculation module is deployed on the GPU accelerator to improve the speed of real-time plaque recognition and calculation.

[0020] A method for dynamic identification of vulnerable plaques using intravascular ultrasound images is applied to the above-mentioned system for dynamic identification of vulnerable plaques using intravascular ultrasound images, and specifically comprises the following steps:

[0021] S1. Acquire intravascular ultrasound image sequence;

[0022] S2. Preprocessing the image sequence, including denoising, image enhancement and standardization operations;

[0023] S3. Input the preprocessed image sequence into the spatiotemporal convolutional neural network to perform multi-frame feature extraction and plaque detection to obtain the evolution characteristics of the plaque in time series;

[0024] S4, input the extracted features into the transfer learning classification submodule to classify and identify the plaque components;

[0025] S5. Calculate the blood flow wall shear force value at the plaque and obtain the corresponding serum IL-6 value, input both into the risk prediction model, and calculate the plaque rupture risk index;

[0026] S6. Mark high-risk plaque areas above a preset threshold in the intravascular ultrasound image.

[0027] Beneficial effects: The system and method for dynamic identification of vulnerable plaques using intravascular ultrasound imaging provided by the present invention have the following beneficial effects:

[0028] This invention pioneers the application of spatiotemporal convolutional neural networks to IVUS sequence image processing. By applying multi-frame 3D convolution kernels in parallel to capture subtle changes in plaque space and time, it can automatically identify thin fibrous caps, tiny calcifications, and lipid cores that are difficult to distinguish using traditional 2D segmentation. This method improves detection sensitivity by approximately 15% compared to single-frame static models, effectively capturing plaque morphological changes caused by the cardiac cycle and catheter rotation, ensuring dynamic and accurate identification and localization of vulnerable plaques during surgery.

[0029] 2. Regarding risk assessment, this invention innovatively integrates local wall shear force and serum IL-6 inflammatory levels with image features extracted by a spatiotemporal convolutional neural network in a deep fusion layer, constructing a highly interpretable multi-source information risk prediction model. This model intuitively presents the plaque rupture index under different WSS-IL-6 combinations through risk matrices and heat maps. This model not only reflects the impact of mechanical shear stimulation on plaque structure but also takes into account the individual inflammatory status of the patient, providing clinicians with a quantitative, comparable, and traceable basis for intraoperative decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for describing the embodiments or the prior art.

[0031] Figure 1 This is the overall framework diagram of the system of the present invention;

[0032] Figure 2 Flow chart of the method of the present invention;

[0033] Figure 3 This is a schematic diagram of marking vulnerable plaque areas in intravascular ultrasound of the present invention;

[0034] Figure 4 Schematic diagram of the spatiotemporal convolutional neural network structure and processing flow of the present invention;

[0035] Figure 5 This is a schematic diagram of the plaque risk heat map overlay of the present invention;

[0036] Figure 6 This is the hemodynamics overlay diagram of the present invention;

[0037] Figure 7 Schematic diagram of plaque rupture risk prediction matrix. DETAILED DESCRIPTION

[0038] The present invention will be described more clearly and completely below by way of a preferred embodiment in conjunction with the accompanying drawings, but the present invention is not limited to the scope of the embodiment.

[0039] like Figure 1As shown, a system for dynamic identification of vulnerable plaques based on intravascular ultrasound images mainly includes two parts: hardware architecture and software architecture. The hardware architecture includes: image acquisition module, deep learning computing unit and annotation interaction module. The image acquisition module acquires intravascular ultrasound sequence images through a catheter and performs preprocessing; the deep learning computing module has a built-in GPU accelerator for deploying and running the neural network model described in the present invention; the annotation interaction module realizes real-time image display and result annotation, and provides a user interaction interface. The software architecture includes functional modules such as plaque recognition engine, shear force calculation module, and biological data fusion interface. The plaque recognition engine is responsible for feature extraction and plaque component classification of intravascular ultrasound sequences, specifically including spatiotemporal feature extraction submodule and transfer learning classification submodule; the shear force calculation module uses a hemodynamic model to calculate the blood wall shear force WSS; the biological data fusion interface is used to input biomarker data such as serum IL-6 of the test subject and fuse it with the image analysis results.

[0040] The image acquisition module consists of an IVUS imaging catheter, a data acquisition card, and a digital signal processing unit. The IVUS catheter incorporates a high-frequency piezoelectric transducer (40–60 MHz) that captures echo signals from vascular cross-sections at a frame rate of 30 fps during catheter rotation and advancement. The acquisition card amplifies and filters the analog radio frequency (RF) signal, converts it into a 12-bit digital quantity, and stores it in polar coordinate format. The digital signal processing unit performs the following operations on the digital image:

[0041] Denoising: Apply adjustable Gaussian filtering (σ = 0.5–1.0) and median filtering to suppress discrete noise;

[0042] Normalization: Map the pixel values ​​of each frame to the interval [0,1] and perform histogram equalization to compensate for gain differences;

[0043] Coordinate conversion: Map the polar coordinate image to the Cartesian coordinate system (256×256 pixels) for subsequent convolution processing;

[0044] Boundary detection: The Canny operator is used to quickly identify the center of the catheter shadow circle, providing a reference for the network to locate the region of interest (ROI).

[0045] The main structure of the deep learning computing module is a spatiotemporal convolutional neural network, which consists of the following substructures:

[0046] The spatiotemporal feature extraction submodule uses a three-dimensional convolutional neural network. The first layer has 1 input channel and 32 output channels; the second layer has 64 output channels; and the third layer has 128 output channels. The convolution step size is set to 1, and the padding method keeps the spatial and temporal dimensions consistent. Convolution operations are performed in the spatial and temporal directions on consecutive frames of intravascular ultrasound images to extract the changing characteristics of plaque morphology in the time series.

[0047] A method for extracting features from intravascular ultrasound image sequences based on a spatiotemporal convolutional neural network was developed to extract information about the spatiotemporal evolution of plaques from multiple consecutive frames of intravascular ultrasound images. The spatiotemporal convolutional neural network effectively captures the changing characteristics of plaque morphology between consecutive frames by performing convolution operations in both spatial and temporal dimensions.

[0048] The transfer learning classification submodule utilizes a deep neural network structure pre-trained on an optical coherence tomography plaque dataset and fine-tuned on limited intravascular ultrasound plaque annotation data.

[0049] A transfer learning-based plaque component recognition model was developed, incorporating limited OCT annotated data into the training process to improve the accuracy of identifying vulnerable plaque components, such as the lipid core, as detected by IVUS. Specifically, IVUS images were preprocessed and enhanced. Then, using a network architecture pretrained on an OCT plaque dataset (e.g., ResNet) as the base model, the model was fine-tuned and retrained on a small amount of IVUS annotated data to automatically classify different plaque components detected by IVUS (e.g., fibrous tissue, lipid core, calcification, etc.).

[0050] like Figure 4 The diagram shows the structure and processing flow of the spatiotemporal convolutional neural network (STN). This diagram illustrates the processing flow of the STN in an intravascular ultrasound (IVUS) sequence: First, multiple preprocessed IVUS image sequences are fed into a series of three-dimensional convolutional layers (3-frame × 3-frame × 3-frame kernels, with the number of channels increasing to 32, 64, and 128 at each layer). Dynamic and structural features of plaques are extracted in parallel from the spatiotemporal domain. A temporal pooling layer then aggregates inter-frame features to suppress cardiac cycle jitter. This layer is then flattened and fed into a fully connected layer to fuse high-dimensional features. Finally, a sigmoid / softmax activation is used to output probability scores for vulnerable and non-vulnerable plaques, achieving accurate binary classification of plaques. This end-to-end network, through transfer learning and fine-tuning mechanisms, can run in real time on a GPU platform, supporting dynamic intraoperative plaque detection.

[0051] The risk prediction module is used to input the shear force and the IL-6 value into a preset risk calculation model to calculate and output a plaque rupture risk index.

[0052] The specific formula of the risk calculation model in the risk prediction module is:

[0053] ,

[0054] in is the blood flow wall shear force at the plaque site to be detected, is the normal reference shear force, is the interleukin-6 concentration in the serum of the corresponding test subject, is the normal reference serum interleukin-6 concentration, 、 and The dimensionless weight coefficients are determined by historical case training. The model combines the standardized values ​​of wall shear force and IL-6 and their interaction terms to quantitatively reflect the joint effect of the two on plaque instability.

[0055] The system first acquires a continuous sequence of IVUS image frames through the image acquisition module. These frames are then fed into the spatiotemporal feature extraction submodule of the plaque recognition engine, which is based on a spatiotemporal convolutional neural network architecture. Specifically, the spatiotemporal convolutional neural network consists of three-dimensional convolutional layers and pooling layers. The convolution kernels span both spatial and temporal dimensions, performing convolution operations simultaneously on N input image frames to extract the spatiotemporal evolutionary features of plaques across multiple frames. The output feature tensor is then fed into the transfer learning classification submodule. This transfer learning classification submodule uses a deep convolutional neural network architecture (such as ResNet) pre-trained on a limited dataset of annotated OCT images to identify plaque component labels (e.g., lipid core, fibrous tissue, calcification). This pre-trained model is then fine-tuned on a small amount of annotated IVUS data to adapt to the resolution and imaging characteristics of IVUS. Finally, this submodule classifies the input IVUS images, outputting the location and composition of the plaque in each frame. In this way, the system leverages the high resolution of OCT to achieve highly accurate identification of vulnerable components such as the lipid core in IVUS images. In parallel, the system obtains the blood flow wall shear force value W at the tested plaque and the inflammatory marker IL-6 concentration of the test subject. The shear force calculation module uses the patient's vascular geometry model and real-time blood flow velocity information to obtain the local wall shear force value through a computational fluid dynamics (CFD) model or a simplified model, such as Figure 6 The figure shows a hemodynamic overlay diagram, with a CFD-simulated flow streamline diagram superimposed on the IVUS cross-sectional image. Blue → red indicates a flow velocity from low to high, illustrating how the shear force calculation module estimates the wall shear force W based on the lumen geometry and flow rate. The biological data fusion interface obtains the IL-6 value from preoperative or intraoperative laboratory results. Then, W and IL-6 are input into the plaque rupture risk prediction matrix for risk calculation. The schematic diagram of the plaque rupture risk prediction matrix is ​​shown in Figure 2. Figure 7As shown, the horizontal and vertical coordinates are the wall shear force level and IL-6 level, respectively, and the corresponding risk index is given in different cells. The calculation formula is the specific formula of the risk calculation model described in the aforementioned invention content. In the formula, the deviation degree of W and IL-6 is compared with the preset threshold through normalization processing, and combined with an interaction term to comprehensively generate the plaque risk index R. The system regards the area above the preset threshold as a high-risk plaque area. Figure 5 Shown is a schematic diagram of the plaque risk heat map overlay. The bottom layer is a grayscale image of the intravascular ultrasound cross-section, showing the echo structure of each layer of the vascular wall; the pseudo-color heat map superimposed on the plaque area indicates the rupture risk of the area, and the color gradually transitions from dark blue (low risk) to orange-red (high risk), making it easy to identify high-risk plaques at a glance.

[0056] During interventional procedures, this system combines these two results: the plaque recognition engine determines plaque location and composition, and the risk prediction module outputs a risk index for each plaque. The interactive annotation module automatically colors high-risk plaque areas on the IVUS image based on risk thresholds, displaying them synchronously with the physician's real-time interface. The system continuously executes these steps, enabling continuous analysis of IVUS sequences. Testing of this system on multiple clinical IVUS sequences has demonstrated a sensitivity of over 94% for vulnerable plaque detection, significantly outperforming traditional single-frame static recognition methods.

[0057] like Figure 2 As shown, a method for dynamic identification of vulnerable plaques using intravascular ultrasound images is applied to a system for dynamic identification of vulnerable plaques using intravascular ultrasound images as described above, and specifically includes the following steps:

[0058] S1. Acquire intravascular ultrasound image sequence;

[0059] S2. Preprocessing the image sequence, including denoising, image enhancement and standardization operations;

[0060] S3. Input the preprocessed image sequence into the spatiotemporal convolutional neural network to perform multi-frame feature extraction and plaque detection to obtain the evolution characteristics of the plaque in time series;

[0061] S4, input the extracted features into the transfer learning classification submodule to classify and identify the plaque components;

[0062] S5. Calculate the blood flow wall shear force value at the plaque and obtain the corresponding serum IL-6 value, input both into the risk prediction model, and calculate the plaque rupture risk index;

[0063] S6. Mark high-risk plaque areas above a preset threshold in the intravascular ultrasound image.

[0064] like Figure 3Shown is a schematic diagram of vulnerable plaque areas marked in intravascular ultrasound. The plaque areas are marked with translucent red and dotted lines and are indicated in English.

[0065] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A system for dynamic identification of vulnerable plaques using intravascular ultrasound imaging, characterized in that: include: An image acquisition module, used for acquiring or receiving intravascular ultrasound image sequence data; a deep learning computing module, connected to the image acquisition module, for deploying a spatiotemporal convolutional neural network model and other neural network models to process the intravascular ultrasound image sequence; An annotation interactive module, used to display intravascular ultrasound images and annotate vulnerable plaque areas in real time based on risk indicators; Shear force calculation module, used to calculate the blood flow wall shear force at the plaque; Biological data fusion interface, used to receive the serum IL-6 inflammatory marker data of the test subject; The risk prediction module is used to input the shear force and the IL-6 value into a preset risk calculation model to calculate and output a plaque rupture risk index.

2. The system for dynamic identification of vulnerable plaques using intravascular ultrasound imaging according to claim 1, characterized in that: A plaque recognition engine runs on the deep learning computing model, which includes a spatiotemporal feature extraction submodule and a transfer learning classification submodule. The spatiotemporal feature extraction submodule takes multiple frames of intravascular ultrasound images as input and extracts the spatiotemporal evolution characteristics of plaques through a spatiotemporal convolutional neural network. The transfer learning classification submodule identifies plaque components based on a neural network model pre-trained on limited optical coherence tomography annotated data.

3. The system for dynamic identification of vulnerable plaques using intravascular ultrasound imaging according to claim 2, characterized in that: The spatiotemporal feature extraction submodule uses a three-dimensional convolutional neural network to perform convolution operations in the spatial and temporal directions on consecutive frames of intravascular ultrasound images to extract the changing characteristics of plaque morphology in the time series.

4. The system for dynamic identification of vulnerable plaques using intravascular ultrasound imaging according to claim 2, characterized in that: The transfer learning classification submodule utilizes a deep neural network structure pre-trained on an optical coherence tomography plaque dataset and fine-tuned on limited intravascular ultrasound plaque annotation data.

5. The system for dynamic identification of vulnerable plaques using intravascular ultrasound imaging according to claim 1, characterized in that: The specific formula of the risk calculation model in the risk prediction module is: ， in is the blood flow wall shear force at the plaque site to be detected, is the normal reference shear force, is the interleukin-6 concentration in the serum of the corresponding test subject, is the normal reference serum interleukin-6 concentration, 、 and Dimensionless weight coefficient determined for historical case training.

6. The system for dynamic identification of vulnerable plaques influenced by intravascular ultrasound according to claim 5, characterized in that: The risk assessment module is used to calculate the risk of and Perform standardization to convert it into a dimensionless value with a mean of 0 and a variance of 1 for input into calculations.

7. The system for dynamic identification of vulnerable plaques using intravascular ultrasound imaging according to claim 1, characterized in that: It further includes a GPU accelerator, and the deep learning calculation module is deployed on the GPU accelerator to improve the speed of real-time plaque recognition and calculation.

8. A method for dynamic identification of vulnerable plaques using intravascular ultrasound imaging, characterized in that: A system for dynamic identification of vulnerable plaques using intravascular ultrasound imaging as described in any one of claims 1 to 7, comprising the following steps: S1. Acquire intravascular ultrasound image sequence; S2. Preprocessing the image sequence, including denoising, image enhancement and standardization operations; S3. Input the preprocessed image sequence into the spatiotemporal convolutional neural network to perform multi-frame feature extraction and plaque detection to obtain the evolution characteristics of the plaque in time series; S4, input the extracted features into the transfer learning classification submodule to classify and identify the plaque components; S5. Calculate the blood flow wall shear force value at the plaque and obtain the corresponding serum IL-6 value, input both into the risk prediction model, and calculate the plaque rupture risk index; S6. Mark high-risk plaque areas above a preset threshold in the intravascular ultrasound image.

Images