Image registration and plaque identification method based on CCTA and IVUS

Abstract

The invention belongs to the technical field of medical engineering artificial intelligence, and relates to an image registration and plaque identification method based on CCTA and IVUS, and the method comprises the steps: obtaining a CCTA image and an IVUS image, and carrying out the preprocessing of the CCTA image and the IVUS image; carrying out registration on the preprocessed CCTA image and the preprocessed IVUS image; a registration image is obtained, sampling is conducted on the registration image, and sampling blocks are obtained; and inputting the image into a preset plaque component prediction model for identification to obtain the component proportion of the tissue in the coronary artery plaque. According to the method, accurate registration of the CCTA coronary straightening image and the IVUS image is achieved, coronary opening calibration and focus area labeling are combined, and the alignment accuracy of the multi-modal image is greatly improved. The lesion component proportion of the CCTA image is automatically identified by using the plaque component prediction model, accurate quantitative expression of lesion components is realized, and dependence of a traditional method on manual analysis is avoided.

Description

Technical Field

[0001] The invention belongs to the technical field of medical engineering artificial intelligence and relates to an image registration and plaque recognition method based on CCTA and IVUS. Background Art

[0002] Common imaging methods for coronary heart disease are invasive coronary angiography (ICA) and coronary CT angiography (CCTA). Although ICA is the gold standard for the assessment of coronary anatomical stenosis, it only shows the contour of the lumen filled with contrast agents, and indirectly reflects the stenosis of the coronary artery through the images of lumen filling defects. The amount of information obtained by angiography is very limited, and there are inevitable defects in the information provided for the assessment of plaques that cause lumen stenosis. Intravascular Ultrasound (IVUS) can intuitively display the size and properties of plaques and provide some quantitative indicators to evaluate the progression of plaques, and can improve the risk prediction of acute coronary syndrome. However, due to the invasiveness of its examination method, its clinical application is still limited. With the rapid development and innovation of imaging technology and post-processing technology, the status and role of non-invasive CCTA in the diagnosis of coronary heart disease have been continuously improved, and it has now become a first-line tool for coronary heart disease screening. However, compared with IVUS, due to the significant limitations of spatial resolution and image resolution caused by hardware limitations of CT equipment or scanning factors, it ultimately leads to a certain gap between it and IVUS in plaque assessment. On the other hand, the post-processing and diagnosis of traditional CCTA rely on manual processing and interpretation by doctors, which is time-consuming and labor-intensive, and is subject to more interference from human subjective factors. Studies have shown that the average time for diagnostic physicians to post-process images and write reports is 13.6 to 18.0 minutes. Summary of the invention

[0003] The purpose of the present invention is to provide a plaque recognition method and system based on CCTA and IVUS to solve the technical problem in the prior art that the identification of the component proportion of tissue within the plaque of CCTA images is too dependent on manual analysis.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions: In a first aspect, the present application discloses an image registration method based on CCTA and IVUS, comprising: Acquire CCTA images and IVUS images, and preprocess them respectively to obtain preprocessed CCTA images and preprocessed IVUS images; The preprocessed CCTA image is registered with the preprocessed IVUS image to obtain a registration map; The CCTA image and the IVUS image are preprocessed respectively, specifically including: segmenting all the coronary arteries from the CCTA image using a coronary vessel segmentation algorithm, and then straightening the coronary arteries to generate a CPR straightening image; determining various components of the plaque on the IVUS image on the IVUS image, and distinguishing and marking the various components; marking the starting point and length of the same lesion on the IVUS image and the CPR straightening image, respectively, to obtain a preprocessed CCTA image and a preprocessed IVUS image; The registering the preprocessed CCTA image with the preprocessed IVUS image to obtain a registration map specifically includes: registering the preprocessed CCTA image with the preprocessed IVUS image using the marked starting point and length information to obtain a registration map.

[0005] In a second aspect, the present application discloses a plaque identification method based on CCTA and IVUS, comprising: Acquire CCTA images and IVUS images, and preprocess them respectively to obtain preprocessed CCTA images and preprocessed IVUS images; The preprocessed CCTA image is registered with the preprocessed IVUS image; a registration map is obtained, and the registration map is sampled to obtain a sampling block; The sampled blocks are input into a preset plaque composition prediction model for identification to obtain the composition ratio of tissues in the coronary artery plaque; The CCTA image and the IVUS image are preprocessed respectively, specifically including: segmenting all the coronary arteries from the CCTA image using a coronary vessel segmentation algorithm, and then straightening the coronary arteries to generate a CPR straightening image; determining various components of the plaque on the IVUS image on the IVUS image, and distinguishing and marking the various components; marking the starting point and length of the same lesion on the IVUS image and the CPR straightening image, respectively, to obtain a preprocessed CCTA image and a preprocessed IVUS image; The registering the preprocessed CCTA image with the preprocessed IVUS image to obtain a registration map specifically includes: registering the preprocessed CCTA image with the preprocessed IVUS image using the marked starting point and length information to obtain a registration map.

[0006] Preferably, the sampling of the registration image to obtain the sampled blocks is specifically as follows: The registration image is sampled sequentially within the lesion range in units of 1 cm to 5 cm and steps of 0.5 cm to 2 cm to obtain sampling blocks.

[0007] Preferably, the various components in the plaque on the IVUS image include: necrotic core, fibrous tissue, fibrous lipids and dense calcium.

[0008] Preferably, the preset plaque composition prediction model includes: Convolutional neural network, used to extract image features at each layer; Recurrent neural network, used to fuse image features between layers to obtain overall features; The fully connected layer is used to regress the proportion of the overall features and obtain the proportion of tissue components in the coronary artery plaque.

[0009] Preferably, the sampling blocks include: IVUS image blocks and CPR straightening image blocks; the method for constructing the plaque component prediction model includes: The ratio of the volume of each component of the lesion in the IVUS image block to the volume of the blood vessels in the sampled block was calculated, and the ratio was used as the gold standard; The convolutional neural network and the recurrent neural network respectively extract image features and fuse image features on the sampled blocks to obtain the overall features; The overall features are input into the fully connected layer, and the fully connected layer realizes the regression of the proportion of each component in the overall features; the regression result of the component proportion is obtained; The component ratio regression results were compared with the gold standard, and the model loss was calculated through the loss function. The back propagation algorithm was used to iteratively update the parameters of the convolutional neural network, recurrent neural network and fully connected layer according to the model loss until the damage converged to obtain the plaque composition prediction model.

[0010] In a third aspect, the present application discloses a coronary artery plaque identification system based on CCTA and IVUS, comprising: A preprocessing unit, used for acquiring CCTA images and IVUS images, and performing preprocessing respectively to obtain preprocessed CCTA images and preprocessed IVUS images; A registration sampling unit is used to register the preprocessed CCTA image with the preprocessed IVUS image; obtain a registration map, sample the registration map, and obtain a sampling block; An identification unit, used to input the sampled blocks into a preset plaque component prediction model for identification, and obtain the composition ratio of tissues in the coronary artery plaque; The CCTA image and the IVUS image are preprocessed respectively, specifically including: segmenting all the coronary arteries from the CCTA image using a coronary vessel segmentation algorithm, and then straightening the coronary arteries to generate a CPR straightening image; determining various components of the plaque on the IVUS image on the IVUS image, and distinguishing and marking the various components; marking the starting point and length of the same lesion on the IVUS image and the CPR straightening image, respectively, to obtain a preprocessed CCTA image and a preprocessed IVUS image; The registering the preprocessed CCTA image with the preprocessed IVUS image to obtain a registration map specifically includes: registering the preprocessed CCTA image with the preprocessed IVUS image using the marked starting point and length information to obtain a registration map.

[0011] Preferably, the preset plaque composition prediction model includes: Convolutional neural network, used to extract image features at each layer; Recurrent neural network, used to fuse image features between layers to obtain overall features; The fully connected layer is used to regress the proportion of the overall features and obtain the proportion of tissue components in the coronary artery plaque.

[0012] In a fourth aspect, the present application discloses an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above-mentioned CCTA and IVUS-based plaque identification methods when executing the computer program.

[0013] In a fifth aspect, the present application discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned CCTA and IVUS-based plaque identification methods are implemented.

[0014] Compared with the prior art, the present invention has the following beneficial effects: The image registration method based on CCTA and IVUS disclosed in this application effectively improves the registration accuracy of CCTA coronary straightening images and IVUS images. The precise registration of CCTA coronary straightening images and IVUS images, combined with coronary artery opening calibration and lesion area annotation, enables the images of the two modalities to be perfectly aligned, providing the possibility for accurate analysis of the lesion area.

[0015] This application uses a plaque component prediction model to automatically identify the tissue components of CCTA images based on lesion area annotations, and outputs the proportion of each component in the tissue; it improves the doctor's work efficiency; and avoids the reliance of traditional methods on manual analysis. This technology significantly enhances the ability to capture local features and process sequence information. Its efficient model design and component analysis method not only improves efficiency and consistency, but also provides a scientific basis for the diagnosis of coronary heart disease, the formulation of treatment plans, and follow-up, and has strong innovation and clinical applicability.

[0016] Furthermore, the present application uses IVUS component ratio as the gold standard to train the model and provide more reliable recognition results. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 is a flow chart of the image registration method of the present invention; Figure 2 is a flow chart of the plaque identification method of the present invention; Figure 3 This is a structural diagram of the plaque component prediction model of the present invention; Figure 4 is a flow chart of an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0020] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0022] In the description of the embodiments of the present invention, it should be noted that if the terms "upper", "lower", "horizontal", "inner", etc. indicate an orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the product of the invention is usually placed when in use, it is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0023] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", which does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0024] In the description of the embodiments of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0025] The present invention is further described in detail below in conjunction with the accompanying drawings: Deep learning technology is a new field in artificial intelligence (AI) research, focusing on establishing and simulating the human brain's process of analyzing and learning large amounts of data. As a data-driven model, the key to deep learning is to establish a neural network that can simulate the human brain for analysis and learning, and imitate the human brain's mechanism to automatically mine and analyze abstract features at all levels in large amounts of data. Through algorithms, machines can learn rules from large amounts of data, thereby intelligently identifying new samples or making predictions about the future. Therefore, AI systems have the potential to significantly shorten the time for image post-processing and generate diagnostic reports instantly, thereby significantly improving the time efficiency of CCTA image post-processing and diagnosis. While speeding up, the accuracy of CCTA image diagnostic analysis is particularly important.

[0026] This application combines the advantages of non-invasive CCTA and IVUS, the gold standard for plaque assessment, and uses the results of coronary lumen stenosis and plaque feature analysis of intracoronary imaging (IVUS) as the gold standard, combined with AI algorithms, to develop a deep learning intelligent coronary artery stenosis and plaque assessment model based on coronary CTA images. This will improve CCTA's ability to identify features related to coronary atherosclerotic plaques, and further provide a faster, more accurate and objective basis for the clinical diagnosis and precise treatment of coronary heart disease.

[0027] See also Figure 1 The present application discloses an image registration method based on CCTA and IVUS, comprising: Step 1: Obtain CCTA images and IVUS images, and preprocess them respectively to obtain preprocessed CCTA images and preprocessed IVUS images; The CCTA image is segmented into all coronary arteries by a coronary vessel segmentation algorithm, and then the coronary arteries are straightened to generate a CPR straightening image; various components of the plaque on the IVUS image are determined on the IVUS image, and the various components are distinguished and marked; the starting point and length of the same lesion are marked on the IVUS image and the CPR straightening image, respectively, to obtain a preprocessed CCTA image and a preprocessed IVUS image; Step 2: Register the preprocessed CCTA image with the preprocessed IVUS image to obtain a registration map; The pre-processed CCTA image and the pre-processed IVUS image are registered with the marked starting point and length information to obtain a registration map.

[0028] In some embodiments, after acquiring the IVUS image, the various components of the plaque on the IVUS image are annotated manually by a doctor or automatically by a medical image post-processing workstation, and different colored masks are used to distinguish them. At the same time, the position of the coronary artery opening is annotated according to the trend of the IVUS image for subsequent registration with the CCTA image. Then the second step is the acquisition and processing of the CCTA image. The CCTA image segments all the coronary arteries through the coronary vessel segmentation algorithm, and then the coronary artery is straightened to generate a CPR straightening map. After that, the position of the coronary artery opening in the CCTA straightening map is manually marked for alignment with the IVUS image. The next third step is to register the IVUS and CCTA straightening images based on the coronary artery opening information, the IVUS image spacing, and the CCTA straightening image sampling spacing.

[0029] See also Figure 2 , the present application discloses a plaque identification method based on CCTA and IVUS, comprising: S1: acquiring CCTA images and IVUS images, and preprocessing them respectively to obtain preprocessed CCTA images and preprocessed IVUS images; S2: registering the preprocessed CCTA image with the preprocessed IVUS image; obtaining a registration map, sampling the registration map, and obtaining a sampling block; S3: Input the sampled blocks into a preset plaque composition prediction model for identification, and obtain the composition ratio of tissues in the coronary artery plaque; The CCTA image and the IVUS image are preprocessed respectively, specifically including: segmenting all the coronary arteries from the CCTA image using a coronary vessel segmentation algorithm, and then straightening the coronary arteries to generate a CPR straightening image; determining various components of the plaque on the IVUS image on the IVUS image, and distinguishing and marking the various components; marking the starting point and length of the same lesion on the IVUS image and the CPR straightening image, respectively, to obtain a preprocessed CCTA image and a preprocessed IVUS image; The pre-processed CCTA image is registered with the pre-processed IVUS image to obtain a registration map, specifically including: registering the pre-processed CCTA image with the pre-processed IVUS image with the marked starting point and length information to obtain a registration map. This application realizes the precise registration of the CCTA coronary straightening image and the IVUS image after pre-processing, and combines the coronary opening calibration and lesion area annotation to greatly improve the alignment accuracy of the multimodal image. The lesion components of the CCTA image are automatically predicted using the plaque component prediction model, which realizes the accurate quantitative analysis of the lesion area and avoids the reliance of traditional methods on manual analysis. This technology significantly enhances the ability to capture local features and process sequence information. Its efficient model design and component analysis method not only improves efficiency and consistency, but also provides a scientific basis for the diagnosis of coronary heart disease, the formulation of treatment plans and follow-up, and has strong innovation and clinical applicability.

[0030] In some embodiments, a plaque identification method based on CCTA and IVUS includes: S1: Acquire CCTA images and IVUS images, and preprocess them respectively to obtain preprocessed CCTA images and preprocessed IVUS images; specifically, including: S101: Segment the CCTA image into all coronary arteries using a coronary vessel segmentation algorithm, and then straighten the coronary arteries to generate a CPR straightening image; S102: determining various components of the plaque on the IVUS image, and distinguishing and marking the various components; S103: Mark the starting point and length of the same lesion on the IVUS image and the CPR straightening image respectively, and obtain a pre-processed CCTA image and a pre-processed IVUS image.

[0031] S2: registering the preprocessed CCTA image with the preprocessed IVUS image; obtaining a registration map, sampling the registration map, and obtaining a sampling block; specifically: S201: the pre-processed CCTA image is registered with the pre-processed IVUS image; a registration map is obtained, and the registration map is sampled to obtain a sampling block, specifically: S202: registering the pre-processed CCTA image with the pre-processed IVUS image using the marked starting point and length information to obtain a registration map; S203: The registration image is sampled sequentially within the lesion range in units of 1 cm to 5 cm and steps of 0.5 cm to 2 cm to obtain sample blocks.

[0032] S3: Input the sampled blocks into a preset plaque composition prediction model for identification to obtain the composition ratio of tissues within the coronary artery plaque.

[0033] In some embodiments, the various components of the plaque on the IVUS image include: necrotic core, fibrous tissue, fibrolipid and dense calcium.

[0034] In some embodiments, see Figure 3 , the preset plaque component prediction model includes: Convolutional neural network, used to extract image features at each layer; Recurrent neural network, used to fuse image features between layers to obtain overall features; The fully connected layer is used to regress the proportion of the overall features and obtain the proportion of tissue components in the coronary artery plaque.

[0035] In some embodiments, the convolutional neural network CNN uses the ResNet structure. ResNet (Residual Neural Network) is a classic convolutional neural network structure that solves the gradient vanishing and network degradation problems in deep neural networks by introducing residual connections. Its core is the residual block, which directly adds the input data to the output of the convolution layer to allow information to propagate across layers, thereby reducing the difficulty of optimizing deep networks.

[0036] In some embodiments, the recurrent neural network RNN ​​uses the LSTM structure. LSTM is an improved recurrent neural network (RNN) structure. The key is to introduce the memory unit (Cell State) and the gate mechanism (Gate Mechanisms), and dynamically control the transmission of information and memory update through the input gate, forget gate and output gate. LSTM can capture important time dependencies in the input sequence, and is particularly suitable for tasks such as time series analysis and natural language processing. Its stability and accuracy in long sequence data processing are far superior to traditional RNN. In this solution, LSTM is used to analyze the structural changes between consecutive vascular contractions.

[0037] A method for constructing a plaque component prediction model, wherein the sampling and cutting includes: IVUS image cutting and CPR straightening image cutting; the method specifically includes: The ratio of the volume of each component of the lesion in the IVUS image block to the volume of the blood vessels in the sampled block was calculated, and the ratio was used as the gold standard; The convolutional neural network and the recurrent neural network respectively extract image features and fuse image features on the sampled blocks to obtain the overall features; The overall features are input into the fully connected layer, and the fully connected layer realizes the regression of the proportion of each component in the overall features; the regression result of the component proportion is obtained; The regression results of the component ratios were compared with the gold standard, and the model loss was calculated through the loss function. The parameters of the convolutional neural network, recurrent neural network, and fully connected layer were iteratively updated using the back propagation algorithm according to the model loss until the damage converged, and the plaque component prediction model was obtained. The IVUS component ratio was used as the gold standard to train the model, providing more reliable prediction results.

[0038] In some embodiments, the method for constructing the plaque composition prediction model specifically comprises the following steps: The ratio of the volume of each component of the lesion within the IVUS image cut to the volume of the blood vessels within the sampled cut was calculated as the gold standard; The convolutional neural network and the recurrent neural network respectively extract image features and fuse image features on the sampled blocks to obtain the overall features; The overall features are input into the fully connected layer, which realizes the regression of the proportion of each component in the overall features and the final recognition result (the ratio of the volume of each component of the lesion to the volume of the blood vessels in the sampled block); The component ratio regression results output by the fully connected layer are compared with the gold standard, and the loss of the model is calculated through the loss function (loss functions include but are not limited to mean squared error (MSE), mean absolute error (MAE), Huber loss (Huber Loss), weighted loss (Weighted Loss) and Hinge loss (Hinge Loss), etc.), and then the back propagation algorithm is used to update the parameters of the convolutional neural network, recurrent neural network and fully connected layer. Finally, the training of the plaque composition prediction model is completed through multiple iterations.

[0039] In some embodiments, the implementation process of the present application is divided into the following two steps. The first is the registration of CCTA images and IVUS images. The CCTA images segment the blood vessels and obtain the straightened images of each blood vessel through the coronary segmentation algorithm and the coronary straightening algorithm. Then, the registration of the CCTA coronary vessel straightening image and the IVUS image is realized by positioning the coronary opening; then, the proportion of various components of the plaque on the IVUS image is used as a label to predict the components of the plaque on the CCTA image, and finally the plaque component prediction and analysis of each plaque on the CCTA image is obtained. The precise registration of the CCTA coronary straightening image and the IVUS image, combined with the calibration of the coronary opening and the annotation of the lesion area, greatly improves the alignment accuracy of the multimodal images. The plaque component prediction model is used to automatically predict the lesion components of the CCTA image, which realizes the accurate quantitative analysis of the lesion area and avoids the reliance of traditional methods on manual analysis. This technology significantly enhances the ability to capture local features and process sequence information. Its efficient model design and component analysis methods not only improve efficiency and consistency, but also provide a scientific basis for the diagnosis, treatment plan formulation and follow-up of coronary heart disease, and have strong innovation and clinical applicability.

[0040] The doctor manually outlines the scope of the IVUS image and divides it into four components: necrotic core, fibrous tissue, fibrolipid, and dense calcium. CCTA images automatically segment the coronary vessels according to the coronary segmentation algorithm and generate straightened CPR images corresponding to each blood vessel. The image positioning is manually annotated by the doctor, and the starting point and length of the same lesion are marked on the IVUS image and the CPR straightened image, respectively, and then the two images are aligned with the annotated starting point and length information. The data is sampled in units of 1 cm and a step length of 0.5 cm. The samples are sampled sequentially within the lesion range. After the sampled blocks are obtained, the ratio of the volume of the four components of the lesion in the IVUS image block to the volume of the blood vessel in the block is calculated as the gold standard, and the CPR straightened image block data is used as the input data. The model uses convolutional neural network (CNN) + recurrent neural network (RNN) to extract image features and regress the proportions of the four components. The model structure and implementation process are shown in the attached figure. Figure 3 Finally, the Pearson correlation coefficient is used as the evaluation index of the regression results, and the performance of the model in the validation set is used as the evaluation of the regression results.

[0041] In some embodiments, the model uses a convolutional neural network combined with a recurrent neural network structure, where the CNN module uses a RESNET structure and the RNN module uses an LSTM structure. After the input image passes through the convolutional neural network, the features of each layer of the image are extracted, and then sent to the recurrent neural network for feature fusion between layers. Finally, the output of the RNN is sent to the fully connected layer as the overall feature to regress the component proportions.

[0042] ResNet (Residual Neural Network) is a classic convolutional neural network structure. It solves the gradient vanishing and network degradation problems in deep neural networks by introducing residual connections. Its core is the residual block, which directly adds the input data to the output of the convolutional layer to allow information to propagate across layers, thereby reducing the difficulty of optimizing deep networks. The design of ResNet supports very deep network structures (such as ResNet-50, ResNet-101), significantly improving the model's feature extraction capabilities and performance while maintaining training stability. Its high efficiency and robustness make it perform well in tasks such as image classification and regression, and it has become a classic model in the field of deep learning.

[0043] LSTM is an improved recurrent neural network (RNN) structure proposed by Hochreiter and Schmidhuber in 1997. It is designed specifically to solve the long-term dependency and gradient vanishing problems in traditional RNN. The key lies in the introduction of cell state and gate mechanisms, which dynamically control the transmission of information and memory updates through input gates, forget gates, and output gates. LSTM can capture important temporal dependencies in the input sequence, and is particularly suitable for tasks such as time series analysis and natural language processing. Its stability and accuracy in long sequence data processing are far superior to traditional RNN. In this solution, LSTM is used to analyze the structural changes between consecutive vascular stretches.

[0044] In some embodiments, as shown in the attached Figure 4As shown, the implementation process of this embodiment is as follows. First, the first part is the processing of IVUS images. After acquiring the IVUS images, the various components of the plaques on the IVUS images are marked on the IVUS images by manual annotation by a doctor or automatic annotation by a medical image post-processing workstation, and different colors of masks are used for distinction. At the same time, the position of the coronary artery opening is marked according to the trend of the IVUS image for subsequent registration with the CCTA image. Then the second step is the acquisition and processing of CCTA images. The CCTA image segments all coronary arteries through the coronary vessel segmentation algorithm, and then the coronary arteries are straightened to generate a CPR straightening map. After that, the position of the coronary artery opening in the CCTA straightening map is manually marked for alignment with the IVUS image. The next third step is to align the IVUS and CCTA straightening images based on the coronary artery opening information, the IVUS image spacing, and the CCTA straightening image sampling spacing. Specifically, the opening is taken as the starting position, and according to the sampling spacing of the IVUS image spacing and the CCTA straightening image, the sampling spacing of 0.5mm is taken as the target. The two images are resampled using the interpolation method, so as to achieve the registration of the IVUS and CCTA images in the sampling scale; the fourth step is to build and train the plaque component prediction model. The model mainly adopts the (CNN+RNN) structure, and other model structures can also be used, such as 3D CNN model, diffusion model, etc. The input of the model is the CCTA straightening image obtained by sampling after registration. In order to reduce the impact of the error between images on the results, we take 1cm length as a unit and 0.5cm as a step length. We sample in sequence within the range of the lesion containing the plaque. After obtaining a series of sampled blocks, the gold standard predicted by the plaque component prediction model is the ratio of the volume of the four components of the lesion in the IVUS image block to the volume of the blood vessel in the block, and the output result is also the ratio of the four tissue components in the plaque. Finally, the recognition of the proportion of the components of the tissue in the plaque is realized on the CCTA image.

[0045] The present application also discloses a coronary artery plaque identification system based on CCTA and IVUS, comprising: A preprocessing unit, used for acquiring CCTA images and IVUS images, and performing preprocessing respectively to obtain preprocessed CCTA images and preprocessed IVUS images; A registration sampling unit is used to register the preprocessed CCTA image with the preprocessed IVUS image; obtain a registration map, sample the registration map, and obtain a sampling block; The identification unit is used to input the sampled blocks into a preset plaque component prediction model for identification, so as to obtain the composition ratio of the tissue in the coronary artery plaque.

[0046] Further preferably, the plaque component prediction model preset in the coronary artery plaque identification system based on CCTA and IVUS includes: Convolutional neural network, used to extract image features at each layer; Recurrent neural network, used to fuse image features between layers to obtain overall features; The fully connected layer is used to regress the proportion of the overall features and obtain the proportion of tissue components in the coronary artery plaque.

[0047] The present application also discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the CCTA and IVUS-based plaque identification method described in any one of the above items are implemented.

[0048] The present application also discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned CCTA and IVUS-based plaque identification methods are implemented.

[0049] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0050] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0051] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.

[0052] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0053] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A CCTA and IVUS-based image registration method, characterized in that: include: Acquire CCTA images and IVUS images, and preprocess them respectively to obtain preprocessed CCTA images and preprocessed IVUS images; The preprocessed CCTA image is registered with the preprocessed IVUS image to obtain a registration map; The CCTA image and the IVUS image are preprocessed respectively, specifically including: segmenting all the coronary arteries from the CCTA image using a coronary vessel segmentation algorithm, and then straightening the coronary arteries to generate a CPR straightening image; determining various components of the plaque on the IVUS image on the IVUS image, and distinguishing and marking the various components; marking the starting point and length of the same lesion on the IVUS image and the CPR straightening image, respectively, to obtain a preprocessed CCTA image and a preprocessed IVUS image; The registering the preprocessed CCTA image with the preprocessed IVUS image to obtain a registration map specifically includes: registering the preprocessed CCTA image with the preprocessed IVUS image using the marked starting point and length information to obtain a registration map.

2. A plaque identification method based on CCTA and IVUS, characterized in that: include: Acquire CCTA images and IVUS images, and preprocess them respectively to obtain preprocessed CCTA images and preprocessed IVUS images; The preprocessed CCTA image is registered with the preprocessed IVUS image to obtain a registration map; the registration map is sampled to obtain a sampling block; The sampled blocks are input into a preset plaque composition prediction model for identification to obtain the composition ratio of tissues in the coronary artery plaque; The CCTA image and the IVUS image are preprocessed respectively, specifically including: segmenting all the coronary arteries from the CCTA image using a coronary vessel segmentation algorithm, and then straightening the coronary arteries to generate a CPR straightening image; determining various components of the plaque on the IVUS image on the IVUS image, and distinguishing and marking the various components; marking the starting point and length of the same lesion on the IVUS image and the CPR straightening image, respectively, to obtain a preprocessed CCTA image and a preprocessed IVUS image; The registering the preprocessed CCTA image with the preprocessed IVUS image to obtain a registration map specifically includes: registering the preprocessed CCTA image with the preprocessed IVUS image using the marked starting point and length information to obtain a registration map.

3. The CCTA and IVUS-based plaque identification method according to claim 2, characterized in that: The sampling of the registration image to obtain the sampling block is specifically as follows: The registration image is sampled sequentially within the lesion range in units of 1 cm to 5 cm and steps of 0.5 cm to 2 cm to obtain sampling blocks.

4. The CCTA and IVUS-based plaque identification method according to claim 2, characterized in that: The various components of the plaque on the IVUS image include: necrotic core, fibrous tissue, fibrolipid and dense calcium.

5. The plaque identification method based on CCTA and IVUS according to claim 2, characterized in that: The preset plaque component prediction model includes: Convolutional neural network, used to extract image features at each layer; Recurrent neural network, used to fuse image features between layers to obtain overall features; The fully connected layer is used to regress the proportion of the overall features and obtain the proportion of tissue components in the coronary artery plaque.

6. The CCTA and IVUS-based plaque identification method according to claim 5, characterized in that: The sampling blocks include: IVUS image blocks and CPR straightening image blocks; the method for constructing the plaque component prediction model includes: The ratio of the volume of each component of the lesion in the IVUS image block to the volume of the blood vessels in the sampled block was calculated, and the ratio was used as the gold standard; The convolutional neural network and the recurrent neural network respectively extract image features and fuse image features on the sampled blocks to obtain the overall features; The overall features are input into the fully connected layer, and the fully connected layer realizes the regression of the proportion of each component in the overall features; the regression result of the component proportion is obtained; The component ratio regression results were compared with the gold standard, and the model loss was calculated through the loss function. The back propagation algorithm was used to iteratively update the parameters of the convolutional neural network, recurrent neural network and fully connected layer according to the model loss until the damage converged to obtain the plaque composition prediction model.

7. A coronary artery plaque identification system based on CCTA and IVUS, characterized in that: include: A preprocessing unit, used for acquiring CCTA images and IVUS images, and performing preprocessing respectively to obtain preprocessed CCTA images and preprocessed IVUS images; A registration sampling unit is used to register the preprocessed CCTA image with the preprocessed IVUS image to obtain a registration map, and to sample the registration map to obtain a sampling block; An identification unit, used to input the sampled blocks into a preset plaque component prediction model for identification, and obtain the composition ratio of tissues in the coronary artery plaque; The CCTA image and the IVUS image are preprocessed respectively, specifically including: segmenting all the coronary arteries from the CCTA image using a coronary vessel segmentation algorithm, and then straightening the coronary arteries to generate a CPR straightening image; determining various components of the plaque on the IVUS image on the IVUS image, and distinguishing and marking the various components; marking the starting point and length of the same lesion on the IVUS image and the CPR straightening image, respectively, to obtain a preprocessed CCTA image and a preprocessed IVUS image; The registering the preprocessed CCTA image with the preprocessed IVUS image to obtain a registration map specifically includes: registering the preprocessed CCTA image with the preprocessed IVUS image using the marked starting point and length information to obtain a registration map.

8. The coronary artery plaque identification system based on CCTA and IVUS according to claim 7, characterized in that: The preset plaque component prediction model includes: Convolutional neural network, used to extract image features at each layer; Recurrent neural network, used to fuse image features between layers to obtain overall features; The fully connected layer is used to regress the proportion of the overall features and obtain the proportion of tissue components in the coronary artery plaque.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the CCTA and IVUS-based plaque identification method as described in any one of claims 2 to 6 when executing the computer program.

10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the CCTA and IVUS-based plaque identification method according to any one of claims 2 to 6.

Images