IVUS and DSA registration method and device and storage medium

Abstract

The invention relates to an IVUS and DSA registration method and device and a storage medium. The method comprises the following steps: respectively loading IVUS image data and DSA image data of an interested blood vessel segment; respectively carrying out feature recognition on the image of the IVUS image data and the image of the DSA image data through a recognition model to obtain the features of the IVUS image data and the features of the DSA image data; and according to the characteristics of the IVUS image data and the characteristics of the DSA image data, registering the IVUS image data and the DSA image data to obtain a registered image. According to the scheme provided by the invention, the full-automatic and high-precision registration of the IVUS and the DSA can be realized, and the robustness of the registration of the IVUS and the DSA is improved.

Description

Technical Field

[0001] This application relates to the field of medical device technology, and in particular to a method, apparatus and storage medium for registering IVUS and DSA. Background Technology

[0002] Intravascular ultrasound (IVUS) is an ultrasound-based imaging technique that uses a miniature ultrasound probe inserted into a blood vessel. The probe performs a 360° scan within the vessel lumen, acquiring high-resolution cross-sectional three-dimensional images of the vessel wall. IVUS provides detailed structural information about the vessel wall, such as plaque composition and distribution, and vessel wall thickness, allowing doctors to better assess diseased vessels. Furthermore, IVUS can acquire real-time three-dimensional images of the blood vessel, facilitating dynamic observation and guidance during surgery. However, IVUS only provides three-dimensional images of the local blood vessels around the probe, resulting in a limited field of view and making it difficult to comprehensively assess the entire vascular system and precisely locate lesions within the vessel.

[0003] Digital subtraction angiography (DSA) is an X-ray-based imaging technique that uses contrast agent injection and digital subtraction to obtain two-dimensional images of blood vessels. DSA can clearly show the morphology and blood flow of blood vessels and is often used to assess lesions such as vascular stenosis and aneurysms. However, DSA's two-dimensional images lack three-dimensional structural information and cannot provide precise details about vascular lesions, making it difficult for doctors to accurately assess the severity of the lesions.

[0004] IVUS and DSA each have their own advantages and disadvantages, and are suitable for different clinical applications. In clinical practice, IVUS and DSA are often used in combination to leverage their respective strengths, providing more comprehensive information on vascular lesions and helping doctors develop more accurate diagnostic and treatment plans.

[0005] In related technologies, registration techniques are used to align IVUS and DSA images, combining the advantages of both to provide more comprehensive information on vascular lesions and help doctors develop more accurate diagnostic and treatment plans. However, the registration process for IVUS and DSA using these technologies is computationally complex and lacks automation. Furthermore, the registration process for IVUS and DSA using these technologies has poor robustness and low accuracy. Summary of the Invention

[0006] To address or partially address the problems existing in related technologies, this application provides an IVUS and DSA registration method, apparatus, and storage medium, which can achieve fully automatic and high-precision IVUS and DSA registration, thereby improving the robustness of IVUS and DSA registration.

[0007] The first aspect of this application provides a method for registering IVUS and DSA, the method comprising: Load the IVUS and DSA image data of the vessel segment of interest, respectively; The features of the IVUS image data and the DSA image data are obtained by performing feature recognition on the images of the IVUS image data and the DSA image data respectively through the recognition model. Based on the characteristics of the IVUS image data and the DSA image data, the IVUS image data and the DSA image data are registered to obtain a registered image.

[0008] Preferably, the step of performing feature recognition on the images of the IVUS image data and the images of the DSA image data respectively through a recognition model to obtain the features of the IVUS image data and the features of the DSA image data includes: The recognition model is used to perform feature recognition on the IVUS image data to identify the IVUS catheter in the IVUS image data, thereby obtaining the IVUS catheter image and IVUS vessel image of the IVUS image data. The recognition model is used to perform feature recognition on the DSA image data to identify the DSA catheter in the DSA image data, thereby obtaining the DSA catheter image and DSA blood vessel image of the DSA image data. The step of registering the IVUS image data and the DSA image data based on the features of the IVUS image data and the features of the DSA image data to obtain a registered image includes: Based on the IVUS catheter image and IVUS vessel image of the IVUS image data, and the DSA catheter image and DSA vessel image of the DSA image data, the IVUS image data is registered with the DSA image data to obtain the registered image.

[0009] Preferably, the step of registering the IVUS image data with the DSA image data based on the IVUS catheter image and IVUS vessel image of the IVUS image data, and the DSA catheter image and DSA vessel image of the DSA image data, to obtain the registered image, includes: Based on the IVUS catheter image and IVUS vessel image of the IVUS image data, the first frame IVUS vessel image, the second frame IVUS vessel image, the third frame IVUS vessel image, ..., the (i-1)th frame IVUS vessel image, and the i-th frame IVUS catheter image of the IVUS image data are obtained, wherein the IVUS image data has I frames of IVUS images, and the i-th frame IVUS image is the i-th frame IVUS catheter image; Based on the DSA catheter image and DSA vessel image of the DSA image data, obtain the coordinates (x1, y1), (x2, y2), (x3, y3), ..., (x... j-1 y j-1 ), (x j y j ), where the (x) j y j () represents the coordinates of the DSA catheter at the catheter orifice; Based on the first frame IVUS vascular image, the second frame IVUS vascular image, the third frame IVUS vascular image, ..., the (i-1)th frame IVUS vascular image, the i-th frame IVUS catheter image, and the coordinates of the vessel centerline of the DSA image data (x1, y1), (x2, y2), (x3, y3), ..., (x... j-1 y j-1 ), (x j y j The IVUS image data is registered with the DSA image data to obtain the registered image.

[0010] Preferably, the step of performing feature recognition on the images of the IVUS image data and the images of the DSA image data respectively through a recognition model to obtain the features of the IVUS image data and the features of the DSA image data further includes: The recognition model is used to perform feature recognition on the IVUS image data to identify IVUS vessel branch points and IVUS vessel stenosis in the IVUS image data, thereby obtaining the location of IVUS vessel branch points and IVUS vessel stenosis in the IVUS image data. The recognition model is used to perform feature recognition on the DSA image data to identify DSA vascular branch points and DSA vascular stenosis in the DSA image data, thereby obtaining the location of DSA vascular branch points and DSA vascular stenosis in the DSA image data.

[0011] Preferably, the method further includes: Based on the registered image, the feature matching error between the features of the IVUS image data and the corresponding features of the DSA image data in the registered image is obtained. The feature matching error includes one or more of the following: catheter position matching error, vascular branch point position matching error, and vascular stenosis position matching error. Based on the feature matching error, adjust the registration of the IVUS image data and the DSA image data in the registered image.

[0012] Preferably, adjusting the registration of the IVUS image data and the DSA image data in the registered image based on the feature matching error includes: An energy function is constructed based on the catheter position matching error, the vascular branch point position matching error, and the vascular stenosis position matching error. The registration of the IVUS image data and the DSA image data in the registered image is adjusted by minimizing the function value of the energy function.

[0013] Preferably, the step of constructing the energy function based on the catheter position matching error, the vessel branch point position matching error, and the vessel stenosis position matching error includes: The energy function is constructed based on the catheter position matching error, the vascular branch point position matching error, the vascular stenosis position matching error, and the weight parameters of the catheter position matching error, the vascular branch point position matching error, and the vascular stenosis position matching error.

[0014] Preferably, the step of performing feature recognition on the images of the IVUS image data and the images of the DSA image data respectively through a recognition model to obtain the features of the IVUS image data and the features of the DSA image data further includes: The IVUS image data is subjected to noise reduction preprocessing; and / or, The DSA image data is then optimized and preprocessed.

[0015] A second aspect of this application provides a registration apparatus for IVUS and DSA, the registration apparatus comprising: Processor; and A memory that stores executable code, which, when executed by the processor, causes the processor to perform the method described above.

[0016] A third aspect of this application provides a computer-readable storage medium having executable code stored thereon, which, when executed by a processor, causes the processor to perform the method described above.

[0017] The technical solution provided in this application may include the following beneficial effects: The technical solution of this application performs feature recognition on the images of IVUS image data and DSA image data through a recognition model to obtain the features of IVUS image data and DSA image data respectively; based on the features of IVUS image data and DSA image data, the IVUS image data and DSA image data are registered to obtain a registered image; it can achieve fully automatic and high-precision IVUS and DSA registration and improve the robustness of IVUS and DSA registration.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0019] The above and other objects, features and advantages of this application will become more apparent from the more detailed description of exemplary embodiments thereof in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments thereof.

[0020] Figure 1 This is a schematic flowchart illustrating the IVUS and DSA registration method in an embodiment of this application; Figure 2 This is another schematic diagram of the IVUS and DSA registration method shown in the embodiments of this application; Figure 3 This is a schematic diagram of the vertical axis image of IVUS image data of the IVUS-DSA registration method shown in the embodiments of this application; Figure 4 This is a schematic diagram of the segmentation of the intima and media of blood vessels in IVUS image data of the IVUS and DSA registration method shown in the embodiments of this application. Figure 5 This is a schematic diagram of a frame of DSA angiography image of the DSA image data of the IVUS and DSA registration method shown in the embodiments of this application; Figure 6 yes Figure 5 A schematic diagram of the binary image of blood vessels corresponding to the DSA angiography image; Figure 7 It is based on Figure 5 A schematic diagram of the binary image of the blood vessel corresponding to the DSA catheter in the DSA angiography image; Figure 8 This is a schematic diagram of the registration image of the IVUS and DSA registration method shown in the embodiments of this application; Figure 9 This is a schematic diagram of the IVUS and DSA registration device shown in the embodiments of this application. Detailed Implementation

[0021] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to make this application more thorough and complete, and to fully convey the scope of this application to those skilled in the art.

[0022] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0023] It should be understood that although the terms "first," "second," "third," etc., may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0024] This application provides a registration method for IVUS and DSA, which can achieve fully automatic and high-precision IVUS and DSA registration and improve the robustness of IVUS and DSA registration.

[0025] The technical solutions of the embodiments of this application are described in detail below with reference to the accompanying drawings.

[0026] Figure 1 This is a schematic flowchart illustrating the IVUS and DSA registration method in an embodiment of this application.

[0027] See Figure 1 A method for registering IVUS and DSA, comprising: Step 101: Load the IVUS image data and DSA image data of the vessel segment of interest, respectively.

[0028] In one embodiment, three-dimensional (3D) IVUS image data of the vessel segment of interest can be loaded from an IVUS device or storage system; two-dimensional (2D) DSA image data of the vessel segment of interest can be obtained from a DSA device or storage system.

[0029] Step 102: Feature recognition is performed on the images of IVUS image data and DSA image data respectively using the recognition model to obtain the features of IVUS image data and DSA image data.

[0030] In one embodiment, the recognition model includes, but is not limited to, classifiers and deep learning models.

[0031] In one embodiment, a recognition model can be used to perform feature recognition on the images of three-dimensional IVUS image data, identifying features such as IVUS catheters and IVUS vessels in the images of three-dimensional IVUS image data.

[0032] In one embodiment, a recognition model can be used to perform feature recognition on the two-dimensional DSA image data to identify features such as DSA catheters and DSA blood vessels in the two-dimensional DSA image data.

[0033] Step 103: Based on the characteristics of the IVUS image data and the DSA image data, register the IVUS image data and the DSA image data to obtain the registered image.

[0034] In one embodiment, based on the features of the IVUS catheter, IVUS blood vessels, etc. in the three-dimensional IVUS image data, an IVUS feature image containing features and an IVUS non-feature image without features can be obtained; based on the features of the DSA catheter, DSA blood vessels, etc. in the two-dimensional DSA image data, a DSA feature image containing features and a DSA non-feature image without features can be obtained; based on the IVUS feature image, IVUS non-feature image, and DSA feature image and DSA non-feature image, the IVUS image data and DSA image data can be registered to obtain a registered image including the registered IVUS image data and DSA image data.

[0035] The IVUS and DSA registration method of this application performs feature recognition on the images of IVUS image data and DSA image data respectively through a recognition model to obtain the features of IVUS image data and DSA image data; based on the features of IVUS image data and DSA image data, the IVUS image data and DSA image data are registered to obtain a registered image; it can achieve fully automatic and high-precision IVUS and DSA registration, and improve the robustness of IVUS and DSA registration.

[0036] Figure 2 This is another schematic diagram of the IVUS and DSA registration method shown in the embodiments of this application. Figure 2 Compared to Figure 1 The technical solution of this application is described in more detail.

[0037] See Figure 2 A method for registering IVUS and DSA, comprising: Step 201: Load the IVUS image data and DSA image data of the vessel segment of interest, respectively.

[0038] In one embodiment, three-dimensional IVUS image data of a vessel segment of interest can be acquired from an IVUS device or storage system. The three-dimensional IVUS image data of the vessel segment of interest is obtained via a three-dimensional ultrasound image of the vessel segment of interest acquired through an ultrasound probe. Figure 3 This is a schematic diagram of the vertical axis image of IVUS image data in the IVUS-DSA registration method shown in the embodiments of this application.

[0039] join Figure 3 The three-dimensional IVUS image data of the segment of interest includes the longitudinal image 301 and the cross-sectional image of the segment of interest.

[0040] In one embodiment, two-dimensional DSA image data of the vessel segment of interest can be acquired from a DSA device or storage system. The two-dimensional DSA image data of the vessel segment of interest is two-dimensional X-ray image data of the vessel segment of interest.

[0041] In one embodiment, the segment of interest can be a portion of a blood vessel, such as a coronary artery, or it can be the entire vessel.

[0042] In one embodiment, the integrity, consistency, data quality, and data format of the loaded IVUS image data and DSA image data can be verified separately to ensure the integrity, consistency, and data quality of the IVUS image data and DSA image data, and to ensure that the data formats of the IVUS image data and DSA image data are consistent.

[0043] In one embodiment, the integrity of the 3D IVUS image data means that the 3D IVUS image data is not lost or damaged during the acquisition and transmission process, and all necessary information is completely recorded. The 3D IVUS image data includes information such as the image dimension, resolution, intensity value of each pixel or voxel, image acquisition parameters, device settings, and timestamps.

[0044] In one embodiment, the consistency of 3D IVUS image data refers to the consistency between the various parts within the 3D IVUS image data and the relationships between the 3D IVUS image data. In the 3D IVUS image data, each slice or layer should be correctly aligned and have coherence, ensuring that their spatial relationship meets expectations. The relationships between data should also be free of contradictions. For example, the metadata of the image and the actual image content should be consistent, ensuring that the entire 3D IVUS image data can correctly and coherently reflect the actual situation.

[0045] In one embodiment, the data quality of 3D IVUS image data refers to the image sharpness, resolution, and contrast.

[0046] In one embodiment, the data format of the three-dimensional IVUS image data can be the internationally recognized DICOM (Digital Imaging and Communications in Medicine) file format, or it can be an internally encrypted data structure file containing information such as the dimensions and pixels of the image.

[0047] In one embodiment, a data verification tool can be used to verify the three-dimensional IVUS image data of the vessel segment of interest. By checking the hash value or integrity check code of the three-dimensional IVUS image data, it can be verified that the three-dimensional IVUS image data has not been lost, modified or damaged during the loading process, and the integrity and consistency of the three-dimensional IVUS image data of the vessel segment of interest can be checked to ensure the integrity and consistency of the three-dimensional IVUS image data.

[0048] In one embodiment, image analysis software can be used to check whether the image sharpness, contrast, noise, and other indicators in the three-dimensional IVUS image data of the blood vessel segment of interest meet the set standards. Through resolution testing, the spatial resolution of the image in the three-dimensional IVUS image data is measured to check whether the spatial resolution of the image in the three-dimensional IVUS image data meets the set resolution, ensuring that the data structure of the three-dimensional IVUS image data is correct and the data quality of the three-dimensional IVUS image data meets the expected standards.

[0049] In one embodiment, if the data format of the three-dimensional IVUS image data of the segment of interest is DICOM file format, the data format of the three-dimensional IVUS image data can be checked by an inspector (e.g., a DICOM inspector) to ensure that the data format of the three-dimensional IVUS image data conforms to the DICOM format specification.

[0050] In one embodiment, if the data format of the three-dimensional IVUS image data of the segment of interest is an internally encrypted data structure file, then a check is performed according to the internally designed data format to ensure that the data format of the three-dimensional IVUS image data conforms to the internally designed data format.

[0051] In one embodiment, the data format of 3D IVUS image data in the data format of an internally encrypted data structure file can be converted to DICOM file format; then the integrity, consistency, data quality and data format of the DICOM file format 3D IVUS image data can be verified.

[0052] In one embodiment, the integrity of two-dimensional DSA image data refers to the fact that the image data is not lost, damaged, or tampered with during the acquisition, transmission, and storage process, including the integrity of the spatial resolution, grayscale level, and pixel data of the image data.

[0053] In one embodiment, the consistency of two-dimensional DSA image data refers to the consistency of the internal logic and structure of the two-dimensional DSA image data. For example, the contrast and brightness of the images in the two-dimensional DSA image data should meet expectations, there should be no contradictions between the images and their metadata (such as patient information and imaging parameters), and the image data should remain consistent across different views or processing steps.

[0054] In one embodiment, the data quality of two-dimensional DSA image data includes the resolution and contrast of the image. The resolution of the image in two-dimensional DSA image data is the spatial resolution of the image, i.e., the sharpness of image details. High-resolution images can display more details but require more storage space. The contrast of the image in two-dimensional DSA image data is the brightness difference between different structures in the image. Images with high contrast can more clearly distinguish different tissues or structures.

[0055] In one embodiment, the data format of the two-dimensional DSA image data may be DICOM format.

[0056] In one embodiment, a data verification tool can be used to verify the two-dimensional DSA image data of the vessel segment of interest. By checking the hash value or integrity check code of the two-dimensional DSA image data, it can be verified that the two-dimensional DSA image data has not been lost, modified or damaged during the loading process, and the integrity and consistency of the two-dimensional DSA image data of the vessel segment of interest can be checked. This ensures that all necessary image information is accurately preserved and that the images in the two-dimensional DSA image data reflect accurate diagnostic information, thus ensuring the integrity and consistency of the two-dimensional DSA image data.

[0057] In one embodiment, the data format of the two-dimensional DSA image data can be checked by an inspector (e.g., a DICOM inspector) to ensure that the data format of the two-dimensional DSA image data conforms to the DICOM format specification.

[0058] In one embodiment, image analysis software can be used to check whether the image sharpness, contrast, noise and other indicators in the two-dimensional DSA image data meet the set standards. Through resolution testing, the spatial resolution of the image in the two-dimensional DSA image data is measured to check whether the spatial resolution of the image in the two-dimensional DSA image data meets the set resolution, thereby ensuring that the data structure of the two-dimensional DSA image data is correct and the data quality of the two-dimensional DSA image data meets the expected standards.

[0059] Step 202: Perform noise reduction preprocessing on the IVUS image data of the vessel segment of interest.

[0060] In one embodiment, each frame of IVUS image data of the vessel segment of interest can be preprocessed with noise reduction to reduce the impact of noise (e.g., speckle noise) on the vessel segment of interest in the IVUS image data, enhance the contrast and detail of each frame of the IVUS image data, and improve the data quality of each frame of the IVUS image data.

[0061] In one embodiment, the noise reduction preprocessing performed on the IVUS image data of the vessel segment of interest may include, but is not limited to, temporal noise reduction and spatial filtering.

[0062] In one embodiment, temporal denoising can be performed on each frame of the 3D IVUS image data of the segment of interest, and spatial filtering can be performed on each frame of the 3D IVUS image data of the segment of interest to remove random noise, speckle noise and other noise from each frame of the 3D IVUS image data, thereby enhancing the contrast and detail of each frame of the 3D IVUS image data, effectively improving the data quality of each frame of the 3D IVUS image data, and effectively reducing the impact of noise on the segment of interest in the 3D IVUS image data.

[0063] In one embodiment, temporal denoising can be performed on each frame of the 3D IVUS image data of the vessel segment of interest, and then spatial filtering can be performed on each frame of the temporally denoised 3D IVUS image data to improve the spatial filtering effect, thereby better enhancing the contrast and detail of each frame of the 3D IVUS image data of the vessel segment of interest, more effectively improving the data quality of each frame of the 3D IVUS image data, and more effectively reducing the impact of noise on the vessel segment of interest in the 3D IVUS image data.

[0064] In one embodiment, time-domain noise reduction may include one or more of moving average filtering, median filtering, and Gaussian filtering.

[0065] In one embodiment, a moving average filter can be applied to each frame of the 3D IVUS image data of the segment of interest, and a sliding window can be used to calculate the average value of the region surrounding each pixel in each frame of the 3D IVUS image data, in order to reduce noise and smooth each frame of the 3D IVUS image data.

[0066] In one embodiment, median filtering can be applied to each frame of the three-dimensional IVUS image data of the segment of interest, and each pixel of each frame of the three-dimensional IVUS image data can be replaced with the median of its neighborhood to effectively remove speckle noise from each frame of the three-dimensional IVUS image data and preserve the edges and details of each frame of the three-dimensional IVUS image data, avoiding blurring.

[0067] In one embodiment, Gaussian filtering can be applied to each frame of the three-dimensional IVUS image data of the segment of interest, and a Gaussian function can be applied to perform a weighted average of each pixel of each frame of the three-dimensional IVUS image data to reduce random noise in each frame of the three-dimensional IVUS image data.

[0068] In one embodiment, spatial filtering may include mean filtering and / or edge-preserving filtering.

[0069] In one embodiment, a mean filter can be used to perform mean filtering on each frame of the 3D IVUS image data of the segment of interest, and to smooth the pixels of each frame of the 3D IVUS image data, so as to reduce the impact of noise on the segment of interest in each frame of the 3D IVUS image data.

[0070] In one embodiment, a bilateral filter can be used to perform edge-preserving filtering on each frame of the 3D IVUS image data of the blood vessel segment of interest, in order to reduce noise in each frame of the 3D IVUS image data, smooth each frame of the 3D IVUS image data, and at the same time preserve the edge information of each frame of the 3D IVUS image data.

[0071] Step 203: Perform optimization preprocessing on the DSA image data of the vessel segment of interest.

[0072] In one embodiment, each frame of DSA image data of the blood vessel segment of interest can be optimized and preprocessed. The optimization and preprocessing includes, but is not limited to, filtering preprocessing and contrast enhancement preprocessing, in order to remove noise from each frame of DSA image data, enhance the contrast of each frame of DSA image data, and improve the data quality of each frame of DSA image data.

[0073] In one embodiment, each frame of the two-dimensional DSA image data of the segment of interest can be preprocessed with filtering, and then each frame of the preprocessed two-dimensional DSA image data can be preprocessed with contrast enhancement to remove high-frequency noise, salt-and-pepper noise and other noise from each frame of the two-dimensional DSA image data, enhance the contrast of each frame of the two-dimensional DSA image data, reduce the impact of noise on the segment of interest in each frame of the two-dimensional DSA image data, retain the edge information of each frame of the two-dimensional DSA image data, and improve the clarity of each frame of the two-dimensional DSA image data.

[0074] In one embodiment, the filtering preprocessing performed on each frame of the two-dimensional DSA image data includes Gaussian filtering preprocessing and / or median filtering preprocessing.

[0075] In one embodiment, a Gaussian filter can be used to preprocess each frame of the two-dimensional DSA image data of the segment of interest by Gaussian filtering, so as to reduce high-frequency noise and other noise in each frame of the two-dimensional DSA image data, remove the influence of noise on the segment of interest in each frame of the two-dimensional DSA image data, smooth each frame of the two-dimensional DSA image data, improve the clarity of each frame of the two-dimensional DSA image data, and retain the edge detail information of each frame of the two-dimensional DSA image data.

[0076] In one embodiment, median filtering preprocessing can be performed on each frame of the two-dimensional DSA image data of the segment of interest to reduce noise such as salt-and-pepper noise in each frame of the two-dimensional DSA image data, remove the influence of noise on the segment of interest in each frame of the two-dimensional DSA image data, smooth each frame of the two-dimensional DSA image data, improve the clarity of each frame of the two-dimensional DSA image data, and retain the edge detail information of each frame of the two-dimensional DSA image data.

[0077] In one embodiment, histogram equalization can be used to perform contrast enhancement preprocessing on each frame of the two-dimensional DSA image data. Based on the grayscale histogram of each frame of the two-dimensional DSA image data, the grayscale histogram of each frame of the two-dimensional DSA image data is adjusted to enhance the overall contrast of each frame of the two-dimensional DSA image data and improve the clarity of each frame of the two-dimensional DSA image data.

[0078] Step 204: The IVUS image data of the segment of interest is characterized by feature recognition through the recognition model, and the IVUS catheter, IVUS catheter image and IVUS vessel image of the IVUS image data are identified, as well as the IVUS vessel branch points and IVUS vessel stenosis of the segment of interest.

[0079] In one embodiment, the IVUS image data can be characterized by a recognition model to identify the IVUS catheter in the IVUS image data, thereby obtaining the IVUS catheter image and the IVUS vessel image.

[0080] In one embodiment, the acquisition process of three-dimensional IVUS image data is determined by whether the IUVS probe has been retracted to the guide catheter orifice to determine whether the acquisition of three-dimensional IVUS image data is to be terminated. Each frame of IVUS image data acquired in the guide catheter is an IVUS catheter image, and each frame of IVUS image acquired in the blood vessel is an IVUS blood vessel image.

[0081] In one embodiment, the recognition model includes, but is not limited to, classifiers obtained through pre-training and deep learning models. Classifiers include, but are not limited to, SVM (Support Vector Machine) and random forest; deep learning models include, but are not limited to, convolutional neural networks (CNN).

[0082] In one embodiment, the recognition model can be pre-trained using IVUS catheter images and vascular images. Through pre-training, the pre-trained recognition model can identify IVUS catheters in IVUS images and classify IVUS images into IVUS catheter images and IVUS vascular images.

[0083] In one embodiment, an IVUS catheter can be identified in each frame of the three-dimensional IVUS image data of the vessel segment of interest (a cross-sectional image of the vessel segment of interest) using a pre-trained recognition model. The model identifies whether each frame of the three-dimensional IVUS image data contains an IVUS catheter, classifies the images of the three-dimensional IVUS image data into IVUS catheter images containing an IVUS catheter during IVUS retraction and IVUS vessel images not containing an IVUS catheter during IVUS retraction, and identifies the IVUS catheter in each frame of the IVUS catheter image to obtain the IVUS catheter position in each frame of the IVUS catheter image. In one embodiment, the IVUS image data of the segment of interest can be characterized by feature recognition using a recognition model to identify IVUS vessel branch points and IVUS vessel stenosis in the segment of interest, thereby obtaining the location of IVUS vessel branch points and IVUS vessel stenosis in the segment of interest.

[0084] In one embodiment, vascular branches appear as smaller vessels connected to the aorta. In images of three-dimensional IVUS data of the segment of interest, vascular branches appear as collateral branches protruding from the aortic wall.

[0085] In one embodiment, a pre-trained recognition model can be used to identify the vascular branches in each frame of the 3D IVUS image data, identify which images in the 3D IVUS image data contain vascular branches and which images do not, classify the images in the 3D IVUS image data, obtain IVUS vascular branch images containing vascular branches and IVUS non-vascular branch images not containing vascular branches, identify the IVUS vascular branch points in the IVUS vascular branch images, obtain the IVUS vascular branch points in each frame of the IVUS vascular branch images, and obtain the IVUS vascular branch point positions in each frame of the IVUS vascular branch images.

[0086] In one embodiment, the IVUS vascular branch point position in each frame of IVUS vascular branch image can be the distance p between each frame of IVUS vascular branch image (assuming there are N frames of 3D IVUS image data, where n represents the frame number, the nth frame of the 3D IVUS image data) and the starting frame (the first frame of the 3D IVUS image data), where, ; In the formula, n represents the frame number of the nth frame of the three-dimensional IVUS image data in which the IVUS vascular branch image is; a represents the retraction speed of the IVUS catheter, in mm / s (millimeters per second); b represents the retraction frame rate, in f / s (frames per second).

[0087] In one embodiment, the IVUS stenosis of the segment of interest can be either a stenotic segment of the segment of interest or a segment of the vessel with vascular disease.

[0088] Figure 4 This is a schematic diagram of the segmentation of the intima and media of blood vessels in IVUS image data of the IVUS and DSA registration method shown in the embodiments of this application.

[0089] In one embodiment, participants Figure 4 A pre-trained recognition model can be used to identify blood vessels in each frame of a 3D IVUS image of a segment of interest. This model can then segment the blood vessels into the intima and media of each frame, obtaining the blood vessel details for each frame of the 3D IVUS image. Figure 4The intima 401 and media 402 are shown. Based on the intima 401 and media 402 of each frame of the three-dimensional IVUS image data, the intima area and media area of ​​the blood vessel in each frame of the three-dimensional IVUS image data are calculated. Based on the intima area and media area of ​​the blood vessel in each frame of the three-dimensional IVUS image data, the degree of vascular stenosis or vascular lesion in each frame of the three-dimensional IVUS image data is calculated. Based on the degree of vascular stenosis or vascular lesion in each frame of the three-dimensional IVUS image data, IVUS vascular stenosis images including vascular stenosis or vascular lesion and IVUS non-vascular stenosis images excluding vascular stenosis and vascular lesion are obtained. The location of IVUS vascular stenosis in each frame of the IVUS vascular stenosis image is identified.

[0090] Step 205: The feature recognition model is used to identify the DSA image data of the segment of interest, and to identify the DSA catheter, DSA catheter image and DSA vessel image of the DSA image data, as well as the DSA vessel branch point and DSA vessel stenosis of the segment of interest.

[0091] In one embodiment, a recognition model can be used to perform feature recognition on the DSA image data, identify the DSA catheter in the DSA image data, and obtain the DSA catheter image and DSA blood vessel image of the DSA image data.

[0092] In one embodiment, a recognition model can be used to identify DSA catheters in two-dimensional DSA image data of the vessel segment of interest, identifying DSA catheter images containing DSA catheters and DSA vessel images not containing DSA catheters in the two-dimensional DSA image data; a tracking algorithm such as a Kalman filter or particle filter is used to track the positional changes of the catheter in the sequence of DSA catheter images to obtain the DSA catheter position in each frame of the DSA catheter image.

[0093] In one embodiment, a recognition model can be used to perform feature recognition on the DSA image data of the segment of interest, identify the DSA vessel branch points and DSA vessel stenosis of the segment of interest in the DSA image data, and obtain the location of the DSA vessel branch points and DSA vessel stenosis of the segment of interest in the DSA image data.

[0094] In one embodiment, in the image of two-dimensional DSA image data, the vascular branch point is the intersection point on the vascular centerline, that is, a point on the vascular centerline with three or more adjacent points.

[0095] In one embodiment, each frame of a two-dimensional DSA image data segmenting the vessel segment of interest can be segmented using a recognition model to obtain a binary image of the vessels in each frame of the two-dimensional DSA image data. Based on the binary image of the vessels in each frame of the two-dimensional DSA image data, a morphological algorithm (e.g., skeletonization) is used to reduce the vessels in the binary image of the vessels to thin lines, preserving the topological structure of the vessels, to obtain the vessel centerline of each frame of the two-dimensional DSA image data. Based on the vessel centerline of each frame of the two-dimensional DSA image data, vessel branch points are identified on the vessel centerline of each frame of the two-dimensional DSA image data, and the images of the two-dimensional DSA image data are classified into DSA vessel branch point images containing vessel branch points and DSA non-vessel branch point images not containing vessel branch points. Based on the vessel centerline of each frame of the DSA vessel branch point image, the DSA vessel branch point position of each frame of the DSA vessel branch point image is obtained.

[0096] In one embodiment, the DSA stenosis of the segment of interest can be either a stenosis of the segment of interest or a segment of the vessel with vascular disease.

[0097] In one embodiment, a target recognition algorithm can be used to identify vascular stenosis or vascular lesions in each frame of two-dimensional DSA image data of the segment of interest through a recognition model. The images of the two-dimensional DSA image data are classified into DSA vascular stenosis images containing vascular stenosis or vascular lesions and DSA non-vascular stenosis images not containing vascular stenosis or vascular lesions. The location of DSA vascular stenosis in the DSA vascular stenosis images is also identified.

[0098] Figure 5 This is a schematic diagram of a frame of DSA angiography image of the DSA image data of the IVUS and DSA registration method shown in the embodiments of this application; Figure 6 yes Figure 5 A schematic diagram of the binary image of blood vessels corresponding to the DSA angiography image; Figure 7 It is based on Figure 5 A schematic diagram of the binary image of the blood vessel corresponding to the DSA catheter in the DSA angiography image.

[0099] See Figures 5-7 It is possible to identify models such as Figure 5 The method involves identifying a single frame of a two-dimensional DSA angiography image of the coronary arteries, as shown, to identify the DSA catheter and DSA vessel within that frame; segmenting the frame to obtain the corresponding data. Figure 6 The binary image of the blood vessel shown corresponds to the DSA catheter in this frame of DSA angiography image, as shown below. Figure 7 The image shows a binary representation of the blood vessels in the DSA catheter.

[0100] Step 206: Based on the IVUS catheter image and IVUS vessel image from the IVUS image data, and the DSA catheter image and DSA vessel image from the DSA image data, register the IVUS image data with the DSA image data to obtain the registered image.

[0101] In one embodiment, the registration of IVUS and DSA image data of the vessel segment of interest can be achieved by registering the IVUS vascular image data of the IVUS image data of the vessel segment of interest with the DSA vascular image data of the same vessel segment, thus obtaining a registered image. During the registration process, the IVUS catheter image data of the three-dimensional IVUS image data and the DSA catheter image data of the two-dimensional DSA image data are considered invalid data and can be excluded.

[0102] In one embodiment, an IVUS catheter frame image transitioning from an IVUS vessel image to an IVUS catheter image can be obtained based on the IVUS catheter image and IVUS vessel image in the IVUS image data; the coordinates of the DSA catheter in the guiding catheter coordinate port can be obtained based on the DSA catheter image and DSA vessel image in the DSA image data; and the IVUS vessel image of the IVUS image data of the vessel segment of interest and the DSA vessel image data of the DSA vessel segment of interest can be registered based on the IVUS catheter frame image and the coordinates of the DSA catheter in the guiding catheter coordinate port, thereby achieving the registration of the IVUS image data of the vessel segment of interest and the DSA image data of the vessel segment of interest, and obtaining a registered image.

[0103] In one embodiment, the IVUS image data can be obtained from the IVUS catheter image and the IVUS vessel image, including the first frame IVUS vessel image, the second frame IVUS vessel image, the third frame IVUS vessel image, ..., the (i-1)th frame IVUS vessel image and the i-th frame IVUS catheter image, wherein the IVUS image data has I frames of IVUS images, and the i-th frame IVUS image is the i-th frame IVUS catheter image.

[0104] In one embodiment, the coordinates (x1, y1), (x2, y2), (x3, y3), ..., (x4, y4) of the vessel centerline in the DSA image data can be obtained from the DSA catheter image and the DSA vessel image data. j-1 y j-1 ), (x j y j ), where (x j y j () represents the coordinates of the DSA catheter at the catheter orifice.

[0105] In one embodiment, the coordinates of the centerline of the IVUS image data can be based on the first frame IVUS vascular image, the second frame IVUS vascular image, the third frame IVUS vascular image, ..., the (i-1)th frame IVUS vascular image, the i-th frame IVUS catheter image, and the centerline coordinates of the DSA image data (x1, y1), (x2, y2), (x3, y3), ..., (x...). j-1 y j-1 ), (x j y j The IVUS image data is registered with the DSA image data to obtain the registered image.

[0106] For example, if the total number of IVUS retraction frames in the 3D IVUS image data is I (i.e., the IVUS image data has I frames of IVUS images), based on the IVUS catheter images and IVUS vessel images in the 3D IVUS image data, the i-th frame IVUS image is determined as the transition from the IVUS vessel image to the IVUS catheter image in the 3D IVUS image data. This i-th frame IVUS image is the i-th frame IVUS catheter image (also called the IVUS catheter frame image). Based on the i-th frame IVUS catheter image, the first i frames of images in the 3D IVUS image data are obtained, i.e., the first frame IVUS vessel image, the second frame IVUS vessel image, the third frame IVUS vessel image, ..., the (i-1)-th frame IVUS vessel image, and the i-th frame IVUS catheter image are obtained. The J coordinates of the vessel centerline in the 2D DSA image data are [(x1, y1), (x2, y2), (x3, y3), ..., (x... J y J In the 2D DSA image data, the coordinates of the DSA catheter at the catheter orifice are (x...). j y j ), based on coordinates (x j y j The first j coordinates of the J coordinates of the vessel centerline obtained from the two-dimensional DSA image data are [(x1, y1), (x2, y2), (x3, y3), ..., (x... j y j Based on the first i frames of the I-frame 3D IVUS image data: the first IVUS vessel image, the second IVUS vessel image, the third IVUS vessel image, ..., the (i-1)th IVUS vessel image, the i-th IVUS catheter image, and the first j coordinates of the J coordinates of the vessel centerline from the 2D DSA image data: (x1, y1), (x2, y2), (x3, y3), ..., (x... j y j), and compare the endpoint frame image (the i-th frame IVUS catheter image) in the 3D IVUS image data with the endpoint coordinates (x, y) of the vessel centerline. j y j Alignment is achieved by registering the IVUS vascular image data of the segment of interest with the DSA vascular image data of the segment of interest, thus obtaining the registered image.

[0107] Step 207: Adjust the registration of IVUS image data and DSA image data in the registered image based on the feature matching error between the features of IVUS image data and the corresponding features of DSA image data in the registered image.

[0108] In one embodiment, an optimization algorithm (e.g., the ICP algorithm) can be used to obtain the feature matching error between the features of the 3D IVUS image data in the registered image (e.g., IVUS catheter, vascular branch point, vascular stenosis) and the corresponding features of the 2D DSA image data (e.g., DSA catheter, vascular branch point, vascular stenosis). Based on the feature matching error, the registration of the IVUS image data and the DSA image data in the registered image is adjusted to make the 3D IVUS image data and the 2D DSA image data accurately aligned. Based on the constraint that the 3D IVUS image data and the 2D DSA image data in the registered image have the same corresponding features, the registration accuracy of the 3D IVUS image data and the 2D DSA image data in the registered image is further improved.

[0109] In one embodiment, the feature matching error between the features of the IVUS image data and the corresponding features of the DSA image data in the registered image can be obtained based on the IVUS image data and DSA image data in the registered image. The feature matching error includes one or more of the following: catheter position matching error, vascular branch point position matching error, and vascular stenosis position matching error. Based on the feature matching error, the registration of the IVUS image data and DSA image data in the registered image is adjusted.

[0110] In one embodiment, an energy function can be constructed based on the matching errors of catheter position, vessel branch point position, and vessel stenosis position between IVUS image data and DSA image data in the registration image. By minimizing the function value of the energy function, the registration of IVUS image data and DSA image data in the registration image can be adjusted, thereby further improving the registration accuracy of three-dimensional IVUS image data and two-dimensional DSA image data in the registration image.

[0111] In one embodiment, an energy function can be constructed based on the catheter position matching error, vascular branch point position matching error, and vascular stenosis position matching error between IVUS image data and DSA image data in the registration image, as well as the weight parameters of the catheter position matching error, vascular branch point position matching error, and vascular stenosis position matching error.

[0112] In one embodiment, the energy function is E; where, E=αE catheter +βE branch +γE stenosis ; In the formula, E catheter It is the catheter position matching error, E branch It is the error in matching the location of the blood vessel branch points, E stenosis α is the matching error of the vascular stenosis location, β is the weighting parameter of the catheter location matching error, γ is the weighting parameter of the vascular branch point location matching error, and the weighting parameters α, β, and γ are used to adjust the relative importance of each part of the matching error.

[0113] In one embodiment, the catheter position matching error E can be calculated based on the IVUS catheter position in the 3D IVUS image data and the corresponding DSA catheter position in the 2D DSA image data. catheter ,in, ; In the formula, It is the location of the o-th IVUS catheter in the 3D IVUS image data. It is the location of the o-th DSA catheter in the two-dimensional DSA image data.

[0114] In one embodiment, the matching error E of the vessel branch point position can be calculated based on the IVUS vessel branch point position in the three-dimensional IVUS image data and the corresponding DSA vessel branch point position in the two-dimensional DSA image data. branch ,in, ; In the formula, It is the location of the m-th IVUS vessel branch point in the 3D IVUS image data. It is the location of the m-th DSA vessel branch point in the two-dimensional DSA image data, where M is the number of vessel branch points.

[0115] In one embodiment, the vascular stenosis location matching error E can be calculated based on the IVUS vascular stenosis location in the three-dimensional IVUS image data and the corresponding DSA vascular stenosis location in the two-dimensional DSA image data. stenosis,in, ; In the formula, This is the l-th IVUS vascular stenosis location in the 3D IVUS imaging data. It represents the l-th DSA vascular stenosis location in the two-dimensional DSA image data, where L is the number of vascular stenosis locations.

[0116] In one embodiment, the registration of 3D IVUS and 2D DSA image data in the registered images can be verified through manual inspection and clinical feedback to ensure that the registered images accurately reflect the actual vascular structure. If the registration of 3D IVUS and 2D DSA image data in the registered images is not ideal, one or more of the following parameters can be adjusted: catheter position matching error, vascular branch point position matching error, vascular stenosis position matching error, weighting parameter α, weighting parameter β, and weighting parameter γ, to adjust the registration of 3D IVUS and 2D DSA image data in the registered images and ensure that the registered images accurately reflect the actual vascular structure.

[0117] Figure 8 This is a schematic diagram of the registration image of the IVUS and DSA registration method shown in the embodiments of this application.

[0118] In one embodiment, after adjusting the registration of IVUS image data and DSA image data in the registered image, the location of vascular stenosis of the segment of interest can be marked on the DSA image of the DSA image data in the registered image. For example... Figure 8 As shown, the registered images include IVUS image 905 (IVUS image data) and DSA image 906 (DSA image data). The location of stenosis in the segment of interest can be marked on the DSA image 906 within the registered images. Point 901 on the DSA image 906 corresponds to line segment 902 on the IVUS image 905, indicating the position of the selected current frame IVUS image on the DSA image. Line segment 903 on the DSA image 906 represents the stenosis.

[0119] See Figure 8 The retraction path of the IVUS catheter within the segment of interest can be displayed on the DSA image 906 of the 2D DSA image data in the registered image, such as... Figure 8In the upper right corner of the DSA image 906, line segment 904 represents the retraction path of the corresponding IVUS catheter within the segment of interest. Doctors or users can view the position of each IVUS image on the corresponding DSA image, or examine the IVUS image corresponding to each position on the DSA image. If the IVUS image shows vascular branches, check if the corresponding DSA image also shows vascular branches. Similarly, if the IVUS image shows vascular stenosis or lesions, check if the corresponding DSA image also shows vascular stenosis or lesions. By examining vascular branches, stenosis, or lesions, the accuracy of the registration between the IVUS and DSA image data can be determined.

[0120] In one embodiment, if both the IVUS image and the corresponding DSA image show vascular branches, the registration is considered accurate. If the IVUS image shows vascular branches but the corresponding DSA image does not, the registration is considered inaccurate. If both the IVUS image and the corresponding DSA image show vascular stenosis, the registration is considered accurate. If both the IVUS image and the corresponding DSA image do not show vascular stenosis, the registration is considered inaccurate.

[0121] In one embodiment, if the registration of IVUS image data and DSA image data is inaccurate, one or more of the following can be adjusted: catheter position matching error, vascular branch point position matching error, vascular stenosis position matching error, weight parameter α, weight parameter β, and weight parameter γ in the energy function. This will adjust the registration of the three-dimensional IVUS image data and the two-dimensional DSA image data in the registered image, ensuring that the registered image can accurately reflect the actual vascular structure.

[0122] The IVUS and DSA registration method of this application embodiment performs feature recognition on the images of IVUS image data and DSA image data respectively through a recognition model to obtain the features of IVUS image data and DSA image data; based on the features of IVUS image data and DSA image data, the IVUS image data and DSA image data are registered to obtain a registered image; this method can achieve fully automatic and high-precision IVUS and DSA registration, improving the robustness of IVUS and DSA registration. The IVUS and DSA registration method of this application embodiment registers high-resolution IVUS image data and DSA image data to obtain a registered image including both high-resolution IVUS image data and DSA image data. This registered image can more intuitively display detailed structural information of the vascular wall and overall vascular morphology information simultaneously. Through this comprehensive information, the registered image provides more comprehensive information on vascular stenosis or vascular lesions for clinical diagnosis and treatment, enabling more accurate identification of vascular lesions such as plaques, stenosis, and aneurysms, precise location of the area of ​​vascular stenosis or vascular lesions, and more comprehensive lesion assessment, thus improving diagnostic accuracy.

[0123] Furthermore, the IVUS and DSA registration method of this application performs noise reduction preprocessing on IVUS image data and / or optimization preprocessing on DSA image data; which can reduce the impact of noise on IVUS and DSA images, improve the contrast and clarity of IVUS and DSA images, improve the data quality of IVUS images in IVUS image data and DSA images in DSA image data, improve the accuracy of feature recognition of IVUS and DSA images, and further improve the accuracy and robustness of IVUS and DSA registration.

[0124] Furthermore, the IVUS and DSA registration method in this application embodiment uses the first frame IVUS vascular image, the second frame IVUS vascular image, the third frame IVUS vascular image, ..., the (i-1)th frame IVUS vascular image, the i-th frame IVUS catheter image from the IVUS image data, and the coordinates of the vessel centerline (x1, y1), (x2, y2), (x3, y3), ..., (x...) from the DSA image data. j-1 y j-1 ), (x j y j This method registers IVUS image data with DSA image data to obtain registered images; it can reduce the amount of data processing required for registering IVUS and DSA image data, and further improve the accuracy and robustness of IVUS and DSA registration.

[0125] Furthermore, the IVUS and DSA registration method of this application embodiment obtains the feature matching error between the features of the IVUS image data and the corresponding features of the DSA image data in the registered image, based on the registered image. The feature matching error includes one or more of the following: catheter position matching error, vascular branch point position matching error, and vascular stenosis position matching error. Based on the feature matching error, the registration of the IVUS image data and DSA image data in the registered image is adjusted. It can adjust the registration of the IVUS image data and DSA image data in the registered image through the feature matching error, and can automatically extract multiple corresponding identical features in the IVUS image data and DSA image data. It adopts multi-scale registration and optimizes the registered image from coarse to fine based on multiple corresponding identical features, reducing manual intervention and improving the efficiency, accuracy, and robustness of IVUS and DSA registration.

[0126] Corresponding to the aforementioned application function implementation method embodiments, this application also provides an IVUS and DSA registration device and corresponding embodiments.

[0127] Figure 9 This is a schematic diagram of the IVUS and DSA registration device shown in the embodiments of this application.

[0128] See Figure 9 The IVUS and DSA registration device 1000 includes a memory 1010 and a processor 1020.

[0129] The processor 1020 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0130] Memory 1010 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. ROM may store static data or instructions required by processor 1020 or other modules of the computer. Permanent storage devices may be read-write storage devices. Permanent storage devices may be non-volatile storage devices that retain stored instructions and data even when the computer is powered off. In some embodiments, permanent storage devices use mass storage devices (e.g., magnetic or optical disks, flash memory) as permanent storage devices. In other embodiments, permanent storage devices may be removable storage devices (e.g., floppy disks, optical drives). System memory may be a read-write storage device or a volatile read-write storage device, such as dynamic random access memory. System memory may store some or all of the instructions and data required by the processor during operation. Furthermore, memory 1010 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (e.g., DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 1010 may include a removable storage device that is readable and / or writable, such as a laser disc (CD), a read-only digital multifunction optical disc (e.g., DVD-ROM, dual-layer DVD-ROM), a read-only Blu-ray disc, an ultra-high density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not contain carrier waves or transient electronic signals transmitted wirelessly or via wired connections.

[0131] The memory 1010 stores executable code, which, when processed by the processor 1020, can cause the processor 1020 to execute part or all of the methods described above.

[0132] Furthermore, the method according to this application can also be implemented as a computer program or computer program product, which includes computer program code instructions for performing some or all of the steps in the method described above.

[0133] Alternatively, this application may also be implemented as a computer-readable storage medium (or a non-transitory machine-readable storage medium or a machine-readable storage medium) storing executable code (or computer program or computer instruction code) thereon, which, when executed by the processor of an IVUS and DSA registration device (or electronic device, or server, etc.), causes the processor to perform part or all of the steps of the method described above according to this application.

[0134] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for registering IVUS and DSA, characterized in that, include: Load the IVUS and DSA image data of the vessel segment of interest, respectively; The features of the IVUS image data and the DSA image data are obtained by performing feature recognition on the images of the IVUS image data and the DSA image data respectively through the recognition model. Based on the characteristics of the IVUS image data and the DSA image data, the IVUS image data and the DSA image data are registered to obtain a registered image.

2. The method according to claim 1, characterized in that, The step of performing feature recognition on the images of the IVUS image data and the images of the DSA image data respectively through recognition models to obtain the features of the IVUS image data and the features of the DSA image data includes: The recognition model is used to perform feature recognition on the IVUS image data to identify the IVUS catheter in the IVUS image data, thereby obtaining the IVUS catheter image and IVUS vessel image of the IVUS image data. The recognition model is used to perform feature recognition on the DSA image data to identify the DSA catheter in the DSA image data, thereby obtaining the DSA catheter image and DSA blood vessel image of the DSA image data. The step of registering the IVUS image data and the DSA image data based on the features of the IVUS image data and the features of the DSA image data to obtain a registered image includes: Based on the IVUS catheter image and IVUS vessel image of the IVUS image data, and the DSA catheter image and DSA vessel image of the DSA image data, the IVUS image data is registered with the DSA image data to obtain the registered image.

3. The method according to claim 2, characterized in that, The step of registering the IVUS image data with the DSA image data based on the IVUS catheter image and IVUS vessel image of the IVUS image data, and the DSA catheter image and DSA vessel image of the DSA image data, to obtain the registered image includes: Based on the IVUS catheter image and IVUS vessel image of the IVUS image data, the first frame IVUS vessel image, the second frame IVUS vessel image, the third frame IVUS vessel image, ..., the (i-1)th frame IVUS vessel image, and the i-th frame IVUS catheter image of the IVUS image data are obtained, wherein the IVUS image data has I frames of IVUS images, and the i-th frame IVUS image is the i-th frame IVUS catheter image; Based on the DSA catheter image and DSA vessel image of the DSA image data, obtain the coordinates (x1, y1), (x2, y2), (x3, y3), ..., (x... j-1 y j-1 ), (x j y j ), where the (x) j y j () represents the coordinates of the DSA catheter at the catheter orifice; Based on the first frame IVUS vascular image, the second frame IVUS vascular image, the third frame IVUS vascular image, ..., the (i-1)th frame IVUS vascular image, the i-th frame IVUS catheter image, and the coordinates of the vessel centerline of the DSA image data (x1, y1), (x2, y2), (x3, y3), ..., (x... j-1 y j-1 ), (x j y j The IVUS image data is registered with the DSA image data to obtain the registered image.

4. The method according to claim 3, characterized in that, The step of performing feature recognition on the IVUS image data and the DSA image data respectively using recognition models to obtain the features of the IVUS image data and the DSA image data further includes: The recognition model is used to perform feature recognition on the IVUS image data to identify IVUS vessel branch points and IVUS vessel stenosis in the IVUS image data, thereby obtaining the location of IVUS vessel branch points and IVUS vessel stenosis in the IVUS image data. The recognition model is used to perform feature recognition on the DSA image data to identify DSA vascular branch points and DSA vascular stenosis in the DSA image data, thereby obtaining the location of DSA vascular branch points and DSA vascular stenosis in the DSA image data.

5. The method according to claim 4, characterized in that, The method further includes: Based on the registered image, the feature matching error between the features of the IVUS image data and the corresponding features of the DSA image data in the registered image is obtained. The feature matching error includes one or more of the following: catheter position matching error, vascular branch point position matching error, and vascular stenosis position matching error. Based on the feature matching error, adjust the registration of the IVUS image data and the DSA image data in the registered image.

6. The method according to claim 5, characterized in that, The step of adjusting the registration of the IVUS image data and the DSA image data in the registered image based on the feature matching error includes: An energy function is constructed based on the catheter position matching error, the vascular branch point position matching error, and the vascular stenosis position matching error. The registration of the IVUS image data and the DSA image data in the registered image is adjusted by minimizing the function value of the energy function.

7. The method according to claim 6, characterized in that, The step of constructing an energy function based on the catheter position matching error, the vascular branch point position matching error, and the vascular stenosis position matching error includes: The energy function is constructed based on the catheter position matching error, the vascular branch point position matching error, the vascular stenosis position matching error, and the weight parameters of the catheter position matching error, the vascular branch point position matching error, and the vascular stenosis position matching error.

8. The method according to claim 1, characterized in that, The step of performing feature recognition on the IVUS image data and the DSA image data respectively using recognition models to obtain the features of the IVUS image data and the DSA image data, prior to which the following steps are included: The IVUS image data is subjected to noise reduction preprocessing; and / or, The DSA image data is then optimized and preprocessed.

9. A registration device for IVUS and DSA, characterized in that, include: processor; as well as A memory having executable code stored thereon, which, when executed by the processor, causes the processor to perform the method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: It stores executable code that, when executed by a processor, causes the processor to perform the method as described in any one of claims 1-8.

Images