Method, apparatus and computing device for registering external and internal vascular images

Abstract

The present invention discloses a method for registering an external vascular image and an endoluminal image. The method includes the steps of: acquiring first feature information of a vascular segment of interest in the endoluminal image of the blood vessel; acquiring second feature information of a reference vessel, which is the entire blood vessel in which the vascular segment of interest is located, in the external vascular image; performing a primary registration between the first feature information and the second feature information to obtain third feature information including external contour information and external branch information of a target vascular segment corresponding to the vascular segment of interest in the external vascular image; and performing a secondary registration between the first feature information and the third feature information to obtain a target registration result. This method can achieve fast and accurate registration between the external image and the endoluminal image. The present invention also provides an apparatus, a computing device, and a storage medium for registering an external vascular image and an endoluminal image of the blood vessel.

Description

[Technical field]

[0001] [CROSS REFERENCE TO RELATED APPLICATIONS] This application claims priority to a Chinese patent application bearing application number 202011418428.X and title "Method, apparatus and computing device for registration of vascular external images and intraluminal images" filed with the China Patent Office on December 7, 2020, the entire disclosure of which is incorporated herein by reference.

[0002] The present invention relates to the medical field, and in particular to a method, apparatus and computing device for registering external and intraluminal images of blood vessels. [Background technology]

[0003] The main imaging methods used in the clinical treatment of coronary heart disease include extraluminal imaging techniques such as coronary angiography, and intraluminal images such as optical coherence tomography (OCT) and intravenous ultrasound (IVUS). Coronary angiography can reflect the two-dimensional anatomical morphology of the coronary artery, allowing the specific location and degree of stenosis of coronary heart disease to be determined, but it is difficult to clearly display the abnormal location in the vascular wall, and cannot provide information such as the morphology and properties of plaque. Intraluminal images can clearly display the cross-section of blood vessels and reflect the tissue structure and fine geometric anatomical morphology of the vascular wall, but intraluminal images cannot well represent the topological structural information of blood vessels. Based on the functional complementarity of coronary angiography and vascular intraluminal images, the registration and fusion of coronary angiography images and intraluminal images can not only observe the spatial location of coronary vessels, but also provide quantification data of the intima-adventitia and plaque of the vascular segment of interest, thereby better assisting clinicians in the clinical assessment of coronary heart disease.

[0004] At present, registration of coronary angiogram images and intraluminal images is mainly realized by a method of 3D reconstruction of coronary angiogram images. For example, Rotger D, Radeva P, and Canero C et al., in a paper titled "Corresponding IVUS and angiogram image data" published in Computers in Cardiology in 2001, proposed that a 3D model of blood vessels is reconstructed using a method of taking two pairs of contrast images at the end points of the catheter's retreat path using a biplanar angiography device, and then fused with IVUS based on the catheter's retreat speed to obtain a 3D model of the coronary artery. In a paper titled "Fusion of 3D QCA and IVUS / OCT" published in the International Journal of Cardiovascular Imaging in 2011, Tu S, Holm N R and Koning G et al. presented a method to select vascular segments of interest from angiograms in two different body positions, perform 3D reconstruction, and then register the OCT / IVUS sequence based on the 3D reconstruction based on a distance matching algorithm (mainly related to vessel length and curvature). Summary of the Invention [Problem to be solved by the invention]

[0005] However, all of the above methods require manually marking the end points in the coronary angiogram of the OCT / IVUS image plane, and manually searching between the angiogram and the OCT / IVUS data requires the operator to have specialized knowledge. In addition, the 3D reconstruction requires exposing the vascular abnormality location in two angiographic positions, which requires a relatively high requirement for angiographic positions, and the algorithm takes a relatively long time.

[0006] In order to overcome the problems of reconstruction by the above two methods, namely, the relatively high requirements for contrast position and the relatively long time required, Wang P, Chen T, Ecabert O and Prummer S et al. proposed in the paper "Image-based device Tracking for the co-registration of angiography and intravascular ultrasound images" published in Med Image Comput Assist Interv in 2011, a method to automatically detect and track the tip of the IVUS sensor on the contrast image to continuously identify the position of the IVUS image in the contrast image, thereby realizing unconstrained registration between the angiography image and the IVUS image.

[0007] However, although this method solves the problem that the reconstruction process requires a relatively high level of postural contrast and takes a relatively long time, it still requires obtaining fluorescence images separately, and the accuracy of detecting and tracking the tip of the IVUS sensor also has a relatively large impact on the registration accuracy.

[0008] An object of the present invention is to provide a method for registering an external image of a blood vessel with an intraluminal image, in order to solve the problem that the conventional technology does not allow for both simplicity and accuracy in registration. [Means for solving the problem]

[0009] In order to solve the above technical problems, an embodiment of the present invention provides a method for registering an external image of a blood vessel with an intraluminal image, comprising: Including the inside diameter and inside length of blood vessels at each part Internal contour information and of the vessel segment of interest Internal branch information and a first acquisition step of acquiring first characteristic information including: Including the outer diameter and outer length of the blood vessel at each point External contour information and the reference vessel External branch information anda primary registration step of performing primary registration on the first feature information and the second feature information to obtain third feature information including external contour information and external branch information of a target blood vessel segment corresponding to a blood vessel segment of interest in the blood vessel external image; and a secondary registration step of performing secondary registration on the first feature information and the third feature information to obtain a target registration result.

[0010] The registration method between the external and intraluminal images of blood vessels according to the above technical solution can fully utilize the blood vessel diameter and branching information, and directly and fully automatically realize the registration between the coronary angiogram and the intraluminal image by using the multi-level registration method. This method only requires detection and segmentation of one frame of coronary angiogram, and does not require manual marking or three-dimensional reconstruction, and has the advantages of being fast, simple, highly accurate, and robust.

[0011] Preferably, the primary registration step includes the steps of: uniformly sampling the outer contour of the reference vessel in the external vascular image, and obtaining a set of reference vessel contour feature points including the vessel outer diameter of each point on the external contour of the reference vessel acquired in the external vascular image and the vessel outer length from each point to the proximal end of the reference vessel based on the external contour information of the reference vessel; uniformly sampling the intraluminal image of the vessel segment of interest in the intraluminal image of the vessel, and obtaining a set of vessel contour feature points including the vessel inner diameter and the vessel inner length at vessel position points corresponding to each frame image in the intraluminal image of the vessel segment of interest based on the internal contour information of the vessel segment of interest, wherein the vessel inner length at each vessel position point is the distance between the point and an origin, and the origin is an end point approaching the proximal end of the vessel segment of interest sampled from the intraluminal image of the vessel; and obtaining third feature information based on the set of reference vessel contour feature points and the set of vessel contour feature points of interest.

[0012] Preferably, the step of obtaining third feature information based on the set of reference vascular contour feature points and the set of vessel contour feature points of interest includes the steps of constructing a reference region based on the set of reference vascular contour feature points and constructing a sliding window based on the set of vessel contour feature points of interest; sliding the sliding window in the reference region and calculating and obtaining a degree of matching between each region in the reference region and the sliding window based on the set of reference vascular contour feature points, the set of vessel contour feature points of interest, external branch information of the reference vessel and internal branch information of the vessel segment of interest; and comparing each degree of matching to obtain the third feature information.

[0013] Preferably, the external branch information of the reference vessel includes the number of each branch vessel of the reference vessel acquired in the external vessel image and the outer length of the branch vessel from the branch point of each branch vessel to the proximal end of the reference vessel, and the internal branch information of the vessel segment of interest includes the number of each branch vessel of the vessel segment of interest acquired in the intraluminal vessel image and the inner length of the branch vessel from each branch vessel to its origin.

[0014] Preferably, the method for calculating the degree of match between each region in the reference region and the sliding window includes the steps of: calculating a diameter matching score based on a vessel outer diameter in the set of reference vessel contour feature points and a vessel inner diameter in the set of vessel contour feature points of interest; calculating a branch matching score based on external branch information of the reference vessel and internal branch information of the vessel segment of interest; constructing a matching function based on the diameter matching score and the branch matching score, and calculating the degree of match by the matching function.

[0015] Preferably, the diameter matching score is:

number

[0016] Preferably, the branch matching score is

number

[0017] Preferably, the matching function is:

number

[0018] Preferably, the range of values ​​that β can take is 0.3 to 0.7.

[0019] Preferably, the vessel segment corresponding to the region in the reference region that has the greatest match with the sliding window is the target vessel segment.

[0020] Preferably, the step of uniformly sampling the outer contour of the reference blood vessel in the outer blood vessel image and obtaining a set of reference blood vessel contour feature points based on the outer contour information of the reference blood vessel includes the steps of uniformly sampling the outer contour curve of the reference blood vessel in the outer blood vessel image and discretizing the outer contour of the reference blood vessel into each point, constructing outer contour feature points for each point by using the outer blood vessel length from each point to the proximal end of the reference blood vessel as a first-dimensional feature and the outer blood vessel diameter at the point as a second-dimensional feature, and obtaining a set of reference blood vessel contour feature points based on all the outer contour feature points.

[0021] Preferably, the step of uniformly sampling the intravascular image of a blood vessel segment of interest in the blood vessel intraluminal image and obtaining a set of blood vessel contour feature points of interest based on internal contour information of the blood vessel segment of interest includes the steps of detecting the intravascular image of the blood vessel intraluminal image using each frame in the sequence of images of the blood vessel intraluminal image as one detection point, constructing internal contour feature points of each frame image using the inner blood vessel length between the blood vessel position point corresponding to each frame image and the starting point as a first-dimensional feature and the inner blood vessel diameter at the blood vessel position point as a second-dimensional feature, and obtaining a set of blood vessel contour feature points of interest based on all the internal contour feature points.

[0022] Preferably, before constructing the reference region and the sliding window, the primary registration step further includes a step of uniformly sampling the external contour of the reference vessel in the external vessel image based on the distance between any two adjacent points in the set of vessel contour feature points of interest.

[0023] Preferably, the primary registration step further includes a step of determining a blood vessel outer diameter and a blood vessel outer length corresponding to each point on the outer contour curve of the target blood vessel segment in the blood vessel outer image based on the third feature information, and determining outer branch information of the target blood vessel segment; a step of constructing outer contour feature points corresponding to each point on the target blood vessel segment by taking the blood vessel outer length as a first-dimensional feature and the blood vessel outer diameter as a second-dimensional feature; and a step of obtaining a target blood vessel contour feature point set based on all the outer contour feature points.

[0024] Preferably, the target registration result is a target registration relationship between the set of vessel contour feature points of interest and the set of target vessel contour feature points.

[0025] Preferably, the secondary registration step comprises the following steps performed in sequence: a) performing normalization processing on a set of blood vessel contour feature points of interest S and a set of target blood vessel contour feature points T, respectively, to obtain a set of blood vessel contour feature points of interest S' and a set of target blood vessel contour feature points T' after normalization processing; b) calculating a feature error between the set of blood vessel contour feature points of interest S' and the set of target blood vessel contour feature points T'; c) determining whether the feature error is smaller than a preset value; if so, determining that the interested vascular contour feature point set S' is the target feature information, and calculating the target registration relationship according to the target feature information and the target vascular contour feature point set T' after normalization; if not, performing the following step; d) calculating and obtaining a registration transformation relationship between the blood vessel contour feature point set S' of interest and the target blood vessel contour feature point set T', and obtaining a new blood vessel contour feature point set S" of interest after performing registration transformation on the blood vessel contour feature point set S' of interest according to the registration transformation relationship; e) updating the set of blood vessel contour feature points of interest S' to a new set of blood vessel contour feature points of interest S", and returning to and executing step b).

[0026] Preferably, after step c) and before step d), the method further includes the step of: if the feature error is greater than or equal to a preset value, determining whether the number of iterations is greater than a preset number; if so, determining that the target vascular contour feature point set S' is the target feature information, and calculating the target registration relationship based on the target feature information and the target vascular contour feature point set T' after the normalization process; if not, adding 1 to the number of iterations, and performing the following step:

[0027] Preferably, the method further comprises, before step b), a step of setting the number of iterations to zero.

[0028] Preferably, the secondary registration step includes performing secondary registration on the first feature information and the third feature information by an iterative nearest neighbor algorithm or a coherent point drift algorithm or a robust point matching algorithm to obtain a target registration result.

[0029] Preferably, the intravascular image is an OCT image or an IVUS image.

[0030] Accordingly, the present invention provides a device for registering an external image of a blood vessel with an internal image of the blood vessel, the device comprising: Including the inside diameter and inside length of blood vessels at each part Internal contour information and of the vessel segment of interest. Internal branch information and a first acquisition module for acquiring first feature information including: Including the outer diameter and outer length of the blood vessel at each point External contour information and The reference blood vessel External branch information and The present invention further provides an apparatus for registering an external vascular image and an intraluminal image, the apparatus comprising: a second acquisition module for acquiring second feature information including: a primary registration module for performing a primary registration on the first feature information and the second feature information to obtain third feature information including external contour information and external branch information of a target vascular segment corresponding to a vascular segment of interest in the external vascular image; and a secondary registration module for performing a secondary registration on the first feature information and the third feature information to obtain a target registration result.

[0031] Preferably, the primary registration module includes: a first sampling unit for uniformly sampling the external contour of the reference vessel in the external vascular image, and obtaining a set of reference vessel contour feature points including a vessel outer diameter and a vessel outer length corresponding to each point on the external contour of the reference vessel acquired in the external vascular image based on the external contour information of the reference vessel; a second sampling unit for sampling the intraluminal image of the vessel segment of interest in the intraluminal image of the vessel, and obtaining a set of vessel contour feature points including a vessel inner diameter and a vessel inner length at a vessel position point corresponding to each frame image in the intraluminal image of the vessel segment of interest based on the internal contour information of the vessel segment of interest, wherein the vessel inner length at each vessel position point is the distance between the point and an origin, and the origin is an end point approaching the proximal end of the vessel segment of interest sampled from the intraluminal image of the vessel; and an acquisition unit for obtaining third feature information based on the reference vessel contour feature point set and the vessel contour feature point set of interest.

[0032] Preferably, the acquisition unit includes a window construction unit for constructing a reference region based on the set of reference vascular contour feature points and constructing a sliding window based on the set of blood vessel contour feature points of interest; a matching degree calculation unit for sliding the sliding window in the reference region and calculating and obtaining a matching degree between each region in the reference region and the sliding window based on the set of reference blood vessel contour feature points, the set of blood vessel contour feature points of interest, outer branch information of the reference blood vessel and inner branch information of the blood vessel segment of interest; and a comparison unit for comparing each matching degree to obtain third feature information.

[0033] Preferably, the external branch information of the reference vessel includes the number of each branch vessel of the reference vessel acquired in the external vessel image and the outer vessel length from the branch point of each branch vessel to the proximal end of the reference vessel, and the internal branch information of the vessel segment of interest includes the number of each branch vessel of the vessel segment of interest acquired in the intraluminal vessel image and the inner vessel length from each branch vessel to the origin.

[0034] Preferably, the method for calculating the matching degree between each region in the reference region and the sliding window by the matching degree calculation unit includes: calculating a diameter matching score based on a vessel outer diameter in the set of reference vessel contour feature points and a vessel inner diameter in the set of vessel contour feature points of interest; calculating a branch matching score based on external branch information of the reference vessel and internal branch information of the vessel segment of interest; constructing a matching function based on the diameter matching score and the branch matching score, and calculating the matching degree by the matching function.

[0035] Preferably, the diameter matching score is:

number

[0036] Preferably, the branch matching score is

number

[0037] Preferably, the matching function is:

number

[0038] Preferably, the range of values ​​that β can take is 0.3 to 0.7.

[0039] Preferably, the vessel segment corresponding to the region in the reference region that has the greatest match with the sliding window is the target vessel segment.

[0040] Preferably, the first sampling unit includes a first discretization subunit for uniformly sampling the outer contour curve of the reference vessel in the external vessel image and discretizing the outer contour of the reference vessel into each point, and a first construction subunit for constructing outer contour feature points of each point by using the outer vessel length from each point to the proximal end of the reference vessel as a first-dimensional feature and the outer vessel diameter at the point as a second-dimensional feature, and for obtaining a set of reference vessel contour feature points based on all the outer contour feature points.

[0041] Preferably, the second sampling unit includes a second framing subunit for detecting a vascular intraluminal image by using each frame in the sequence of vascular intraluminal images as one detection point, and a second construction subunit for constructing internal contour feature points of each frame image by using the inner vascular length between the vascular position point corresponding to each frame image and the origin as a first-dimensional feature and the inner vascular diameter at the vascular position point as a second-dimensional feature, and for obtaining a set of vascular contour feature points of interest based on all the internal contour feature points.

[0042] Preferably, the primary registration module further includes a distance calculation unit for calculating the distance between two adjacent points of the set of vessel contour feature points of interest before the window construction unit constructs the reference region and the sliding window, and the first sampling unit is further used to uniformly re-sample the external contour of the reference vessel in the external vessel image based on the distance.

[0043] Preferably, the first sampling unit is further used to determine a blood vessel outer diameter and a blood vessel outer length corresponding to each point on the outer contour curve of the target blood vessel segment in the blood vessel outer image based on the third feature information, and to determine outer branch information of the target blood vessel segment, take the blood vessel outer length as the first-dimensional feature and the blood vessel outer diameter as the second-dimensional feature to construct outer contour feature points corresponding to each point on the target blood vessel segment, and obtain a target blood vessel contour feature point set based on all outer contour feature points.

[0044] Preferably, the target registration result is a target registration relationship between the set of vessel contour feature points of interest and the set of target vessel contour feature points.

[0045] Preferably, the secondary registration module comprises a normalization unit for performing normalization processing on the set of blood vessel contour feature points of interest S and the set of target blood vessel contour feature points T, respectively, to obtain the set of blood vessel contour feature points of interest S' and the set of target blood vessel contour feature points T' after normalization processing; an error calculation unit for receiving the set of blood vessel contour feature points of interest S' and the set of target blood vessel contour feature points T' after normalization processing output from the normalization unit and calculating a feature error between the set of blood vessel contour feature points of interest S' and the set of target blood vessel contour feature points T', wherein, when the feature error between the set of blood vessel contour feature points of interest S' after normalization processing and the set of target blood vessel contour feature points T' becomes equal to or greater than a preset value, an error calculation unit for receiving the set of updated blood vessel contour feature points of interest S' output from the update unit and calculating the feature error between the set of updated blood vessel contour feature points of interest S' and the set of target blood vessel contour feature points T'; the error determination unit determines whether the feature error is smaller than a preset value; a registration relationship calculation unit determines that the vessel contour feature point set of interest S' is target feature information when the error determination unit determines that the feature error is smaller than the preset value, and calculates the target registration relationship based on the target feature information and the target vessel contour feature point set T' after normalization; a transformation relationship calculation unit obtains a registration transformation relationship between the vessel contour feature point set of interest S' and the target vessel contour feature point set T' when the error determination unit determines that the feature error is equal to or greater than the preset value, and obtains a new vessel contour feature point set of interest S" after registration transformation of the vessel contour feature point set of interest S' according to the registration transformation relationship; and an update unit updates the vessel contour feature point set of interest S' to the new vessel contour feature point set of interest S" obtained by the transformation relationship calculation unit.

[0046] Preferably, the secondary registration module further includes an iteration number judgment unit for judging whether the iteration number is greater than a preset number when the error judgment unit judges that the feature error is equal to or greater than the preset value, and an accumulation unit for adding 1 to the iteration number when the iteration number judgment unit judges that the iteration number is smaller than the preset number. The registration relationship calculation unit is further used for determining that the target vascular contour feature point set S' is target feature information when the iteration number judgment unit determines that the iteration number is greater than the preset number, and calculating a target registration relationship based on the target feature information and the target vascular contour feature point set T' after normalization processing.

[0047] Preferably, the secondary registration module further includes an initial unit for setting the number of iterations to zero before the error calculation unit calculates the feature error between the set S' of vascular contour feature points of interest after normalization processing and the set T' of target vascular contour feature points.

[0048] Preferably, the secondary registration module performs secondary registration on the first feature information and the third feature information by an iterative nearest neighbor algorithm, or a coherent point drift algorithm, or a robust point matching algorithm, to obtain a target registration result.

[0049] Preferably, the intravascular image is an OCT image or an IVUS image.

[0050] Accordingly, the present invention further provides a computing device comprising a processor for implementing various instructions and a memory for storing a plurality of instructions that are loaded by the processor to perform the above-mentioned method for registering external and intraluminal vascular images.

[0051] The computing device using the above technical means can realize fast and accurate registration of the external vascular image and the intraluminal image. Accordingly, the present invention further provides a storage medium storing a plurality of instructions to be loaded by a processor to execute the above method for registering the external vascular image and the intraluminal image.

[0052] The storage medium using the above technical means can realize fast and accurate registration of the external and internal vascular images. [Brief description of the drawings]

[0053] [Figure 1] 1 is a flowchart showing a method for registering an external blood vessel image and an intraluminal blood vessel image according to the present invention. [Diagram 2] 2 is a flow chart showing a primary registration step in the method for registering an external vascular image and an intraluminal vascular image of the present invention. [Diagram 3] 4 is a flow chart showing a secondary registration step in the method for registering an external vascular image and an intraluminal vascular image of the present invention. [Figure 4] 1 is a structural block diagram showing a registration device for an external blood vessel image and an intraluminal blood vessel image of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0054] Hereinafter, the embodiments of the present invention will be described with specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Although the description of the present invention is introduced together with the preferred embodiment, this does not represent that the features of the present invention are limited to only the embodiment. Quite the contrary, the purpose of introducing the invention together with the embodiment is to cover other options or modifications that can be expanded based on the scope of the claims of the present invention. In order to allow a deep understanding of the present invention, the following description includes many specific details. The present invention may be implemented without using these details. In addition, some specific details are omitted in the description to avoid confusing or obscuring the focus of the present invention. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features of the embodiments can be combined with each other. It should be noted that, in this specification, similar symbols and letters represent similar items in the following drawings, and therefore, once an item is defined in one drawing, it is not necessary to further define and interpret it in the following drawings. Terms such as "first", "second", etc. are merely for distinguishing the description, and cannot be understood to indicate or imply relative importance. The terms "proximal end" and "distal end" refer to relative positions and relate to the distance of a blood vessel from a main artery, with the end closer to the main artery being the proximal end and the end farther from the main artery being the distal end.

[0055] In order to make the objectives, technical means and advantages of the present invention clearer, the embodiments of the present invention will be described in more detail below with reference to the drawings.

[0056] As shown in FIG. 1, the present invention discloses a method for registering an external vascular image and an intraluminal vascular image, which includes the following steps: A first obtaining step S1: obtaining first feature information of a vessel segment of interest in a vessel intraluminal image, the first feature information including inner contour information and inner branch information of the vessel segment of interest. Specifically, the vessel segment of interest may be a vessel of a certain segment in which an abnormality appears with respect to a normal vessel. In an embodiment of the present invention, the vessel intraluminal image may be an OCT image or an IVUS image. That is, the internal contour information and the internal branch information are acquired from an intraluminal image such as OCT / IVUS. The internal contour information may include the vessel inner diameter and vessel inner length at various points in the vessel. Among them, the vessel inner diameter refers to the vessel diameter obtained based on the intraluminal image. The vessel inner length is the length along the vessel axis from a certain point in the vessel to the starting point. Specifically, the starting point is an end point approaching the proximal end of the vessel segment of interest sampled in the vessel intraluminal image. Also, since the OCT forms an image by retracting from the distal end to the proximal end, the starting point may be said to be the retraction end point of the OCT. The internal branch information may include the number of the internal branch (the number includes information on which branch on the vessel the branch belongs to) and the position information of the branch with respect to the vessel segment of interest.

[0057] Regarding the acquisition of the first characteristic information of the vessel segment of interest acquired in the intraluminal image of blood vessel, the intraluminal image may be an image including only the already separated vessel segment of interest acquired directly, or may be an image corresponding to the vessel segment of interest selected from the intraluminal image of one segment of blood vessel. The present invention is not limited thereto. The source of the intraluminal image may be directly introduced related data, may be acquired by real-time configuration and connection from other resource libraries, or may be acquired after searching from a stored image database based on information such as a user's name. The present invention is not limited thereto. The intraluminal image data is preferably in DICOM (Digital Imaging and Communications in Medicine) format. DICOM has the following advantages: It covers almost all information exchange protocols such as collection, archiving, communication, display and inquiry of medical digital images. It defines an object set including a set of various types of medical diagnostic images and their related analysis, report and other information with an open interconnection architecture and an object-oriented method. It defines service classes and command sets used for information transmission and exchange, and standard responses of messages. It details the technology of marking various types of information objects. It provides service support for use in network environments (OSI or TCP / IP). It structurally defines conformance statements of manufacturers. By using the DICOM format, the exchange of medical image information can be greatly simplified, and related collaboration with other medical application systems such as HIS and RIS can be easily achieved.

[0058] A second acquisition step S2: acquiring second feature information of a reference vessel in the vessel external image, where the reference vessel is the entire vessel in which the vessel segment of interest is located, and the second feature information includes external contour information and external branch information of the reference vessel.

[0059] Specifically, the vascular external image may be a coronary angiography image, reflecting the contour information of the outside of the blood vessel. Of course, those skilled in the art can understand that the external image of the blood vessel segment corresponding to the blood vessel segment of interest may be directly collected, but the applicant has found that the external image information collected in this step of the present embodiment is the external image information of the entire blood vessel in which the blood vessel segment of interest is located (i.e., the entire blood vessel from the proximal end point of the blood vessel to the distal end point of the blood vessel), which appears to be redundant information, but is actually advantageous to further improve the overall processing efficiency in combination with the subsequent processing steps.

[0060] Specifically, the external contour information may include the vessel outer diameter and vessel outer length at each point of the vessel. The vessel outer diameter is the vessel diameter obtained based on the vessel outer image. The vessel outer length is the length along the vessel axis from a point on the vessel outer contour to the proximal end of the vessel. The external branch information may include the number of the external branch vessel (the number includes information on which branch on the vessel the branch belongs to) and the position information of the external branch vessel relative to the reference vessel.

[0061] Primary registration step S3: performing primary registration on the first feature information and the second feature information to obtain third feature information, the third feature information including external contour information and external branch information of a target vascular segment corresponding to the vascular segment of interest in the vascular external image, where the vascular segment of interest mentioned in the text is located inside the blood vessel, and the target vascular segment refers to the vascular segment located outside the blood vessel corresponding to the vascular segment of interest.

[0062] That is, in order to grasp the situation of the vascular segment of interest more comprehensively, it is necessary to know the information of the inside and outside of the vascular segment of interest respectively, and the second feature information is the external contour information of the entire reference vessel. Therefore, the first feature information and the second feature information can be registered to obtain the external contour information of the desired target vascular segment.

[0063] Secondary registration step S4: A secondary registration is performed on the first feature information and the third feature information to obtain a target registration result.

[0064] Preferably, the target registration result may be target feature information whose feature error with the third feature information satisfies a preset condition, or the target registration result may be a target registration relationship between the first feature information and the third feature information, or the target registration result includes the target feature information and also includes the target registration relationship between the first feature information and the third feature information.

[0065] In the first registration step, the first feature information and the second feature information are roughly registered to obtain the external contour information of the required blood vessel of interest. Therefore, there may be a mismatch or a relatively large deviation between the first feature information and the third feature information obtained in step S3, so it is necessary to perform a second registration to further match the first feature information and the third feature information. However, in the method for registering an external image of a blood vessel and an intraluminal image provided by the present invention, the order of the first acquisition step and the second acquisition step can be exchanged, that is, the first feature information of the blood vessel segment of interest can be acquired first, and then the second feature information of the entire blood vessel in which the blood vessel segment of interest is located can be acquired, or the second feature information of the entire blood vessel in which the blood vessel segment of interest is located can be acquired first, and then the first feature information of the blood vessel segment of interest can be acquired. Furthermore, the first feature information of the blood vessel segment of interest and the second feature information of the entire blood vessel in which the blood vessel segment of interest is located can be acquired simultaneously. The present invention is not limited to the order of acquiring the first feature information of the blood vessel segment of interest and the second feature information of the entire blood vessel in which the blood vessel segment of interest is located.

[0066] Specifically, the method for registering external and intraluminal images of blood vessels mentioned in the present invention can be applied to the registration of external and intraluminal images of coronary artery vessels, such as the left rotator artery, the left anterior descending coronary artery, and the right coronary artery.

[0067] The registration method between the external and intraluminal images of blood vessels provided by the present invention fully utilizes the diameter and branching information of blood vessels, and realizes the direct and fully automatic registration between the coronary angiogram and the intraluminal image by using the multi-order registration method. The method only requires detection and segmentation of one frame of coronary angiogram, and does not require manual marking or three-dimensional reconstruction, and has the advantages of being fast, simple, highly accurate, and highly robust. The robustness here mainly refers to the degree to which the method is affected by the data required for calculation. High robustness means that the registration method is less affected by the variation of data, that is, the registration method is highly stable.

[0068] Furthermore, in the process of acquiring OCT / IVUS intraluminal images, various physiological or physical factors may cause perturbation of the OCT / IVUS probe, so that the acquisition plane of the OCT / IVUS image is not perpendicular to the vascular surface, which will affect the accuracy of the subsequent primary registration and secondary registration. To avoid this effect, angle calibration can be performed on the acquired OCT / IVUS image before the primary registration to improve the accuracy of the rough registration and fine registration. Specifically, the angle of the OCT / IVUS image can be calibrated using the method proposed in the paper "Research on angle calibration integrating coronary artery CAG three-dimensional reconstruction model" by Zhao Haisheng, Yang Feng, Lin Mudan et al., published in Science, Technology and Engineering, 2015, 15(36):84-90.

[0069] Specifically, as shown in FIG. 2, the primary registration step S3 specifically includes the following steps:

[0070] Step S31: In the external vascular image, uniform sampling is performed on the external contour of the reference vessel, and a set of reference vessel contour feature points is obtained based on the external contour information of the reference vessel, and the set of reference vessel contour feature points includes the vessel outer diameter of each point on the external contour of the reference vessel acquired in the external vascular image and the vessel outer length from each point to the proximal end of the reference vessel.

[0071] Specifically, in this step, the process of obtaining a set of reference blood vessel contour feature points may include a step of uniformly sampling the outer contour curve of the reference blood vessel in the external blood vessel image and discretizing the outer contour of the reference blood vessel into each point, a step of constructing outer contour feature points for each point by taking the outer blood vessel length from each point to the proximal end of the reference blood vessel as a first-dimensional feature and the outer blood vessel diameter at that point as a second-dimensional feature, and a step of obtaining a set of reference blood vessel contour feature points based on all the outer contour feature points.

[0072] Preferably, when sampling the external vascular image, each pixel point in the external image is taken as one sampling point, and the external image is sampled uniformly to obtain the external vascular diameter and external vascular length corresponding to each pixel point on the external vascular contour curve. Of course, the external image may be sampled uniformly at intervals of one, two or more pixel points, but this is not limited thereto.

[0073] Furthermore, the coronary artery in the external image may be segmented by a conventional segmentation algorithm or a deep learning algorithm, or a method of fusing the prediction result obtained by deep learning with a priori knowledge designed based on the conventional method, and information such as the outer length and outer diameter of the blood vessel may be calculated according to the segmentation result. For example, in the patent with application number CN202010134217.7, the coronary artery is cut by a deep learning model method to obtain information such as the length and diameter.

[0074] Step S32: In the vascular intraluminal image, uniform sampling is performed on the vascular intraluminal image of the vascular segment of interest, and a set of vascular contour feature points of interest is obtained based on the internal contour information of the vascular segment of interest, and the set of vascular contour feature points of interest includes the vascular inner diameter and vascular inner length at the vascular position points corresponding to each frame image in the vascular intraluminal image of the vascular segment of interest.

[0075] Specifically, the intravascular image is a sequence image, and one frame of the image corresponds to a cross-sectional image of a blood vessel at one location inside the blood vessel, that is, the intravascular image is composed of cross-sectional images of blood vessels at various locations inside the blood vessel. In general, the frequency of capturing OCT / IVUS intravascular images is 100 frame / s or 180 frame / s, and the retraction speed of the catheter is 18 mm / s, 20 mm / s, or 36 mm / s, and the structure inside the blood vessel is captured by retracting the catheter inside the blood vessel. When sampling the intravascular image uniformly, the intravascular image of each frame may be taken as one sample point, and the blood vessel inner diameter and blood vessel inner length at each position inside the blood vessel corresponding to the intravascular image of each frame may be obtained.

[0076] Specifically, the process of obtaining the set of blood vessel contour feature points of interest may include the steps of detecting a blood vessel intraluminal image by using each frame in the sequence of blood vessel intraluminal images as one detection point, constructing internal contour feature points of each frame image by using the blood vessel inner length between the blood vessel position point corresponding to each frame image and the starting point as a first-dimensional feature and the blood vessel inner diameter at the blood vessel position point as a second-dimensional feature, and obtaining the set of blood vessel contour feature points of interest based on all the internal contour feature points. However, except for the above-mentioned method of obtaining the set of blood vessel contour feature points of interest by detecting the blood vessel intraluminal image of each frame, it is also possible to obtain the set of blood vessel contour feature points of interest by a method of detecting every other frame or detecting frames at fixed intervals. Specifically, detecting is performed uniformly for the blood vessel intraluminal image sequence at intervals of one frame, two frames, or multiple frames, constructing internal contour feature points of each frame image by using the blood vessel inner length between the blood vessel position point corresponding to each detection point and the starting point as a first-dimensional feature and the blood vessel inner diameter at the blood vessel position point as a second-dimensional feature, and obtaining the set of blood vessel contour feature points of interest based on all the internal contour feature points.

[0077] Furthermore, by using the method mentioned in the Chinese patent with patent application number CN201710438256.4 to segment blood vessels, contour information such as blood vessel inner length and blood vessel inner diameter can be obtained.

[0078] In other words, when constructing the reference vessel contour feature point set, first, the outer contour curve of the reference vessel is discretized, and each point on the reference vessel contour curve is represented by a two-dimensional vector, the first-dimensional feature of the two-dimensional vector is the vessel outer length of the point, and the second-dimensional feature of the two-dimensional vector is the vessel outer diameter of the point, and then the two-dimensional vector set corresponding to each point is the reference vessel contour feature point set. Similarly, the construction process of the vessel contour feature point set of interest is similar, but since the OCT / IVUS intraluminal image is originally a sequence image of each discrete frame, there is no need to perform discretization processing on the intraluminal image, and each frame image is directly taken as a sample point, and similarly, the vessel position information corresponding to each frame image can be represented by a two-dimensional vector, the first-dimensional feature of which can be the vessel inner length of the vessel point corresponding to the corresponding frame image, and the second-dimensional feature of which can be the vessel inner diameter of the point, and then the two-dimensional vector set corresponding to each frame image can be the vessel contour feature point set of interest. In addition, each point in the set of reference vascular contour feature points and the set of vascular contour feature points of interest may be represented not only by a two-dimensional vector but also by a three-dimensional vector or a vector of more dimensions. When represented by a vector of three or more dimensions, each dimensional feature in the vector may be feature information such as length, diameter, blood vessel cross-sectional area, branch blood vessel diameter, branch opening direction, etc.

[0079] Step S33: Obtain third feature information based on the set of reference vessel contour feature points and the set of vessel contour feature points of interest.

[0080] Furthermore, a primary registration may be performed on the set of reference vascular contour feature points and the set of interested vascular contour feature points according to a sliding window fast target detection method to obtain third feature information. Specifically, the primary registration process may include the following steps:

[0081] Step S331: construct a reference region based on a set of reference vascular contour feature points, and construct a sliding window based on a set of vascular contour feature points of interest.

[0082] Preferably, the start point of the vessel segment of interest in the intraluminal image (i.e., the end point approaching the proximal end point of the reference vessel) may be the left boundary of the sliding window, and the end point of the vessel segment of interest in the intraluminal image (i.e., the end point approaching the distal end point of the reference vessel) may be the right boundary of the sliding window. The boundaries of the sliding region are the proximal end point and the distal end point of the reference vessel, respectively.

[0083] Step S332: Slide the sliding window in the reference region, and calculate the matching degree between each region in the reference region and the sliding window based on the reference vascular contour feature point set, the interest vascular contour feature point set, the external branch information of the reference vessel, and the internal branch information of the interest vascular segment.

[0084] Specifically, the external branch information of the reference vessel may include the number of each branch vessel of the reference vessel acquired in the vessel external image and the vessel outer length from the branch point of each branch vessel to the proximal end of the reference vessel, and the internal branch information of the vessel segment of interest may include the number of each branch vessel of the vessel segment of interest acquired in the vessel intraluminal image and the vessel inner length from each branch vessel to the origin. Since each feature point in the set of reference vessel contour feature points and the set of vessel contour feature points of the vessel of interest are arranged in order, the branch numbers are also automatically sorted according to the position of each branch on the vessel contour.

[0085] Specifically, in the process of sliding, the left boundary of the sliding window may be overlapped with the proximal end point of the reference region, and then the sliding window may be gradually slid from the proximal end point of the reference region to its distal end point. The right boundary of the sliding window may be overlapped with the distal end point of the reference region, and then the sliding window may be gradually slid from the distal end point of the reference region to the proximal end point of the reference region. Here, the sliding direction is not limited.

[0086] Specifically, in the sliding process, each time one point in the reference region is moved to the sliding window, the matching degree between the sliding window and the sub-region in the reference region corresponding to the sliding window may be calculated once. That is, in the sliding process, the total number of calculations of the matching degree is equal to the difference between the total number of points in the set of reference vascular contour feature points and the total number of points in the set of vascular contour feature points of interest. Of course, each time two or more points are moved to the sliding window in the reference region, the matching degree between the sliding window and the sub-region in the reference region corresponding to the sliding window may be calculated once. That is, in the present invention, it is sufficient that the calculated matching degrees can realize the primary registration of the first feature information and the second feature information. There is no specific requirement as to how many matching degrees are used.

[0087] Furthermore, the target evaluation function may be used to calculate the matching degree between the coronary artery reference vessel segment in the external image of the current reference region and the coronary artery vessel segment of interest in the OCT / IVUS intraluminal image, and the matching degree may be calculated according to the diameter information and the branch information. Of course, the above calculation of the matching degree is for the case where each point in the feature point set is a two-dimensional vector, and when each point in the feature point set is a three-dimensional or higher vector, the matching degree may be further calculated according to the diameter information, the branch information and other information contained in the vector, and is not limited thereto.

[0088] Furthermore, in the process of sliding, in order to make it easier to calculate the matching degree between the sliding window and each sub-region in the corresponding region, the interval between each point in the sliding window may be set to be equal to the interval between each point in the reference region. Therefore, the present invention may further include a step of uniformly re-sampling the external contour of the reference vessel in the external vessel image according to the distance between any two adjacent points in the set of the vessel contour feature points of interest before constructing the reference region and the sliding window. Specifically, when the intraluminal image is an OCT or IVUS image, the distance between two adjacent points in the set of the vessel contour feature points of interest may be calculated according to the catheter retraction speed and the OCT / IVUS imaging frequency, and then the distance may be used as the sampling step size to uniformly re-sample the external contour of the reference vessel in the external vessel image. Specifically, the distance between two adjacent points in the set of the vessel contour feature points of interest is equal to the product of the catheter retraction speed and the time required to capture one frame of the intraluminal image. Since the time required to capture one frame of an intraluminal image is equal to the reciprocal of the capture frequency, the distance is calculated by dividing the catheter retraction speed by the capture frequency.

[0089] Furthermore, the method for calculating the matching degree between each region in the reference region and the sliding window includes the following steps:

[0090] A diameter matching score is calculated based on the outer diameter of the blood vessel in the set of reference blood vessel contour feature points and the inner diameter of the blood vessel in the set of interest blood vessel contour feature points. The diameter matching score is calculated as follows:

number

[0091] Based on the external branch information of the reference vessel and the internal branch information of the vessel segment of interest, a branch matching score is calculated. The branch matching score is:

number

[0092] Based on the diameter matching score and the branch matching score, a matching function is constructed, and the match degree is calculated by the matching function. Specifically, the matching function is

number

[0093] Step S333: Compare the match degrees to obtain third feature information. Specifically, the matching degree between the sliding window and each subregion in the reference region is calculated according to equation (3), in which the vascular segment corresponding to the subregion in the reference region having the maximum matching degree with the sliding window is the target vascular segment, and the third feature information is a set of target vascular contour feature points corresponding to the target vascular segment.

[0094] That is, in one embodiment of the present invention, the primary registration step combines the diameter information and branch information to perform coarse registration on the external image and the OCT / IVUS intraluminal image using a sliding window fast target detection method. This method has a simple calculation process, solves the problem of manually locating the end points of the corresponding OCT / IVUS image blood vessels in the external coronary artery angiogram, reduces the manual operation steps, and improves the automation degree of the algorithm.

[0095] Furthermore, the primary registration step S3 may further include a step of establishing a set of target vascular contour feature points, as follows.

[0096] Based on the third feature information, a blood vessel outer diameter and a blood vessel outer length corresponding to each point on the outer contour curve of the target blood vessel segment in the blood vessel outer image are determined, and outer branch information of the target blood vessel segment is determined. The blood vessel outer length is taken as the first-dimensional feature, and the blood vessel outer diameter is taken as the second-dimensional feature to construct outer contour feature points corresponding to each point on the target blood vessel segment, and a set of target blood vessel contour feature points is obtained based on all the outer contour feature points.

[0097] Furthermore, the secondary registration step S4 uses a registration algorithm that combines diameter information and branch information based on the rough matching result of the primary registration step S3 to repeatedly calculate the set of interested vessel contour feature points and the set of target vessel contour feature points, so as to minimize the difference between the two sets of contour feature points, thereby achieving the purpose of precise registration. Preferably, the target registration result may be selected as a corresponding registration relationship between the first feature information and the third feature information. Specifically, the corresponding registration relationship may be calculated by the set of targeted vessel contour feature points and the set of interested vessel contour feature points, that is, the corresponding registration relationship between the first feature information and the third feature information may specifically be a target registration relationship between the set of targeted vessel contour feature points and the set of interested vessel contour feature points.

[0098] Specifically, as shown in FIG. 3, in the secondary registration process, a set of interest vessel contour feature points in the OCT / IVUS intraluminal image is denoted by S, and a set of target vessel contour feature points in the external contrast image is denoted by T. The secondary registration step S4 may include the following steps:

[0099] a) A normalization process is performed on the set of blood vessel contour feature points of interest S and the set of target blood vessel contour feature points T, respectively, to obtain a set of blood vessel contour feature points of interest S' and a set of target blood vessel contour feature points T' after normalization process. b) Calculate the feature error between the target blood vessel contour feature point set S' and the target blood vessel contour feature point set T'. Specifically, the feature error calculation formula is:

number

[0100] In concrete implementation, because the preset value is set small, no matter how many times the above iterative process is repeated, the calculated feature error will always be greater than the preset value, which will lead to an infinite loop and affect the registration efficiency. To solve this problem, the applicant discovered that in the iterative process, the feature error between the updated set of interest vascular contour feature points and the set of target vascular contour feature points gradually decreases with the increase in the number of iterations, so in one embodiment of the present invention, a maximum number of iterations is set to avoid the iterative process from entering an infinite loop. That is, even if the feature error is not smaller than the preset value in the iterative process, when the final cumulative number of iterations reaches a certain value, it is deemed that the purpose of precise registration has been achieved.

[0101] Specifically, a judgment condition may be added between step c) and step d), which may include the following content: if the feature error is equal to or greater than a preset value, judge whether the number of repetitions is greater than a preset number, if so, it is deemed to have been registered, and the interested vascular contour feature point set S' is determined to be the target feature information, and the target registration relationship is calculated according to the target feature information and the target vascular contour feature point set T' after normalization, if not, add 1 to the number of repetitions and execute the next step. Specifically, the preset number may be set according to actual circumstances, for example, set to 100 times.

[0102] In addition, in order to facilitate the calculation and determination of the number of iterations, the number of iterations may be set to zero before step b).The above process of solving the problem by iterative calculation minimizes the difference between the two sets of contour feature points, thereby realizing one-to-one correspondence between each point in the feature point sets corresponding to the external image of the target vessel segment and the endoluminal image of the vessel segment of interest, respectively.

[0103] As can be seen from the above, after the secondary registration is performed through the above steps, not only can a corresponding registration relationship between the first feature information and the third feature information be obtained, but also a target feature information whose feature error with the third feature information satisfies a preset condition can be obtained.

[0104] Furthermore, in the secondary registration step, secondary registration can be further performed on the first feature information and the third feature information by an iterative nearest neighbor algorithm, or a coherent point drift algorithm, or a robust point matching algorithm to obtain a target registration result.

[0105] The secondary registration step uses a non-rigid registration algorithm that combines diameter information and branch information, minimizes the feature difference between the two point sets through an iterative optimization process, obtains mapping parameters between the two point sets, and finds the correspondence between the two point sets to achieve registration. The diameter and branch information can directly reflect the contour topology features of the vascular tissue, and compared with the method of registration by positioning the tip of the IVUS sensor, this method has higher stability.

[0106] The registration method between external and intraluminal vascular images provided by the present invention realizes accurate registration of multi-modality vascular images by using the contour information based on external coronary artery images and intraluminal OCT / IVUS images through the method of multiple registrations (coarse and fine registration). This method realizes one-stop fully automatic multi-modality image registration, reduces the number of manual operation steps, and avoids errors caused by human factors to a large extent. In addition, the registration process does not require 3D reconstruction of external vascular images, which reduces the complexity and time consumption of the registration algorithm.

[0107] Accordingly, as shown in FIG. 4, the present invention further provides a registration device for a vascular external image and an intraluminal image, which includes a first acquisition module 1, a second acquisition module 2, a primary registration module 3 and a secondary registration module 4. The first acquisition module 1 is used to acquire first feature information of a vascular segment of interest in a vascular intraluminal image, the first feature information including inner contour information and inner branch information of the vascular segment of interest. The second acquisition module 2 is used to acquire second feature information of a reference vessel in a vascular external image, the reference vessel being the whole vessel in which the vascular segment of interest is located, and the second feature information including outer contour information and outer branch information of the reference vessel. The primary registration module 3 is used to perform primary registration on the first feature information and the second feature information to obtain third feature information, the third feature information including outer contour information and outer branch information of a target vessel segment corresponding to the vascular segment of interest in the vascular external image. The secondary registration module 4 is used to perform secondary registration on the first feature information and the third feature information to obtain a target registration result.

[0108] Preferably, the target registration result may be target feature information whose feature error with the third feature information satisfies a preset condition. Or, the target registration result may be a target registration relationship between the first feature information and the third feature information. Moreover, the target registration result may further include the target feature information and the target registration relationship between the first feature information and the third feature information.

[0109] Specifically, the primary registration module 3 may include: a first sampling unit 31 for uniformly sampling the external contour of the reference vessel in the external vascular image, and obtaining a set of reference vessel contour feature points including a vessel outer diameter and a vessel outer length corresponding to each point on the external contour of the reference vessel acquired in the external vascular image based on the external contour information of the reference vessel; and a second sampling unit 32 for sampling the intraluminal image of the vessel of an interest vessel segment in the intraluminal image, and obtaining a set of vessel contour feature points including a vessel inner diameter and a vessel inner length at a vessel position point corresponding to each frame image in the intraluminal image of the vessel of the interest vessel segment based on the internal contour information of the vessel segment of interest, where the vessel inner length at each vessel position point is the distance between the point and an origin, and the origin is an end point approaching the proximal end of the vessel segment of interest collected from the intraluminal image of the vessel; and an acquisition unit 33 for obtaining third feature information based on the reference vessel contour feature point set and the vessel contour feature point set of interest.

[0110] Preferably, when the first sampling unit 31 samples the external vascular image, it may uniformly sample the external image by treating each pixel point in the external image as one sampling point, and obtain the external vascular diameter and external vascular length corresponding to each pixel point on the external vascular contour curve.

[0111] Preferably, the intravascular image is an OCT image or an IVUS image. In general, the OCT / IVUS intravascular image is captured at a frequency of 100 frame / s or 180 frame / s, and the catheter is retracted at a speed of 18 mm / s, 20 mm / s, or 36 mm / s, to capture the internal structure of the blood vessel by retracting the catheter inside the blood vessel. When the second sampling unit 32 uniformly samples the intravascular image, the intravascular image of each frame may be taken as one sample point, and the blood vessel inner diameter and blood vessel inner length at each position inside the blood vessel corresponding to the intravascular image of each frame may be obtained.

[0112] Preferably, the acquisition unit 33 specifically includes a window construction unit 331 for constructing a reference region based on a set of reference vascular contour feature points and constructing a sliding window based on a set of interested vascular contour feature points; a matching degree calculation unit 332 for sliding the sliding window into the reference region and calculating a matching degree between each region in the reference region and the sliding window according to the reference vascular contour feature point set, the interested vascular contour feature point set, the outer branch information of the reference vessel and the inner branch information of the interested vascular segment; and a comparison unit 333 for comparing each matching degree to obtain third feature information.

[0113] Preferably, the external branch information of the reference vessel includes the number of each branch vessel of the reference vessel acquired in the external vessel image and the outer length of the branch vessel from the bifurcation point of each branch vessel to the proximal end of the reference vessel, and the internal branch information of the vessel segment of interest includes the number of each branch vessel of the vessel segment of interest acquired in the intraluminal vessel image and the inner length of the vessel from each branch vessel to its origin.

[0114] Preferably, the method for calculating the matching degree between each region in the reference region and the sliding window by the matching degree calculation unit includes: calculating a diameter matching score based on a vessel outer diameter in the set of reference vessel contour feature points and a vessel inner diameter in the set of vessel contour feature points of interest; calculating a branch matching score based on external branch information of the reference vessel and internal branch information of the vessel segment of interest; constructing a matching function based on the diameter matching score and the branch matching score, and calculating the matching degree by the matching function.

[0115] Preferably, the diameter matching score is:

number

[0116] Preferably, the branch matching score is

number

[0117] Preferably, the matching function is:

number

[0118] Preferably, the range of values ​​that β can take is 0.3 to 0.7.

[0119] Preferably, the vessel segment corresponding to the region in the reference region that has the greatest match with the sliding window is the target vessel segment.

[0120] Preferably, the first sampling unit 31 includes a first discretization subunit for uniformly sampling the outer contour curve of the reference blood vessel in the external blood vessel image and discretizing the outer contour of the reference blood vessel into each point, a first construction subunit for constructing outer contour feature points of each point by using the outer blood vessel length from each point to the proximal end of the reference blood vessel as a first-dimensional feature and the outer blood vessel diameter at the point as a second-dimensional feature, and obtaining a set of reference blood vessel contour feature points based on all the outer contour feature points.

[0121] Preferably, the second sampling unit 32 includes a second framing subunit for detecting a blood vessel intraluminal image by using each frame in the image of the blood vessel intraluminal image sequence as one detection point, a second construction subunit for constructing internal contour feature points of each frame image by using the blood vessel inner length between the blood vessel position point corresponding to each frame image and the origin as a first-dimensional feature and the blood vessel inner diameter at the blood vessel position point as a second-dimensional feature, and obtaining a blood vessel contour feature point set of interest based on all internal contour feature points. However, the second framing subunit may detect the blood vessel intraluminal image for each frame so that the second construction subunit obtains the blood vessel contour feature point set of interest, or may detect the blood vessel intraluminal image every other frame or for frames at a fixed interval so that the second construction subunit obtains the blood vessel contour feature point set of interest.

[0122] Specifically, when the first sampling unit 31 constructs the reference vessel contour feature point set, the first discretization subunit discretizes the outer contour curve of the reference vessel, the first construction subunit represents each point on the reference vessel contour curve as a two-dimensional vector, the first-dimensional feature of the two-dimensional vector is the outer vessel length of the point, and the second-dimensional feature of the two-dimensional vector is the outer vessel diameter of the point, and the two-dimensional vector set corresponding to each point is the reference vessel contour feature point set. Similarly, the second sampling unit 32 has a similar construction process of the interested vessel contour feature point set, but since the OCT / IVUS intraluminal image is originally a discrete frame-by-frame sequence image, there is no need to perform discretization processing on the intraluminal image, and the second framing subunit directly constructs each frame image as a sample point. The second construction subunit may represent the vascular position information corresponding to each frame image as a two-dimensional vector, the first-dimensional feature of which may be the vascular inner length of the vascular point corresponding to the corresponding frame image, and the second-dimensional feature of which may be the vascular inner diameter of the point, and then the second construction subunit may take the set of two-dimensional vectors corresponding to each frame image as a set of vascular contour feature points of interest.

[0123] Each point in the set of reference blood vessel contour feature points and the set of blood vessel contour feature points of interest may be represented not only by a two-dimensional vector but also by a three-dimensional vector or a vector of more dimensions. When represented by a vector of three or more dimensions, the feature of each dimension in the vector may be feature information such as length, diameter, blood vessel cross-sectional area, branch blood vessel diameter, branch opening direction, etc.

[0124] Preferably, the primary registration module 3 further includes a distance calculation unit for calculating the distance between two adjacent points in the set of vascular contour feature points of interest before the window construction unit constructs the reference region and the sliding window, and the first sampling unit 31 is further used to uniformly re-sample the external contour of the reference vessel in the external vascular image based on the distance.

[0125] Specifically, when the intraluminal image is an OCT or IVUS image, the distance calculation unit may calculate the distance between two adjacent points in the set of vascular contour feature points of interest according to the retraction speed of the catheter and the imaging frequency of OCT / IVUS, and the distance is the catheter retraction speed divided by the imaging frequency. The first sampling unit 31 may further use the distance as a sampling step size to uniformly sample the external contour of the reference vessel in the external vascular image. In this way, the interval between each point in the sliding window is set to be equal to the interval between each point in the reference region, so that it is easier to calculate the matching degree between the sliding window and each sub-region in the corresponding region during the sliding process.

[0126] Preferably, the first sampling unit 31 is further used to determine, based on the third feature information, a blood vessel outer diameter and a blood vessel outer length corresponding to each point on the outer contour curve of the target blood vessel segment in the blood vessel outer image, and determine the outer branch information of the target blood vessel segment, take the blood vessel outer length as the first-dimensional feature and the blood vessel outer diameter as the second-dimensional feature to construct outer contour feature points corresponding to each point on the target blood vessel segment, and obtain a target blood vessel contour feature point set based on all the outer contour feature points.

[0127] Preferably, the target registration result may be selected as a corresponding registration relationship between the first feature information and the third feature information, specifically, the corresponding registration relationship may be calculated by the target vascular contour feature point set and the interested vascular contour feature point set, that is, the corresponding registration relationship between the first feature information and the third feature information may specifically be a target registration relationship between the target vascular contour feature point set and the interested vascular contour feature point set.

[0128] Preferably, the secondary registration module 4 includes a normalization unit 41 for performing normalization processing on the set of blood vessel contour feature points of interest S and the set of target blood vessel contour feature points T, respectively, to obtain the set of blood vessel contour feature points of interest S' and the set of target blood vessel contour feature points T' after normalization processing; an error calculation unit 42 for receiving the set of blood vessel contour feature points of interest S' and the set of target blood vessel contour feature points T' after normalization processing output from the normalization unit and calculating a feature error between the set of blood vessel contour feature points of interest S' and the set of target blood vessel contour feature points T', the error calculation unit 42 receiving the set of blood vessel contour feature points of interest after update output from the update unit and calculating the feature error between the set of blood vessel contour feature points of interest after update S' and the set of target blood vessel contour feature points T' when the feature error between the set of blood vessel contour feature points of interest after normalization processing S' and the set of target blood vessel contour feature points T' becomes equal to or larger than a preset value; the error determination unit 43 determining whether the feature error is smaller than a preset value; a registration relationship calculation unit 46 determining that the blood vessel contour feature point set of interest S' is target feature information when the error determination unit 43 determines that the feature error is smaller than the preset value, and calculating a target registration relationship based on the target feature information and the target blood vessel contour feature point set T' after normalization processing; a transformation relationship calculation unit 44 obtaining a registration transformation relationship between the blood vessel contour feature point set of interest S' and the target blood vessel contour feature point set T' when the error determination unit 43 determines that the feature error is equal to or greater than the preset value, and obtaining a new blood vessel contour feature point set of interest S" after registration transformation of the blood vessel contour feature point set of interest S' according to the registration transformation relationship; and an update unit 45 updating the blood vessel contour feature point set of interest S' to the new blood vessel contour feature point set of interest S" obtained by the transformation relationship calculation unit 44.

[0129] Specifically, the preset value can be set according to actual circumstances, and can generally be set to 0.01, that is, if the feature error is less than 0.01, it is considered that the registration has been completed.

[0130] Preferably, the secondary registration module 4 may further include: an iteration number judgment unit for judging whether the iteration number is greater than a preset number when the error judgment unit judges that the feature error is greater than or smaller than a preset value; and an accumulation unit for adding 1 to the iteration number when the iteration number judgment unit judges that the iteration number is smaller than the preset number.

[0131] If the iteration count judgment unit determines that the iteration count is greater than the preset count, the registration relation calculation unit 46 further determines that the interested vascular contour feature point set S' is the target feature information, and uses it to calculate the target registration relation according to the target feature information and the target vascular contour feature point set T' after normalization. Specifically, the preset count can be set according to actual circumstances, for example, 100 times.

[0132] Preferably, the secondary registration module 4 further includes an initial unit for setting the number of iterations to zero before the error calculation unit calculates the feature error between the vascular contour feature point set of interest S' and the target vascular contour feature point set T' after the normalization process.

[0133] Preferably, the secondary registration module 4 performs secondary registration on the first feature information and the third feature information by an iterative nearest neighbor algorithm or a coherent point drift algorithm or a robust point matching algorithm to obtain a target registration result.

[0134] The device for registering external and intraluminal images of blood vessels provided by the present invention can directly and automatically register coronary angiograms and intraluminal images by using the method of multi-order registration according to the diameter and branching information of blood vessels. The device only needs to perform detection and segmentation on one frame of coronary angiogram, and does not require manual marking or 3D reconstruction, and has the advantages of being fast, simple, highly accurate, and robust.

[0135] The present invention further provides a computing device comprising a processor for implementing various instructions and a memory for storing a plurality of instructions that are loaded by the processor to perform the above-mentioned method for registering an external vascular image and an intraluminal vascular image. The computing device according to the above technical solution can realize fast and accurate registration of the external and intraluminal images of blood vessels.

[0136] The embodiment of the present invention further discloses a storage medium storing a number of instructions, which is loaded by a processor to execute the above-mentioned registration method for the external and intraluminal images of blood vessels. The storage medium according to the above technical solution can realize fast and accurate registration of the external and intraluminal images of blood vessels.

[0137] Each embodiment disclosed herein may be implemented in hardware, software, firmware, or a combination of these implementations. Examples of the present application may be implemented as a computer program or program code executed on a programmable system, the programmable system comprising at least one processor, a storage system (comprising volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. The program code may be used to input instructions to perform the functions described herein and generate output information. The output information may be used to output one or more output devices in a known manner. For purposes of the present application, a processing system may comprise any system having a processor, such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0138] The program code may be implemented in a high level or object oriented programming language for communicating with a processing system. If necessary, the program code may also be implemented in assembly language or machine language. In fact, the mechanisms described herein are not limited in scope to any particular programming language. In any case, the language may be a compiled or interpreted language.

[0139] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented by instructions loaded or stored on one or more transitory or non-transitory machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed over a network or other computer-readable media. Thus, a machine-readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including, but not limited to, floppy disks, laser disks, optical disks, read-only memories (CD-ROMs), magneto-optical disks, read-only memories (ROM), random access memories (RAM), erasable programmable read-only memories (EPROM), electrically erasable programmable read-only memories (EEPROM), magnetic or optical cards, flash memory, or tangible machine-readable memory for transmitting information in an electrical, optical, acoustic or other form of propagated signal (e.g., carrier, infrared signal, digital signal, etc.) over the Internet. Thus, a machine-readable medium includes any type of machine-readable medium for storing or transmitting electronic instructions or information in a form readable by a machine (e.g., a computer).

[0140] In the drawings, some structural or method features may be shown in a particular arrangement and / or order. However, such a particular arrangement and / or sorting may not be required. In some embodiments, these features may be arranged differently than the manner and / or order shown in the illustrative figures. Also, the inclusion of structural or method features in a particular figure does not imply that such features are required in any embodiment. And, in some embodiments, these features may not be included or may be combined with other features. However, each module / unit mentioned in each device embodiment of the present application is a logic module / unit, and physically, one logic module / unit may be one physical module / unit, may be a part of one physical module / unit, and may be realized by a combination of multiple physical modules / units. The physical implementation manner of these logic modules / units themselves is not the most important, and it is the combination of functions realized by these logic modules / units that is the key to solving the technical problem presented in the present application. Furthermore, in order to highlight the innovative aspects of the present application, each of the above device embodiments of the present application does not incorporate modules / units that are not closely related to solving the technical problem presented in the present application, and it is not intended to represent that there are no other modules / units in the above device embodiments.

[0141] Although the present invention has been illustrated and described with reference to some preferred embodiments of the present invention, those skilled in the art should clearly understand that the above is a further detailed description of the present invention in combination with specific embodiments, and cannot recognize that the specific implementation of the present invention is limited only to these descriptions. Those skilled in the art may make various changes thereto in form and detail, including some simple deductions or substitutions, without departing from the spirit and scope of the present invention.

Claims

1. A method for registering an external image and an intraluminal image of a blood vessel, comprising the steps of: a first acquisition step of acquiring first feature information including inner contour information including a blood vessel inner diameter and an inner blood vessel length at each point in a blood vessel segment of interest in a blood vessel intraluminal image and inner branch information of the blood vessel segment of interest; a second acquisition step of acquiring second feature information including external contour information including a blood vessel outer diameter and a blood vessel outer length at each point in a reference blood vessel, which is an entire blood vessel in which the blood vessel segment of interest is located, and external branch information of the reference blood vessel; a primary registration step of performing a primary registration on the first feature information and the second feature information to obtain third feature information including external contour information and external branch information of a target blood vessel segment corresponding to the blood vessel segment of interest in the blood vessel external image; a secondary registration step of performing a secondary registration on the first feature information and the third feature information to obtain a target registration result; A method for registering an external blood vessel image and an intraluminal blood vessel image, comprising:

2. The primary registration step comprises: A step of uniformly sampling the outer contour of the reference blood vessel in the blood vessel external image, and obtaining a reference blood vessel contour feature point set including a blood vessel outer diameter of each point on the outer contour of the reference blood vessel acquired in the blood vessel external image and a blood vessel outer length from each point to a proximal end of the reference blood vessel based on the outer contour information of the reference blood vessel; a step of uniformly sampling the intraluminal image of the vessel of interest in the intraluminal image of the vessel of interest, and obtaining a set of vessel of interest contour feature points including a vessel inner diameter and a vessel inner length at a vessel position point corresponding to each frame image in the intraluminal image of the vessel of interest based on internal contour information of the vessel of interest, wherein the vessel inner length at each vessel position point is the distance between the point and an origin point, and the origin point is an end point approaching a proximal end of the vessel of interest sampled from the vessel of interest image; obtaining the third feature information based on the set of reference vessel contour feature points and the set of vessel contour feature points of interest.

2. The method for registering an external blood vessel image and an intraluminal blood vessel image according to claim 1.

3. The step of obtaining the third feature information based on the set of reference vascular contour feature points and the set of interested vascular contour feature points includes: constructing a reference region according to the set of reference vascular contour feature points, and constructing a sliding window according to the set of vascular contour feature points of interest; Sliding the sliding window in the reference region, and calculating a matching degree between each region in the reference region and the sliding window according to the reference vessel contour feature point set, the interested vessel contour feature point set, the outer branch information of the reference vessel, and the inner branch information of the interested vessel segment; comparing the match degrees to obtain the third feature information; 3. The method for registering an external blood vessel image and an intraluminal blood vessel image according to claim 2, further comprising:

4. 4. The method for registering an external vascular image and an intraluminal image according to claim 3, wherein the external branch information of the reference vessel includes a number of each branch vessel of the reference vessel acquired in the external vascular image and an outer length of the branch vessel from a bifurcation point of each branch vessel to a proximal end of the reference vessel, and the internal branch information of the vessel segment of interest includes a number of each branch vessel of the vessel segment of interest acquired in the intraluminal vascular image and an inner length of the branch vessel from each branch vessel to the origin.

5. A method of calculating the degree of match between each region in the reference region and the sliding window is as follows: calculating a diameter matching score based on a vessel outer diameter of the set of reference vessel contour feature points and a vessel inner diameter of the set of vessel contour feature points of interest; calculating a branch matching score based on the external branch information of the reference vessel and the internal branch information of the vascular segment of interest; constructing a matching function based on the diameter matching score and the branch matching score, and calculating the match degree according to the matching function; 5. The method for registering an external blood vessel image and an intraluminal blood vessel image according to claim 4, further comprising:

6. The diameter matching score is [0010] where S d represents the diameter matching score, N represents the total number of feature points in the sliding window, and d Ｓi represents the blood vessel inner diameter corresponding to the i-th feature point in the sliding window, and d ｔi The method for registering an external blood vessel image and an intraluminal blood vessel image according to claim 5 , wherein i represents an external diameter of a blood vessel corresponding to the i-th feature point in the reference region.

7. The branch matching score is [0025] where S b represents the branch matching score, M represents the total number of branches in the sliding window, and dist i The method for registering an external vascular image and an intraluminal vascular image according to claim 5, wherein i represents a distance between an i-th branch vessel in the sliding window and a branch vessel closest to the i-th branch vessel in the reference region.

8. The matching function is [0030] where S represents the matching function, and S d represents the diameter matching score, and S b 6. The method of claim 5, wherein: β represents a branch matching score; β represents a weighting coefficient, and 0<β<1.

9. 9. The method for registering an external blood vessel image and an intraluminal blood vessel image according to claim 8, wherein the range of values ​​that the parameter β can take is 0.3 to 0.

7.

10. The method for registering an external vascular image and an intraluminal vascular image according to claim 8, wherein the vascular segment corresponding to the region in the reference region that has the greatest degree of match with the sliding window is the target vascular segment.

11. The step of uniformly sampling the outer contour of the reference blood vessel in the outer blood vessel image and obtaining a set of reference blood vessel contour feature points based on the outer contour information of the reference blood vessel includes: uniformly sampling an outer contour curve of a reference blood vessel in the outer blood vessel image to discretize the outer contour of the reference blood vessel into individual points; constructing an outer contour feature point for each point by using the outer vessel length from each point to the proximal end of the reference vessel as a first-dimensional feature and the outer vessel diameter of the point as a second-dimensional feature; and obtaining the set of reference vascular contour feature points based on all of the external contour feature points.

12. The step of uniformly sampling the intraluminal image of the blood vessel segment of interest in the blood vessel intraluminal image and obtaining a set of blood vessel contour feature points based on internal contour information of the blood vessel segment of interest in the blood vessel intraluminal image includes: detecting the intravascular image by setting each frame in the sequence of intravascular images as one detection point; constructing an internal contour feature point for each frame image by using a blood vessel inner length between the blood vessel position point corresponding to each frame image and the starting point as a first-dimensional feature and a blood vessel inner diameter at the blood vessel position point as a second-dimensional feature; and obtaining the set of vessel contour feature points of interest based on all of the internal contour feature points.

13. Prior to constructing the reference region and the sliding window, the primary registration step comprises: The method for registering an external vascular image and an intraluminal image according to claim 3, further comprising a step of uniformly sampling the external contour of the reference vessel in the external vascular image based on the distance between any two adjacent points of the set of blood vessel contour feature points of interest.

14. The primary registration step comprises: determining a blood vessel outer diameter and a blood vessel outer length corresponding to each point on the outer contour curve of the target blood vessel segment in the blood vessel outer image according to the third feature information, and determining outer branch information of the target blood vessel segment; constructing outer contour feature points corresponding to each point on the target vessel segment, with the vessel outer length as a first-dimensional feature and the vessel outer diameter as a second-dimensional feature; The method for registering an external vascular image and an intraluminal vascular image according to claim 2, further comprising the step of: obtaining a set of target vascular contour feature points based on all the external contour feature points.

15. The method of claim 14, wherein the target registration result is a target registration relationship between the set of interest vessel contour feature points and the set of target vessel contour feature points.

16. The secondary registration step comprises the following steps, performed in sequence: a) performing a normalization process on the set of blood vessel contour feature points of interest S and the set of target blood vessel contour feature points T, respectively, to obtain a set of blood vessel contour feature points of interest S' and a set of target blood vessel contour feature points T' after normalization process; b) calculating a feature error between the set of vessel contour feature points of interest S′ and the set of target vessel contour feature points T′; c) determining whether the feature error is smaller than a preset value, and if so, determining that the interested vessel contour feature point set S' is target feature information, and calculating the target registration relationship according to the target feature information and the target vessel contour feature point set T' after normalization processing; if not, executing the following step; d) calculating and obtaining a registration transformation relationship between the set of blood vessel contour feature points of interest S' and the set of target blood vessel contour feature points T', and obtaining a new set of blood vessel contour feature points of interest S" after performing registration transformation on the set of blood vessel contour feature points of interest S' according to the registration transformation relationship; e) updating the set of blood vessel contour feature points of interest S' to a new set of blood vessel contour feature points of interest S", and returning to step b); The method for registering an external blood vessel image and an intraluminal blood vessel image according to claim 15, further comprising:

17. After step c) and before step d), 17. The method of claim 16, further comprising the steps of: determining whether a number of iterations is greater than a predetermined number when the feature error is equal to or greater than the predetermined value; if so, determining that the set of interested vascular contour feature points S' is the target feature information; and calculating the target registration relationship based on the target feature information and the set of target vascular contour feature points T' after normalization; if not, adding 1 to the number of iterations, and performing the following step:

18. The method for registering an external vascular image and an intraluminal vascular image according to claim 17, further comprising the step of setting the number of iterations to zero before step b).

19. The method for registering an external vascular image and an intraluminal vascular image according to claim 1, characterized in that the secondary registration step includes a step of performing secondary registration on the first feature information and the third feature information by an iterative nearest neighbor algorithm, a coherent point drift algorithm, or a robust point matching algorithm to obtain the target registration result.

20. An apparatus for registering an external image and an intraluminal image of a blood vessel, comprising: a first acquisition module for acquiring first feature information including inner contour information including a vessel inner diameter and an inner vessel length at each point in a vessel segment of interest in a vessel intraluminal image and inner branch information of the vessel segment of interest; a second acquisition module for acquiring second feature information including external contour information including a blood vessel outer diameter and a blood vessel outer length at each point in a reference blood vessel, which is an entire blood vessel in which the blood vessel segment of interest is located, and external branch information of the reference blood vessel; a primary registration module for performing a primary registration on the first feature information and the second feature information to obtain third feature information including external contour information and external branch information of a target vessel segment corresponding to the vessel segment of interest in the vessel external image; and a secondary registration module for performing secondary registration on the first feature information and the third feature information to obtain a target registration result.

21. 1. A computing device comprising: A processor for implementing various instructions; a memory for storing a plurality of instructions, when loaded by said processor, for carrying out the method for registering an external vascular image and an intraluminal vascular image according to any one of claims 1 to 19; 1. A computing device comprising:

22. A storage medium having stored thereon a plurality of instructions which, when loaded by a processor, executes the method for registering an external vascular image and an intraluminal vascular image according to any one of claims 1 to 19.

Images