CAG and IVUS precise linkage analysis method and system based on 3D reconstruction technology
Through the CAG and IVUS precise linkage analysis method based on 3D reconstruction technology, the problem of low image fusion accuracy and inability to accurately locate the spatial location of the lesions in the prior art is solved, and the rapid and accurate diagnosis and precise positioning of cardiovascular diseases are achieved, which improves the Diagnostic efficiency.
Patent Information
- Application Number
- CN202011633284.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-12-31
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2040-12-31
AI Technical Summary
Existing coronary angiography and intravascular ultrasound image fusion techniques have problems with low image fusion accuracy and inability to accurately locate the spatial location of the lesions when diagnosing cardiovascular diseases. Especially under the influence of heartbeat, the blood vessel center line cannot represent IVUS guidance. Silk trajectory, resulting in low diagnostic efficiency.
Using CAG and IVUS precision linkage analysis method based on 3D reconstruction technology, the IVUS guide wire trajectory, discrete point interpolation, three-dimensional reconstruction and curve fitting are extracted through image preprocessing and interactive segmentation to establish the guide wire length and the IVUS sequence. The mapping relationship of the imaging image and the ultrasound image in the vascular system can be accurately linked.
It improves the accuracy of image fusion, can quickly and accurately diagnose vascular lesions and performs precise positioning, greatly improving the diagnostic efficiency of cardiovascular diseases and reducing operator intervention.
Smart Images

Figure CN112669449B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of medical image processing and medical equipment, and in particular to a CAG and IVUS precise linkage analysis method and system based on 3D reconstruction technology. Background Art
[0002] Cardiovascular diseases have become the main cause of harm to human health and death. With the deepening of population aging, the prevalence of cardiovascular diseases in my country has also increased year by year. Detecting whether coronary artery lesions such as atherosclerosis or stenosis occur in the early stage of cardiovascular disease is of great significance for the early treatment and intervention control of coronary heart disease.
[0003] There are two main methods for early diagnosis of coronary heart disease: coronary angiography (CAG) and intravascular ultrasound (IVUS). Among them, coronary angiography can more clearly reveal the anatomical malformation of the coronary artery and the location, degree and range of its obstructive lesions. It is currently the only diagnostic method that can directly observe the morphology of the coronary artery and is known as the "gold standard".
[0004] However, in actual clinical applications, due to the influence of contrast agent filling and projection angle, coronary angiography usually cannot make a correct judgment on lesions at the opening or bifurcation, and there is no way to diagnose some critical lesions. In this case, intravascular ultrasound IVUS is used for auxiliary examination. IVUS is not affected by the projection position. During the imaging process, the IVUS catheter can be sent to different branch vessels respectively, which can not only observe the morphology of the lumen, but also the structure or lesions of the wall, accurately judge the degree and range of lesion accumulation, and detect unstable plaques, spontaneous dissections and intramural hematomas. However, although IVUS can detect the cross-sectional lesions through sequence frames after imaging, which is conducive to making a more accurate judgment on the condition, it cannot know the spatial location information of the lesions. Therefore, coronary angiography is still needed for interventional treatment after diagnosis.
[0005] In the prior art, there are related studies on fusing cardiovascular images (CAG) and intracardiac ultrasound images (IVUS) to perform three-dimensional reconstruction of coronary arteries, such as the three-dimensional reconstruction method, device, equipment and storage medium of coronary blood vessels disclosed in the Chinese patent document with publication number CN107392994A, which includes: preprocessing the coronary angiography (CAG) image, extracting the edge contour of the blood vessel and the two-dimensional guide wire, and performing inner and outer membrane segmentation on the intravascular ultrasound image (IVUS); translating the two-dimensional guide wires in the CAG images located in the first angiography plane and the second angiography plane to the same starting point, constructing a vertical intersecting surface, and setting the intersection line as the three-dimensional guide wire; arranging each frame of IVUS image at equal intervals on the three-dimensional guide wire and rotating it to be perpendicular to the tangent vector at the corresponding position; rotating the IVUS image in the plane perpendicular to the tangent vector and back-projecting it on the CAG image, determining the optimal orientation angle according to the back-projection and the distance from the edge contour of the blood vessel to the three-dimensional guide wire, and finally reconstructing the blood vessel surface.
[0006] However, the basic method adopted by existing related studies is to use the centerline of the blood vessel to replace the IVUS guide wire. Due to the influence of the heartbeat, the blood vessels in the acquired angiography images will also move, and the IVUS guide wire is a rigid structure. Its trajectory in the blood vessel does not coincide with the centerline of the blood vessel, that is, the centerline of the blood vessel cannot represent the trajectory of the IVUS guide wire. Therefore, in order to improve the fusion quality of the two images, it is necessary to restore the retraction trajectory of the ultrasonic transducer required for IVUS imaging as much as possible. The ultrasonic transducer is retracted along the IVUS guide wire, so when performing a three-dimensional reconstruction operation, restoring the trajectory of the IVUS guide wire can improve the accuracy of the fusion of the two images, thereby assisting clinical diagnosis. Summary of the invention
[0007] The present invention provides a method and system for accurate linkage analysis of cardiovascular images (CAG) and intracardiac ultrasound images (IVUS) based on 3D reconstruction technology, which can combine the advantages of two different images for diagnosis, quickly and accurately obtain the status of vascular lesions and perform accurate positioning and disease assessment. The entire operation process can be completed with only a small amount of intervention by the operator.
[0008] In order to achieve the above-mentioned invention object, the following technical scheme is adopted:
[0009] A CAG and IVUS precise linkage analysis method based on 3D reconstruction technology, comprising the steps of:
[0010] 1) In the angiographic image sequences under two different angiographic viewing angles, a single angiographic image with a clear IVUS guide wire trajectory was selected and image preprocessing was performed;
[0011] 2) Interactively segment and extract the IVUS guide wire in the preprocessed single angiography image, and extract the two-dimensional trajectory coordinates of the IVUS guide wire from the two images respectively;
[0012] 3) Perform discrete point interpolation on the two sets of extracted two-dimensional trajectory coordinates, and perform three-dimensional reconstruction after obtaining the same number of discrete points;
[0013] 4) Perform curve fitting on the coordinate points obtained by three-dimensional reconstruction, calculate the trajectory length of the IVUS guide wire withdrawal, establish a mapping relationship between the guide wire length and the IVUS sequence, and realize the linkage analysis between the angiography image and the intravascular ultrasound image.
[0014] Preferably, the image preprocessing method in step 1) is: image denoising and enhancement based on stochastic resonance.
[0015] Preferably, the steps of segmenting the IVUS guide wire in the angiography image include: selecting a seed point, generating a path and triggering a segmentation end event; during the relevant operations of segmenting the guide wire in the angiography image, when the seed point selection position is not effective, the previous seed point can be revoked and a new seed point can be regenerated, and a new guide wire trajectory can be generated by clicking to ensure the overall segmentation effect of the guide wire.
[0016] In the present invention, the operations related to the IVUS guide wire in the segmentation angiography image are preferably all completed by the operator operating the mouse, such as setting the seed point, moving the mouse, generating the path, triggering the segmentation end event, etc. Different events are distinguished by mouse movement, left button, right button or double button, etc. In another preferred embodiment, the operations related to the IVUS guide wire in the segmentation angiography image can also be achieved semi-automatically or fully automatically.
[0017] Preferably, in step 2), an intelligent algorithm based on graph theory is used to interactively segment and extract the image, the cost function in the intelligent scissors algorithm commonly used in interactive segmentation is improved, and different image features are selected to construct the cost function: the image is regarded as a weighted directed graph, and each pixel in the image is a node in the directed graph. The expression for determining the edge weight is:
[0018] l(p,q)=w C gf C (q)+w S gf S (q)+w G gf G (q)+w Gpq gf Gpq (p,q)+w H gf H (q)
[0019] Where p is a pixel in the graph, q is its adjacent pixel, l(p,q) is the edge weight of the edge connecting pixel p and pixel q, and w C , w S , w G , w Gpq and w H They are weight coefficients, f C (q), f S (q), f G (q), f Gpq (p,q) and f H (q) are the image feature value cost functions based on different image features.
[0020] In the present invention, in order to increase the speed of finding the minimum cost path in the image, preferably, a Dijkstra path finding algorithm is used to generate a path based on the seed points.
[0021] Preferably, when performing three-dimensional reconstruction in step 3), the damped least squares Levenberg-Marquard algorithm is used to optimize the system matrix. This method obtains a result with global convergence by referring to the trust region method when solving the problem of local convergence of nonlinear least squares problems. In addition, the trust region method is used to ensure that the optimization result will not fall into the local optimum, thereby ensuring the accuracy of three-dimensional reconstruction.
[0022] Preferably, the corresponding relationship between the two images is determined by the linkage analysis through the mapping relationship between the current sequence of IVUS and the distance moved by the related points on the back-projected angiographic image. There are three ways to calculate the distance, as shown below:
[0023]
[0024]
[0025] Distance (mm) = withdrawal time × IVUS withdrawal rate
[0026] According to the distance calculation formula, the IVUS sequence frame corresponding to each point on the IVUS guide wire in the angiography image can be determined. When the observation point of interest moves on the angiography image, the IVUS sequence frame at the corresponding position will also move; when the IVUS sequence frame changes, the corresponding point on the angiography image will also change accordingly. That is, if any one of the observation point on the IVUS guide wire in the angiography image, the IVUS sequence frame, and the corresponding position of the IVUS longitudinal section changes, the other two will also change accordingly.
[0027] The linkage analysis of the present invention is not only a corresponding linkage analysis operation between the cardiovascular angiography image and the cardiovascular intra-ultrasound image, but also can perform a linkage analysis operation between the longitudinal section of the cardiovascular intra-ultrasound image and the two. The longitudinal section of the cardiovascular intra-ultrasound image can be read from a related cardiovascular intra-ultrasound image imaging instrument to jointly complete the linkage analysis, assist in the diagnosis of related diseases, and quickly and accurately locate the disease.
[0028] The present invention also provides a CAG and IVUS precise linkage analysis system based on 3D reconstruction technology, comprising:
[0029] Memory, storing computer-related executable instructions, computer-related programs, and various data required in the linkage analysis process;
[0030] The processor executes the above-mentioned CAG and IVUS precise linkage analysis method according to the data in the memory in steps.
[0031] In the present invention, when the system imports cardiovascular ultrasound images for linkage analysis, all operations are completed by the mouse, and linkage analysis of cardiovascular angiography images, cardiovascular ultrasound image sequences and their longitudinal sections can be achieved. When the mouse operates one of the objects, the corresponding other two images will also change at the same time. By locating an observation point, the information of the three types of images can be displayed accordingly.
[0032] Compared with the prior art, the present invention has the following beneficial effects:
[0033] The precise linkage analysis system of the present invention can effectively combine the advantages of cardiovascular angiography images and intracardiac ultrasound images. By directly restoring the motion trajectory of the ultrasonic transducer used for IVUS imaging, the improved intelligent scissors algorithm is used for interactive segmentation to directly extract the IVUS guide wire trajectory in the angiography image, rather than using the center line of the blood vessel or generating curves in the blood vessel to replace the IVUS guide wire trajectory. The improved intelligent scissors algorithm has a faster segmentation speed, and the curve edge is smoother and has fewer jagged edges. When solving the objective function in three-dimensional reconstruction, the trust region method is used. During the iteration process, the parameters are continuously updated according to the ratio of the actual descent amount to the predicted descent amount, so that it has global convergence, and then the three-dimensional IVUS guide wire is reconstructed. Finally, based on the mapping relationship between the guide wire length and the IVUS sequence, the linkage analysis between the cardiovascular angiography image, the intracardiac ultrasound image sequence and its longitudinal section can be accurately realized, and the interaction is convenient, which greatly improves the diagnostic efficiency of related diseases. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1It is a schematic diagram of the principle of accurate linkage analysis of cardiovascular images (CAG) and intracardiac ultrasound images (IVUS) based on 3D reconstruction technology in an embodiment of the present invention;
[0035] Figure 2 A technical flow chart of linkage analysis in an embodiment of the present invention;
[0036] Figure 3 A schematic diagram of segmentation operation of angiography images in an embodiment of the present invention;
[0037] Figure 4 4 is a schematic diagram of a three-dimensional reconstruction module in an embodiment of the present invention. DETAILED DESCRIPTION
[0038] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the present invention is not limited to the specific embodiments disclosed below.
[0039] See also Figures 1 to 4 The specific implementation steps of the CAG and IVUS precise linkage analysis method based on 3D reconstruction technology in this embodiment are as follows:
[0040] 1) Import two cardiovascular angiography image sequences generated by a single-plane angiography system under different angiography viewing angles, and select a single angiography image with a clear IVUS guide wire trajectory, such as Figure 2 shown.
[0041] 2) Perform image preprocessing operations, such as random resonance denoising and enhancement on the contrast image, to reduce the noise in the image and make the interesting part of the image easier to segment, such as Figure 2 shown.
[0042] 3) Use the mouse to interactively extract the IVUS guide wire trajectory in the image, and extract the two-dimensional trajectory coordinates of the IVUS guide wire, such as Figure 2 Along the edge of the guide wire track in the contrast image, click a seed point with the left mouse button. The seed point will be displayed as a blue dot on the image. Then move the mouse. The path of the mouse movement will automatically fit on the IVUS guide wire track, generating a green line until the segmentation of the entire guide wire track is completed. Figure 3 shown.
[0043] 4) For the two sets of IVUS guide wire trajectory coordinate points extracted, discrete point interpolation operations are performed to interpolate the same number of discrete points, such as Figure 2 shown.
[0044] 5) Based on the two sets of two-dimensional coordinate points obtained in step 4), three-dimensional reconstruction of the coordinate points is performed. The three-dimensional reconstruction method uses two-dimensional coordinate points as the primitives for stereo matching and three-dimensional reconstruction, and needs to be matched and reconstructed point by point. At the same time, the parameter optimization operation of the system's geometric transformation matrix is also required.
[0045] 6) For the three-dimensional coordinate points obtained in step 5), a curve fitting operation is performed, and the length of the entire IVUS guide wire withdrawal route is calculated using the B-spline fitting method in computer graphics, such as Figure 2 shown.
[0046] 7) According to the retracement path length obtained in step 6), the corresponding IVUS image sequence is imported, and based on the correspondence between the total retracement path length and the IVUS sequence, the linkage analysis among the angiography image (CAG), the IVUS image sequence and the IVUS longitudinal section is realized. After obtaining the expected effect, exit the system.
[0047] In this embodiment, the process of operating the mouse to perform interactive segmentation operations adopts an intelligent algorithm based on graph theory. The algorithm improves the cost function in the original intelligent scissors algorithm. The new cost function is constructed by a series of new image feature value functions, which maximizes the cost value difference between the edge part and the non-edge part. The image is regarded as a weighted directed graph to set weights for the edges formed by each pixel point on the image and its adjacent pixels. The expression is as follows:
[0048] l(p,q)=w C gf C (q)+w S gf S (q)+w G gf G (q)+w Gpq gf Gpq (p,q)+w H gf H (q)
[0049] Where p is a pixel in the graph, q is its adjacent pixel, l(p,q) is the edge weight of the edge connecting pixel p and pixel q, and w C , w S , w G , w Gpq and w H They are weight coefficients, f C (q), f S (q), f G (q), f Gpq (p,q) and f H(q) are respectively the image feature value cost function of pixel q based on the Canny operator, the image feature cost value function of pixel q based on the Scharr operator, the gradient amplitude value of pixel q, the gradient transformation adjustment function of pixel p and pixel q, and the histogram adjustment function of pixel q.
[0050] like Figure 3 As shown in the figure, by calculating the weight values of each pixel point on the image and the surrounding pixels, the entire weighted directed graph can be determined. Each seed point set by the user through mouse operation is a point in this weighted directed graph. The shortest path algorithm can be used to find the shortest path between the seed points set by the user. As long as the seed point setting is reasonable, the shortest path found is the path with the minimum cost, that is, the retracement trajectory path of the IVUS guide wire to be segmented.
[0051] In addition, in order to speed up the calculation of the shortest path, this embodiment adopts a fast Dijkstra path-finding algorithm, which enables the mouse to quickly find a path that satisfies the minimum cost when moving, and serves as the retraction trajectory path of the IVUS guide wire.
[0052] In this embodiment, in order to obtain images at the same cardiac moment, ECG gating method is used to collect data.
[0053] In this embodiment, in order to perform three-dimensional reconstruction, it is necessary to obtain the imaging parameters of the imaging equipment, including the distance between the emission source and the image intensifier, the distance between the emission source and the target, the field of view, the radiation magnification, the pixel width, the left and right angles, the front and back angles, etc.
[0054] In this embodiment, when performing three-dimensional reconstruction, in order to ensure the correspondence between points under two perspectives in the three-dimensional reconstruction, the two sets of discrete coordinate points interactively extracted from two different perspectives are combined into the same number of discrete points through the B-spline interpolation method, and then the objective function to be optimized is obtained according to the rotation matrix, and the objective function is solved.
[0055] The objective function is a nonlinear least squares problem. This embodiment uses the damped least squares Levenberg-Marquardt method to optimize and solve it, and solves its possible local convergence and other problems through the trust region method. During the iteration process, the parameters are continuously updated according to the ratio of the actual descent to the predicted descent, so that it has global convergence and will not fall into the local optimum. The result is more robust, which is the key to ensuring the accuracy of three-dimensional reconstruction.
[0056] In this embodiment, various parameters and information required in all steps can be obtained from the Tag information in the DICOM sequence corresponding to the cardiovascular angiography image (CAG) and the intracardiac ultrasound image (IVUS).
[0057] In another embodiment, a precise linkage analysis system of cardiovascular angiography (CAG) and intracardiac ultrasound (IVUS) images based on 3D reconstruction technology comprises:
[0058] Memory, storing computer-related executable instructions, computer-related programs, and various data required in the linkage analysis process;
[0059] The processor can be used in combination with the memory and can run the system program, execute the relevant operation instructions, and complete the above-mentioned CAG and IVUS precise linkage analysis method step by step;
[0060] Input and output devices to meet the needs of interaction with users, including mouse, monitor or other touch input devices.
[0061] The above description is only an example of a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A CAG and IVUS precise linkage analysis method based on 3D reconstruction technology, characterized in that: Includes steps: 1) In the angiographic image sequences under two different angiographic viewing angles, a single angiographic image with a clear IVUS guide wire trajectory was selected and image preprocessing was performed; 2) Interactively segment and extract the IVUS guide wire in the preprocessed single angiography image, and extract the two-dimensional trajectory coordinates of the IVUS guide wire from the two images respectively; 3) Perform discrete point interpolation on the two sets of extracted two-dimensional trajectory coordinates, and perform three-dimensional reconstruction after obtaining the same number of discrete points; 4) Perform curve fitting on the coordinate points obtained by 3D reconstruction, calculate the trajectory length of the IVUS guide wire withdrawal, establish a mapping relationship between the guide wire length and the IVUS sequence, and realize the linkage analysis between the angiography image and the intravascular ultrasound image; The corresponding relationship between the two images is determined by the mapping relationship between the current sequence of IVUS and the distance moved by the back-projected related points on the angiographic image. There are three ways to calculate the distance, as shown below: Distance (mm) = withdrawal time × IVUS withdrawal rate According to the distance calculation formula, the IVUS sequence frame corresponding to each point on the IVUS guide wire in the angiography image can be determined. When the observation point of interest moves on the angiography image, the IVUS sequence frame at the corresponding position and the longitudinal section where it is located will also move; when the IVUS sequence frame changes, the corresponding point on the angiography image will also change accordingly.
2. The CAG and IVUS precise linkage analysis method based on 3D reconstruction technology according to claim 1, characterized in that: The image preprocessing method in step 1) is: image denoising and enhancement based on stochastic resonance.
3. The CAG and IVUS precise linkage analysis method based on 3D reconstruction technology according to claim 1, characterized in that: The steps of segmenting the IVUS guide wire in the angiography image include: selecting a seed point, generating a path, and triggering a segmentation end event; When the seed point selection result is not good, you can cancel the previous seed point and regenerate a new seed point.
4. The CAG and IVUS precise linkage analysis method based on 3D reconstruction technology according to claim 3, characterized in that: In step 2), the intelligent scissors algorithm is used to interactively segment and extract the image: the image is regarded as a weighted directed graph, and each pixel in the image is a node in the directed graph. The expression for determining the edge weight is: l(p,q)=w C ·f C (q)+w S ·f S (q)+w G ·f G (q)+w Gpq ·f Gpq (p,q)+w H ·f H (q) Where p is a pixel in the graph, q is its adjacent pixel, l(p,q) is the edge weight of the edge connecting pixel p and pixel q, and w C , w S , w G , w Gpq and w H They are weight coefficients, f C (q), f S (q), f G (q), f Gpq (p,q) and f H (q) are the image feature value cost functions based on different image features.
5. The CAG and IVUS precise linkage analysis method based on 3D reconstruction technology according to claim 3, characterized in that: Based on the seed points, a path is generated using the Dijkstra pathfinding algorithm.
6. The CAG and IVUS precise linkage analysis method based on 3D reconstruction technology according to claim 1, characterized in that: When performing three-dimensional reconstruction in step 3), the damped least squares Levenberg-Marquard algorithm is used to optimize the system matrix.
7. A CAG and IVUS precise linkage analysis system based on 3D reconstruction technology, characterized in that: include: Memory, storing computer-related executable instructions, computer-related programs, and various data required in the linkage analysis process; The processor executes the CAG and IVUS precise linkage analysis method as described in any one of claims 1 to 6 according to the data in the memory.
