Blood vessel three-dimensional reconstruction method based on single-view coronary angiography sequence and related equipment

Abstract

The invention provides a blood vessel three-dimensional reconstruction method based on a single-angle coronary angiography sequence and related equipment. The method comprises the following steps: acquiring a single-angle coronary angiography image sequence; segmenting each frame of coronary angiography image to obtain a corresponding blood vessel structure image; extracting a 2D center line of a blood vessel in each frame of blood vessel structure image, marking a plurality of key points on the 2D center line of one frame of blood vessel structure image, and identifying a corresponding position of each key point in other blood vessel structure images; each key point divides the blood vessel into a plurality of sections, the form of the 2D center line of the blood vessel in the three-dimensional space is determined based on the difference of each section of blood vessel in each frame of blood vessel structure image, and the 3D center line of the blood vessel is obtained; and constructing a three-dimensional blood vessel by taking the 3D center line of the blood vessel as the center According to the method, the smooth coronary artery three-dimensional structure can be reconstructed only depending on the single-view coronary artery angiography sequence.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional reconstruction of blood vessels, and in particular to a three-dimensional reconstruction method of blood vessels based on a single-view coronary angiography sequence and related equipment. Background Art

[0002] With the advancement of science and technology, medical imaging technologies such as X-rays, CT (computed tomography), and MRI (magnetic resonance imaging) are increasingly being used in the diagnosis of cardiovascular diseases. These medical images provide doctors with detailed views of the internal structure of the patient's heart and blood vessels to determine the severity and possible causes of the disease. In the direction of coronary angiography, it is difficult for doctors to further determine the extent of vascular lesions through single-view angiography. Usually, it is necessary to perform three-dimensional reconstruction of blood vessels based on coronary angiography images to more accurately display the location, degree, and length of coronary stenosis, so as to more intuitively reflect the patient's coronary artery morphology and facilitate the doctor's observation and evaluation; the three-dimensional reconstruction of blood vessels can also provide accurate positioning information for interventional surgery. Doctors can also simulate the surgical process on the three-dimensional reconstructed images to evaluate the surgical risks and feasibility, thereby formulating a more reasonable surgical plan.

[0003] At present, vascular 3D reconstruction technology usually requires medical images from at least two perspectives to provide spatial information in order to construct the three-dimensional structure of the blood vessels. This method requires estimating the matching relationship between these perspectives, which has certain inaccuracies. Moreover, image data is sometimes limited, and coronary angiography sometimes cannot meet the multi-perspective conditions, which limits the implementation of vascular 3D reconstruction technology. For example, a single-arm angiography system can only obtain coronary angiography images from one angle at a time. If multi-perspective imaging is required, it is necessary to perform angiography from multiple angles at different time points. Due to the complex movement of the heart and the continuous dilation and contraction of blood vessels, vascular 3D reconstruction in this case will introduce certain errors. Summary of the invention

[0004] The purpose of the present invention is to provide a method and related equipment for three-dimensional reconstruction of blood vessels based on a single-view coronary angiography sequence to solve the above-mentioned problem. A relatively smooth three-dimensional structure of the coronary artery can be reconstructed by relying only on a single-view coronary angiography sequence.

[0005] The present invention proposes a method for three-dimensional reconstruction of blood vessels based on a single-view coronary angiography sequence, comprising the following steps:

[0006] Acquire a single-angle coronary angiography image sequence;

[0007] Segment each frame of coronary angiography image to obtain the corresponding vascular structure image;

[0008] Extracting the 2D centerline of the blood vessel in each frame of the blood vessel structure image, marking a number of key points on the 2D centerline of one frame of the blood vessel structure image, and identifying the corresponding positions of each key point in other blood vessel structure images;

[0009] Each key point divides the blood vessel into several segments, and the shape of the 2D centerline of the blood vessel in three-dimensional space is determined based on the difference between each segment of the blood vessel in each frame of the blood vessel structure image, thereby obtaining the 3D centerline of the blood vessel;

[0010] Construct a three-dimensional blood vessel with the 3D centerline of the blood vessel as the center.

[0011] In one embodiment, the segmentation of each frame of the coronary angiography image to obtain the corresponding vascular structure image specifically includes:

[0012] The vascular structure in the coronary angiography image is defined as the foreground, and the surrounding environment of the blood vessels is defined as the background;

[0013] The foreground and background are separated and the vascular structure image is extracted.

[0014] In one embodiment, when identifying the corresponding positions of each key point in other vascular structure images, there are the following two situations:

[0015] If the position of the key point identified in other vascular structure images falls on the 2D centerline of the blood vessel, then the position is taken as the corresponding key point;

[0016] If the position of the key point identified in other vascular structure images does not fall on the 2D centerline of the blood vessel, the point on the 2D centerline closest to the position is selected as the corresponding key point.

[0017] In one embodiment, determining the shape of the 2D centerline of the blood vessel in three-dimensional space based on the difference between each blood vessel segment in each frame of the blood vessel structure image to obtain the 3D centerline of the blood vessel specifically includes:

[0018] Calculating the shortening rate of each blood vessel segment based on the difference between each blood vessel segment in each frame of the blood vessel structure image, wherein the shortening rate is used to characterize the shortening degree of the blood vessel in the plane where the image is located;

[0019] The curvature of the 2D centerline of the blood vessel in three-dimensional space is determined according to the shortening rate of each segment of the blood vessel, and the 3D centerline of the blood vessel is obtained.

[0020] In one embodiment, determining the curvature of the 2D centerline of the blood vessel in three-dimensional space according to the shortening rate of each segment of the blood vessel to obtain the 3D centerline of the blood vessel specifically includes:

[0021] Set the shortening rate threshold;

[0022] Determine the bending direction of each blood vessel segment: determine the bending direction of the blood vessel segments on both sides thereof based on the blood vessel segment that meets the shortening rate threshold, wherein the bending direction is bending inward or outward of the image;

[0023] Determine the bending angle of each blood vessel segment: the bending angle is the angle between the blood vessel and the plane where the image is located, and the blood vessel bending angle is obtained according to the shortening rate of the blood vessel;

[0024] The 3D centerline of the blood vessel is obtained based on the 2D centerline of the blood vessel and the bending direction and bending angle of each segment of the blood vessel.

[0025] In one embodiment, the determining of the bending directions of the blood vessel segments on both sides based on the blood vessel segment that meets the shortening rate threshold specifically includes:

[0026] The blood vessel segments on both sides of the blood vessel segment that meets the shortening rate threshold have opposite bending directions, that is, of the blood vessel segments on both sides of the blood vessel segment that meets the shortening rate threshold, one blood vessel segment bends inwardly of the image, and the other blood vessel segment bends outwardly of the image;

[0027] The bending directions of all the blood vessel segments between every two blood vessel segments that meet the shortening rate threshold are the same, that is, all the blood vessel segments between every two blood vessel segments that meet the shortening rate threshold are bent inward or outward of the image.

[0028] In one embodiment, the differences between the blood vessel segments in each frame of the blood vessel structure image include any one or more of length difference, contour difference, and angle difference.

[0029] In one embodiment, constructing a three-dimensional blood vessel with the 3D centerline of the blood vessel as the center specifically includes:

[0030] Mark multiple points of interest on the 2D centerline of the blood vessel;

[0031] Obtaining the blood vessel diameter at each point of interest through the blood vessel structure image;

[0032] With each corresponding interest point on the 3D center line of the blood vessel as the center, draw a circle with a diameter equal to the diameter of the blood vessel, and each circle forms a series of parallel contour lines;

[0033] Mark a number of sampling points evenly and correspondingly on each circle;

[0034] Constraints are set to connect the sampling points on adjacent contour lines to form the three-dimensional surface of the blood vessel.

[0035] The present invention also proposes a 3D blood vessel reconstruction device based on a single-view coronary angiography sequence, which is used to implement the 3D blood vessel reconstruction method based on a single-view coronary angiography sequence as described above, comprising:

[0036] An input module, used for inputting a coronary angiography image sequence of a single angle;

[0037] A blood vessel segmentation module is used to segment the blood vessel structure from the coronary angiography image to obtain a blood vessel structure image;

[0038] A centerline extraction module, used to extract the 2D centerline of the blood vessel in the blood vessel structure image;

[0039] A custom setting module for marking key points on the 2D centerline and dividing the blood vessels into several segments;

[0040] A tracking module, used to track the corresponding positions of key points in the image sequence;

[0041] A calculation module, used for solving the 3D centerline of the blood vessel according to the difference between each segment of the blood vessel in each frame of the blood vessel structure image;

[0042] A vascular three-dimensional imaging module is used to construct a three-dimensional surface of a blood vessel with the 3D centerline of the blood vessel as the center;

[0043] The output module is used to output the three-dimensional image of the blood vessel.

[0044] The present invention also proposes a computer device, including a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the above-mentioned method for three-dimensional reconstruction of blood vessels based on a single-view coronary angiography sequence.

[0045] The present invention also proposes a computer storage medium, on which a computer program is stored. When the computer program is executed by a processor, the three-dimensional blood vessel reconstruction method based on a single-view coronary angiography sequence as described above is implemented.

[0046] Compared with the prior art, the method for three-dimensional reconstruction of blood vessels based on a single-view coronary angiography sequence and the related equipment of the present invention have the following beneficial effects:

[0047] 1) The present invention can reconstruct a relatively smooth three-dimensional structure of the coronary artery by relying only on a single-view coronary angiography sequence, thereby helping doctors to better obtain vascular structure information and further diagnose the degree of vascular lesions.

[0048] 2) The present invention makes full use of centerline tracking and foreshortening information to estimate the depth information of the 2D centerline, which is used to calculate the three-dimensional blood vessel centerline, and can obtain a more accurate three-dimensional coronary artery model. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a schematic flow chart of a method for three-dimensional reconstruction of blood vessels based on a single-view coronary angiography sequence according to an embodiment of the present invention;

[0050] Figure 2aA coronary angiography image according to an embodiment of the present invention;

[0051] Figure 2b To use segmentation algorithm Figure 2a The vascular structure image obtained from the coronary angiography image;

[0052] Figure 2c For extraction Figure 2b 2D centerline image of blood vessels in vascular structure images;

[0053] Figure 3 A schematic diagram of the PIPs++ model structure according to an embodiment of the present invention;

[0054] Figure 4 A schematic diagram of the bending angle of a certain blood vessel section according to an embodiment of the present invention;

[0055] Figure 5 A schematic diagram of continuous bending of a blood vessel section in the yz plane according to an embodiment of the present invention;

[0056] Figure 6 This is a schematic diagram of the shortest diagonal method according to an embodiment of the present invention;

[0057] Figure 7 It is a schematic structural diagram of a blood vessel three-dimensional reconstruction device based on a single-view coronary angiography sequence according to an embodiment of the present invention.

[0058] Reference numerals

[0059] 100. A vascular three-dimensional reconstruction device, 11. An input module, 12. A vascular segmentation module, 13. A centerline extraction module, 14. A custom setting module, 15. A tracking module, 16. An operation module, 17. A vascular three-dimensional imaging module, 18. An output module. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the present invention more understandable, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments of the specification. It should be noted that many specific details are set forth in the following description to facilitate a full understanding of the present invention, but the present invention can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present invention, so the present invention is not limited by the specific embodiments disclosed below.

[0061] Secondly, "one embodiment" or "an embodiment" in this application refers to a specific feature, structure or characteristic that can be included in at least one implementation of the present invention. The "in one embodiment" and "an embodiment" that appear in different places in this specification do not all refer to the same embodiment, nor are they separate or selective embodiments that are mutually exclusive with other embodiments. The terms "including" and "comprising" indicate the presence of the claimed features, but do not exclude the presence of one or more other features. The term "and / or" used in this application includes any and all combinations of one or more of the relevant listed items.

[0062] The present invention proposes a method for three-dimensional reconstruction of blood vessels based on a single-view coronary angiography sequence. Figure 1 , including the following steps:

[0063] A coronary angiography image sequence at a single angle (i.e., a single-view coronary angiography sequence) is acquired, wherein the coronary angiography image sequence includes coronary angiography images covering at least one complete cardiac cycle, such as Figure 2a As shown, the number should be as large as possible to facilitate the subsequent calculation of various bending angles;

[0064] Each frame of coronary angiography image is segmented to obtain the corresponding vascular structure image, see Figure 2b ;

[0065] Extract the 2D centerline of the blood vessel in each frame of the blood vessel structure image, such as Figure 2c As shown, a number of key points are marked on the 2D center line of one frame of the vascular structure image, and the corresponding positions of each key point in other vascular structure images are identified;

[0066] Each key point divides the blood vessel into several segments, and based on the differences of each segment of the blood vessel in each frame of the blood vessel structure image (including any one or more of length difference, contour difference, and angle difference), the shape of the 2D centerline of the blood vessel in three-dimensional space is determined to obtain the 3D centerline of the blood vessel;

[0067] Construct a three-dimensional blood vessel with the 3D centerline of the blood vessel as the center.

[0068] When the above-mentioned 3D reconstruction method of blood vessels is used to perform 3D reconstruction of 2D blood vessels based on a certain frame of vascular structure images, the static 3D structure of the blood vessels at that moment can be obtained; and if the 3D reconstruction of blood vessels is performed based on the vascular structure images of all frames, the 3D morphological changes of blood vessels at different time points or in different states can be obtained, that is, dynamic 3D blood vessels. Static 3D reconstruction of blood vessels can clearly display the anatomical structure and morphology of blood vessels, which helps doctors to accurately diagnose and evaluate vascular lesions. It can usually be used to analyze lesions such as stenosis, dilation, aneurysm, and plan surgical routes. Dynamic 3D reconstruction of blood vessels can capture the 3D morphological changes of blood vessels at different time points or in different states, so as to have a more comprehensive understanding of the function and physiological state of blood vessels, and help doctors more accurately evaluate the hemodynamic parameters of blood vessels, such as blood flow velocity, blood flow, etc., and predict the development trend of vascular lesions.

[0069] The method used to segment each frame of the coronary angiography image to obtain the corresponding vascular structure image is a segmentation algorithm, and the specific operating principle is: define the vascular structure in the coronary angiography image as the foreground and the surrounding environment of the blood vessel as the background; then, use the segmentation algorithm to distinguish the foreground from the background, and extract the vascular structure image. Among them, the segmentation algorithm refers to an algorithm that can segment an image into a foreground and a background, or distinguish different objects or regions in an image. The segmentation algorithm may perform segmentation based on features such as color, texture, shape, and edge. In medical image processing, the segmentation algorithm is often used to extract specific anatomical structures, such as blood vessels, organs, or tumors.

[0070] The segmentation algorithm can be either a traditional image segmentation algorithm or a segmentation algorithm based on a neural network. Traditional image segmentation algorithms include the Level Set algorithm, Conditional Random Field (CRF), etc., while segmentation algorithms based on neural networks include UNet, VNet, etc.

[0071] In one embodiment of the present invention, the UNet algorithm is used to process the coronary angiography image for image segmentation: by annotating the vascular segmentation data set and training the network to achieve image segmentation, a vascular structure image is obtained, where the foreground (vascular structure) is 1 and the background is 0.

[0072] Methods for extracting the 2D centerline of the blood vessel in each frame of the vascular structure image include but are not limited to manual calibration, topological refinement, distance transformation, AI network training, etc. The manual calibration method is to manually observe and analyze the vascular image and use marking tools (such as a mouse, a brush, etc.) to manually draw the centerline of the blood vessel. This method relies on the operator's experience and accuracy and is suitable for accurate processing of a small number of images or verifying the accuracy of other automatic methods. The topological refinement method is based on the morphological operation of the image. By gradually stripping off the pixels at the edge of the image until only the pixels representing the vascular skeleton, i.e., the centerline, remain, the core of this method is to keep the connectivity and topological structure of the blood vessel unchanged while removing unnecessary detail information. The operation process of the distance transformation method is mainly: first determine the foreground (blood vessel) and background in the image, then calculate the distance from each foreground pixel to the nearest background pixel. Through the distance transformation, the vascular image can be converted into a distance image, in which the pixels on the centerline usually have the largest distance value (the farthest distance to the background). Then, the position of the centerline is determined by finding the local maximum value or a specific threshold in the distance image. The AI ​​network training method mainly uses deep learning technology to automatically identify the centerline of blood vessels by training neural network models. This method requires a large amount of labeled data for training. The model can learn the morphology, structure and possible variations of blood vessels. In the inference stage, the model can automatically extract the centerline of new blood vessel images with high accuracy and robustness.

[0073] In one embodiment of the present invention, a topological thinning method is used to extract the 2D centerline of the vascular structure image obtained by segmentation using the UNet algorithm. The specific operation steps are as follows:

[0074] 1) Set the loop flag to 0, use the structure element to corrode the vascular structure image, and retain the result after corrosion;

[0075] 2) If there is still a point of 1 in the result after corrosion (binary pixel value of blood vessel area), the binary image is not completely corroded, and the loop flag is set to 1 (indicating continued looping);

[0076] 3) Use the structural element to perform an opening operation on the eroded image (erode first and then expand) to eliminate small white areas;

[0077] 4) Subtract the image before and after the opening operation to obtain the eliminated white area. The white area here generally refers to very thin lines that cannot expand back after being mistakenly eroded, and it is considered to be part of the skeleton. By subtracting the image before and after the opening operation, the eroded white area can be obtained;

[0078] 5) Union the white area with the skeleton obtained in the previous cycle to get a new skeleton;

[0079] 6) assigning the corrosion result to the original segmented vascular structure image;

[0080] 7) Repeat steps 1 to 6 until the image is eroded into an empty set;

[0081] 8) The final skeleton obtained is the refined 2D centerline.

[0082] The method used to identify the corresponding positions of each key point in other vascular structure images is a tracking algorithm. A tracking algorithm refers to a method for tracking the position of a target in a sequence of continuous frame images to determine the position, motion trajectory and other related attributes of the target object in different time periods.

[0083] The tracking algorithm can be a tracking algorithm based on traditional features or a tracking algorithm based on deep learning. The tracking algorithms based on traditional features include feature point matching algorithm, optical flow matching algorithm, etc. The tracking algorithms based on deep learning include STARK algorithm, RAFT algorithm, PIPs++ model, etc.

[0084] The feature point matching algorithm extracts significant feature points (such as corner points, edge points or spots) in the image and tracks the movement of the target through the correspondence between these feature points in different frames. These feature points are usually determined by calculating the local gradient, curvature or other image properties of the image. Then, the algorithm will try to find corresponding points that match these feature points in subsequent frames to estimate the displacement and possible deformation of the target. The optical flow matching algorithm is a target tracking method based on image brightness changes. The algorithm assumes that the brightness of pixels in the image remains unchanged between consecutive frames, and calculates the motion vector (i.e., optical flow) of each pixel based on this, and estimates the target's motion trajectory by comparing the brightness changes of pixels in adjacent frames.

[0085] The main component of STARK is Transformer (the transformer implemented by DETRQ is used in the code). The encoder in Transformer is used to model the global spatiotemporal feature correlation between the target object and the search area, and the decoder in Transfommer is used to learn query embedding to predict the spatial position of the target object. RAFT (Recurrent Appearance Flow) is an optical flow estimation framework based on recurrent neural network (RNN). It achieves accurate pixel-level matching by iteratively updating the correspondence. The model uses a feature pyramid structure to handle motions of different scales and merges previous optical flow estimates at each iteration to effectively utilize historical information and enhance the understanding of long-term dependencies. PIPs++ is designed to address the limitations of existing point tracking algorithms on long sequences. It uses an advanced neural network architecture, which can achieve stable and accurate point trajectory prediction in complex environments and can process video sequences up to thousands of frames.

[0086] In one embodiment of the present invention, a PIPs++ model is used to track key points on a 2D centerline.

[0087] Before putting it into use, the PIPs++ model needs to be trained. The specific training process is as follows:

[0088] 1) Data acquisition: collecting multiple groups of coronary angiography sequences, each group of coronary angiography sequences includes several frames of coronary angiography images. In this embodiment, the frame rate of the coronary angiography images is 15 fps;

[0089] 2) Data annotation: Annotate two points on the main branch vessels, and annotate the corresponding positions of the two points every 3-4 frames in the sequence. Divide the annotated images into training sets and test sets. The training set contains 2,000 sets of images and corresponding points, and the test set contains 500 sets of images and corresponding points.

[0090] 3) PIPs++ model training optimization: The training of the PIPs++ model is based on a single card gtx4090. Set training parameters: Set the batch size batchsize to 16, which means that in each training iteration, the network will process 16 images at the same time. Input each group of images in the training set into the PIPs++ model. The training set input is 3 images at different times, each of which is marked with 2 feature points. Use the 2 coordinate points of one image to predict the coordinates of the corresponding points of the other 2 images. Calculate the loss function based on the predicted position points and the pre-marked feature position points, and iterate the parameters of the tracking neural network through the loss function until the loss function is less than the preset threshold, and judge that the tracking neural network training is completed. In this embodiment, the loss function is L1smooth loss, admw is used as the optimizer, the learning rate is set to 5e-5, and the weight regularization coefficient is set to 0.001.

[0091] 4) PIPs++ model testing: Each group of images in the test set is input into the trained PIPs++ model to evaluate the performance of the trained PIPs++ model.

[0092] Use the trained PIPs++ model to track key points on the 2D centerline. Figure 3 , the specific operations are:

[0093] ResNet50 is used to extract the feature information in each frame of the image and output the feature map corresponding to each frame of the image;

[0094] Based on the feature map, for each marked coordinate point, bilinear interpolation is used to extract the feature vector F corresponding to the current coordinate point from the feature map;

[0095] Based on the feature vector F, feature map and current coordinate point, multi-scale cropping is performed on the neighborhood of the current coordinate point to generate a feature block, and the feature block is multiplied with the feature map of other frames to construct a correlation matrix C. The correlation matrix C is used to calculate the correlation between different frames, and contains information about the number of frames T, the neighborhood size P and the number of pyramid scales L, and its size is (T*P*P*L);

[0096] Based on the feature vector F, the correlation matrix C and the current coordinate point, the coordinate dx and the feature increment df are obtained through the multi-layer perceptron MLP. The coordinates and features are iterated multiple times using the obtained increments to obtain the coordinate point x after the current coordinate point is tracked. k =x k-1 +dx.

[0097] When using the tracking algorithm to identify the corresponding positions of each key point in other vascular structure images, there are two situations:

[0098] If the position of the key point identified in other vascular structure images falls on the 2D centerline of the blood vessel, then the position is taken as the corresponding key point;

[0099] If the position of the key point identified in other vascular structure images does not fall on the 2D centerline of the blood vessel, the point on the 2D centerline closest to the position is selected as the corresponding key point.

[0100] Based on the differences between each segment of the blood vessel in each frame of the blood vessel structure image, the shape of the 2D centerline of the blood vessel in the three-dimensional space is determined to obtain the 3D centerline of the blood vessel. Taking the length difference as an example, the specific operation process includes:

[0101] Calculating the shortening rate of each blood vessel segment based on the length of each blood vessel segment in each frame of the blood vessel structure image, wherein the shortening rate is used to characterize the shortening degree of the blood vessel in the plane where the image is located;

[0102] The curvature of the 2D centerline of the blood vessel in three-dimensional space is determined according to the shortening rate of each segment of the blood vessel, and the 3D centerline of the blood vessel is obtained.

[0103] The calculation principle of the shortening rate of each segment of the blood vessel is as follows: mark several key points P on the 2D center line of a certain image n , the corresponding key point P can also be obtained in other images using the tracking algorithm n Each key point divides the blood vessel into segments. For a certain segment, k P k+1 , the corresponding segment P in other frames can be calculated k 'P k+1 ', among the corresponding line segments of all frames, the longest one is x k max Then, for a certain blood vessel in a certain frame, based on its length in the current image and its maximum length in all frame images, its shortening rate can be calculated to characterize its shortening degree relative to the plane where the image is located.

[0104] The following is a detailed description of the basic principle of determining the curvature of the 2D centerline of a blood vessel in three-dimensional space based on the shortening rate of each segment of the blood vessel:

[0105] The plane where the acquired image is located is the xy plane, and the positive direction of the z axis is perpendicular to the plane where the image is located and inward, and the xyz rectangular coordinate system of the coronary blood vessel three-dimensional space is established. It is assumed that when a certain blood vessel is the longest in all frame images, its direction is parallel to the xy plane (image plane), perpendicular to the z axis, and the bending angle is 0; when it is in other images that are not the longest, it means that it has bent in the z axis direction, and its shortening rate is related to its deflection angle in the z axis direction, such as Figure 4As shown in the figure, there are two bending directions for a blood vessel segment, one is in the positive direction of the z axis (perpendicular to the image and toward the inside of the image), and the other is in the negative direction of the z axis (perpendicular to the image and toward the outside of the image). Combined with other blood vessel segments, the 3D centerline of a blood vessel in an image is a multi-segment broken line in the zy plane, see Figure 5 .

[0106] The shortening rate R of a blood vessel can be expressed as: x k is the length of the blood vessel in a certain frame of image, is the maximum length of the blood vessel in all frame images. At this time, the calculation formula of the corresponding blood vessel bending angle β is: β = arccos (R), such as Figure 4 Of course, the shortening rate of blood vessels can also be expressed as Accordingly, the calculation formula for the bending angle should also be changed to β=arccos(1-R).

[0107] Since the inclination of each segment has directional ambiguity, the direction of the inclination of each segment needs to be determined. The inclination direction of each segment of blood vessels between two segments of non-shortened blood vessels (blood vessels parallel to the xy plane) remains consistent, making the curvature of the blood vessels smoother. Each time a segment of non-shortened blood vessels (blood vessels parallel to the xy plane) is passed, the inclination direction of the blood vessels reverses. Figure 6 As shown, before point A, the slope of the vascular segment is negative, the length of the blood vessel in segment AB reaches the local maximum without shortening, the slope of the blood vessel inclination between segments BC is positive, the length of the blood vessel in segment CD reaches the local maximum without shortening, and the slope of the blood vessel inclination becomes negative again after point D.

[0108] Then, after determining the curvature of the 2D centerline of the blood vessel in three-dimensional space according to the shortening rate of each segment of the blood vessel, the specific process of obtaining the 3D centerline of the blood vessel is as follows:

[0109] 1) Setting a shortening rate threshold. This is because the number of blood vessel segments that reach the maximum length in one frame of image is small. In order to facilitate blood vessel segmentation, a threshold needs to be set to select several blood vessel segments close to the maximum length to determine the bending direction;

[0110] 2) Determine the bending direction of each blood vessel segment: determine the bending direction of the blood vessel segments on both sides of the blood vessel segment that meets the shortening rate threshold as a reference, and the bending direction is bending inward or outward of the image;

[0111] 3) Determine the bending angle of each blood vessel segment: The bending angle is the angle between the blood vessel and the plane where the image is located. The blood vessel bending angle is obtained based on the shortening rate of the blood vessel;

[0112] 4) Based on the coordinates of the 2D centerline of the blood vessel and the bending direction and bending angle of each segment of the blood vessel, combined with the ratio of physical coordinates to pixel coordinates spacing and the coordinates in the z-axis direction, the three-dimensional spatial coordinates of the 3D centerline of the blood vessel can be obtained to obtain the 3D centerline.

[0113] The bending directions of the vascular segments on both sides are determined based on the vascular segment that meets the shortening rate threshold, specifically including:

[0114] The blood vessel segments on both sides of the blood vessel segment that meets the shortening rate threshold have opposite bending directions, that is, of the blood vessel segments on both sides of the blood vessel segment that meets the shortening rate threshold, one blood vessel segment bends inwardly of the image, and the other blood vessel segment bends outwardly of the image;

[0115] The bending directions of all the blood vessel segments between every two blood vessel segments that meet the shortening rate threshold are the same, that is, all the blood vessel segments between every two blood vessel segments that meet the shortening rate threshold are bent inward or outward of the image.

[0116] When the shortening rate R of the blood vessel is expressed as: When , the shortening rate threshold is the set minimum value, that is, the shortening rate of the blood vessel must reach this minimum value to meet the requirement. Then, when used to determine the 3D centerline, first determine which blood vessel segments have a shortening rate that reaches the local maximum value. For example, if the shortening rate threshold is set to 0.95, that is, determine the blood vessel segments with a shortening rate greater than 0.95, and the inclination direction of the blood vessel 2D centerline between these blood vessel segments remains consistent. Every time a blood vessel segment with a shortening rate greater than 0.95 is passed, the inclination direction of the blood vessel 2D centerline is reversed. When the shortening rate of the blood vessel is expressed as When, correspondingly, the shortening rate threshold is the set maximum value, that is, first determine which blood vessel segments have shortening rates less than the shortening rate threshold, and then determine the bending directions and bending angles of other blood vessels based on these blood vessel segments.

[0117] The above method of solving the 3D centerline based on the 2D centerline ignores the elastic contraction ability of the blood vessel itself, and attributes all the contraction of the blood vessel in the xy plane to its bending in the Z plane. This error is very small and can generally be ignored. However, in order to obtain a more accurate three-dimensional vascular structure, the obtained vascular 3D centerline can also be post-processed: the length of the 3D centerline of each frame of the vascular 3D centerline image is calculated, and the longest 3D centerline is found as a reference. The elastic shortening rate of the blood vessels corresponding to other 3D centerline images is calculated, and then they are extended proportionally to compensate for the elastic shortening of the blood vessels.

[0118] The constructing of a three-dimensional blood vessel with the 3D centerline of the blood vessel as the center specifically includes:

[0119] 1) Mark multiple points of interest on the 2D centerline of the vessel.

[0120] 2) Obtain the blood vessel diameter at each point of interest through the blood vessel structure image: According to the blood vessel structure image, calculate the image width corresponding to each point of interest marked on the 2D center line. Assume that the interest point P k The width of the blood vessel is d k , then the diameter of the blood vessel at that point on the 3D centerline is d k *spacing, spacing is the ratio of real physical distance to pixel distance.

[0121] 3) With each corresponding interest point on the 3D center line of the blood vessel as the center, draw a circle with a diameter equal to the diameter of the blood vessel, and each circle forms a series of parallel contour lines: Specifically, the interest point P k The axial direction is set to P k-1 P k+1 Direction, in the vertical direction, i.e. radial direction, with P k Draw a circle with diameter d as the center k *Spacing circles, repeat the above operation for all points of interest on the center line to make multiple circles. The continuous three-dimensional circles form a series of parallel contour lines.

[0122] 4) Mark a number of sampling points evenly on each circle.

[0123] 5) Set constraints to connect the sampling points on adjacent contour lines to form the three-dimensional surface of the blood vessel.

[0124] The constraints are set because there are many ways to connect adjacent contours. In order to make the reconstructed surface smooth and natural, some constraints need to be added, including but not limited to the maximum volume criterion, the minimum surface area criterion, the shortest diagonal method, etc.

[0125] The following uses the shortest diagonal method as an example to demonstrate the process of connecting adjacent contour lines to form a three-dimensional surface of a blood vessel: Figure 6 As shown, assuming that the sampling points on a circle are {S i}, the next sampling point on the circle is {T j}, first calculate the point closest to point Si as T j , S i T j The third point can be S i+1 or T j+1 , if S i T j+1 <=S i+1 T j , then select T j+1 Make the third point to form a triangle S i T j T j+1 , otherwise select S i+1 . Therefore, it can be found in the circle {S i} and {T j} are connected by triangles. Repeat the above algorithm to construct triangles at other centerline positions, and a complete blood vessel surface can be constructed outside the 3D centerline.

[0126] In order to improve the efficiency of 3D construction of blood vessels, the key points used to segment the blood vessels can be directly used as interest points when constructing the 3D surface of the blood vessels.

[0127] The present invention also proposes a 3D blood vessel reconstruction device 100 based on a single-view coronary angiography sequence, which is used to implement the 3D blood vessel reconstruction method based on a single-view coronary angiography sequence as described above, see Figure 7 , including an input module 11, a blood vessel segmentation module 12, a centerline extraction module 13, a custom setting module 14, a tracking module 15, a calculation module 16, a blood vessel three-dimensional imaging module 17 and an output module 18.

[0128] The input module 11 is used to input a sequence of coronary angiography images at a single angle. The vascular segmentation module 12 is used to segment the vascular structure from the coronary angiography image to obtain a vascular structure image. The centerline extraction module 13 is used to extract the 2D centerline of the blood vessel in the vascular structure image. The custom setting module 14 is used to mark key points on the 2D centerline and divide the blood vessel into several segments. The tracking module 15 is used to track the corresponding positions of the key points in the image sequence. The calculation module 16 is used to solve the 3D centerline of the blood vessel based on the differences between each segment of the blood vessel in each frame of the vascular structure image. For example, when solving the 3D centerline of the blood vessel based on the length differences between each segment of the blood vessel in each frame of the vascular structure image, the shortening rate of each segment of the blood vessel can be solved to obtain the 3D centerline of the blood vessel. The vascular three-dimensional imaging module 17 is used to construct a three-dimensional surface of the blood vessel with the 3D centerline of the blood vessel as the center. The output module 18 is used to output a three-dimensional image of the blood vessel.

[0129] When 3D blood vessel reconstruction is performed based on the 2D blood vessel of a certain frame of blood vessel structure image, the output module 18 outputs a static 3D blood vessel structure image at that moment; when 3D blood vessel reconstruction is performed based on the vascular structure images of all frames, the output module 18 can output a dynamic 3D blood vessel image.

[0130] The present invention also proposes a computer device, including a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the above-mentioned method for three-dimensional reconstruction of blood vessels based on a single-view coronary angiography sequence.

[0131] The present invention also proposes a computer storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for three-dimensional reconstruction of blood vessels based on a single-view coronary angiography sequence as described above is implemented.

[0132] It should be noted that, in the present application, unless otherwise clearly specified and limited, "connection" and similar terms should be understood in a broad sense. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, an indirect connection through an intermediate medium, or the internal connection of two components. Technical personnel in the field can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0133] The present invention has the following beneficial effects:

[0134] 1) The present invention can reconstruct a relatively smooth three-dimensional structure of the coronary artery by relying only on a single-view coronary angiography sequence, thereby helping doctors to better obtain vascular structure information and further diagnose the degree of vascular lesions.

[0135] 2) The present invention makes full use of centerline tracking and foreshortening information to estimate the depth information of the 2D centerline, which is used to calculate the three-dimensional blood vessel centerline, and can obtain a more accurate three-dimensional coronary artery model.

[0136] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software module may reside in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read and write information from / to the storage medium. In an alternative, a storage medium may be integrated into a processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and the storage medium may reside in a user terminal as discrete components.

[0137] In one or more exemplary embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented as a computer program product in software, each function may be stored on or transmitted by a computer-readable medium as one or more instructions or codes. Computer-readable media include both computer storage media and communication media, including any medium that facilitates the transfer of a computer program from one place to another. Storage media may be any available medium that can be accessed by a computer. As an example and not limitation, such a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer. Any connection is also properly referred to as a computer-readable medium. For example, if the software is transmitted from a website, a server, or other remote source using a coaxial cable, a fiber optic cable, a twisted pair, a digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of the medium. Disk and disc as used herein include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, wherein disk often reproduces data magnetically, while disc reproduces data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0138] The method steps of the present application shown in a plurality of different exemplary embodiments are only exemplary. Although only several embodiments are described in detail in this disclosure, it should be easily understood by the personnel referring to this disclosure that many modifications are possible (e.g., structures, parameter values, etc.) without actually departing from the novel teachings and advantages of the subject matter described in this application. Therefore, all such modifications are intended to be included within the scope of the present invention. The order or sequence of any process or method steps can be changed or reordered according to alternative embodiments. Without departing from the spirit of the present invention and its essence, those skilled in the art can make various corresponding modifications and changes according to the present invention, but these corresponding modifications and changes should all fall into protection scope of the present invention.

Claims

1. A method for three-dimensional reconstruction of blood vessels based on a single-view coronary angiography sequence, characterized in that: The steps include: Acquire a single-angle coronary angiography image sequence; Segment each frame of coronary angiography image to obtain the corresponding vascular structure image; Extracting the 2D centerline of the blood vessel in each frame of the blood vessel structure image, marking a number of key points on the 2D centerline of one frame of the blood vessel structure image, and identifying the corresponding positions of each key point in other blood vessel structure images; Each key point divides the blood vessel into several segments, and the shape of the 2D centerline of the blood vessel in three-dimensional space is determined based on the difference between each segment of the blood vessel in each frame of the blood vessel structure image, thereby obtaining the 3D centerline of the blood vessel; Construct a three-dimensional blood vessel with the 3D centerline of the blood vessel as the center.

2. The method for three-dimensional reconstruction of blood vessels based on a single-view coronary angiography sequence according to claim 1, characterized in that: The segmentation of each frame of the coronary angiography image to obtain the corresponding vascular structure image specifically includes: The vascular structure in the coronary angiography image is defined as the foreground, and the surrounding environment of the blood vessels is defined as the background; The foreground and background are separated and the vascular structure image is extracted.

3. The method for three-dimensional reconstruction of blood vessels based on a single-view coronary angiography sequence according to claim 1, characterized in that: When identifying the corresponding positions of each key point in other vascular structure images, there are two situations: If the position of the key point identified in other vascular structure images falls on the 2D centerline of the blood vessel, then the position is taken as the corresponding key point; If the position of the key point identified in other vascular structure images does not fall on the 2D centerline of the blood vessel, the point on the 2D centerline closest to the position is selected as the corresponding key point.

4. The method for three-dimensional reconstruction of blood vessels based on a single-view coronary angiography sequence according to claim 1, characterized in that: Based on the differences between each segment of the blood vessel in each frame of the blood vessel structure image, the shape of the 2D centerline of the blood vessel in the three-dimensional space is determined to obtain the 3D centerline of the blood vessel, specifically including: Calculating the shortening rate of each blood vessel segment based on the difference between each blood vessel segment in each frame of the blood vessel structure image, wherein the shortening rate is used to characterize the shortening degree of the blood vessel in the plane where the image is located; The curvature of the 2D centerline of the blood vessel in three-dimensional space is determined according to the shortening rate of each segment of the blood vessel, and the 3D centerline of the blood vessel is obtained.

5. The method for three-dimensional reconstruction of blood vessels based on a single-view coronary angiography sequence according to claim 4, characterized in that: Determining the curvature of the 2D centerline of the blood vessel in three-dimensional space according to the shortening rate of each segment of the blood vessel to obtain the 3D centerline of the blood vessel specifically includes: Set the shortening rate threshold; Determine the bending direction of each blood vessel segment: determine the bending direction of the blood vessel segments on both sides thereof based on the blood vessel segment that meets the shortening rate threshold, wherein the bending direction is bending inward or outward of the image; Determine the bending angle of each blood vessel segment: the bending angle is the angle between the blood vessel and the plane where the image is located, and the blood vessel bending angle is obtained according to the shortening rate of the blood vessel; The 3D centerline of the blood vessel is obtained based on the 2D centerline of the blood vessel and the bending direction and bending angle of each segment of the blood vessel.

6. The method for three-dimensional reconstruction of blood vessels based on a single-view coronary angiography sequence according to claim 5, characterized in that: The step of determining the bending directions of the blood vessel segments on both sides based on the blood vessel segment that meets the shortening rate threshold specifically includes: The blood vessel segments on both sides of the blood vessel segment that meets the shortening rate threshold have opposite bending directions, that is, of the blood vessel segments on both sides of the blood vessel segment that meets the shortening rate threshold, one blood vessel segment bends inwardly of the image, and the other blood vessel segment bends outwardly of the image; The bending directions of all the blood vessel segments between every two blood vessel segments that meet the shortening rate threshold are the same, that is, all the blood vessel segments between every two blood vessel segments that meet the shortening rate threshold are bent inward or outward of the image.

7. The method for three-dimensional reconstruction of blood vessels based on a single-view coronary angiography sequence according to claim 1, characterized in that: The differences between each segment of the blood vessel in each frame of the blood vessel structure image include: any one or more of a length difference, a contour difference, and an angle difference.

8. The method for three-dimensional reconstruction of blood vessels based on a single-view coronary angiography sequence according to claim 1, characterized in that: The constructing of a three-dimensional blood vessel with the 3D centerline of the blood vessel as the center specifically includes: Mark multiple points of interest on the 2D centerline of the blood vessel; Obtaining the blood vessel diameter at each point of interest through the blood vessel structure image; With each corresponding interest point on the 3D center line of the blood vessel as the center, draw a circle with a diameter equal to the diameter of the blood vessel, and each circle forms a series of parallel contour lines; Mark a number of sampling points evenly and correspondingly on each circle; Constraints are set to connect the sampling points on adjacent contour lines to form the three-dimensional surface of the blood vessel.

9. A 3D blood vessel reconstruction device based on a single-view coronary angiography sequence, used to implement the 3D blood vessel reconstruction method based on a single-view coronary angiography sequence as described in any one of claims 1 to 8, characterized in that: include: An input module, used for inputting a coronary angiography image sequence of a single angle; A blood vessel segmentation module is used to segment the blood vessel structure from the coronary angiography image to obtain a blood vessel structure image; A centerline extraction module, used to extract the 2D centerline of the blood vessel in the blood vessel structure image; A custom setting module for marking key points on the 2D centerline and dividing the blood vessels into several segments; A tracking module, used to track the corresponding positions of key points in the image sequence; A calculation module, used for solving the 3D centerline of the blood vessel according to the difference between each segment of the blood vessel in each frame of the blood vessel structure image; A vascular three-dimensional imaging module is used to construct a three-dimensional surface of a blood vessel with the 3D centerline of the blood vessel as the center; The output module is used to output the three-dimensional image of the blood vessel.

10. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the method for three-dimensional reconstruction of blood vessels based on a single-view coronary angiography sequence as described in any one of claims 1 to 8.

11. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for three-dimensional reconstruction of blood vessels based on a single-view coronary angiography sequence as described in any one of claims 1 to 8 is implemented.

Images