Method and device for automatically reconstructing a three-dimensional model of a coronary vessel

Abstract

The application relates to a method and device for automatically reconstructing a coronary three-dimensional model, the method comprising the following steps: obtaining a first angle image, a second angle image, a first angle key frame and a second angle key frame based on a DSA image sequence, performing key point pair detection on the first angle key frame and the second angle key frame to obtain a matching point pair, obtaining a two-angle projection relationship before correction according to the first angle image and the second angle image information, and obtaining a projection relationship after correction in combination with the matching point pair; performing semantic segmentation on the first angle key frame and the second angle key frame to obtain a first angiography angle graph and a second angiography angle graph of a target blood vessel; obtaining a first center line, a first contour line, a second center line and a second contour line according to the first angiography angle graph and the second angiography angle graph; and combining the projection relationship after correction to obtain a three-dimensional center line and a three-dimensional contour of the target blood vessel, so that an accurate three-dimensional center line and three-dimensional contour of the target blood vessel are obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical image processing, in particular to a method and device for automatically reconstructing a three-dimensional model of a coronary artery. BACKGROUND

[0002] Coronary arteries can provide oxygen and nutrients to the heart to meet the metabolic needs of the myocardium. When the morphology of the coronary arteries changes, the blood supply to the heart will be affected, which may cause angina pectoris, myocardial infarction, and even sudden death. Therefore, obtaining the true morphology of the coronary arteries is crucial for the diagnosis of coronary heart disease.

[0003] Angiography is an interventional detection method, and Digital Subtraction Angiography (DSA) is a method of subtracting the image after injection of a contrast agent from the image before injection, eliminating bone and soft tissue images, and making the blood vessels clearly visible. It was once considered the gold standard for diagnosing coronary artery stenosis. However, coronary angiography is a two-dimensional image, and the traditional basis for diagnosis is the degree of stenosis in two dimensions. Due to the influence of the angle of the angiogram and the reduction of the projection, it is difficult to directly obtain the true three-dimensional morphology of the blood vessels from a two-dimensional image alone, and therefore it is not possible to quantitatively evaluate the impact of stenosis on the physiological function of the coronary artery, and the severity of the lesion cannot be accurately evaluated, which may lead to inadequate or excessive treatment.

[0004] In recent years, techniques have been developed to obtain three-dimensional morphological parameters of the coronary arteries using DSA images, and three-dimensional reconstruction of the coronary vessels can provide more accurate coronary morphology and assist in quantitatively evaluating the impact of lesions on the physiological function of the coronary artery.

[0005] Depending on the number of input image files, existing methods for obtaining a three-dimensional model of the coronary artery based on coronary angiography images can be divided into two categories. One method uses a single-angle image input to obtain a three-dimensional model of the blood vessel, and the other method uses two (or more) DSA images at two (or more) angles to reconstruct the target blood vessel.

[0006] In the first method, a single DSA image at a single angle is used to reconstruct the target blood vessel, which requires the use of the centerline, contour information of a single two-dimensional blood vessel, and the projection parameters of the image file. The calculation process is simple and the calculation time is short, but the information it represents is basically consistent with two dimensions, and cannot eliminate the effects of projection reduction and eccentric stenosis. The calculation accuracy is poor.

[0007] In the second method, taking reconstruction of a double-angle coronary three-dimensional model as an example, due to reasons such as respiration, heartbeats, equipment errors and patient movement, the projection parameter information recorded in the image files cannot be directly used for back projection reconstruction between different angle images, and the projection relationship needs to be corrected to obtain the correct matching relationship of the target blood vessels between angles. The existing correction method is mainly a manual method, which mainly selects matching feature points manually, corrects the projection parameters by using an optimization algorithm, obtains the corrected projection relationship, and performs back projection reconstruction. In addition, the user also needs to manually select the image files of the same target blood vessels under double angles, select key frames (clear diastolic phase frames are selected) and select the start and end points of the target blood vessels. Compared with the first method, this method eliminates the influence of projection reduction and eccentric stenosis, but the accuracy depends on the accuracy of manual point selection, the calculation speed is slow, and too many manual operations are required. SUMMARY

[0008] Therefore, it is necessary to provide a method and device for automatically reconstructing a coronary three-dimensional model in view of the above technical problems.

[0009] The method for automatically reconstructing a coronary three-dimensional model comprises the following steps.

[0010] Obtain a DSA image sequence of a coronary blood vessel, which comprises a plurality of angiographic angles, and automatically perform screening and classification on the DSA image sequence in sequence to obtain a first angle image, a second angle image, a first angle key frame and a second angle key frame.

[0011] Perform key point pair detection on the first angle key frame and the second angle key frame to obtain a matching point pair, obtain a two-angle projection relationship before correction according to the first angle image and the second angle image information, correct the projection relationship in combination with the matching point pair, and obtain a corrected projection relationship.

[0012] Perform semantic segmentation on the first angle key frame and the second angle key frame to obtain a binary image of a target blood vessel at a first angiographic angle and a second angiographic angle, which are a first angiographic angle image and a second angiographic angle image respectively; and obtain a first center line, a first contour line, a second center line and a second contour line according to the first angiographic angle image and the second angiographic angle image.

[0013] Reconstruct a three-dimensional center line and a three-dimensional contour of a target blood vessel according to the first center line, the contour line, the second center line, the contour line and the corrected projection relationship.

[0014] Optionally, the key point pair detection on the first angle key frame and the second angle key frame to obtain the matching point pair specifically comprises the following steps.

[0015] The trained target detection model is used to detect the matching point pairs from the first angle key frame and the second angle key frame.

[0016] Optionally, the training process of the target detection model comprises:

[0017] training a target detection model for recognizing key points;

[0018] training a regression model for recognizing the corresponding relationship between the key point pairs;

[0019] concatenating the two networks to correct the bias of the key points with low confidence.

[0020] Optionally, the key points include proximal key points, bifurcation key points and distal key points.

[0021] Optionally, the regression model is used for combined regression of three groups of key points corresponding to the first angle key frame and the second angle key frame.

[0022] Optionally, the input of the concatenated two networks is the first angle key frame and the second angle key frame, and the target detection model processes the key frames in two channels respectively, and the output is the coordinate positions of three groups of key point pairs.

[0023] Optionally, the projection relationship before correction is obtained according to the first angle image and the second angle image information, and the projection relationship after correction is obtained by combining the matching point pairs and correcting the projection relationship, which specifically comprises:

[0024] obtaining the projection relationship of the two angle images before correction according to the angiographic information provided by the first angle image and the second angle image information;

[0025] correcting the projection relationship using an optimization algorithm according to the matching point pairs and the projection relationship before correction to obtain the projection relationship after correction.

[0026] Optionally, the DSA image sequence is automatically screened and classified in sequence to obtain the first angle image, the second angle image, the first angle key frame and the second angle key frame:

[0027] using the trained first depth model to screen the DSA image sequence to obtain the key frame;

[0028] using the trained second depth model to classify the DSA image sequence based on the key frame to obtain the DSA images of the first angiographic angle and the second angiographic angle of the target blood vessel, and the corresponding key frames are the first angle key frame and the second angle key frame.

[0029] Optionally, the second deep model is a four-classification model, used for classifying the key frame into: left anterior descending branch (LAD), left circumflex branch (LCX), right coronary artery (RCA), and other categories of images not belonging to the target blood vessel.

[0030] The trained second deep model obtains the input key frame and outputs probability values of each classification.

[0031] The application also provides a device for automatically reconstructing a coronary three-dimensional model, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method for automatically reconstructing a coronary three-dimensional model.

[0032] The method for automatically reconstructing a coronary three-dimensional model has at least the following effects:

[0033] The application can automatically screen and classify DSA image sequences, automatically obtain first-angle images, second-angle images, first-angle key frames, and second-angle key frames of a target blood vessel, correct the projection relationship of the first-angle images and the second-angle images according to the key point pairs automatically obtained by target detection on the first-angle key frames and the second-angle key frames, no longer rely on manual point selection, and obtain more reliable corrected projection relationships. According to the first contrast angle map and the second contrast angle map obtained by semantic segmentation on the first-angle key frames and the second-angle key frames, combined with the corrected projection relationship, back projection reconstruction can be performed to obtain an accurate three-dimensional center line and three-dimensional contour of the target blood vessel. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 FIG. 1 is a flowchart of the method for automatically reconstructing a coronary three-dimensional model according to an embodiment of the application;

[0035] Figure 2 FIG. 2 is a flowchart of the method for automatically reconstructing a coronary three-dimensional model according to an embodiment of the application;

[0036] Figure 3 FIG. 3 is a schematic diagram of a DSA image sequence classified by a second deep model according to an embodiment of the application;

[0037] Figure 4 FIG. 4 is a schematic diagram of a key point pair detection model according to an embodiment of the application;

[0038] Figures 5-6 FIG. 5 is a schematic diagram of a key point pair detection result of two different contrast angle key frames according to an embodiment of the application;

[0039] Figure 7 FIG. 6 is a first contrast angle map (binary map) according to an embodiment of the application;

[0040] Figure 8Second contrast angle map (binary map) in an embodiment of the present application;

[0041] Figure 9 Structural diagram of the three-dimensional profile of the target blood vessel in an embodiment of the present application;

[0042] Figure 10 Flowchart of the method for obtaining the coronary flow reserve score in an embodiment of the present application;

[0043] Figure 11 Flowchart of the method for obtaining the coronary flow reserve score in an embodiment of the present application;

[0044] Figure 12 Flowchart of the sub-step of step S100 in the method for obtaining the coronary flow reserve score in an embodiment of the present application;

[0045] Figure 13 DSA image sequence classified by the second depth model in an embodiment of the present application;

[0046] Figure 14 Diagram of receiving the user-specified target blood vessel in an embodiment of the present application;

[0047] Figure 15 Key frame of the first contrast angle in an embodiment of the present application;

[0048] Figure 16 Key frame of the second contrast angle in an embodiment of the present application;

[0049] Figure 17 First contrast angle map (binary map) in an embodiment of the present application;

[0050] Figure 18 Second contrast angle map (binary map) in an embodiment of the present application;

[0051] Figure 19 Flowchart of the sub-step of step S200 in the method for obtaining the coronary flow reserve score in an embodiment of the present application;

[0052] Figure 20 Diagram of generating the three-dimensional center line according to the first center line and the second center line back projection in an embodiment of the present application;

[0053] Figure 21 Diagram of obtaining the cross-sectional profile through the cross section of the first contrast angle map and the second contrast angle map in an embodiment of the present application;

[0054] Figure 22 Structural diagram of the first profile line of the first contrast angle map in an embodiment of the present application;

[0055] Figure 23 A structural schematic diagram of a second profile line of a second contrast angle graph in an embodiment of the present application;

[0056] Figure 24 A structural schematic diagram of a three-dimensional center line of a target blood vessel in an embodiment of the present application;

[0057] Figure 25 A structural schematic diagram of a three-dimensional profile of a target blood vessel in an embodiment of the present application;

[0058] Figures 26-27 A schematic diagram of key frames of two different contrast angles with matching points marked;

[0059] Figure 28 A schematic diagram of a distribution of a coronary flow reserve fraction along a three-dimensional center line obtained in an embodiment of the present application;

[0060] Figure 29 An internal structural diagram of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description herein is for describing particular embodiments only and is not intended to be limiting of the application. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0062] In the present application, the terms "first", "second", etc. are only used for the purpose of description and can not be understood as indicating or implying relative importance or implicitly indicating the number or order of the technical features indicated. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "a plurality of" is at least two, for example, two, three, etc., unless otherwise specifically limited.

[0063] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0064] Referring to Figure 1 , an embodiment of the present application provides a method for automatically reconstructing a coronary three-dimensional model, comprising steps S10-S40. Among them:

[0065] Step S10, obtaining a DSA image sequence of a coronary vessel including a plurality of projection angles, automatically screening and classifying the DSA image sequence in sequence to obtain a first angle image, a second angle image and a first angle key frame and a second angle key frame;

[0066] Step S20, performing key point pair detection on the first angle key frame and the second angle key frame to obtain a matching point pair, obtaining a two-angle projection relationship before correction according to the first angle image and the second angle image information, correcting the projection relationship in combination with the matching point pair to obtain a projection relationship after correction;

[0067] Step S30, performing semantic segmentation on the first angle key frame and the second angle key frame to obtain a binary graph of the target vessel at the first projection angle and the second projection angle, which are respectively a first projection angle graph and a second projection angle graph; obtaining a first center line, a first contour line, a second center line and a second contour line according to the first projection angle graph and the second projection angle graph;

[0068] Step S40, reconstructing a three-dimensional center line and a three-dimensional contour of the target vessel according to the first center line, the contour line, the second center line and the contour line, and the projection relationship after correction.

[0069] The method for automatically reconstructing a coronary three-dimensional model provided in the embodiment takes into account automation and calculation accuracy. In the embodiment, the DSA image processing in the prior art is improved. Compared with the single-angle two-dimensional reconstruction method in the prior art, the influence of projection reduction and eccentric stenosis is eliminated, the calculation accuracy is improved, compared with the double-angle three-dimensional reconstruction method in the prior art, the manual operation is reduced, the efficiency is improved, the calculation accuracy is improved without relying on manual selection of matching point pairs.

[0070] Referring to Figures 1-2 , obtaining a coronary DSA image file, screening to obtain a diastolic clear frame as a key frame, classifying the DSA image file to obtain a target vessel category, obtaining double-angle image files of the same target vessel, i.e. a first angle image and a second angle image, corresponding key frames, i.e. a first angle key frame and a second angle key frame (corresponding to step S10).

[0071] Performing key point pair detection on the first angle key frame and the second angle key frame to obtain a matching point pair, obtaining a two-angle projection relationship before correction according to the first angle image and the second angle image file information, correcting the projection relationship in combination with the matching point pair to obtain a projection relationship after correction (corresponding to step S20).

[0072] The first angle key frame and the second angle key frame are subjected to semantic segmentation to obtain a binary image of the target blood vessel at a first angiography angle and a second angiography angle, i.e., a first angiography angle image and a second angiography angle image; according to the first angle image and the second angle image, a two-dimensional contour line and a two-dimensional center line are extracted to obtain a first center line, a first contour line, a second center line and a second contour line (corresponding to step S30).

[0073] According to the first center line, the contour line, the second center line, the contour line and the corrected projection relationship, a three-dimensional center line and a three-dimensional contour of the target blood vessel are reconstructed (corresponding to step S40).

[0074] In step S20, the trained target detection model is used to detect key points in the first angle key frame and the second angle key frame to obtain a matching point pair.

[0075] Referring to Figures 4-6 The training process of the target detection model includes: training a target detection model for recognizing key points; training a regression model for recognizing the corresponding relationship between key point pairs; and cascading the two networks to correct the bias of key points with low confidence.

[0076] Specifically, the key points of the target detection model include proximal key points, bifurcation key points and distal key points, the network structure adopts an end-to-end YoLov3, the input is a single-channel grayscale image of 512*512, and the output is a candidate box and a confidence of a target region. The training data needs to label the key frame images of each image sequence, and each image needs to label the positions of 3 key points, i.e., proximal key point S, bifurcation key point M and distal key point E. The actual corresponding target region is a rectangular box with a size of 28*28 centered on the key point. The training data should select the corresponding positions of the same main blood vessels suitable for observation. The recommended key point marking positions for the recommended angiography angles of LAD, LCX and RCA are recommended, and three different target detection models are trained for the three main blood vessels LAD, LCX and RCA.

[0077] The regression model is used for recognizing the corresponding relationship between three groups of key point pairs corresponding to the first angle key frame and the second angle key frame.

[0078] The two networks are cascaded to correct the bias of key points with low confidence. The input is the first angle key frame and the second angle key frame, the target detection model processes the key frames of the two channels respectively, and the output is the coordinate positions of the S, M and E three groups of key point pairs.

[0079] The projection relationship of the two-angle images before correction is obtained according to the matching point pairs and the projection relationship before correction, and the projection relationship is corrected using an optimization algorithm to obtain the projection relationship after correction.

[0080] Specifically, the correction process is as follows: the three-dimensional estimated point coordinates obtained by back-projection of the matching points under the projection relationship before correction are calculated, the two-dimensional estimated point coordinates are obtained by re-projecting the three-dimensional estimated points onto the two-angle images, the coordinate difference between the two-dimensional estimated points and the corresponding matching points is calculated, and the difference is used as an optimization target to iteratively optimize the projection parameters to obtain the projection relationship after correction.

[0081] In step S10, the DSA image sequence is automatically screened and classified in sequence to obtain the first-angle image, the second-angle image, the first-angle key frame and the second-angle key frame. The actual operation process includes using a deep learning method to sequentially complete screening (finding key frames belonging to diastolic clear frames) and image classification (classification belonging to target blood vessels) based on all coronary artery blood vessel DSA image sequences of the same patient. The DSA image sequence can include different target blood vessel files, but at least two different contrast angle image files of the target blood vessel type are required for three-dimensional reconstruction, i.e., at least the first contrast angle and the second contrast angle used below.

[0082] The trained first deep model is used to screen the DSA image sequence to obtain the key frame.

[0083] Specifically, the first deep model is used to find the most clear frame of the target blood vessel diastolic end in each image of the DSA image sequence using a deep learning method. Based on the key frame finding algorithm of deep learning, a first deep model (binary classification model) based on key frame images and non-key frame image data needs to be trained first, and then the trained model is used for inference to output the frame corresponding to the highest probability value of the key frame position probability in the binary classification probability as the key frame of the image.

[0084] The trained second deep model is used to classify the DSA image sequence based on the key frame to obtain the first contrast angle and the second contrast angle of the target blood vessel DSA image.

[0085] The second deep model is a four-classification model used to classify the key frame into: left anterior descending branch LAD, left circumflex branch LCX, right coronary artery RCA, and other categories of images not belonging to the target blood vessel; the trained second deep model obtains the input key frame and outputs the probability value of each classification.

[0086] Specifically, the key frame image is input into the second deep model, and the image file is classified using a deep learning method to obtain the type of the target blood vessel in the image sequence. Based on the blood vessel classification algorithm of deep learning, a four-classification model based on LAD (left anterior descending branch), LCX (left circumflex branch), RCA (right coronary artery) and other images not belonging to the target blood vessel is first needed to be trained, wherein LAD, LCX and RCA are taken from the images corresponding to the key frames of the images obtained in the previous step (step S110), and the other class images include any one of the images with poor image quality and other images that are not coronary arteries. Then, the trained second deep model is used for inference to output the probability value corresponding to each class in the four classification, and the class with the highest probability is the target blood vessel type corresponding to the image.

[0087] Referring to Figure 10 In an embodiment of the present application, a method for obtaining a coronary flow reserve score is also provided, comprising steps S100-S500. Among them:

[0088] In step S100, a DSA image sequence of a coronary vessel including multiple contrast angles is obtained, and the DSA image sequence is automatically screened, classified and semantically segmented in sequence to obtain a binary image of a target blood vessel at a first contrast angle and a second contrast angle, which are a first contrast angle image and a second contrast angle image respectively, both of which include the starting point of the target blood vessel.

[0089] In step S200, a three-dimensional center line and a three-dimensional contour of the target blood vessel are reconstructed according to the first contrast angle image and the second contrast angle image, and then an equivalent three-dimensional diameter of the three-dimensional contour is obtained.

[0090] In step S300, the equivalent three-dimensional diameter is fitted to obtain a reference diameter.

[0091] In step S400, a first time difference of the contrast agent from the starting point to the ending point participating in the reconstruction of the three-dimensional center line in the first contrast angle image is obtained along the time sequence of the DSA image sequence, a second time difference of the contrast agent from the starting point to the ending point participating in the reconstruction of the three-dimensional center line in the second contrast angle image is obtained, and the length of the three-dimensional center line and the mean value of the first time difference and the second time difference are used to obtain the blood flow velocity.

[0092] In step S500, the three-dimensional center line, the equivalent three-dimensional diameter, the reference diameter and the blood flow velocity are combined to obtain the pressure drop value of the blood passing through the target blood vessel, and the coronary distal flow reserve score is obtained in combination with the coronary ostial pressure.

[0093] The method for obtaining the coronary flow reserve fraction provided by the embodiment takes into account automation and calculation accuracy. In the embodiment, the prior art is improved for DSA image processing. Compared with the prior art of single-angle two-dimensional flow reserve fraction calculation method, the accuracy of obtaining the flow reserve fraction is improved. Compared with the prior art of double-angle three-dimensional flow reserve fraction calculation method, the efficiency is improved.

[0094] The first contrast angle map and the second contrast angle map are used for back projection reconstruction in the embodiment, so that the three-dimensional center line, the three-dimensional profile and the equivalent three-dimensional diameter of the target blood vessel can be obtained. According to the correspondence between the first contrast angle map and the second contrast angle map and the three-dimensional center line reconstruction, the first time difference and the second time difference are obtained through the first contrast angle map and the second contrast angle map and the DSA image time sequence, respectively. The mean value of the two and the three-dimensional center line length are used to obtain the blood flow velocity. The numerical value of the obtained blood flow velocity is more reliable. In other embodiments, each embodiment can further include a third contrast angle map, which also participates in the reconstruction of the three-dimensional center line and the calculation of the time difference, so as to further enhance the reliability of the obtained three-dimensional parameters.

[0095] It can be understood that the "double angle" in each embodiment refers to the first contrast angle and the second contrast angle. The angle difference between the first contrast angle and the second contrast angle is greater than or equal to twenty-five degrees, so as to sufficiently obtain the spatial information of the target blood vessel. The target blood vessel includes one of the left anterior descending branch LAD, the left circumflex branch LCX and the right coronary artery RCA.

[0096] Referring to Figure 10 and Figure 11 , a coronary DSA image file is obtained, a diastolic end clear frame is screened as a key frame, the DSA image file is classified to obtain a target blood vessel category, double-angle image files of the same target blood vessel are obtained, and double-angle clear frame segmentation maps, i.e., the first angle map and the second angle map, are obtained by segmenting the key frames of the double-angle image files (corresponding to step S100).

[0097] According to the first angle map and the second angle map, a two-dimensional profile line and a two-dimensional center line are extracted, and three-dimensional parameters of the target blood vessel are obtained by back projection reconstruction. The three-dimensional parameters include a three-dimensional center line, a three-dimensional profile and an equivalent three-dimensional diameter (corresponding to step S200). The equivalent three-dimensional diameter is fitted to obtain a reference diameter (corresponding to step S300).

[0098] The two-angle image files are segmented frame by frame, and the blood vessel length-time curve is obtained by combining the image timing, the average time difference t of the two-angle blood vessel length reaching the starting point and the ending point of the target blood vessel is obtained, the blood flow rate is calculated by using the three-dimensional center line length and the time t (corresponding to step S400), and then the target blood vessel pressure drop ΔP is calculated, and the blood flow reserve fraction 1-ΔP / Pa is calculated by combining the cardiac coronary orifice pressure Pa (corresponding to step S500).

[0099] In step S100, the DSA image sequence is automatically screened, classified and semantically segmented in sequence to obtain the binary graph of the target blood vessel at the first and second contrast angles. The actual operation process includes using a deep learning method to sequentially complete screening (finding key frames belonging to diastolic clear frames), image classification (classification belonging to the target blood vessel), and semantic segmentation (image segmentation) based on all coronary blood vessel DSA image sequences of the same patient. The DSA image sequence can include different target blood vessel files, but at least two different contrast angle image files of the target blood vessel type can be used for three-dimensional reconstruction, i.e., at least the first contrast angle and the second contrast angle used below.

[0100] The coronary artery in each embodiment of the present application is the coronary artery, and the binary graph may, for example, mark the target blood vessel region as white and the non-target blood vessel region as black. The DSA image sequence is digital subtraction angiography, which includes image records of the entire contrast process.

[0101] Referring to Figures 12-18 , step S100 specifically includes steps S110-S130. Wherein:

[0102] In step S110, a trained first deep model is used to screen the DSA image sequence to obtain key frames.

[0103] Specifically, using the first deep model, a deep learning method is used to find the most clear frame of the target blood vessel diastolic end in each image of the DSA image sequence. Based on the key frame finding algorithm of deep learning, a first deep model (binary classification model) based on key frame image and non-key frame image data needs to be trained first, and then the trained model is used for inference to output the frame corresponding to the highest probability value of the key frame position probability as the key frame of the image.

[0104] In step S120, a trained second deep model is used to classify the DSA image sequence based on the key frames to obtain the first and second contrast angle DSA images of the target blood vessel.

[0105] In step S120, the second deep model is a four-classification model for classifying the key frame into: left anterior descending branch LAD, left circumflex branch LCX, right coronary artery RCA, and other images not belonging to the target blood vessel; the trained second deep model obtains the input key frame and outputs the probability value of each classification.

[0106] Specifically, the key frame image is input into the second deep model, the image file is classified using the deep learning method, and the type of the target blood vessel in the image sequence is obtained. Based on the deep learning-based blood vessel classification algorithm, a four-classification model based on LAD (left anterior descending branch), LCX (left circumflex branch), RCA (right coronary artery), and other images not belonging to the target blood vessel is first trained, wherein LAD, LCX, and RCA are all taken from the image corresponding to the key frame obtained in the previous step (step S110), and the other images include any one of poor image quality and other images not of the coronary artery. Then, the trained second deep model is used for inference, and the probability value corresponding to each classification in the four classifications is output, wherein the classification with the highest probability is the target blood vessel type corresponding to the image.

[0107] In step S130, the trained semantic segmentation model matched with the target blood vessel is used to segment the DSA image key frame of the first and second angiographic angles of the target blood vessel, and the target blood vessel binary graph of the first and second angiographic angles is obtained.

[0108] Since there are three target blood vessels, three semantic segmentation models are trained accordingly. The data for training the three segmentation models is derived from the annotation of the blood vessel contour of each frame of the original image. Then, the trained semantic segmentation model is used for inference, and the key frame is segmented. In the following step S400, the semantic segmentation model matched with the target blood vessel is used to segment the DSA image frame by frame at the first and second angiographic angles. Details are described below.

[0109] The above steps S110 to S130 realize the process of automatically processing the DSA image sequence to obtain the target blood vessel binary graph at the first and second angiographic angles.

[0110] Referring to Figures 19-27 In step S200, the three-dimensional center line and three-dimensional contour of the target blood vessel are reconstructed according to the first and second angiographic angle graphs, and the equivalent three-dimensional diameter of the three-dimensional contour is obtained, specifically including:

[0111] In step S210, the first center line and the first contour line are obtained according to the first angiographic angle graph, and the second center line and the second contour line are obtained according to the second angiographic angle graph.

[0112] Step S220, according to the first center line and the second center line, a three-dimensional center line of the target blood vessel is obtained, and the three-dimensional center line includes a starting point and an ending point of the target blood vessel.

[0113] In step S220, the starting point of the three-dimensional center line is the starting point of the target blood vessel, and the ending point of the three-dimensional center line corresponds to the ending point of the first center line and / or the second center line.

[0114] Step S230, for any point on the three-dimensional center line, after matching the first center line and the second center line respectively, four profile points corresponding to the point on the first contour line and the second contour line are obtained, a cross-sectional profile is generated, a three-dimensional profile of the target blood vessel is obtained in combination with the three-dimensional center line and the cross-sectional profile, and an equivalent three-dimensional diameter of the three-dimensional profile is obtained based on the three-dimensional center line and the three-dimensional profile.

[0115] Referring to Figure 20 , in step S210, the binary images of the target blood vessel key frames of the double angles (the first contrast angle image and the second contrast angle image) are used to extract the target blood vessel center line and the contour line. In step S220, the target blood vessel center line and the contour line of the key frames of the double angles are used to sequentially perform center line matching and boundary matching according to the double-angle projection relationship.

[0116] Referring to Figures 21-25 , in step S230, first, the three-dimensional center line point cloud coordinates are reconstructed according to the double-angle center line back projection. According to the three-dimensional center line points and the profile points of the double angles in the normal plane of the three-dimensional center line, the point cloud of each cross section of the blood vessel (cross-sectional profile) is generated, and the three-dimensional profile is the collection of the cross-sectional profiles along the three-dimensional center line. As shown in Figure 21 , for any point on the three-dimensional center line, the center line point 10 on the first center line under the first contrast angle (angle 1) is matched to obtain, and the center line point 20 on the second center line under the second contrast angle (angle 2) is matched to obtain. The left profile point 11 and the right profile point 12 of the center line point 10 on the first contour line are obtained, and the left profile point 21 and the right profile point 22 of the center line point 20 on the second contour line are obtained, a total of four profile points. Further, the three-dimensional blood vessel cross-sectional fitting profile, i.e., the cross-sectional profile, is obtained. Figures 22-25 Therefore, the above process is displayed through the first contrast angle image and the second contrast angle image.

[0117] Referring to Figures 26-27 , in other embodiments, the projection relationship can also be corrected by manually selecting at least one pair of matching points, that is, at least one pair of matching points is selected in the key frame of the first contrast angle and the key frame of the second contrast angle, so as to correct the matching relationship of the three-dimensional center line.

[0118] In step S300, the equivalent three-dimensional diameter is fitted to obtain the reference diameter, specifically including: the equivalent three-dimensional diameter is fitted for the first time to obtain the first fitted diameter, the narrow section of the equivalent three-dimensional diameter is excluded according to the first fitted diameter, and the equivalent three-dimensional diameter excluding the narrow section is fitted for the second time to obtain the second fitted diameter, which is the reference diameter. By excluding the narrow section through the first fitting, the reference diameter can be closer to the true diameter of the normal blood vessel.

[0119] In step S300, the distribution of the reference diameter along the three-dimensional center line is obtained by twice linear fitting based on the distribution curve of the equivalent three-dimensional diameter along the three-dimensional center line. The specific fitting method is: the equivalent three-dimensional diameter is first fitted for the first time to obtain the first fitted diameter, at this time, the points in the equivalent three-dimensional diameter below the first fitted diameter are considered to be in the relatively narrow position of the blood vessel and should not participate in the fitting, so the points above the first fitted diameter (i.e. excluding the narrow section) are taken to perform the second linear fitting to obtain the final fitting line, i.e. the distribution of the reference diameter along the three-dimensional center line.

[0120] In step S400, the first time difference of the contrast agent from the starting point to the ending point of the three-dimensional center line participating in reconstruction in the first contrast angle image is obtained, specifically including:

[0121] The semantic segmentation model matched with the target blood vessel is used to segment the DSA image of the first contrast angle frame by frame, and the blood vessel length-time curve of the first contrast angle is obtained based on the segmentation result;

[0122] The key frame under the first contrast angle and the target blood vessel binary graph of all previous frames are selected, and the center line length corresponding to the target blood vessel of all selected frames is calculated, i.e. on the blood vessel length-time curve of the first contrast angle. On the blood vessel length-time curve, search forward from the frame position of the first contrast angle image along the time sequence until the blood vessel length reaches the starting point of the target blood vessel to obtain the starting frame n1, find the image with a blood vessel length of zero as the starting frame n1 (i.e. when the contrast agent first reaches the starting point of the target blood vessel), and search backward from the starting frame n1 until the blood vessel length reaches the ending frame n2 of the first center line participating in reconstruction, find the image with a blood vessel length reaching the key frame length as the ending frame n2 (i.e. when the contrast agent first reaches the end of the target blood vessel). Since the contrast agent first enters the target blood vessel at n1 and first reaches the tail of the key frame at n2, the first time difference is the time difference between the starting frame n1 and the ending frame n2.

[0123] The difference between the start frame n1 and the end frame n2 divided by the number of frames per second c of the video file can obtain the time difference of the contrast agent flowing through the start point and the end point of the target blood vessel: t1=(n2-n1) / c. The scheme for obtaining the second time difference of the contrast agent from the start point to the end point participating in the reconstruction of the three-dimensional center line in the second contrast angle diagram is the same, and the second time difference t2 is obtained. The average time difference is t=(t1+t2) / 2. In combination with the length L of the three-dimensional center line, the average speed of the contrast agent flowing through the start and end points of the target blood vessel can be obtained: v=L / t. The average flow rate of blood is approximately equal to the average flow rate of the contrast agent.

[0124] In step S500, in combination with the three-dimensional center line (obtained in step S200), the equivalent three-dimensional diameter (obtained in step S200), the reference diameter (obtained in step S300), and the blood flow rate (obtained in step S400), the pressure drop value of the blood passing through the target blood vessel is obtained, and the coronary distal blood flow reserve fraction is obtained in combination with the coronary ostium pressure.

[0125] Specifically, the distribution of the coronary blood flow reserve fraction along the three-dimensional center line is obtained in combination with the coronary ostium pressure, and then the coronary distal blood flow reserve fraction is obtained by formula (1-ΔP / Pa), wherein ΔP is the pressure drop value of the blood passing through the target blood vessel, and Pa is the coronary ostium pressure.

[0126] Referring to Figure 28 In step S500, the three-dimensional model parameters of the target blood vessel, the reference diameter, and the blood flow rate are used to calculate the pressure drop value ΔP of the blood passing through the target blood vessel according to hemodynamics, which at least considers the expansion pressure drop, the viscous pressure drop, and the Bernoulli change term, and then the distribution of the coronary blood flow reserve fraction along the center line and the value of the coronary distal blood flow reserve fraction can be obtained by 1-ΔP / Pa according to the coronary ostium pressure Pa.

[0127] The method for obtaining the coronary blood flow reserve fraction provided by each embodiment can automatically complete the double-angle three-dimensional blood flow reserve fraction calculation in steps S100-S500, uses all coronary DSA images of the same patient as input, uses a deep learning method according to the required target blood vessel type, and automatically completes image file selection, key frame selection, and target blood vessel binary image acquisition. It is not necessary to manually select the start and end points of the target blood vessel, adjust the blood vessel contour, count the frames, reconstruct the three-dimensional model, and calculate the blood flow reserve fraction. It can assist the interventional physician in judging whether the coronary stenosis will cause myocardial ischemia, and according to the reconstruction, accurate three-dimensional size information of the blood vessel is obtained to help the interventional physician select a suitable stent size, so that the treatment of coronary stenosis is more simple and efficient.

[0128] The method for obtaining the coronary blood flow reserve fraction provided by each embodiment has the following technical advantages in steps S100-S500.

[0129] (1) Based on the deep learning method, the key frame (clear diastolic frame) is automatically screened from the DSA image sequence, the target blood vessel double-angle image is obtained by classifying the image according to the key frame, and the target blood vessel binary image is obtained by segmenting the key frame of the image, which greatly shortens the time used in the whole calculation process, and does not need the user to manually select the target blood vessel image, the key frame and the starting point and ending point of the target blood vessel, thereby reducing the tedious manual or semi-automatic manual modification operation. The method for obtaining the coronary flow reserve fraction provided in each embodiment directly extracts the contour line from the target blood vessel binary image obtained by the deep learning method, and compared with the method of using the original DSA image information to find the contour line in other traditional algorithms, the effect is better.

[0130] (2) The reference diameter (second reference diameter) of the blood vessel is calculated by using the twice fitting method, which eliminates the influence of the stenosis section of the blood vessel on the fitting, and the reference diameter is more accurate.

[0131] (3) The center line length of the target blood vessel is calculated by using the target blood vessel binary image obtained by the deep learning method before the key frame, the time of blood passing through the target blood vessel is obtained according to the change of the center line length, and the blood flow velocity of the target blood vessel can be calculated according to the three-dimensional center line length of the target blood vessel of the key frame, without the user manually selecting the frame number of the blood flowing through the starting point and the ending point of the target blood vessel, and the whole process is completely automated. In addition, compared with the method of using a fixed blood flow rate, the calculation result is also more accurate.

[0132] (4) The double-angle reconstructed three-dimensional model is used to calculate the blood flow reserve fraction calculation method, which reduces the manual operation and shortens the calculation time, and ensures the accuracy of the blood flow reserve fraction calculation.

[0133] It should be understood that, although Figure 10 the steps in the flowchart are shown in order according to the arrows, these steps are not necessarily executed in order according to the arrows. Unless otherwise specified in this article, the execution of these steps has no strict order limitation, and these steps can be executed in other order. Moreover, Figure 10 at least part of the steps in the flowchart can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or sub-steps or stages of other steps.

[0134] In one embodiment, a computer device, which can be a server, is provided, and the internal structure diagram thereof can be as shown in Figure 29The computer device includes a processor, a memory, a network interface and a database connected by a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium. The database of the computer device is configured to store DSA image sequence data. The network interface of the computer device is configured to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement a method for obtaining a coronary flow reserve score.

[0135] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the following steps:

[0136] In step S100, a DSA image sequence of a coronary vessel is obtained, including a plurality of contrast angles. The DSA image sequence is automatically screened, classified and semantically segmented in sequence to obtain a binary image of a target vessel at a first contrast angle and a second contrast angle, respectively a first contrast angle image and a second contrast angle image, both of which include a starting point of the target vessel.

[0137] In step S200, a three-dimensional centerline and a three-dimensional contour of the target vessel are reconstructed according to the first contrast angle image and the second contrast angle image, and an equivalent three-dimensional diameter of the three-dimensional contour is obtained.

[0138] In step S300, the equivalent three-dimensional diameter is fitted to obtain a reference diameter.

[0139] In step S400, a first time difference of contrast agent from a starting point to an ending point participating in reconstruction of the three-dimensional centerline in the first contrast angle image is obtained along the time sequence of the DSA image sequence, a second time difference of contrast agent from the starting point to the ending point participating in reconstruction of the three-dimensional centerline in the second contrast angle image is obtained, and a blood flow velocity is obtained using a length of the three-dimensional centerline and a mean value of the first time difference and the second time difference.

[0140] In step S500, a pressure drop value of blood passing through the target vessel is obtained in combination with the three-dimensional centerline, the equivalent three-dimensional diameter, the reference diameter and the blood flow velocity, and a coronary distal flow reserve score is obtained in combination with a coronary ostial pressure.

[0141] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a readable storage medium of a non-volatile computer device. When the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, storage, databases, or other media in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0142] The technical features of the above embodiments can be combined in any way. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not contradict, they should be considered within the scope of the present disclosure. When the technical features in different embodiments are embodied in the same figure, it can be considered that the figure also discloses the combination of the embodiments involved.

[0143] The above embodiments only express several implementation manners of the present application, and the description is specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are within the scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A method for automatically reconstructing a three-dimensional model of a coronary artery, characterized in that, include: A DSA image sequence of coronary arteries, including multiple angiographic angles, is obtained. The DSA image sequence is then automatically filtered and classified to obtain a first-angle image, a second-angle image, and a first-angle keyframe and a second-angle keyframe. Keypoint pair detection is performed on the first angle keyframe and the second angle keyframe to obtain matching point pairs. The projection relationship between the two angles is then corrected based on the information from the first angle image and the second angle image. The projection relationship is then corrected using the matching point pairs to obtain the corrected projection relationship. The keypoints include near-end keypoints, bifurcation keypoints, and far-end keypoints. Semantic segmentation is performed on the first angle keyframe and the second angle keyframe to obtain binary images of the first angiography angle and the second angiography angle of the target blood vessel, which are the first angiography angle image and the second angiography angle image, respectively; the first center line, the first contour line, the second center line and the second contour line are obtained based on the first angiography angle image and the second angiography angle image. Based on the first centerline, the contour line, the second centerline, the contour line, and the corrected projection relationship, the three-dimensional centerline and three-dimensional contour of the target blood vessel are reconstructed.

2. The method according to claim 1, characterized in that, Key point pair detection is performed on the first angle keyframe and the second angle keyframe to obtain matching point pairs, specifically including: Using the trained target detection model, key point pairs are detected in the first angle keyframe and the second angle keyframe to obtain matching point pairs.

3. The method according to claim 2, characterized in that, The training process of an object detection model includes: Train a target detection model that identifies key points; Train a regression model to identify the correspondence between keypoint pairs; By cascading the two networks, bias correction is performed on key points with low confidence.

4. The method according to claim 3, characterized in that, The regression model is used to perform combined regression on the three sets of key points corresponding to the first angle keyframe and the second angle keyframe.

5. The method according to claim 3, characterized in that, The two cascaded networks take the first angle keyframe and the second angle keyframe as inputs. The target detection model processes the keyframes of the two channels separately and outputs the coordinate positions of three pairs of key points.

6. The method according to claim 1, characterized in that, Based on the first angle image and the second angle image information, the projection relationship of the two angles before correction is obtained. Combined with the corrected projection relationship of the matching point pairs, the corrected projection relationship is obtained, specifically including: The projection relationship of the first two angle images is obtained based on the imaging information inherent in the first angle image and the second angle image information. Based on the matching point pairs and the projection relationship before correction, an optimization algorithm is used to correct the projection relationship and obtain the corrected projection relationship.

7. The method according to claim 1, characterized in that, The DSA image sequence is automatically filtered and classified sequentially to obtain a first-angle image, a second-angle image, and first-angle keyframes and second-angle keyframes: Using the first trained deep model, the DSA image sequence is filtered to obtain keyframes; Using the trained second depth model, the DSA image sequence is classified based on the keyframes to obtain DSA images of the target blood vessel at the first and second angiography angles, with the corresponding keyframes being the first angle keyframe and the second angle keyframe.

8. The method according to claim 7, characterized in that, The second depth model is a four-class classification model, used to classify the keyframes into: left anterior descending artery (LAD), left circumflex artery (LCX), right coronary artery (RCA), and other categories of images that do not belong to the target vessel; The trained second deep model obtains the input keyframes and outputs the probability values ​​for each category.

9. An apparatus for automatically reconstructing a three-dimensional model of a coronary artery, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method for automatically reconstructing a three-dimensional model of the coronary artery as described in any one of claims 1 to 8.

Images