A vascular guidewire tracking method based on Res-Unet and hidden Markov model

Abstract

The present invention discloses a vascular guidewire tracking method based on Res-Unet and a hidden Markov model, comprising the following steps: S1, training a preoperative neural network model; S2, 2D / 3D alignment; S3, construction and introduction of an HMM probability model; the tip tracking and roadmap used in the present invention can be performed continuously using fluoroscopic imaging, reducing the use of X-ray imaging and contrast agents when obtaining position information, and reducing the radiation dose received by doctors and patients; the present invention provides a 3D roadmap next to the 2D roadmap, which will provide more information for doctors with difficult cases. The present invention constructs an HMM model to track the guidewire tip, and the nature of the tracking method provides temporal consistency of the tracking results, thereby being robust to changes in the field of view, etc.

Description

Technical Field

[0001] The present invention relates to the field of intelligent medical image processing, and in particular to a vascular guidewire tracking method based on Res-Unet and a hidden Markov model. Background Art

[0002] Vascular interventional surgery involves the use of specialized catheters, guidewires, and other instruments along blood vessels to reach the site of vascular lesions for diagnosis and treatment. It is a form of minimally invasive surgery. Interventional physicians operate catheters, guidewires, and other instruments in an operating room, free from the burden of lead aprons and minimizing radiation exposure. During vascular interventional surgery, physicians need to know the position of the guidewire tip relative to the patient's blood vessels to facilitate further procedures.

[0003] Currently, most procedures use contrast agents to visualize blood vessels, guiding guidewire insertion. Doctors use X-rays to visually visualize the blood vessels and the guidewire tip in 2D images, empirically estimating the guidewire's approximate position within the vessel. A computer automatically identifies the guidewire, performs image segmentation, and aligns it with the pre-operative 3D scan.

[0004] However, contrast agents cannot be used continuously due to their toxicity. In addition, the projected vessel shapes are sometimes difficult to interpret. To have a continuous roadmap, doctors use 2D images of X-ray images and also overlay 3D models, such as from preoperative CTA. Unfortunately, such 2D and 3D roadmaps are usually static, and the registration between the 3D vessel model extracted from CTA / MRA and the 2D image is not simple. For example, during cardiac vascular guidewire insertion, the movement caused by breathing and catheter stiffness can cause movement and deformation of the vasculature, thus invalidating the roadmap. Summary of the Invention

[0005] The method provided by the present invention can be performed continuously using fluoroscopic imaging, thereby reducing the use of X-ray imaging and contrast agents when obtaining position information, and reducing the radiation dose received by doctors and patients.

[0006] To achieve this object, the present invention provides the following technical solutions:

[0007] The present invention provides a vascular guidewire tracking method based on Res-Unet and a hidden Markov model, which is characterized by comprising the following steps:

[0008] S1, preoperative neural network model training;

[0009] S2, 2D / 3D registration;

[0010] S3. Construction and import of HMM probability model.

[0011] Preferably, step S1 includes the following steps:

[0012] S11, obtaining X-ray 2D image data of the vascular guidewire, determining the area where the guidewire is located, and zooming in on the image of the area, then marking some points on the guidewire, fitting a spline curve with these points, obtaining a final mask, and forming a network training dataset;

[0013] S12. Build the Res-Unet network model and input the data set for deep convolution training. Based on the traditional image segmentation Unet network model, add a long-hop and short-hop residual network model to make the network have better performance for deeper network convolution.

[0014] Preferably, in step S11, the guide wire is labeled using labelme.

[0015] In the present invention, labelme annotation means: based on the existing coronary guidewire image, the guidewire width is determined to be approximately 1 to 2 pixels, and the gradient pixels between the guidewire and the background are approximately 3 to 4 pixels. First of all, because the guidewire width is narrow, if the polygon method is used when labelme depicts the area, the manual error will cause the width of the guidewire to be wide and narrow, so this method is excluded. Labelme supports the line annotation method linestrip, which can draw multiple line segments together as labels when annotating. This method can solve the problem of different guidewire annotation widths. In addition, the default line width in the labelme source code is specially changed to 2 to better adapt to the annotation of this project.

[0016] Preferably, step S2 includes:

[0017] S21. Perform 3D DCTA scans on the patient's vascular area before surgery to obtain a 3D model of the lesion area, and use algorithms to extract the 3D vascular tree and centerline;

[0018] S22: During the operation, the X-ray 2D image taken in real time during the operation is input into the Res-Unet network model, and the position information of the guide wire in the 2D image is output;

[0019] S23. The 2D image information is registered with the preoperative 3D image.

[0020] Preferably, step S23 includes: firstly projecting the 3D vascular tree from the world coordinate system to the image coordinate system, converting the 3D point from the 3D world coordinate system to the X-ray detector plane coordinate system, and then projecting it to the 2D image plane, assuming that the 3D guidewire tip position Pi, Ci is the 2D guidewire tip position, in the world coordinate system, the line of projection Ci, that is, the line from the X-ray source to Ci on the X-ray image / detector plane, intersects with the plane parallel to the image plane and containing the 3D tip Pi, the intersection point is represented by point li, and this point will be used as τ The starting point of the 3D catheter tip for the registration search, the center of rotation of τ is the same point li, T1 is the conversion of the 3D world coordinate system to the X-ray detector plane coordinate system, T2 is the cone beam projection transformation matrix, due to the geometric structure of the C-arm (surgical projection instrument), both transformations are known, T3 is the conversion from the world coordinate system to the coordinate system centered on li, T4 is the conversion from the coordinate system centered on li to the world coordinate system, and T5 is the translation along the Pi to li line in the world coordinate system. Therefore, the parameterization of the projection transformation is as follows:

[0021] T=T1*T2*T3*T4*T5

[0022] After conversion, the guidewire stereotactic information is obtained.

[0023] Preferably, step S3 includes:

[0024] S31, constructing an HMM probability model, where the time frame is consistent with the intraoperative X-ray 2D image acquisition frame, the state is the position information of the guidewire end in the 3D vascular tree, and the vascular tree is discretized into a set of 3D points P = {P1, ...P n}, the catheter tip is at position P i The probability that the HMM is in state S i probability;

[0025] S32, displaying the guidewire position information in real time on the 3D image to provide the doctor with good surgical guidance information, and at the same time inputting the current state information into the HMM model for updating;

[0026] S33. The Viterbi algorithm in HMM outputs the guidewire route map at that time and the predicted tracking of the guidewire in the absence of contrast agent.

[0027] Preferably, the Viterbi algorithm in step S33 includes: recording the optimal sequence of each observation label at the current moment, as shown in the figure below. Assuming that the optimal path from time 0 to time t has been saved at time t, then at time t+1, it is only necessary to calculate the optimal path from t to t+1, that is, to calculate the Viterbi path from the initial state position of the first image to the current image. First, according to the score max[δ t-1 (j)a ji ]. The state with the best score will be selected to evaluate its observation score Ot (i), therefore, only the most probable tip position will be displayed and other end position points will be discarded.

[0028] Preferably, define the observation function O t (t) represents the quality of 3D / 2D registration, and the centerline of the 3D catheter path V i ={v1,...V i}, then the expression is: Among them, M(C t , v i ) is the metric for 3D / 2D registration, σ a Controls the scaling of the registration metric.

[0029] Preferably, the function δ t (i) is the tip at discrete time point t at position P in the tree i The expression is δ t (i) = max[δ t-1 (j)a ji ]O t (i);

[0030] Define the state transition probability matrix A, which describes the state transition probability matrix A of the catheter tip between two time points, from a point P in the vascular tree i Move to another point P j All transition probabilities a ij

[0031]

[0032] Where D(P i , P j ) is the distance between two points, σ a Controls the speed and distance the guidewire tip moves.

[0033] In summary, the Res-Unet network model of the present invention is formed by combining the advantages of Resnet (residual network) and Unet network, and each submodule of UNet is replaced by a form with residual connection and dense connection. Dense connection is to use the output of a certain layer in the submodule as part of the input of several subsequent layers, and the input of a certain layer comes from the combination of the outputs of several previous layers. By training the Res-Unet neural network, a good guidewire segmentation model is given. After realizing the position information of the guidewire in the picture, the image algorithm determines the position information of the cardiovascular system according to the position of the guidewire in the picture and matches it with the preoperative three-dimensional model.

[0034] The method of the present invention is based on a hidden Markov model (HMM), where the states of the model are the possible positions of the catheter tip within a 3D vascular tree. The transition from state to state becomes the probability of the catheter tip moving from one position to another. Information from X-ray 2D images is used to track the catheter tip within a 3D vascular tree extracted from preoperative computed tomography (CTA). The HMM is used to calculate the probability distribution of the 3D catheter tip position. The probability distribution is updated based on the previous distribution and the 3D / 2D registration results of the 3D vascular tree and the 2D catheter centerline extracted from the X-ray images during the interventional procedure. The guidewire tip with the highest probability is 2D / 3D registered. The HMM is updated after the observation score is obtained to obtain real-time data information.

[0035] Compared with the prior art, the present invention has the following beneficial effects and significant improvements:

[0036] 1. The method of the present invention effectively reduces the use of X-ray imaging and contrast agents during surgery, reduces the radiation dose received by doctors and patients, and ensures the real-time tracking of the guidewire;

[0037] 2. The fan method provided by the present invention provides a 3D roadmap next to the 2D roadmap, which will provide more information for doctors with difficult cases.

[0038] 3. The nature of the tracking method of the present invention provides temporal consistency of the tracking results, and is therefore robust to changes in the field of view, etc.

[0039] 4. The tip tracking and road mapping of the present invention can be performed continuously using fluoroscopic imaging, without the need for contrast agents, and without other features other than the guidewire. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for the embodiments of the present invention.

[0041] Obviously, the drawings described below are only drawings of some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work, but these other drawings also belong to the drawings required for use in the embodiments of the present invention.

[0042] Figure 1 This is a flow chart of the method in Example 2 of the present invention;

[0043] Figure 2 Schematic diagram of the 2D guidewire recognition network structure model in Example 2 of the present invention;

[0044] Figure 3 This is a flowchart of 3D vascular guidewire positioning in Example 2 of the present invention;

[0045] Figure 4 This is the HMM model diagram in Example 2 of the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution, beneficial effects and significant improvements of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention.

[0047] Obviously, all the embodiments described are only partial embodiments of the present invention, rather than all embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative work are within the scope of protection of the present invention.

[0048] What needs to be understood is:

[0049] For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0050] It should also be noted that the following specific embodiments may be combined with each other, and the same or similar concepts or processes therein may not be repeated in some embodiments.

[0051] The technical solution of the present invention is described in detail below with reference to specific embodiments.

[0052] Example 1

[0053] A vascular guidewire tracking method based on Res-Unet and hidden Markov model includes the following steps:

[0054] S1, preoperative neural network model training;

[0055] S2, 2D / 3D registration;

[0056] S3. Construction and import of HMM probability model.

[0057] In this embodiment, step S1 includes the following steps:

[0058] S11, obtaining X-ray 2D image data of the vascular guidewire, determining the area where the guidewire is located, and zooming in on the image of the area, then marking some points on the guidewire, fitting a spline curve with these points, obtaining a final mask, and forming a network training dataset;

[0059] S12. Build the Res-Unet network model and input the data set for deep convolution training. Based on the traditional image segmentation Unet network model, add a long-hop and short-hop residual network model to make the network have better performance for deeper network convolution.

[0060] In this embodiment, in step S11, the guidewire is labeled using labelme.

[0061] In this embodiment, step S2 includes:

[0062] S21. Perform 3D DCTA scans on the patient's vascular area before surgery to obtain a 3D model of the lesion area, and use algorithms to extract the 3D vascular tree and centerline;

[0063] S22: During the operation, the X-ray 2D image taken in real time during the operation is input into the Res-Unet network model, and the position information of the guide wire in the 2D image is output;

[0064] S23. The 2D image information is registered with the preoperative 3D image.

[0065] In this embodiment, step S23 includes: first projecting the 3D vascular tree from the world coordinate system to the image coordinate system, converting the 3D point from the 3D world coordinate system to the X-ray detector plane coordinate system, and then projecting it to the 2D image plane. Let the 3D guidewire tip position Pi, Ci be the 2D guidewire tip position, and in the world coordinate system, the line of projection Ci, that is, the line from the X-ray source to Ci on the X-ray image / detector plane, intersects with the plane parallel to the image plane and containing the 3D tip Pi, and the intersection is represented by point li, which will be used as τ is the starting point of the 3D catheter tip for the registration search, the rotation center of τ is the same point li, T1 is the conversion of the 3D world coordinate system to the X-ray detector plane coordinate system, T2 is the cone beam projection transformation matrix, due to the geometric structure of the C-arm (surgical projection instrument), both transformations are known, T3 is the conversion from the world coordinate system to the coordinate system centered on li, T4 is the conversion from the coordinate system centered on li to the world coordinate system, and T5 is the translation along the Pi to li line in the world coordinate system. Therefore, the parameterization of the projection transformation is as follows:

[0066] T=T1*T2*T3*T4*T5

[0067] After conversion, the guidewire stereotactic information is obtained.

[0068] In this embodiment, step S3 includes:

[0069] S31, constructing an HMM probability model, where the time frame is consistent with the intraoperative X-ray 2D image acquisition frame, the state is the position information of the guidewire end in the 3D vascular tree, and the vascular tree is discretized into a set of 3D points P = {P1, ...P n}, the catheter tip is at position P i The probability that the HMM is in state S i probability;

[0070] S32, displaying the guidewire position information in real time on the 3D image to provide the doctor with good surgical guidance information, and at the same time inputting the current state information into the HMM model for updating;

[0071] S33. The Viterbi algorithm in HMM outputs the guidewire route map at that time and the predicted tracking of the guidewire in the absence of contrast agent.

[0072] In this embodiment, the Viterbi algorithm in step S33 includes: recording the optimal sequence of each observation label at the current time, as shown in the figure below. Assuming that the optimal path from time 0 to time t has been saved at time t, then at time t+1, it is only necessary to calculate the optimal path from t to t+1, that is, to calculate the Viterbi path from the initial state position of the first image to the current image. First, according to the score max[δ t-1 (j)a ji ]. The state with the best score will be selected to evaluate its observation score O t (i), therefore, only the most probable tip position will be displayed and other end position points will be discarded.

[0073] In this embodiment, the observation function O is defined t (t) represents the quality of 3D / 2D registration, and the centerline of the 3D catheter path V i ={v1,...V i}, then the expression is: Among them, M(C t , v i ) is the metric for 3D / 2D registration, σ a Controls the scaling of the registration metric.

[0074] In this embodiment, the function δ t (i) is the tip at discrete time point t at position P in the tree i The expression is δ t (i) = max[δ t-1 (j)a ji ]O t (i);

[0075] Define the state transition probability matrix A, which describes the state transition probability matrix A of the catheter tip between two time points, from a point P in the vascular tree i Move to another point P j All transition probabilities a ij

[0076]

[0077] Where D(P i , P j ) is the distance between two points, σ aControls the speed and distance the guidewire tip moves.

[0078] Example 2

[0079] This embodiment provides a PCI surgery vascular guidewire tracking method based on Res-Unet and Hidden Markov Model. The specific process is as follows: Figure 1 shown.

[0080] First, we prepared a model training dataset. To ensure comprehensiveness of the guidewire tip dataset, we obtained folders containing case images from hospitals. Some contain over 100 sequences, with each sequence averaging dozens of frames. However, the image features of a single case are often unique. Three sequences were selected for each case, with 10 images extracted from each sequence at a specific step size, resulting in 30 images per case. Sequence selection ensured that the guidewire position, including on the main coronary artery, at the bifurcation, on the branch, and at the end of the branch, was accurately captured. Furthermore, the contrast agent was observed to be present and absent.

[0081] Then use labelme to annotate. According to the existing coronary guidewire images, it is determined that the guidewire width is about 1 to 2 pixels, and the gradient pixels between the guidewire and the background are about 3 to 4 pixels. First of all, because the guidewire width is narrow, if the polygon method is used when labelme to depict the area, the manual error will cause the width of the annotated guidewire to be wide and narrow, so this method is excluded. Labelme supports the line annotation method linestrip, which can draw multiple line segments together as labels when annotating. This method can solve the problem of different guidewire annotation widths. In addition, the default line width in the labelme source code is specially changed to 2 to better adapt to the annotation of this project.

[0082] Finally, the saved json annotation files are batch-generated into png binary images through python code to complete the input of the data set.

[0083] The guidewire segmentation method based on Res-Unet includes a dataset module, a network model, and a training module. The dataset module defines the image and label paths, reads the original images and labels, and converts them to RGB, converting them to the square image aspect ratio required by the network. The network model includes defining a basic 3x3 convolutional block, a downsampling module (using convolution to downsample by a factor of 2, with the same number of channels), an upsampling module (increasing the feature map size by a factor of 2 and halving the number of channels), and ResNet50 (alleviating the vanishing gradient problem in deep networks).

[0084] The network backbone performs four downsampling passes, four upsampling passes, and a forward pass (upsampling can be done by cutting out the middle of the downsampling process and then concatenating it). The output is a prediction whose size is the same as the input. Residual skipping is implemented between basic blocks to enable deeper convolutions.

[0085] The training module includes defining the device input (CPU, graphics card training), network model selection, and binary cross entropy loss. After preparation, run the training module directly and output the trained model and effect graph. Use the trained model to output 2D image guide wire end information, such as Figure 2 As shown, it is a schematic diagram of the 2D guidewire recognition network structure model.

[0086] 3D vascular tree model

[0087] like Figure 3 Shown is the flowchart of 3D vascular guidewire positioning.

[0088] First, a preoperative 3D scan of the heart is performed to extract the coronary centerline. The coronary opening point and seed point are detected first, and then the seed points are traversed to track the centerline. Two detector models are used to detect the two opening points and 200 seed points of the left and right coronary arteries, respectively. The detector model is trained based on a convolutional neural network to learn the relationship between anatomical structure and proximity. For the opening point detector, the proximity is calculated based on the point 16mm away from the opening point. For the seed point detector, the proximity is calculated based on the point 4mm away from the centerline. The proximity of the remaining points is defaulted to 0.

[0089] Then, the 200 seed points are traversed, and the tracker model is used to predict the direction and radius of the current point. Forward and reverse iterative tracking is performed with the radius as the step size. The tracker model is also trained based on a convolutional neural network, and it learns the relationship between anatomical structure and direction and radius. The number of direction categories is set to 422.

[0090] A discriminator model is used to determine whether to stop tracking. The discriminator model is trained based on a convolutional neural network to learn the relationship between anatomical structure and whether tracking should stop. Points at the end of the centerline are considered positive samples (tracking stopped), and points near the centerline are considered negative samples (tracking did not stop).

[0091] If tracking reaches the vicinity of the opening point (i.e. the point at the head of the center line), tracking stops.

[0092] Combined with the image boundary algorithm, a 3D vascular tree model is obtained.

[0093] 2D / 3D pairing

[0094] First, the 3D vascular tree is projected from the world coordinate system to the image coordinate system, and the 3D points are converted from the 3D world coordinate system to the X-ray detector plane coordinate system, and then projected to the 2D image plane. Let the 3D guidewire end position P be i , C i is the 2D guidewire end position, in the world coordinate system, projection C i The line (i.e. the line from the X-ray source to C on the X-ray image / detector plane)i The line parallel to the image plane and containing the 3D tip P i The intersection point is defined by point l i Indicates that this point will be the starting point of the 3D catheter tip for the τ registration search. In addition, the rotation center of τ is the same point l i T1 is the conversion of 3D world coordinate system to X-ray detector plane coordinate system, T2 is the cone beam projection transformation matrix, due to the geometric structure of C-arm (surgical projection instrument), both transformations are known, T3 is the conversion from world coordinate system to l i The transformation of the coordinate system centered on l i The conversion from the coordinate system centered on the world coordinate system to the world coordinate system is as follows: i to l i Line translation, therefore, the parameterization of the projective transformation is as follows:

[0095] T=T1*T2*T3*T4*T5

[0096] After conversion, the guidewire stereotactic information is obtained.

[0097] Constructing HMM model

[0098] When there is no surgical imaging information, construct an HMM model, such as Figure 4 shown.

[0099] The time frame is consistent with the intraoperative X-ray 2D image acquisition time frame, and the state is the position information of the guidewire end in the 3D vascular tree. The vascular tree is discretized into a set of 3D points P = {P1, ...P n The catheter tip is at position P. i The probability that the HMM is in state S i probability.

[0100] Definition of state transition probability matrix A: The state transition probability matrix A describes the state transition probability matrix A of the catheter tip from a point P in the vascular tree between two time points (i.e., between two subsequent 2D X-ray images). i Move to another point P j (State S i to S j ) all transition probabilities a ij .

[0101]

[0102] Where D(P i , P j ) is the distance between two points, σ a Controls the speed and distance the guidewire tip moves.

[0103] Define the observation function Ot (t) represents the quality of 3D / 2D registration, and the centerline of the 3D catheter path V i ={v1,...V i}, then the expression is:

[0104]

[0105] Where M(C t , v i ) is the metric for 3D / 2D registration, σ a Controls the scaling of the registration metric.

[0106] Function δ t (i) is the tip at discrete time point t at position P in the tree i The expression is

[0107] δ t (i) = max[δ t-1 (j)a ji ]O t (i).

[0108] The Viterbi algorithm is a dynamic programming algorithm that finds the optimal sequence of all observations. The optimal sequence of each observation label is recorded at the current moment, as shown in the figure below. Assuming that the optimal path from 0 to t has been saved at time t, then at time t+1, only the optimal path from t to t+1 needs to be calculated, that is, the Viterbi path from the initial state position of the first image to the current image is calculated. First, according to the score max[δ t-1 (j)a ji ]. The state with the best score will be selected to evaluate its observation score O t (i). Therefore, only the most probable tip position will be displayed and other end position points will be discarded.

[0109] In the description of the above manual:

[0110] The descriptions of terms such as "this embodiment", "an embodiment of the present invention", "as shown in...", "further", "a further improved technical sub-scheme", etc., mean that the specific features, structures, materials or characteristics described in the embodiment or example are included in at least one embodiment or example of the present invention; in this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example, and the specific features, structures, materials or characteristics described may be combined or combined in an appropriate manner in any one or more embodiments or examples; in addition, a person of ordinary skill in the art may combine or combine different embodiments or examples and features of different embodiments or examples described in this specification without causing any contradiction.

[0111] Finally, it should be noted that:

[0112] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same.

[0113] Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein, and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention. Non-essential improvements, adjustments or replacements made by those skilled in the art based on the contents of this specification are all within the scope of protection required by the present invention.

Claims

1. A vascular guidewire tracking method based on Res-Unet and Hidden Markov Model, characterized in that: The following steps are involved: S1, preoperative neural network model training; S2, 2D / 3D registration; S3, construction and import of HMM probability model; Step S1 includes the following steps: S11, obtaining X-ray 2D image data of the vascular guidewire, determining the area where the guidewire is located, and zooming in on the image of the area, then marking some points on the guidewire, fitting a spline curve with these points, obtaining a final mask, and forming a network training dataset; S12. Build a Res-Unet network model and input the data set for deep convolution training. Based on the traditional image segmentation Unet network model, add a long-hop and short-hop residual network model to make the network have better performance for deeper network convolution; Step S3 includes: S31, constructing an HMM probability model, where the time frame is consistent with the intraoperative X-ray 2D image acquisition frame, the state is the position information of the guidewire end in the 3D vascular tree, and the vascular tree is discretized into a set of 3D points P = {P1, ...P n }, the catheter tip is at position P i The probability that the HMM is in state S i probability; S32, displaying the guidewire position information in real time on the 3D image to provide the doctor with good surgical guidance information, and at the same time inputting the current state information into the HMM model for updating; S33, the Viterbi algorithm in the HMM outputs the guidewire route map at that time and the predicted tracking of the guidewire in the absence of contrast agent; The Viterbi algorithm in step S33 includes: recording the optimal sequence of each observation label at the current moment, as shown in the figure below. Assuming that the optimal path from time 0 to time t has been saved at time t, then at time t+1, it is only necessary to calculate the optimal path from t to t+1, that is, to calculate the Viterbi path from the initial state position of the first image to the current image. First, according to the score max[δ t-1 (j)a ji ], the state with the best score will be selected to evaluate its observation score O t (i), therefore, only the most probable tip position will be displayed, and other end position points will be discarded; Define the observation function O t (t) represents the quality of 3D / 2D registration, and the centerline of the 3D catheter path V i ={v1,...V i }, then the expression is: Among them, M(C t , v i ) is the metric for 3D / 2D registration, σ s Control the scaling of the registration metric; Function δ t (i) is the tip at discrete time point t at position P in the tree i The fraction of t (i) = max[δ t-1 (j)a ji ]O t (i); Define the state transition probability matrix A, which describes the state transition probability matrix A of the catheter tip between two time points, from a point P in the vascular tree i Move to another point P j All transition probabilities a ij Where D(P i , P j ) is the distance between two points, σ a Controls the speed and distance the guidewire tip moves.

2. A vascular guidewire tracking method based on Res-Unet and Hidden Markov Model according to claim 1, characterized in that: In step S11, the guidewire is labeled using labelme.

3. The vascular guidewire tracking method based on Res-Unet and Hidden Markov Model according to claim 1, characterized in that: Step S2 includes: S21. Perform 3D DCTA scans on the patient's vascular area before surgery to obtain a 3D model of the lesion area, and use algorithms to extract the 3D vascular tree and centerline; S22: During the operation, the X-ray 2D image taken in real time during the operation is input into the Res-Unet network model, and the position information of the guide wire in the 2D image is output; S23. The 2D image information is registered with the preoperative 3D image.

4. The vascular guidewire tracking method based on Res-Unet and Hidden Markov Model according to claim 3, characterized in that: Step S23 includes: first projecting the 3D vascular tree from the world coordinate system to the image coordinate system, converting the 3D point from the 3D world coordinate system to the X-ray detector plane coordinate system, and then projecting it to the 2D image plane, assuming the 3D guidewire tip position Pi, Ci is the 2D guidewire tip position, in the world coordinate system, the line of projection Ci, that is, the line from the X-ray source to Ci on the X-ray image / detector plane, intersects with the plane parallel to the image plane and containing the 3D tip Pi, the intersection is represented by point li, which will serve as the starting point of the 3D catheter tip for τ registration search, the rotation center of τ is the same point li, T1 is the 3D world coordinate system to the X-ray detector plane coordinate system, T2 is the cone beam projection transformation matrix, due to the geometric structure of the C-arm surgical projection instrument, these two transformations are known, T3 is the transformation from the world coordinate system to the coordinate system centered on li, T4 is the transformation from the coordinate system centered on li to the world coordinate system, and T5 is the translation along the Pi to li line in the world coordinate system, so the parameterization of the projection transformation is as follows: T=T1*T2*T3*T4*T5 After conversion, the guidewire stereotactic information is obtained.

Images