Automatic Early Warning Method, Device and Medium for Instrument Damage Risk of Vascular Intervention Robot

Abstract

The present invention belongs to the technical field of minimally invasive interventional surgery, and discloses an automatic early warning method, device and medium for the risk of instrument damage of a vascular interventional robot. The method includes: obtaining angiography images in real time; determining the edge position and branch position of the vascular tree based on the angiography images, and determining the current position of the guide wire head; monitoring the distance from the current position to the edge position and / or branch position; and if the distance is less than a preset distance, giving an automatic early warning. By the above method, it is possible to monitor the movement of the guide wire head in real time during PCI surgery, and give an early warning in a timely manner according to the movement situation, thereby reducing the risk of accidental blood vessel damage during PCI surgery by doctors.

Description

Technical Field

[0001] The present invention relates to the technical field of minimally invasive interventional surgery, and particularly to an automatic early warning method, device and medium for the risk of instrument damage of a vascular interventional robot. Background Art

[0002] Currently, the main vascular interventional robot products on the market include CorPath GRX of Corindus (approved by the FDA for marketing in 2016), and R-One of RoboCath (approved by the CE for marketing in 2019), etc. When these vascular interventional robot products are used in PCI operations, doctors control the movement of the guide wire through a joystick, so as to assist doctors in placing stents. However, in the traditional technology, there is no early warning for the damage risk. Therefore, when using a vascular interventional robot for PCI surgery, it is easy for the guide wire to touch and act on the intima of the blood vessel, resulting in iatrogenic arterial dissection and endangering the life of the patient. Summary of the Invention

[0003] The main purpose of the present invention is to provide an automatic early warning method, device and medium for the risk of instrument damage of a vascular interventional robot, aiming to solve the technical problem that the prior art lacks early warning for the damage risk during PCI surgery.

[0004] To achieve the above purpose, the present invention provides an automatic early warning method for the risk of instrument damage of a vascular interventional robot, and the method includes the following steps:

[0005] Obtain angiography images in real time;

[0006] Based on the angiography images, determine the edge positions and branch positions of the vascular tree, and determine the current position of the head of the guide wire;

[0007] Monitor the distance from the current position to the edge position and / or branch position;

[0008] If the distance is less than a preset distance, give an automatic early warning.

[0009] Optionally, the determining the edge positions and / or branch positions of the vascular tree based on the angiography images includes:

[0010] Based on the angiography images, obtain the vascular tree segmentation images;

[0011] Based on the vascular tree segmentation images, extract the centerlines of each vascular tree;

[0012] Based on the vascular tree segmentation images and the centerlines of each vascular tree, determine the edge positions of the vascular tree;

[0013] Based on the centerlines of the respective vascular trees, determine the intersection points of the respective vascular trees, thereby obtaining the positions of the branch points of the vascular trees.

[0014] Optionally, based on the angiography image, determine the current position of the guide wire tip, including:

[0015] Input the angiography image into the Encoder of the trained first U-net to extract abstract features;

[0016] Input the abstract features into the Decoder of the trained first U-net to predict the probability map of the part belonging to the guide wire in the angiography image;

[0017] Based on a threshold, convert the probability map into a binary segmentation result;

[0018] Based on the binary segmentation result, determine the current position of the guide wire tip through position information.

[0019] Optionally, before inputting the angiography image into the Encoder of the trained first U-net to extract abstract features, further include:

[0020] S2010. Input the first angiography image into the Encoder of the first U-net to extract abstract features, where the real guide wire part is marked in the first angiography image;

[0021] S2011. Input the abstract features into the Decoder of the first U-net to predict the probability map of the part belonging to the guide wire in the first angiography image;

[0022] S2012. Based on the probability map and the real guide wire part, calculate the first prediction error;

[0023] S2013. Update the parameters in the first U-net through backpropagation based on the first prediction error;

[0024] S2014. After taking the second angiography image as the new first angiography image, repeat steps S2010 - S2014 until the first prediction error is less than a preset value to obtain the trained first U-net.

[0025] Optionally, based on the angiography image, determine the current position of the guide wire tip, including:

[0026] Input the angiography image into the Encoder of the trained second U-net to extract abstract features;

[0027] Input the abstract features into the Decoder of the trained second U-net to predict the displacement of each point in the angiography image to the guide wire head, obtaining a displacement map;

[0028] Calculate the displaced position of each point in the angiography image based on the displacement map and form a voting map;

[0029] Take the position with the highest score in the voting map as the current position of the guide wire head.

[0030] Optionally, before inputting the angiography image into the Encoder of the trained U-net to extract abstract features, it further includes:

[0031] S2020: Input the third angiography image into the Encoder of the second U-net to extract abstract features, where the true distance from each point to the guide wire head is marked in the third angiography image;

[0032] S2021: Input the abstract features into the Decoder of the second U-net to predict the displacement of each point in the third angiography image to the guide wire head, obtaining a predicted displacement map;

[0033] S2022: Compare the displacement of each point to the guide wire head in the predicted displacement map with the true distance from each point to the guide wire head, and calculate the second prediction error;

[0034] S2023: Update the parameters in the second U-net through backpropagation based on the second prediction error;

[0035] S2024: After taking the fourth angiography image as the new third angiography image, repeat steps S2020 - S2024 until the second prediction error is less than a preset value to obtain the trained second U-net.

[0036] Optionally, determining the current position of the guide wire head based on the angiography image includes:

[0037] Input the angiography image into the trained CNN to extract abstract features;

[0038] Input the abstract features into a fully connected layer and predict the current position of the guide wire head through regression.

[0039] Optionally, before inputting the angiography image into the trained CNN to extract abstract features, it further includes:

[0040] S2030. The fifth angiography image is input into the CNN to extract abstract features, where the position of the real guide wire head is marked on the fifth angiography image;

[0041] S2031. The abstract features are input into the fully connected layer, and the position of the predicted guide wire head is obtained through regression prediction;

[0042] S2032. The position of the measured guide wire head is compared with the position of the real guide wire head, and the third prediction error is calculated;

[0043] S2033. The parameters in the CNN are updated through backpropagation based on the third prediction error;

[0044] S2034. After using the sixth angiography image as the new fifth angiography image, steps S2030 - S2034 are repeatedly executed until the third prediction error is less than the preset value, and a trained CNN is obtained.

[0045] In addition, to achieve the above object, the present invention also proposes an automatic warning device for the risk of instrument damage of a vascular intervention robot. The automatic warning device for the risk of instrument damage of a vascular intervention robot includes: a memory, a processor, and an automatic warning program for the risk of instrument damage of a vascular intervention robot stored on the memory and executable on the processor. The automatic warning program for the risk of instrument damage of a vascular intervention robot is configured to implement the steps of the automatic warning method for the risk of instrument damage of a vascular intervention robot as described above.

[0046] In addition, to achieve the above object, the present invention also proposes a storage medium with an automatic warning program for the risk of instrument damage of a vascular intervention robot stored thereon. When the automatic warning program for the risk of instrument damage of a vascular intervention robot is executed by a processor, the steps of the automatic warning method for the risk of instrument damage of a vascular intervention robot as described above are implemented.

[0047] By obtaining the angiography image in real time, the present invention can timely determine the latest position of the guide wire head during the interventional surgery, and can also determine the edge position and branch position of the vascular tree based on the angiography image. By comparing the latest position of the guide wire head with the edge position and branch position of the blood vessel respectively, the distance from the current position to the edge position and / or branch position can be monitored. When this distance is less than the preset distance, an automatic warning can be issued. Through the above method, the movement of the guide wire head can be monitored in real time during the PCI surgery, and an early warning can be issued in a timely manner according to the movement situation, thereby reducing the risk of accidental blood vessel damage during the PCI surgery by doctors. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1It is a schematic structural diagram of an automatic early warning device for the risk of instrument damage of a vascular intervention robot in the hardware operating environment involved in the embodiment solution of the present invention;

[0049] Figure 2 It is a schematic flowchart of the first embodiment of the automatic early warning method for the risk of instrument damage of the vascular intervention robot of the present invention;

[0050] Figure 3 It is a schematic flowchart of the second embodiment of the automatic early warning method for the risk of instrument damage of the vascular intervention robot of the present invention;

[0051] Figure 4 It is a schematic flowchart of the third embodiment of the automatic early warning method for the risk of instrument damage of the vascular intervention robot of the present invention;

[0052] Figure 5 It is a schematic flowchart of the fourth embodiment of the automatic early warning method for the risk of instrument damage of the vascular intervention robot of the present invention;

[0053] Figure 6 It is a structural block diagram of the first embodiment of the automatic early warning device for the risk of instrument damage of the vascular intervention robot of the present invention;

[0054] Figure 7 They are the angiography image, the vascular tree segmentation image, the vascular centerline image of the vascular tree, and the edge image of the vascular tree in an embodiment of the automatic early warning method for the risk of instrument damage of the vascular intervention robot of the present invention;

[0055] Figure 8 It is the early warning method in an embodiment of the automatic early warning method for the risk of instrument damage of the vascular intervention robot of the present invention.

[0056] The realization, functional features, and advantages of the object of the present invention will be further described in conjunction with the embodiments with reference to the accompanying drawings. Detailed Embodiment

[0057] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0058] Refer to Figure 1 , Figure 1 It is a schematic structural diagram of an automatic early warning device for the risk of instrument damage of a vascular intervention robot in the hardware operating environment involved in the embodiment solution of the present invention.

[0059] Such as Figure 1As shown in the figure, the automatic early warning device for the risk of instrument damage of the vascular intervention robot may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard). Optionally, the user interface 1003 may further include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0060] Those skilled in the art can understand that Figure 1 the structure shown in the figure does not constitute a limitation on the automatic early warning device for the risk of instrument damage of the vascular intervention robot, and may include more or fewer components than shown in the figure, or combine some components, or have different component arrangements.

[0061] As Figure 1 shown, the memory 1005, as a storage medium, may include an operating system, a network communication module, a user interface module, and an automatic early warning program for the risk of instrument damage of the vascular intervention robot.

[0062] In Figure 1 the automatic early warning device for the risk of instrument damage of the vascular intervention robot shown in the figure, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the automatic early warning device for the risk of instrument damage of the vascular intervention robot of the present invention may be arranged in the automatic early warning device for the risk of instrument damage of the vascular intervention robot. The automatic early warning device for the risk of instrument damage of the vascular intervention robot calls the automatic early warning program for the risk of instrument damage of the vascular intervention robot stored in the memory 1005 through the processor 1001 and executes the automatic early warning method for the risk of instrument damage of the vascular intervention robot provided by the embodiments of the present invention.

[0063] The embodiments of the present invention provide an automatic early warning method for the risk of instrument damage of a vascular intervention robot. Referring to Figure 2 , Figure 2Schematic flowchart of the first embodiment of an automatic early warning method for instrument damage risk of a vascular intervention robot according to the present invention.

[0064] In this embodiment, the automatic early warning method for instrument damage risk of the vascular intervention robot includes the following steps:

[0065] Step S10: Obtain angiography images in real time.

[0066] Among them, the angiography image can be an image formed by taking coronary angiography through X-rays, usually in DICOM format.

[0067] Specifically, percutaneous coronary intervention (PCI) surgery, also known as cardiac stent surgery, is a procedure where a guide wire is advanced through a punctured blood vessel to reach the arterial opening, and a stent is delivered to the required location using a special delivery system, and then the guide wire is placed and withdrawn to complete the surgery. During the PCI procedure, the guide wire is constantly moving, so it is necessary to obtain angiography images in real time to determine the latest position of the guide wire tip. During PCI surgery, contrast agent is continuously injected into the blood vessel, which can make the blood vessels appear clearer in the real-time obtained angiography images.

[0068] Step S20: Based on the angiography image, determine the edge position and branch position of the vascular tree, and determine the current position of the guide wire tip.

[0069] It should be noted that the vascular tree refers to multiple blood vessels, including the aortic arch vessels and branch vessels, etc.

[0070] Among them, the current position of the guide wire tip is the latest position of the guide wire tip, which can be the current position coordinates of the guide wire tip. The edge position of the vascular tree can be the edge position coordinates of the vascular tree, and the branch position of the vascular tree can be the branch position coordinates.

[0071] In one embodiment, based on the angiography image, obtain a vascular tree segmentation image; based on the vascular tree segmentation image, extract the centerlines of each blood vessel; based on the vascular tree segmentation image and the centerlines of each vascular tree, determine the edge position of the vascular tree; based on the centerlines of each blood vessel, determine the intersection points of each vascular tree, so as to obtain the branch position of the vascular tree.

[0072] As Figure 7 described, Figure 7 (a) is an angiography image, Figure 7 (b) is a vascular tree segmentation image, Figure 7 (c) is an image of the vascular centerlines of the vascular tree, and Figure 7 (d) is an edge image of the vascular tree.

[0073] Specifically, it is necessary to perform vascular segmentation on angiography images, that is, to segment blood vessels from angiography images, and then obtain vascular tree segmentation images. Deep learning methods can be used, such as using U-Net to segment vascular trees from angiography images, or traditional image processing methods can also be used, such as using thresholding methods to segment vascular trees from angiography images. Among them, the segmented vascular trees can include aortic arch vessels and branch vessels, etc. Then, the vascular centerlines of each blood vessel are extracted from the vascular tree segmentation images. The vascular centerlines of each vascular tree can be extracted by the shortest path method, or a topology-preserving erosion operation can be performed on the vascular masks (masks) of each blood vessel in the vascular tree segmentation images until each vascular mask is refined into a vascular centerline with a single-voxel width. Since each blood vessel is interconnected, the centerlines of each blood vessel intersect, and thus the position coordinates of the vascular branch can be determined based on the intersection points of each blood vessel. The segmentation image can also be eroded and subtracted according to the centerlines of each vascular tree to obtain the edge of the vascular tree, and then the edge position coordinates of the vascular tree can be determined.

[0074] Step S30: Monitor the distance from the current position to the edge position and / or the branch position.

[0075] Specifically, the current position of the guide wire head refers to the current position coordinates of the guide wire head, the edge position of the vascular tree refers to the edge position coordinates of the vascular tree, and the branch position of the vascular tree refers to the branch position coordinates of the vascular tree. The distances from the current position of the guide wire head to the edge position of the vascular tree and to the branch position can be determined based on the coordinates between each pair.

[0076] Step S40: If the distance is less than a preset distance, an automatic warning is issued.

[0077] It should be noted that the preset distance is set in advance. This preset distance is determined based on multiple experimental data. When the position of the guide wire head is greater than the preset distance from the edge position of the blood vessel or the branch position of the blood vessel, a warning is issued in a timely manner, so that the doctor has enough time to readjust the direction of the guide wire head, thereby reducing the risk of accidental damage to blood vessels during the doctor's PCI operation. For example Figure 8As shown, the warning methods include but are not limited to sound warning and visual warning. Among them, the sound warning can achieve automatic warning by emitting a beeping sound or a warning message, and the visual warning can be carried out by emitting a pop-up prompt. The content of the pop-up prompt can be a text reminder or a partial enlarged view of the position of the guide wire head. Multiple preset distances can also be set. Different preset distances correspond to different warning levels, and different warning levels adopt different warning methods. For example, when the position of the guide wire head is less than the first preset distance, that is, the distance from the edge position and / or the branch position is relatively far, the automatic warning can be completed only through the warning message. When the position of the guide wire head is less than the second preset distance, that is, the distance from the edge position and / or the branch position is relatively close, the automatic warning can be jointly completed by emitting the warning message and the beeping sound.

[0078] In this embodiment, by obtaining the angiography image in real time, it is possible to timely determine the latest position of the guide wire head during the interventional operation, and based on the angiography image, determine the edge position and the branch position of the vascular tree. By comparing the latest position of the guide wire head with the edge position of the blood vessel and the branch position of the blood vessel respectively, the distance from the current position to the edge position and / or the branch position can be monitored. When this distance is less than the preset distance, an automatic warning can be issued. Through the above method, it is possible to monitor the movement of the guide wire head in real time during the PCI operation and give a timely warning according to the movement situation, which not only avoids the patient from suffering from iatrogenic arterial dissection and endangering the patient's life due to the guide wire touching and acting on the edge position of the blood vessel, that is, the intima of the blood vessel, but also avoids the patient from suffering from iatrogenic plaque rupture and forming thrombus and endangering the patient's life due to the guide wire touching and acting on the branch position of the blood vessel, that is, the plaque. Based on the above, the present invention can reduce the risk of accidental injury to blood vessels during the PCI operation by doctors.

[0079] Reference Figure 3 , Figure 3 is a schematic flowchart of the second embodiment of an automatic warning method for the risk of instrument damage of a vascular intervention robot according to the present invention.

[0080] Based on the above first embodiment, in this embodiment of the automatic warning method for the risk of instrument damage of a vascular intervention robot, based on the angiography image, obtaining the current position of the guide wire head includes:

[0081] Step S201: Input the angiography image into the Encoder of the trained first U-net to extract abstract features; input the abstract features into the Decoder of the trained first U-net to predict the probability map of the part belonging to the guide wire in the angiography image; convert the probability map into a binary segmentation result based on a threshold; based on the binary segmentation result, determine the current position of the guide wire head through position information.

[0082] It should be noted that the first U-net is the U-net network.

[0083] Specifically, the threshold can be set in advance. For example, the threshold is set to 0.5. The pixel points in the probability map with a threshold greater than 0.5 are classified as the guide wire part, and the pixel points in the probability map with a threshold less than or equal to 0.5 are classified as the non-guide wire part, obtaining a binary segmentation result. The position of the guide wire head is determined from the binary segmentation result, and then the current position coordinates of the guide wire head are calculated based on the determined position information.

[0084] Among them, the steps to obtain the trained first U-net include:

[0085] Step S2010: Input the first angiography image into the Encoder of the first U-net to extract abstract features, where the real guide wire part is marked in the first angiography image.

[0086] It should be noted that both the first angiography image and the second angiography image are angiography images marked with the real guide wire part. The first angiography image and the second angiography image are both sample data used to train the first U-net, and the first angiography image and the second angiography image in each round of training are different.

[0087] Step S2011: Input the abstract features into the Decoder of the first U-net to predict the probability map of the part belonging to the guide wire in the first angiography image.

[0088] Step S2012: Calculate the first prediction error based on the probability map and the real guide wire part.

[0089] Specifically, the predicted guide wire part in the first angiography image is determined according to the probability map, and the predicted guide wire part is compared with the real guide wire part to calculate the first prediction error.

[0090] Step S2013: Update the parameters in the first U-net by backpropagation based on the first prediction error.

[0091] S2014. After using the second angiography image as the new first angiography image, repeat steps S2010 - S2014 until the first prediction error is less than the preset value, obtaining the trained first U-net.

[0092] Specifically, the preset value is determined in advance and can be set according to the desired network state to be trained. When the first prediction error is less than the preset value, it can be considered that the first U-net is already trained at this time, that is, the trained first U-net is obtained.

[0093] In this embodiment, a method based on a deep learning network, especially a method based on the U-net network, is used to monitor the head position of the guide wire in the angiography image. It can not only quickly monitor the head position of the guide wire, but also effectively improve the accuracy of monitoring.

[0094] Reference Figure 4 , Figure 4 FIG. is a schematic flowchart of the third embodiment of an automatic early warning method for instrument damage risk of a vascular intervention robot according to the present invention.

[0095] Based on the above first embodiment, for the automatic early warning method for instrument damage risk of the vascular intervention robot in this embodiment, based on the angiography image, the current position of the guide wire head is obtained, including:

[0096] Step S202: Input the angiography image into the Encoder of the trained second U-net to extract abstract features; input the abstract features into the Decoder of the trained second U-net to predict the displacement of each point in the angiography image to the guide wire head, obtaining a displacement map; calculate the position of each point in the angiography image after displacement based on the displacement map, and form a voting map; take the position with the highest score in the voting map as the current position of the guide wire head.

[0097] It should be noted that the second U-net is the U-net network.

[0098] Specifically, the displacement of each point in the angiography image to the guide wire head is predicted , obtaining a displacement map. Then, according to the displacement map, calculate the position of each point in the angiography image after displacement, and then record 1 point for this position. After each point in the angiography image is displaced, a voting map will be finally formed. Take the position coordinates with the highest score in the voting map as the current position coordinates of the guide wire head.

[0099] Among them, the steps to obtain the trained second U-net include:

[0100] Step S2020: Input the third angiography image into the Encoder of the second U-net to extract abstract features, where the distance from each real point to the guide wire head is marked in the third angiography image.

[0101] It should be noted that both the third angiography image and the fourth angiography image are marked with the distance from each real point to the guide wire head. The third angiography image and the fourth angiography image are both sample data used to train the second U-net, and the third angiography image and the fourth angiography image in each round of training are different.

[0102] Step S2021: Input the abstract features into the Decoder of the second U-net to predict the displacement of each point in the third angiography image to the tip of the guidewire, obtaining a predicted displacement map.

[0103] Specifically, the predicted displacement map corresponds to the displacement of each point in the angiography image to the tip of the guidewire. 。

[0104] Step S2022: Compare the displacement of each point in the predicted displacement map to the tip of the guidewire with the actual distance of each point to the tip of the guidewire, and calculate the second prediction error.

[0105] Specifically, obtain the distance of each point to the tip of the guidewire based on the displacement of each point in the predicted displacement map, and then compare it with the actual distance of each point to the tip of the guidewire to calculate the second prediction error.

[0106] Step S2023: Update the parameters in the second U-net through backpropagation based on the second prediction error.

[0107] Step S2024: After taking the fourth angiography image as the new third angiography image, repeat steps S2020 - S2024 until the second prediction error is less than a preset value, obtaining a trained second U-net.

[0108] Specifically, the preset value is determined in advance and can be set according to the desired network state to be trained. When the second prediction error is less than the preset value, it can be considered that the second U-net is already trained at this time, that is, a trained second U-net is obtained.

[0109] In this embodiment, a method based on a deep learning network, especially a method based on the U-net network, is used to monitor the position of the tip of the guidewire in the angiography image. It can not only quickly monitor the position of the tip of the guidewire but also effectively improve the accuracy of monitoring.

[0110] Reference Figure 5 , Figure 5 is a schematic flowchart of the fourth embodiment of an automatic early warning method for instrument damage risk of a vascular intervention robot according to the present invention.

[0111] Based on the above first embodiment, in this embodiment of the automatic early warning method for instrument damage risk of a vascular intervention robot, based on the angiography image, the current position of the tip of the guidewire is obtained, including:

[0112] Step S203: Input the angiography image into the trained CNN to extract abstract features; input the abstract features into the fully connected layer and predict the current position of the tip of the guidewire through regression.

[0113] Among them, the steps to obtain the trained CNN include:

[0114] Step S2030: The fifth angiography image is input into the CNN to extract abstract features, where the position of the real guide wire head is marked on the fifth angiography image.

[0115] It should be noted that both the fifth angiography image and the sixth angiography image are marked with the position of the real guide wire head. The fifth angiography image and the sixth angiography image are both sample data used to train the CNN, and the fifth angiography image and the sixth angiography image in each round of training are different.

[0116] Step S2031: Input the abstract features into the fully connected layer and obtain the position of the predicted guide wire head through regression prediction.

[0117] Step S2032: Compare the position of the measured guide wire head with the position of the real guide wire head and calculate the third prediction error.

[0118] Step S2033: Update the parameters in the CNN through backpropagation based on the third prediction error.

[0119] Step S2034: After taking the sixth angiography image as the new fifth angiography image, repeat steps S2030 - S2034 until the third prediction error is less than the preset value, and obtain the trained CNN.

[0120] Specifically, the preset value is determined in advance and can be set according to the desired network state to be trained. When the third prediction error is less than the preset value, it can be considered that the CNN is already trained at this time, that is, the trained CNN is obtained.

[0121] In this embodiment, a method based on a deep learning network, especially a method based on a CNN network, is used to monitor the position of the guide wire head in the angiography image, which can not only quickly monitor the position of the guide wire head, but also effectively improve the accuracy of monitoring.

[0122] In addition, an embodiment of the present invention also provides a storage medium, on which an automatic early warning program for the risk of damage to vascular intervention robot instruments is stored. When the automatic early warning program for the risk of damage to vascular intervention robot instruments is executed by a processor, it realizes the steps of the automatic early warning method for the risk of damage to vascular intervention robot instruments as described above.

[0123] Refer to Figure 6 , Figure 6 which is the structural block diagram of the first embodiment of the automatic early warning device for the risk of damage to vascular intervention robot instruments of the present invention.

[0124] AsFigure 6 As shown in Figure 6 , the automatic early warning device for the risk of damage to the vascular intervention robot instrument proposed in the embodiment of the present invention includes:

[0125] An acquisition module 601, configured to acquire angiography images in real time;

[0126] A determination module 602, configured to determine the edge position and branch position of the vascular tree based on the angiography image, and determine the current position of the guide wire head;

[0127] A monitoring module 603, configured to monitor the distance from the current position to the edge position and / or branch position;

[0128] An early warning module 604, further configured to automatically give an early warning when the distance is less than a preset distance.

[0129] In this embodiment, by acquiring angiography images in real time, it is possible to timely determine the latest position of the guide wire head during the interventional operation, and based on the angiography image, determine the edge position and branch position of the vascular tree. By comparing the latest position of the guide wire head with the edge position of the blood vessel and the branch position of the blood vessel respectively, the distance from the current position to the edge position and / or branch position can be monitored. When this distance is less than the preset distance, an automatic early warning can be given. In the above manner, it is possible to monitor the movement of the guide wire head in real time during the PCI operation and give an early warning in a timely manner according to the movement situation, thereby reducing the risk of accidental damage to blood vessels by doctors during the PCI operation.

[0130] In one embodiment, the determination module 602 is further configured to:

[0131] Acquire a vascular tree segmentation image based on the angiography image;

[0132] Extract the centerlines of each vascular tree based on the vascular tree segmentation image;

[0133] Determine the edge position of the vascular tree based on the vascular tree segmentation image and the centerlines of each vascular tree;

[0134] Determine the intersection points of each vascular tree based on the centerlines of each vascular tree, so as to obtain the branch positions of the vascular tree.

[0135] In one embodiment, the determination module 602 is further configured to:

[0136] Input the angiography image into the Encoder of the trained first U-net to extract abstract features;

[0137] Input the abstract features into the Decoder of the trained first U-net to predict the probability map of the part belonging to the guide wire in the angiography image;

[0138] Convert the probability map into a binary segmentation result based on a threshold;

[0139] Based on the binary segmentation result, determine the current position of the guide wire head through position information.

[0140] In one embodiment, the determining module 602 is further configured to:

[0141] Input the angiography image into the Encoder of the trained first U-net to extract abstract features;

[0142] Input the abstract features into the Decoder of the trained first U-net to predict the probability map of the part belonging to the guide wire in the angiography image;

[0143] Convert the probability map into a binary segmentation result based on a threshold;

[0144] Based on the binary segmentation result, determine the current position of the guide wire head through position information.

[0145] In one embodiment, the determining module 602 is further configured to:

[0146] Input the first angiography image into the Encoder of the first U-net to extract abstract features, where the real guide wire part is marked in the first angiography image;

[0147] Input the abstract features into the Decoder of the first U-net to predict the probability map of the part belonging to the guide wire in the first angiography image;

[0148] Calculate a first prediction error based on the probability map and the real guide wire part;

[0149] Update the parameters in the first U-net through backpropagation based on the first prediction error to obtain the trained first U-net.

[0150] In one embodiment, the determining module 602 is further configured to:

[0151] Input the angiography image into the Encoder of the trained U-net to extract abstract features;

[0152] Input the abstract features into the Decoder of the trained U-net to predict the distance from each point in the angiography image to the guide wire head, obtaining a displacement map;

[0153] Calculate the displaced position of each point in the angiography image based on the displacement map and form a voting map;

[0154] Take the position with the highest score in the voting map as the current position of the guide wire head.

[0155] In one embodiment, the determining module 602 is further configured to:

[0156] Input the third angiography image into the Encoder of the second U-net to extract abstract features, wherein the distance from each real point to the guide wire head is marked in the third angiography image;

[0157] Input the abstract features into the Decoder of the U-net to predict the distance from each point in the third angiography image to the guide wire head, and obtain a predicted displacement map;

[0158] Compare the distance from each point in the predicted displacement map to the guide wire head with the distance from each real point to the guide wire head, and calculate the second prediction error;

[0159] Update the parameters in the second U-net by backpropagation based on the second prediction error to obtain a trained second U-net.

[0160] In one embodiment, the determining module 602 is further configured to:

[0161] Input the angiography image into the trained CNN to extract abstract features;

[0162] Input the abstract features into the fully connected layer and predict the current position of the guide wire head through regression.

[0163] In one embodiment, the determining module 602 is further configured to:

[0164] Input the fifth angiography image into the CNN to extract abstract features, wherein the position of the real guide wire head is marked in the fifth angiography image;

[0165] Input the abstract features into the fully connected layer and obtain the predicted position of the guide wire head through regression prediction;

[0166] Compare the position of the measured guide wire head with the position of the real guide wire head, and calculate the third prediction error;

[0167] Update the parameters in the CNN by backpropagation based on the third prediction error to obtain a trained CNN.

[0168] It should be understood that the above is only an example and does not constitute any limitation to the technical solution of the present invention. In specific applications, those skilled in the art can set according to needs, and the present invention does not limit this.

[0169] It should be noted that the workflow described above is only illustrative and does not limit the protection scope of the present invention. In actual applications, those skilled in the art can select some or all of them according to actual needs to achieve the purpose of the solution of this embodiment, and no limitation is made here.

[0170] In addition, for the technical details not described in detail in this embodiment, reference can be made to the automatic early warning method for instrument damage risk of the vascular intervention robot provided in any embodiment of the present invention, which will not be elaborated here.

[0171] In addition, it should be noted that in this article, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or system. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, article or system including that element.

[0172] The serial numbers of the above embodiments of the present invention are only for description and do not represent the superiority or inferiority of the embodiments.

[0173] Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as a read-only memory (ROM) / RAM, magnetic disk, optical disk), and includes several instructions for causing a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0174] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. An automatic early warning method for the risk of damage to vascular interventional robot instruments, characterized in that, The automatic warning method includes: Obtaining angiography images in real time; Based on the angiography images, determining the edge positions and branch positions of the vascular tree, and determining the current position of the guide wire head; Monitoring the distance from the current position to the edge position and / or branch position; If the distance is less than a preset distance, an automatic warning is given; The determining the edge positions and / or branch positions of the vascular tree based on the angiography images includes: Obtaining a vascular tree segmentation image based on the angiography images; Extracting the centerlines of each vascular tree based on the vascular tree segmentation image; Determining the edge positions of the vascular tree based on the vascular tree segmentation image and the centerlines of each vascular tree; Determining the intersection points of each vascular tree based on the centerlines of each vascular tree, thereby obtaining the branch positions of the vascular tree; The determining the current position of the guide wire head based on the angiography images includes: Inputting the angiography images into the Encoder of the trained second U-net to extract abstract features; Inputting the abstract features into the Decoder of the trained second U-net to predict the displacement of each point in the angiography images to the guide wire head, obtaining a displacement map; Calculating the displaced positions of each point in the angiography images based on the displacement map and forming a voting map; Taking the position with the highest score in the voting map as the current position of the guide wire head; Before inputting the angiography images into the Encoder of the trained U-net to extract abstract features, it further includes: S2020. Inputting the third angiography image into the Encoder of the second U-net to extract second abstract features, where the true distance from each point to the guide wire head is marked in the third angiography image; S2021. Inputting the second abstract features into the Decoder of the second U-net to predict the displacement of each point in the third angiography image to the guide wire head, obtaining a predicted displacement map; S2022. Comparing the displacement of each point to the guide wire head in the predicted displacement map with the true distance from each point to the guide wire head, and calculating a second prediction error; S2023. Updating the parameters in the second U-net based on the second prediction error through backpropagation; S2024. After taking the fourth angiography image as the new third angiography image, repeating steps S2020 - S2024 until the second prediction error is less than a preset value, obtaining the trained second U-net.

2. The method according to claim 1, characterized in that The determining the current position of the guide wire head based on the angiography images includes: Inputting the angiography images into the Encoder of the trained first U-net to extract abstract features; Inputting the abstract features into the Decoder of the trained first U-net to predict the probability map of the part belonging to the guide wire in the angiography images; Converting the probability map into a binary segmentation result based on a threshold; Based on the binary segmentation result, determining the current position of the guide wire head through position information.

3. The method according to claim 2, wherein Before inputting the angiography image into the Encoder of the trained first U-net to extract abstract features, the following steps are further included: S2010: Input the first angiography image into the Encoder of the first U-net to extract the first abstract features, where the real guide wire part is marked in the first angiography image; S2011: Input the first abstract features into the Decoder of the first U-net to predict the probability map of the part belonging to the guide wire in the first angiography image; S2012: Calculate the first prediction error based on the probability map and the real guide wire part; S2013: Update the parameters in the first U-net through backpropagation based on the first prediction error; S2014: After taking the second angiography image as the new first angiography image, repeat steps S2010 - S2014 until the first prediction error is less than the preset value to obtain the trained first U-net.

4. The method according to claim 1, characterized in that Based on the angiography image, determining the current position of the guide wire head includes: Input the angiography image into the trained CNN to extract abstract features; Input the abstract features into the fully connected layer and predict the current position of the guide wire head through regression.

5. The method according to claim 4, wherein Before inputting the angiography image into the trained CNN to extract abstract features, the following steps are further included: S2030: Input the fifth angiography image into the CNN to extract abstract features, where the position of the real guide wire head is marked in the fifth angiography image; S2031: Input the abstract features into the fully connected layer and obtain the predicted position of the guide wire head through regression prediction; S2032: Compare the position of the measured guide wire head with the position of the real guide wire head and calculate the third prediction error; S2033: Update the parameters in the CNN through backpropagation based on the third prediction error; S2034: After taking the sixth angiography image as the new fifth angiography image, repeat steps S2030 - S2034 until the third prediction error is less than the preset value to obtain the trained CNN.

6. An automatic early warning device for the risk of damage to vascular interventional robot instruments, characterized in that, The automatic early warning device for the instrument damage risk of the vascular intervention robot includes: a memory, a processor, and an automatic early warning program for the instrument damage risk of the vascular intervention robot stored on the memory and operable on the processor. The automatic early warning program for the instrument damage risk of the vascular intervention robot is configured to implement the steps of the method for automatically early warning the instrument damage risk of the vascular intervention robot as described in any one of claims 1 to 5.

7. A storage medium, characterized in that, The storage medium stores an automatic early warning program for the instrument damage risk of the vascular intervention robot. When the automatic early warning program for the instrument damage risk of the vascular intervention robot is executed by the processor, it implements the steps of the method for automatically early warning the instrument damage risk of the vascular intervention robot as described in any one of claims 1 to 5.

Images