A method, system, and storage medium for guiding a coronary intervention procedure

Abstract

Embodiments of the present specification provide a method, system and storage medium for guiding a coronary intervention procedure. The method comprises: acquiring a left coronary angiography image sequence and a right coronary angiography image sequence of a target object, the left coronary angiography image sequence and the right coronary angiography image sequence being captured by a digital subtraction angiography device in a first stage of an intervention procedure of the target object and containing an intervention device; determining a plurality of pairs of angiography images corresponding to a plurality of heartbeat phases from the left coronary angiography image sequence and the right coronary angiography image sequence; generating, for each heartbeat phase, a heart coronary segmentation image based on its corresponding pair of angiography images; acquiring a real-time fluoroscopy image of the target object, the real-time fluoroscopy image being captured by the digital subtraction angiography device in a second stage of the intervention procedure and containing the intervention device; and generating a coronary roadmap for guiding the intervention procedure based on the real-time fluoroscopy image and the heart coronary segmentation images corresponding to the plurality of heartbeat phases.

Description

Technical Field

[0001] This specification relates to the field of interventional surgery technology, and in particular to a method, system, and storage medium for guiding coronary interventional procedures. Background Technology

[0002] In coronary interventional procedures, especially for patients with chronic total occlusion (CTO), surgeons typically need to guide a guidewire through the occluded vessel. Due to the complexity and invisibility of the occluded segment, surgeons rely primarily on experience and touch in a "blind" operation. This approach may result in the guidewire failing to pass smoothly through the occluded segment, thus prolonging the procedure and even leading to surgical failure.

[0003] Therefore, a protocol is needed to guide coronary interventional procedures, which can improve the safety and success rate of these procedures. Summary of the Invention

[0004] This specification provides one or more embodiments of a method for guiding coronary interventional surgery. The method includes: acquiring left and right coronary angiography image sequences of a target subject, the left and right coronary angiography image sequences being acquired by a digital subtraction angiography device during a first stage of the interventional surgery and including the interventional device; determining multiple angiography image pairs corresponding to multiple cardiac phases from the left and right coronary angiography image sequences, each angiography image pair including one frame of left coronary angiography image and one frame of right coronary angiography image corresponding to one cardiac phase; generating a segmented coronary artery image based on the corresponding angiography image pair for each cardiac phase; acquiring a real-time fluoroscopic image of the target subject, the real-time fluoroscopic image being acquired by a digital subtraction angiography device during a second stage of the interventional surgery and including the interventional device; and generating a coronary artery path map for guiding the interventional surgery based on the real-time fluoroscopic image and the multiple segmented coronary artery images corresponding to multiple cardiac phases.

[0005] This specification provides one or more embodiments of a system for guiding coronary interventional procedures. The system includes: a first acquisition module for acquiring left and right coronary angiography image sequences of a target subject, the left and right coronary angiography image sequences being acquired by a digital subtraction angiography device during the first stage of the interventional procedure and including the interventional device; a determination module for determining multiple angiography image pairs corresponding to multiple cardiac phases from the left and right coronary angiography image sequences, each angiography image pair including one frame of left coronary angiography image and one frame of right coronary angiography image corresponding to one cardiac phase; a segmentation module for generating a segmented coronary artery image for each cardiac phase based on its corresponding angiography image pair; a second acquisition module for acquiring real-time fluoroscopic images of the target subject, the real-time fluoroscopic images being acquired by a digital subtraction angiography device during the second stage of the interventional procedure and including the interventional device; and a generation module for generating a coronary artery path map for guiding the interventional procedure based on the real-time fluoroscopic images and the segmented coronary artery images corresponding to multiple cardiac phases.

[0006] One or more embodiments of this specification provide a computer-readable storage medium that stores computer instructions that, when read by a computer, enable the computer to perform a method for guiding a coronary intervention procedure. Attached Figure Description

[0007] This specification will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting; in these embodiments, the same reference numerals denote the same structures, wherein:

[0008] Figure 1 This is a schematic diagram illustrating the application scenario of a system for guiding coronary interventional procedures, based on some embodiments of this specification.

[0009] Figure 2 This is a schematic diagram of a system for guiding coronary interventional procedures, according to some embodiments of this specification;

[0010] Figure 3 This is a flowchart illustrating a method for guiding coronary interventional procedures according to some embodiments of this specification;

[0011] Figure 4A This is a schematic diagram illustrating the determination of multiple angiographic image pairs corresponding to multiple cardiac phases according to some embodiments of this specification;

[0012] Figure 4B This is another schematic diagram illustrating the determination of multiple angiographic image pairs corresponding to multiple heartbeat phases according to some embodiments of this specification;

[0013] Figure 5 This is a flowchart illustrating a method for generating segmented images of the coronary arteries of the heart, according to some embodiments of this specification;

[0014] Figure 6 This is a flowchart illustrating a method for generating a coronary artery path diagram for guiding interventional procedures, according to some embodiments of this specification;

[0015] Figure 7 This is a schematic diagram of a coronary artery pathway according to some embodiments of this specification;

[0016] Figure 8 This is a flowchart illustrating a method for determining a target imaging image that matches a real-time fluoroscopic image, according to some embodiments of this specification;

[0017] Figure 9 This is a flowchart illustrating a method for determining the first position of the catheter tip according to some embodiments of this specification;

[0018] Figure 10 This is a schematic diagram of the first position of the catheter tip according to some embodiments of this specification. Detailed Implementation

[0019] To more clearly illustrate the technical solutions of the embodiments in this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some examples or embodiments of this specification. For those skilled in the art, these drawings can be applied to other similar scenarios without creative effort. Unless obvious from the context or otherwise specified, the same reference numerals in the drawings represent the same structures or operations.

[0020] It should be understood that the terms “system,” “device,” “unit,” and / or “module” used herein are one way to distinguish different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0021] Unless the context clearly indicates an exception, words such as "a," "an," "a kind," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0022] Flowcharts are used in this specification to illustrate the operations performed by the system according to embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0023] Figure 1 These are schematic diagrams illustrating application scenarios for guiding coronary artery interventional procedures, based on some embodiments of this specification. For example... Figure 1 As shown, the application scenario 100 of the system for guiding coronary artery interventional surgery may include a processing device 110, a network 120, a terminal device 130, a storage device 140, a scanning device 150, and an interventional device 160.

[0024] The processing device 110 can process data and / or information obtained from the terminal device 130, the storage device 140, and the scanning device 150. For example, the processing device 110 can acquire left coronary angiography image sequences, right coronary angiography image sequences, and real-time fluoroscopic images acquired by the scanning device 150. As another example, the processing device 110 can determine multiple angiography image pairs corresponding to multiple cardiac phases from the left and right coronary angiography image sequences, and generate coronary artery segmentation images based on the angiography image pairs. Yet another example, the processing device 110 can generate a coronary artery path map for guiding interventional procedures based on real-time fluoroscopic images and coronary artery segmentation images corresponding to multiple cardiac phases.

[0025] In some embodiments, processing device 110 may be integrated into scanning device 150. In some embodiments, processing device 110 may be a single server or a group of servers. In some embodiments, processing device 110 may be local or remote. Processing device 110 may be directly connected to terminal device 130, storage device 140, and scanning device 150 to access stored or acquired information and / or data. In some embodiments, processing device 110 may be implemented on a cloud platform. By way of example only, cloud platforms may include private clouds, public clouds, hybrid clouds, community clouds, distributed clouds, internal clouds, multi-tiered clouds, etc., or any combination thereof. In some embodiments, processing device 110 may be a distributed group of servers, which may include multiple server nodes.

[0026] Network 120 may include any suitable network that facilitates the exchange of information and / or data within application scenario 100. In some embodiments, one or more components of application scenario 100 (e.g., terminal device 130, processing device 110, storage device 140, or scanning device 150) may transmit information and / or data with one or more other components of application scenario 100 via network 120. For example, processing device 110 may acquire left coronary angiography image sequences, right coronary angiography image sequences, and real-time fluoroscopic images from scanning device 150 and / or storage device 140 via network 120.

[0027] In some embodiments, network 120 can be any one or more of wired or wireless networks. In some embodiments, the network can be various topologies such as point-to-point, shared, centralized, or a combination of multiple topologies.

[0028] Terminal device 130 may include mobile device 130-1, tablet computer 130-2, laptop computer 130-3, etc., or any combination thereof. In some embodiments, terminal device 130 may interact with other components in application scenario 100 via network 120. In some embodiments, terminal device 130 may receive information and / or instructions input by a user and send the received information and / or instructions to processing device 110 via network 120. For example, terminal device 130 may receive instructions from a user (such as a medical professional), retrieve and display a coronary artery path diagram from scanning device 150 and / or storage device 140 via network 120 to the user.

[0029] Storage device 140 can store data and / or instructions. In some embodiments, storage device 140 can store data acquired from processing device 110, terminal device 130, and / or scanning device 150. For example, storage device 140 can store data acquired by scanning device 150 (such as left coronary angiography image sequences, right coronary angiography image sequences, real-time fluoroscopic images, etc.). As another example, storage device 140 can store data acquired by processing device 110 (such as cardiac coronary artery segmentation images). In some embodiments, storage device 140 can store data and / or instructions used by processing device 110 to perform the exemplary methods described in this specification. For example, storage device 140 can store instructions from processing device 110 to perform the methods shown in the flowcharts. In some embodiments, storage device 140 can include a mass storage device, a removable storage device, volatile read-write memory, read-only memory (ROM), etc., or any combination thereof. In some embodiments, storage device 140 can be implemented on a cloud platform. In some embodiments, storage device 140 can be part of processing device 110.

[0030] The scanning device 150 is a device that can be used to acquire medical images (e.g., left coronary angiography image sequence, right coronary angiography image sequence, real-time fluoroscopic images). In some embodiments, the scanning device 150 may be a digital subtraction angiography (DSA) device.

[0031] Interventional device 160 is a surgical device used for coronary artery interventional procedures. In some embodiments, interventional device 160 may include, but is not limited to, guidewires, catheters, balloons, stents, etc. A guidewire is a tool used to guide other interventional devices to the target lesion location; it is a long, flexible metal wire with a certain degree of support. A catheter is a tool used to guide other substances or devices into the target blood vessel; it is a hollow tubular instrument. The catheter may include an angiography catheter for delivering contrast agents into the blood vessel.

[0032] The above description is for illustrative purposes only, and actual application scenarios may vary. It should be noted that application scenario 100 is provided for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can make various modifications or variations based on the description in this specification. However, these modifications and variations will not depart from the scope of this specification.

[0033] Figure 2 This is a schematic diagram of modules of a system 200 for guiding coronary artery interventional procedures, according to some embodiments of this specification. Figure 2 As shown, the system 200 for guiding coronary artery interventional surgery may include a first acquisition module 210, a determination module 220, a segmentation module 230, a second acquisition module 240, and / or a generation module 250.

[0034] The first acquisition module 210 can be used to acquire the left and right coronary angiography image sequences of the target object. The left and right coronary angiography image sequences are acquired by a digital subtraction angiography device during the first stage of the interventional procedure on the target object and include the interventional device itself. A detailed description of the first acquisition module 210 can be found in step 310.

[0035] The determining module 220 can be used to determine multiple angiographic image pairs corresponding to multiple cardiac phases from a left coronary angiographic image sequence and a right coronary angiographic image sequence. In some embodiments, each angiographic image pair includes one frame of left coronary angiographic image and one frame of right coronary angiographic image corresponding to one cardiac phase. In some embodiments, the determining module 220 can perform one or more of the following operations: extracting phase feature information of each frame of angiographic image in the left and right coronary angiographic image sequences; matching the left and right coronary angiographic image sequences based on the phase feature information to determine multiple angiographic image pairs. In some embodiments, the determining module 220 may perform one or more of the following operations: for each frame of angiography in the left coronary angiography image sequence and the right coronary angiography image sequence, process the frame of angiography using a cardiac phase determination model to determine the cardiac phase corresponding to the frame of angiography and its confidence level, wherein the cardiac phase determination model is a machine learning model; for each cardiac phase, determine at least one frame of left coronary angiography and at least one frame of right coronary angiography corresponding to the cardiac phase based on the cardiac phase corresponding to each frame of angiography; select the left coronary angiography image with the highest confidence level from the at least one frame of left coronary angiography; select the right coronary angiography image with the highest confidence level from the at least one frame of right coronary angiography; and use the selected left coronary angiography and right coronary angiography images as a pair of angiography images corresponding to the cardiac phase. A detailed description of the determining module 220 can be found in step 320.

[0036] The segmentation module 230 can be used to generate a segmented image of the coronary arteries for each cardiac phase, based on its corresponding angiographic image pair. In some embodiments, the segmentation module 230 can perform one or more of the following operations: acquiring the left coronary artery segmentation image and the right coronary artery segmentation image corresponding to the left coronary artery angiographic image and the right coronary artery angiographic image in the angiographic image pair; extracting the centerline of the left coronary artery segmentation image and the right coronary artery segmentation image respectively to obtain the centerline of the left coronary artery and the centerline of the right coronary artery; registering the left coronary artery angiographic image and the right coronary artery angiographic image based on the centerline of the left coronary artery and the centerline of the right coronary artery to determine the deformation field; and fusing the left coronary artery segmentation image and the right coronary artery segmentation image according to the deformation field to generate a segmented image of the coronary arteries. For a detailed description of the segmentation module 230, please refer to step 330.

[0037] The second acquisition module 240 can be used to acquire real-time fluoroscopic images of the target object. In some embodiments, the real-time fluoroscopic images are acquired by a digital subtraction angiography device during the second stage of the interventional procedure and include the interventional device. A detailed description of the second acquisition module 240 can be found in step 340.

[0038] The generation module 250 can be used to generate a coronary artery path map for guiding interventional surgery based on a real-time fluoroscopic image and coronary artery segmentation images corresponding to multiple cardiac phases. In some embodiments, the generation module 250 can perform one or more of the following operations: determining a target angiography image matching the real-time fluoroscopic image from a left coronary angiography image sequence and a right coronary angiography image sequence, wherein the real-time fluoroscopic image corresponds to the left or right coronary artery; segmenting the catheter tip in the target angiography image to determine a first position of the catheter tip; segmenting the catheter tip in the real-time fluoroscopic image to determine a second position of the catheter tip; converting the coronary artery segmentation image corresponding to the target angiography image based on the first and second positions of the catheter tip to obtain a real-time coronary artery segmentation image; and overlaying the real-time coronary artery segmentation image onto the real-time fluoroscopic image to obtain a coronary artery path map. In some embodiments, the generation module 250 may perform one or more of the following operations: acquiring a reference contrast image in the image sequence to which the target contrast image belongs, whose acquisition time is earlier than that of the target contrast image; performing catheter tip segmentation on the reference contrast image and the target contrast image respectively to obtain corresponding reference catheter tip segmentation images and target catheter tip segmentation images; determining the position coordinates of the catheter tip based on the target catheter tip segmentation image; determining whether the position coordinates are correct based on the reference catheter tip segmentation image; and, in response to the correct position coordinates of the catheter tip, using the position coordinates as the first position of the catheter tip. In some embodiments, the real-time fluoroscopic image includes multiple frames of real-time fluoroscopic images acquired sequentially. In some embodiments, the generation module 250 may perform one or more of the following operations: for multiple frames of first real-time fluoroscopic images in multiple real-time fluoroscopic images, determine multiple frames of first target angiography images matching the first real-time fluoroscopic images from the left coronary angiography image sequence and the right coronary angiography image sequence, wherein the acquisition time of the multiple frames of first real-time fluoroscopic images meets a preset condition; determine the heartbeat pattern of the target object based on the heartbeat phase corresponding to the multiple frames of first target angiography images; for a second real-time fluoroscopic image in multiple real-time fluoroscopic images, determine a second target angiography image matching the second real-time fluoroscopic image based on the heartbeat pattern and the multiple frames of first target angiography images, wherein the acquisition time of the second real-time fluoroscopic image is later than that of the multiple frames of first real-time fluoroscopic images. In some embodiments, the generation module 250 may perform one or more of the following operations: extract phase feature information from the multiple frames of first real-time fluoroscopic images and the second real-time fluoroscopic images; determine whether the heartbeat pattern of the target object has changed when the second real-time fluoroscopic image is acquired based on the phase feature information; in response to determining that the heartbeat pattern has not changed, determine a second target angiography image matching the second real-time fluoroscopic image based on the heartbeat pattern and the multiple frames of first target angiography images. For a detailed description of the generation module 250, please refer to step 350.

[0039] In some embodiments, the first acquisition module 210, the determination module 220, the segmentation module 230, the second acquisition module 240, and / or the generation module 250 may be different modules in a system, or a single module may implement the functions of two or more of the above-mentioned modules. For example, the modules may share a storage module, or each module may have its own storage module. Such variations are all within the scope of protection of this specification.

[0040] Figure 3 This is a flowchart illustrating a method for guiding coronary artery interventional procedures according to some embodiments of this specification. In some embodiments, procedure 300 may be implemented by a system (such as processing device 110) for guiding coronary artery interventional procedures. Figure 3 As shown, process 300 includes the following steps 310-350. Steps 310-330 constitute the first stage (preparation stage) of the interventional procedure, used to acquire coronary artery segmentation images based on coronary angiography images. Steps 340-350 constitute the second stage (operative procedure stage) of the interventional procedure, used to acquire a coronary artery path map for real-time guidance of the interventional procedure based on real-time fluoroscopic images. It should be noted that the first and second stages may be performed without time intervals or with a certain time interval.

[0041] Step 310: Obtain the left coronary angiography image sequence and the right coronary angiography image sequence of the target object. In some embodiments, step 310 is performed by the first acquisition module 210.

[0042] The target population refers to patients who require coronary interventional procedures. For example, patients with chronic total occlusion (CTO) of the coronary arteries. The coronary arteries include the left and right coronary arteries.

[0043] The left coronary angiography image sequence and the right coronary angiography image sequence each consist of multiple frames of left coronary angiography images and multiple frames of right coronary angiography images arranged in chronological order, respectively. The left coronary angiography image (or right coronary angiography image) is an image of the left coronary artery (or right coronary artery) acquired after the injection of contrast agent. The left coronary angiography image sequence (or right coronary angiography image sequence) may contain multiple (e.g., 3) cardiac cycles corresponding to multiple frames of left coronary angiography images (or right coronary angiography images).

[0044] In some embodiments, the left and right coronary angiography image sequences are acquired by a digital subtraction angiography (DSA) device during the first stage of the interventional procedure on the target patient and include the interventional device. A detailed description of the interventional device can be found in [link to relevant documentation]. Figure 1And related descriptions. In some embodiments, the interventional device is a catheter.

[0045] Specifically, after the puncture needle successfully punctures the target, a guidewire is inserted through the puncture needle into the arterial sheath, and then a catheter is inserted through the arterial sheath. Further, the catheter is gradually advanced to the opening of the left (or right) coronary artery at the root of the aorta, and contrast agent is injected into the left (or right) coronary artery. Using a digital subtraction angiography (DSA) device, multiple frames of left coronary angiography images (or multiple frames of right coronary angiography images) containing the interventional device are continuously acquired at preset time intervals.

[0046] The first phase refers to the period after the interventional procedure begins, during which angiographic images of the left and right coronary arteries are acquired. For example, the interventional procedure can be considered to have begun once the patient enters the operating room. Alternatively, it can be considered to have begun after the patient is anesthetized. The first phase will last for several heartbeat cycles. For example, if the left and right coronary angiographic image sequences each contain angiographic images spanning three heartbeat cycles, the first phase will last at least six heartbeat cycles.

[0047] Step 320 involves determining multiple angiographic image pairs corresponding to multiple cardiac phases from the left and right coronary angiographic image sequences. In some embodiments, step 320 is performed by the determining module 220.

[0048] The phases of a heartbeat are the different stages that occur in sequence during a complete heartbeat. For example, the phases of a heartbeat cycle can include atrial systole, ventricular systole, atrial diastole, and ventricular diastole.

[0049] Each angiography image pair contains one frame of left coronary angiography and one frame of right coronary angiography corresponding to a cardiac phase. For example, angiography image pair A1 contains left coronary angiography image A corresponding to the atrial systole phase. L Image A of right coronary angiography R .

[0050] Figure 4A This is a schematic diagram illustrating the determination of multiple angiographic image pairs corresponding to multiple cardiac phases, according to some embodiments of this specification. For example... Figure 4AAs shown, the determining module 220 can extract phase feature information from each frame of the left and right coronary angiography image sequences. Phase feature information reflects the phase of the heart, and its value changes with heartbeat. For example, phase feature information may include the heart outline, coronary artery ostium size, etc. In some embodiments, the determining module 220 can extract the phase feature information of each frame of the angiography image using feature extraction algorithms (e.g., histogram of oriented gradients, Fourier transform, etc.) and image processing models (e.g., convolutional neural network models, segmentation models, etc.).

[0051] Furthermore, the determination module 220 can match the left coronary angiography image sequence and the right coronary angiography image sequence based on the phase feature information to determine multiple angiography image pairs.

[0052] First, the determining module 220 can determine the corresponding cardiac phase for each frame of left coronary angiography and each frame of right coronary angiography based on phase feature information. For example, it can obtain reference values ​​for the phase features corresponding to each cardiac phase, and determine the corresponding cardiac phase based on the reference values ​​and the phase feature information of each frame of left coronary angiography (or right coronary angiography). Alternatively, it can determine the corresponding phase for each frame of left coronary angiography (or right coronary angiography) based on the changing patterns of the phase feature information corresponding to multiple frames of left coronary angiography (or multiple frames of right coronary angiography) in a left coronary angiography image sequence (or right coronary angiography image sequence). Then, the determining module 220 can match the left and right coronary angiography images corresponding to the same cardiac phase into a single angiography image pair.

[0053] For the same cardiac phase, there may be multiple corresponding left coronary angiography images and multiple corresponding right coronary angiography images. For example, the atrial systole phase may correspond to three left coronary angiography images and three right coronary angiography images across three cardiac cycles. As an example, the determining module 220 can fuse the multiple left coronary angiography images and multiple right coronary angiography images corresponding to the same cardiac phase separately (e.g., homogenize the grayscale values ​​of pixels) to obtain fused left coronary angiography images and fused right coronary angiography images, which serve as the angiography image pairs corresponding to that cardiac phase. As another example, the determining module 220 can input the phase feature information corresponding to the multiple left coronary angiography images and multiple right coronary angiography images corresponding to the same cardiac phase into the matching model. The matching model can calculate the phase feature matching degree between each left coronary angiography image and each other right coronary angiography image, and determine the left coronary angiography image and right coronary angiography image corresponding to the maximum phase feature matching degree as an angiography image pair.

[0054] In some embodiments of this specification, multiple angiography image pairs are determined by matching the left coronary angiography image sequence and the right coronary angiography image sequence based on phase feature information. Regardless of changes in the patient's heart rate, rhythm, etc., the corresponding angiography image pair can be accurately found through phase feature information, thereby improving the universality and applicability of this method in different patient groups.

[0055] Figure 4B This is another schematic diagram illustrating the determination of multiple angiographic image pairs corresponding to multiple cardiac phases, as shown in some embodiments of this specification. For example... Figure 4B As shown, the determining module 220 can process each frame of angiography in the left and right coronary angiography image sequences using a cardiac phase determination model to determine the corresponding cardiac phase and its confidence level. The confidence level can be a decimal between 0 and 1; a higher confidence level indicates a higher probability that the angiography frame belongs to the corresponding cardiac phase.

[0056] Specifically, the cardiac phase determination model can extract features (such as depth features) from the angiographic image, then map these features to multiple confidence levels corresponding to multiple cardiac phases, and determine the cardiac phase corresponding to the frame of the angiographic image with the highest confidence level. For example, taking a left coronary angiographic image A... L Input the cardiac phase determination model, and the cardiac phase determination module can output a left coronary angiography image A. L If the confidence levels for atrial systole, ventricular systole, atrial diastole, and ventricular diastole are 0.5, 0.1, 0.2, and 0.1, respectively, then the left coronary angiography image A can be identified. L The corresponding cardiac phase and its confidence level are atrial systole and 0.5.

[0057] The heartbeat phase determination model is a machine learning model. In some embodiments, the heartbeat phase determination model may be, but is not limited to, a combination of one or more of the following: support vector machine model, convolutional neural network (CNN) model, and recurrent neural network (RNN) model. In some embodiments, the heartbeat phase determination model may be trained based on a sample training set. The training sample set may include multiple training samples and multiple training labels corresponding to the multiple training samples. The training samples may be historical coronary angiography images, and the training labels may be the heartbeat phases corresponding to the historical coronary angiography images. The training labels may be manually labeled.

[0058] Furthermore, for each cardiac phase, based on the cardiac phase corresponding to each angiographic image, at least one left coronary angiographic image and at least one right coronary angiographic image are determined corresponding to the cardiac phase. The left coronary angiographic image with the highest confidence is selected from the at least one left coronary angiographic image, and the right coronary angiographic image with the highest confidence is selected from the at least one right coronary angiographic image. These selected left and right coronary angiographic images are then used as the angiographic image pair corresponding to the cardiac phase. For example, as... Figure 4B As shown, the cardiac phase "atrial systole" corresponds to 3 frames of left coronary angiography images (confidence levels of 0.8, 0.9, and 0.85, respectively) and 3 frames of right coronary angiography images (confidence levels of 0.9, 0.95, and 0.89, respectively). Therefore, the left coronary angiography image with a confidence level of 0.9 and the right coronary angiography image with a confidence level of 0.95 can be selected as the angiography image pair corresponding to "atrial systole".

[0059] In some embodiments of this specification, a machine learning model (heartbeat phase determination model) is used to process each frame of angiography image. This model can learn a large number of angiography image data features and capture more subtle and complex image features, thereby accurately identifying the heartbeat phase corresponding to each frame of image and improving the accuracy of heartbeat phase determination. At the same time as determining the heartbeat phase, a confidence level is given, which reflects the reliability of the model for the determined heartbeat phase. This allows images that are more consistent with the actual heartbeat phase to be selected from many possible images, further enhancing the reliability of the results.

[0060] The determination module 220 can acquire the heartbeat signal of the target subject during interventional surgery using a heartbeat signal acquisition device (such as an ECG device). In some embodiments, the determination module 220 can divide the heartbeat signal of the target subject into multiple heartbeat phases. The heartbeat signal is data reflecting the heartbeat state and related physiological information, such as electrocardiogram signals and pulse signals. The heartbeat signal is acquired simultaneously with multiple frames of left coronary angiography images and multiple frames of right coronary angiography images.

[0061] Specifically, the determination module 220 marks the corresponding cardiac phase for each frame of left coronary angiography and each frame of right coronary angiography based on the characteristics of the heartbeat signal. For example, the cardiac phase corresponding to the acquisition time is divided according to the peak value and waveform of the heartbeat signal, and then the corresponding cardiac phase is marked based on the acquisition time corresponding to each frame of left and right coronary angiography. Then, left and right coronary angiography images in the same cardiac phase are matched into an angiography image pair.

[0062] Step 330: For each cardiac phase, generate a segmented image of the coronary arteries based on its corresponding angiographic image pair. In some embodiments, step 330 is performed by the segmentation module 230.

[0063] A segmented image of the coronary arteries is an image that segments the coronary arteries of the heart, including the left and right coronary arteries. For a detailed description of generating segmented images of the coronary arteries based on their corresponding angiographic image pairs, please refer to [link to relevant documentation]. Figure 5 And its related descriptions.

[0064] Step 340: Acquire a real-time perspective image of the target object. In some embodiments, step 340 is performed by the second acquisition module 240.

[0065] The real-time fluoroscopic image of the target object is a unilateral coronary artery image of the heart without contrast agent injection. For example, a real-time fluoroscopic image of the left coronary artery or a real-time fluoroscopic image of the right coronary artery. When coronary artery surgery is required on the left coronary artery, the real-time fluoroscopic image is the left coronary artery. When coronary artery surgery is required on the right coronary artery, the real-time fluoroscopic image is the right coronary artery.

[0066] In some embodiments, real-time fluoroscopic images are acquired by a digital subtraction angiography device during the second stage of an interventional procedure on the target subject and include the interventional device.

[0067] Specifically, after the puncture needle successfully punctures the target object, a guidewire is inserted through the puncture needle into the arterial sheath, and then a catheter is inserted through the arterial sheath. Further, real-time fluoroscopic images containing the interventional device are acquired in a coronary artery direction (left or right) using a digital subtraction angiography (DSA) system.

[0068] During the acquisition of real-time fluoroscopic images, no contrast agent was injected into the target object. Therefore, based on the real-time fluoroscopic images, only the interventional device can be observed, and the coronary arteries of the heart and the relative positions between the interventional device and the coronary arteries cannot be observed.

[0069] Step 350: Based on real-time fluoroscopic images and segmented coronary artery images corresponding to multiple cardiac phases, a coronary artery pathway map for guiding interventional procedures is generated. In some embodiments, step 350 is performed by the generation module 250.

[0070] Coronary artery path mapping is an image that displays the real-time relative position between the interventional device and the target coronary artery. It is used to guide doctors to operate the interventional device to the target location in the coronary artery in order to clear the blocked segment of the target vessel. Figure 7 This is a schematic diagram of a coronary artery pathway according to some embodiments of this specification. For example... Figure 7 As shown, the coronary artery roadmap includes the coronary vessels and interventional devices of the target object.

[0071] For a detailed description of generating coronary artery pathway maps for guiding interventional procedures, please refer to [link to relevant documentation]. Figure 6 And its related descriptions.

[0072] In some embodiments of this specification, during the first stage (preparation stage) of the interventional procedure, a segmented image of the coronary arteries, including the left and right coronary arteries, is obtained based on the left and right coronary angiography image sequences containing the interventional device. In the second stage (operative stage), the complete segmented coronary artery image and a real-time fluoroscopic image of one coronary artery are fused to obtain a complete coronary artery path map. This allows the physician to observe the real-time relative position between the interventional device and the coronary arteries based on the path map. Furthermore, obtaining a coronary artery path map based on a real-time fluoroscopic image of one coronary artery can simultaneously reflect information from both coronary arteries. On the one hand, it eliminates the need to acquire a real-time fluoroscopic image of the other coronary artery, reducing image acquisition time, enhancing image real-time performance, and improving surgical efficiency. On the other hand, it allows the physician to fully utilize information from collateral circulation to assess the real-time effect of the interventional device on clearing occluded vessels, avoiding damage to coronary vessels, and improving the safety and success rate of the procedure.

[0073] Furthermore, the coronary artery segmentation images are generated based on angiographic images acquired during the interventional procedure. Compared to obtaining coronary artery segmentation images via computed tomography angiography (CTA) a few days before the procedure, the coronary artery segmentation images in this manual better reflect the real-time vascular structure during the patient's interventional procedure, reducing errors caused by time differences and improving the accuracy and success rate of the procedure. On the other hand, although CTA can provide three-dimensional vascular images, it relies on traditional three-dimensional reconstruction algorithms, which may not be accurate enough when dealing with complex vascular structures and may introduce errors during the process. Generating coronary angiographic images using DSA technology has higher accuracy.

[0074] Figure 5 This is a flowchart illustrating a method for generating segmented images of the coronary arteries of the heart according to some embodiments of this specification. In some embodiments, process 500 may be used to implement step 330. Figure 5 As shown, process 500 includes the following steps.

[0075] Step 510: Obtain the left coronary artery segmentation image and the right coronary artery segmentation image corresponding to the left coronary artery angiography image and the right coronary artery angiography image in the angiography image pair.

[0076] The left coronary artery segmentation image (or right coronary artery segmentation image) is an image mask that segments the left coronary artery (or right coronary artery) of the heart. In some embodiments, the segmentation module 230 can segment the left coronary artery (or right coronary artery) from the left coronary artery segmentation image (or right coronary artery segmentation image) based on a segmentation model to obtain the left coronary artery segmentation image (or right coronary artery segmentation image).

[0077] In some embodiments, the segmentation model can be a traditional segmentation algorithm model. For example, traditional segmentation algorithms may include, but are not limited to, one or more combinations of thresholding, region growing, and edge detection methods. In some embodiments, the segmentation model can be an image segmentation algorithm model combined with a specific tool. For example, image segmentation algorithms combined with a specific tool may include, but are not limited to, one or more combinations of genetic algorithms, wavelet analysis, wavelet transform, and active contour models. In some embodiments, the segmentation model is a neural network model. For example, the segmentation model may include, but is not limited to, one or more combinations of fully convolutional networks (FCN) models and masked region-based convolutional neural networks (Mask R-CNN) models.

[0078] Step 520: Extract the center lines from the segmented images of the left and right coronary arteries to obtain the center lines of the left and right coronary arteries.

[0079] The centerline is a line segment formed by the center point of the cross-section of a blood vessel, and it is a core feature of the vascular structure.

[0080] In some embodiments, the segmentation module 230 can extract the corresponding centerline (i.e., the element belonging to the centerline in the coronary artery segmentation image) from the left coronary artery segmentation image (or the right coronary artery segmentation image) using a centerline extraction algorithm.

[0081] In some embodiments, the centerline extraction algorithm may be a combination of at least one or more of the following: direct centerline tracking algorithm, minimum path algorithm, skeletonization algorithm, distance transformation method, and centerline extraction algorithm based on neural network model.

[0082] Step 530: Based on the center line of the left coronary artery and the center line of the right coronary artery, register the left coronary angiography image and the right coronary angiography image to determine the deformation field.

[0083] Registration refers to the process of matching pixels in the left and right coronary angiography images one-to-one.

[0084] The deformation field is a vector field that describes the pixel correspondence and changes between the left and right coronary angiography images. Each vector in the field can represent the displacement of a pixel between the left and right coronary angiography images.

[0085] Specifically, the center lines of the left and right coronary arteries are mapped back to the left and right coronary angiography images, respectively. Then, the center lines of the left and right coronary arteries in the left and right coronary angiography images are aligned, thus registering the left and right coronary angiography images. For each pixel in the left coronary angiography image, based on the pixel's position and its corresponding position in the registered pixel in the right coronary angiography image, the displacement vector of that pixel is determined, thereby determining the deformation field.

[0086] Step 540: The left coronary artery segmentation image and the right coronary artery segmentation image are fused according to the deformation field to generate a cardiac coronary artery segmentation image.

[0087] Specifically, the segmented image of the left coronary artery (or the segmented image of the right coronary artery) is deformed according to the deformation field. Then, the deformed segmented image is fused with the segmented image of the right coronary artery (or the segmented image of the left coronary artery) to generate a segmented image of the coronary arteries of the heart. The fusion process integrates the segmented images of the left and right coronary arteries based on the information of the deformation field to generate a complete segmented image of the coronary arteries of the heart containing information about the left and right coronary arteries. For example, displaying complete images of the left and right coronary arteries on the same image under the same cardiac phase, and obtaining the spatial mapping relationship or registration relationship of the complete images of the left and right coronary arteries in the fused image based on the registration relationship / registration information of the center lines of the left and right coronary arteries. As an example only, the fusion process can be to superimpose the gray values ​​of corresponding pixels in the two segmented images based on the spatial mapping relationship or registration relationship. The fusion process can also include fusion based on the connection points of the left and right coronary arteries in the heart, fusion based on machine learning models, etc., and the embodiments in this specification are not limited to these methods.

[0088] In some embodiments of this specification, the centerline can reflect the direction and shape characteristics of the left and right coronary arteries. Based on the centerlines of the left and right coronary arteries in the segmented images of the left and right coronary arteries, registration of the left and right coronary angiography images can be performed. This allows for registration of the left and right coronary angiography images based on a small amount of feature data, thereby improving registration efficiency and accuracy. Since the segmented images of the left and right coronary arteries contain relatively little information, mapping the centerlines of the left and right coronary arteries back to the left and right coronary angiography images, and determining the deformation field based on these images, can allow the deformation field to contain more information, thereby improving the accuracy of the cardiac coronary artery segmentation images.

[0089] Figure 6 This is a flowchart illustrating a method for generating a coronary artery pathology for guiding interventional procedures, according to some embodiments of this specification. In some embodiments, process 600 may be used to implement step 350. Figure 6 As shown, process 600 may include the following steps.

[0090] Step 610: Determine the target angiography image that matches the real-time fluoroscopic image from the left coronary angiography image sequence and the right coronary angiography image sequence.

[0091] The real-time fluoroscopic image corresponds to either the left or right coronary artery. The direction of the coronary artery in the real-time fluoroscopic image is the same as the direction in which the image is acquired. For example, if the real-time fluoroscopic image is acquired from the direction of the left coronary artery, then the real-time fluoroscopic image corresponds to the left coronary artery, and its direction is the direction of the left coronary artery.

[0092] The target angiographic image is an angiographic image that is in the same cardiac phase and corresponds to the same coronary artery direction as the real-time fluoroscopic image. The method for determining the cardiac phase of the target angiographic image is similar to the method for determining the cardiac phase of the coronary angiographic image in step 320, and will not be repeated here. The coronary artery direction of the target angiographic image is the coronary artery direction contained within the target angiographic image. For example, if the target angiographic image is a left coronary artery angiographic image, then its coronary artery direction is the left coronary artery direction.

[0093] Specifically, based on the cardiac phase of the target angiographic image, angiographic image pairs corresponding to the same cardiac phase are matched from multiple angiographic image pairs. Then, based on the coronary artery direction, the corresponding left or right coronary artery angiographic image is determined from the angiographic image pairs as the target angiographic image.

[0094] For example, based on the cardiac phase "atrial systole" of the target angiographic image, the angiographic image pair A1(A1) corresponding to "atrial systole" is matched from multiple angiographic image pairs. L A R If the real-time fluoroscopic image is a real-time fluoroscopic image of the left coronary artery, then the target angiographic image is the angiographic image A of the left coronary artery in image A1. L If the real-time fluoroscopic image is a real-time fluoroscopic image of the right coronary artery, then the target angiographic image is the angiographic image A of the right coronary artery in image pair A1. R .

[0095] In some embodiments, the real-time fluoroscopic images include multiple frames of real-time fluoroscopic images acquired sequentially. The generation module 350 can match the multiple frames of real-time fluoroscopic images with corresponding multi-frame target angiography images based on heart rate patterns. A detailed description of matching multiple frames of real-time fluoroscopic images with corresponding multi-frame target angiography images can be found in [reference needed]. Figure 8 And its related descriptions.

[0096] Step 620: Segment the catheter tip in the target angiography image to determine the first position of the catheter tip.

[0097] The first position refers to the location of the catheter tip in the target angiographic image. For a detailed description of determining the first position of the catheter tip, please refer to [link to relevant documentation]. Figure 9 And its related descriptions.

[0098] Step 630: Segment the catheter head in the real-time fluoroscopic image to determine the second position of the catheter head.

[0099] The second position is the current real-time position of the catheter tip in the real-time fluoroscopic image. The method for determining the second position of the catheter tip is similar to the method for determining the first position of the catheter tip in step 620, and will not be described again here.

[0100] Step 640: Based on the first and second positions of the catheter tip, the coronary artery segmentation image corresponding to the target angiography image is converted to obtain a real-time coronary artery segmentation image.

[0101] The real-time coronary artery segmentation image is a segmented image of the coronary arteries registered with a real-time fluoroscopic image. Specifically, the generation module 350 determines the positional offset of the catheter tip based on the first and second positions of the catheter tip, and then performs translation, rotation, or other geometric transformations on the coronary artery segmentation image corresponding to the target angiography image according to the offset to obtain the real-time coronary artery segmentation image.

[0102] Step 650: The real-time coronary artery segmentation image is superimposed on the real-time fluoroscopic image to obtain the coronary artery path map. Specifically, the grayscale values ​​of corresponding pixels in the real-time coronary artery segmentation image and the real-time fluoroscopic image can be superimposed to obtain the coronary artery path map.

[0103] In some embodiments of this specification, angiographic images of the same cardiac phase and the same coronary artery direction are selected as target angiographic images. This ensures that the selected target angiographic image is highly consistent with the coronary artery status in the real-time fluoroscopic image. Based on the positional offset of the catheter tip in the offline image (i.e., the target angiographic image) and the real-time image (real-time fluoroscopic image), the segmented image of the coronary artery is converted. This achieves dynamic matching and updating of the pre-acquired segmented image of the coronary artery with the current real-time fluoroscopic image. This allows the surgeon to obtain the coronary artery structure information corresponding to the current catheter position in real time during the operation, thereby helping the surgeon to better plan the catheter's advancement path, avoid damage to the coronary arteries, and improve the safety and success rate of the operation.

[0104] Figure 8 This is a flowchart illustrating a method for determining a target contrast image that matches a real-time fluoroscopic image, according to some embodiments of this specification. In some embodiments, process 800 may be used to implement step 610. For example... Figure 8 As shown, process 800 may include the following steps.

[0105] Step 810: For the multiple frames of the first real-time fluoroscopic images in the multiple frames of the real-time fluoroscopic images, determine the multiple frames of the first target angiography images that match the first real-time fluoroscopic images from the left coronary angiography image sequence and the right coronary angiography image sequence.

[0106] Multi-frame real-time fluoroscopic images are real-time fluoroscopic images acquired continuously at multiple acquisition points, providing dynamic information about catheter movement and cardiac motion. For example, multiple frames of real-time fluoroscopic images can be acquired based on a preset time interval (such as 0.1 seconds).

[0107] The first real-time perspective image is a real-time perspective image acquired earlier. In some embodiments, the acquisition time of the multiple first real-time perspective images meets preset conditions. The preset conditions may include: the acquisition time of the multiple first real-time perspective images is within the first half of the acquisition time of the multiple real-time perspective images, and / or the acquisition time of the multiple first real-time perspective images covers at least N (e.g., 3) heartbeat cycles.

[0108] Each frame of the first target angiography image in the multi-frame first target angiography image is an angiography image in the same cardiac phase and in the same coronary artery direction as the first real-time fluoroscopy image. The method for determining each frame of the first target angiography image that matches each frame of the first real-time fluoroscopy image is similar to the method for determining the target angiography image that matches the real-time fluoroscopy image in step 610, and will not be described again here.

[0109] Step 820: Determine the heartbeat pattern of the target object based on the heartbeat phases corresponding to multiple frames of the first target imaging images.

[0110] The method for determining the cardiac phase corresponding to the first target angiography image is similar to that for determining the cardiac phase of the coronary angiography image in step 320, and will not be repeated here.

[0111] The heartbeat pattern of a target object reflects the regularity of the heart in a certain state of motion, including the duration of each cardiac phase. When the state of the heart changes, the heartbeat pattern changes accordingly. For example, if the target object suddenly begins vigorous exercise from a resting state, the duration of each cardiac phase shortens.

[0112] As an example only, with a heart rate of 0.8s, 24 frames of the first real-time fluoroscopic images were acquired within 3 heart rate cycles at a preset time interval of 0.1s. The corresponding heart rate phases of the 24 frames of the first target angiography images are P1 (atrial systole) / P2 (ventricular systole) / P2 / P3 (atrial diastole) / P3 / P4 (ventricular diastole) / P4 / P4 / P1 / P2 / P2 / P3 / P3 / P4 / P4 / P1 / P2 / P2 / P3 / P3 / P4 / P4 / P4. Thus, the following heart rate movement pattern of the target object can be observed: the durations of P1 (atrial systole), P2 (ventricular systole), P3 (atrial diastole), and P4 (ventricular diastole) are 0.1s, 0.2s, 0.2s, and 0.3s, respectively.

[0113] Step 830: For the second real-time fluoroscopic image in the multi-frame real-time fluoroscopic images, based on the heartbeat motion pattern and the multi-frame first target angiography images, determine the second target angiography image that matches the second real-time fluoroscopic image.

[0114] The second real-time perspective image is the one acquired later than the first real-time perspective images. For example, the first real-time perspective images are acquired during the first 1-3 heartbeat cycles, while the second real-time perspective images are acquired during the 4th-6th heartbeat cycles.

[0115] Sub-step 832: Extract phase feature information from multiple frames of the first real-time perspective image and the second real-time perspective image.

[0116] The method for extracting phase feature information from multiple frames of the first and second real-time fluoroscopic images is similar to the method for extracting phase feature information from each frame of the left and right coronary angiography image sequences in step 320, and will not be described again here.

[0117] Sub-step 834: Based on the phase feature information, determine whether the heartbeat movement pattern of the target object has changed during the acquisition of the second real-time perspective image.

[0118] Specifically, by comparing the phase feature information of multiple frames of the first real-time fluoroscopic image with the phase feature information of the second real-time fluoroscopic image, it is determined whether the heartbeat pattern of the target object has changed during the acquisition of the second real-time fluoroscopic image. For example, if the difference between the phase feature information K1 of the first real-time fluoroscopic image and the phase feature information K2 of the second real-time fluoroscopic image corresponding to the "atrial systole" exceeds the change threshold, it indicates that the heartbeat pattern has changed.

[0119] Sub-step 836, in response to determining that the heartbeat pattern has not changed, determines a second target imaging image that matches the second real-time fluoroscopic image based on the heartbeat pattern and multiple frames of the first target imaging image.

[0120] Specifically, if the heartbeat pattern remains unchanged, it indicates that the target object's heartbeat is relatively stable, and the second target contrast image within the same heartbeat phase is similar to the first target contrast image. Therefore, the heartbeat phase of multiple frames of second real-time fluoroscopic images and their corresponding second target contrast images can be determined based on the heartbeat pattern. Continuing the example above, the phases corresponding to the 8 frames of second real-time fluoroscopic images acquired in the 4th heartbeat cycle should be P1 / P2 / P2 / P3 / P3 / P4 / P4 / P4, respectively. Therefore, the 8 frames of first target contrast images corresponding to P1 / P2 / P2 / P3 / P3 / P4 / P4 / P4 can be determined as 8 frames of second target contrast images matching the 8 frames of second real-time fluoroscopic images.

[0121] Sub-step 838, in response to the determination of a change in the heart rate pattern, involves determining multiple second target angiographic images matching the second real-time fluoroscopic images from the left and right coronary angiography image sequences. The method for determining the multiple second target angiographic images matching the second real-time fluoroscopic images from the left and right coronary angiography image sequences is similar to the method for determining the multiple first target angiographic images in step 810, and will not be repeated here. That is, when the heart rate pattern changes, it is necessary to re-match the second real-time fluoroscopic images and the angiography image sequences to ensure the accuracy of the determined second target angiographic images.

[0122] In some embodiments of this specification, when the heartbeat pattern of the target object does not change during the acquisition of the first real-time fluoroscopic image and the second real-time fluoroscopic image, the second target angiography image is determined based on the heartbeat pattern and multiple frames of the first target angiography image. Otherwise, multiple frames of the second target angiography image matching the second real-time fluoroscopic image are re-determined from the left coronary angiography image sequence and the right coronary angiography image sequence. This ensures timely detection of sudden changes in heartbeat pattern and improves the adaptability and reliability of the method for determining the second target angiography image.

[0123] In some embodiments of this specification, after determining multiple first target angiographic images that match the first real-time fluoroscopic image from the left coronary angiography image sequence and the right coronary angiography image sequence, a second target angiographic image is determined based on the heartbeat pattern and the multiple first target angiographic images. This can eliminate the need for matching the first target angiographic image and improve the efficiency of determining the second target angiographic image.

[0124] Figure 9 This is a flowchart illustrating a method for determining a first position of the catheter tip according to some embodiments of this specification. In some embodiments, process 900 may be used to implement step 620. Figure 9 As shown, process 900 may include the following steps.

[0125] Step 910: Obtain a reference contrast image from the image sequence to which the target contrast image belongs, whose acquisition time is earlier than that of the target contrast image.

[0126] A reference angiographic image is an angiographic image used to determine whether the coordinate position of the catheter tip is correct. For example, in the image sequence to which the target angiographic image belongs (i.e., the left coronary angiographic image sequence or the right coronary angiographic image sequence of the target object), if the target angiographic image is the angiographic image acquired at acquisition time T2, then the angiographic image corresponding to the acquisition time before T2 (e.g., T1) in the image sequence can be selected as the reference angiographic image.

[0127] Step 920: Perform catheter tip segmentation on the reference angiography image and the target angiography image respectively to obtain the corresponding reference catheter tip segmentation image and target catheter tip segmentation image.

[0128] The reference catheter tip segmentation image is an image segmented from the reference angiographic image. The target catheter tip segmentation image is an image segmented from the target angiographic image.

[0129] Specifically, the catheter tip can be segmented from the reference angiography image (or the target angiography image) based on the segmentation model to obtain a segmented image of the reference catheter tip (or the segmented image of the target catheter tip). For a detailed description of the segmentation model, please refer to the relevant description in step 330.

[0130] Step 930: Determine the position coordinates of the catheter tip based on the segmented image of the target catheter tip.

[0131] For example, the geometric center / centroid W of the catheter tip can be used as the coordinate position (Wcx, Wcy) of the target catheter tip segmentation image coordinate system as the position coordinate of the catheter tip.

[0132] Step 940: Based on the segmented image of the reference catheter tip, determine whether the position coordinates are correct.

[0133] First, similar to determining the coordinate position of the catheter tip, the reference position coordinates of the catheter tip can be based on the coordinate position (Wtx, Wty) of the geometric center / centroid W of the catheter tip in the coordinate system of the reference catheter tip segmentation image.

[0134] Furthermore, a reference position list is generated based on the reference position coordinates corresponding to the segmented reference catheter head images of multiple frames. The reference position list records at least one reference position coordinate and the corresponding frame number of the segmented reference catheter head image.

[0135] Furthermore, if the distance between the current catheter tip position coordinates and any reference position coordinate in the reference position list is less than a distance threshold, then the frame number of the corresponding reference catheter tip segmented image is obtained. If the frame number is greater than the frame number threshold, then the position coordinates of the catheter tip can be determined to be correct.

[0136] Step 950: In response to the correct position coordinates of the catheter tip, the position coordinates are set as the first position of the catheter tip.

[0137] Figure 10 This is a schematic diagram of the first position of the catheter tip according to some embodiments of this specification, such as... Figure 10As shown, if the position coordinates Wt(Wtx,Wty) of the catheter tip are correct, then the position coordinates Wt(Wtx,Wty) of the catheter tip can be used as the first position of the catheter tip; if the position coordinates W't(W'tx,W'ty) of the catheter tip are incorrect, then the position coordinates W't(W'tx,W'ty) of the catheter tip cannot be used as the first position of the catheter tip.

[0138] In some embodiments of this specification, a reference catheter tip segmentation image is used to determine whether the position coordinates determined based on the target catheter tip segmentation image are correct. The correlation between the preceding and following images can be used for mutual verification, effectively reducing position determination errors caused by single image segmentation errors. This improves the accuracy of determining the first position of the catheter tip and provides a reliable foundation for subsequent analysis and operations based on the catheter tip position.

[0139] The beneficial effects that the embodiments of this specification may bring include, but are not limited to: (1) In the first stage of interventional surgery (preparation stage), based on the left coronary angiography image sequence and the right coronary angiography image sequence containing the interventional device, a segmented image of the coronary arteries containing the left and right coronary arteries is obtained. In the second stage of interventional surgery (operation stage), the complete segmented image of the coronary arteries and the real-time fluoroscopic image of one side of the coronary artery are fused to obtain a complete coronary artery path map, which allows the doctor to observe the real-time relative position between the interventional device and the coronary artery based on the coronary artery path map. In addition, the coronary artery path map obtained based on the real-time fluoroscopic image of one side of the coronary artery can reflect the information of both sides of the coronary artery at the same time. On the one hand, it is not necessary to obtain the real-time fluoroscopic image of the other side of the coronary artery, which reduces the image acquisition time, enhances the real-time nature of the image, and improves the efficiency of the operation. On the other hand, it allows the doctor to make full use of the information of collateral circulation to judge the real-time effect of the interventional device on the unblocking of the occluded blood vessel, avoid damage to the coronary artery, and improve the safety and success rate of the operation. (2) Based on the phase feature information, the left coronary angiography image sequence and the right coronary angiography image sequence are matched to determine multiple angiography image pairs. Regardless of changes in the patient's heart rate, rhythm, etc., the corresponding angiography image pairs can be accurately found through the phase feature information, thereby improving the universality and applicability of this method in different patient groups. (3) A machine learning model (heartbeat phase determination model) is used to process each frame of angiography image. This model can learn a large number of angiography image data features and capture more subtle and complex image features, thereby accurately identifying the heartbeat phase corresponding to each frame of image and improving the accuracy of heartbeat phase determination. At the same time as determining the heartbeat phase, a confidence level is given, reflecting the reliability of the determined heartbeat phase by the model. This allows the model to select images that are more consistent with the actual heartbeat phase from among many possible images, further enhancing the reliability of the results. (4) Selecting angiographic images with the same cardiac phase and the same coronary artery direction as the target angiographic image ensures that the selected target angiographic image is highly consistent with the coronary artery status in the real-time fluoroscopic image. Based on the positional offset of the catheter tip in the offline image (i.e., the target angiographic image) and the real-time image (real-time fluoroscopic image), the coronary artery segmentation image is converted, realizing the dynamic matching and updating of the pre-acquired coronary artery segmentation image with the current real-time fluoroscopic image. This allows the surgeon to obtain the coronary artery structure information corresponding to the current catheter position in real time during the operation, thereby helping the surgeon to better plan the catheter's advancement path, avoid damage to the coronary arteries, and improve the safety and success rate of the operation. (5) Using the reference catheter tip segmentation image to determine whether the position coordinates determined based on the target catheter tip segmentation image are correct can be verified by using the correlation between the preceding and following images. This effectively reduces the position determination error caused by the segmentation error of a single image, thereby improving the accuracy of determining the first position of the catheter tip and providing a reliable basis for subsequent analysis and operation based on the catheter tip position.(6) After determining multiple first target angiographic images matching the first real-time fluoroscopic image from the left and right coronary angiographic image sequences, the second target angiographic image is determined based on the heartbeat pattern and the multiple first target angiographic images. This eliminates the need for matching and improves the efficiency of determining the second target angiographic image. (7) When the heartbeat pattern of the target object does not change during the acquisition of the first and second real-time fluoroscopic images, the second target angiographic image is determined based on the heartbeat pattern and the multiple first target angiographic images. Otherwise, multiple second target angiographic images matching the second real-time fluoroscopic image are re-determined from the left and right coronary angiographic image sequences. This ensures timely detection of sudden changes in heartbeat pattern and improves the adaptability and reliability of the method for determining the second target angiographic image.

[0140] The basic concepts have been described above. Obviously, for those skilled in the art, the detailed disclosure above is merely illustrative and does not constitute a limitation of this specification. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this specification. Such modifications, improvements, and corrections are suggested in this specification and therefore remain within the spirit and scope of the exemplary embodiments described herein.

[0141] Furthermore, this specification uses specific terms to describe embodiments thereof. For example, "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Moreover, certain features, structures, or characteristics in one or more embodiments of this specification can be appropriately combined.

[0142] Furthermore, unless expressly stated in the claims, the order of processing elements and sequences, the use of numbers and letters, or other names described in this specification are not intended to limit the order of the processes and methods described herein. Although various examples have been discussed in the foregoing disclosure of some embodiments of the invention that are currently considered useful, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments; rather, the claims are intended to cover all modifications and equivalent combinations that conform to the spirit and scope of the embodiments described herein. For example, while the system components described above can be implemented using hardware devices, they can also be implemented solely using software solutions, such as installing the described system on existing servers or mobile devices.

[0143] Similarly, it should be noted that, in order to simplify the description disclosed herein and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of embodiments in this specification may sometimes combine multiple features into a single embodiment, drawing, or description thereof. However, this method of disclosure does not imply that the subject matter of this specification requires more features than those mentioned in the claims. In fact, the embodiments contain fewer features than all the features of a single embodiment disclosed above.

[0144] In some embodiments, numbers describing the quantity of components and attributes are used. It should be understood that such numbers used in the description of embodiments are modified in some examples with the terms "approximately," "approximately," or "generally." Unless otherwise stated, "approximately," "approximately," or "generally" indicates that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may be changed depending on the characteristics required by individual embodiments. In some embodiments, numerical parameters should take into account specified significant digits and employ a general method of digit reservation. Although the numerical ranges and parameters used to confirm their breadth of range in some embodiments of this specification are approximate values, in specific embodiments, such values ​​are set as precisely as feasible.

[0145] For each patent, patent application, patent application publication, and other material, such as articles, books, specifications, publications, and documents, referenced in this specification, the entire contents of which are incorporated herein by reference. This excludes historical application documents that are inconsistent with or conflict with the content of this specification, as well as documents that limit the broadest scope of the claims in this specification (currently or subsequently appended to this specification). It should be noted that in the event of any inconsistency or conflict between the descriptions, definitions, and / or terminology used in the supplementary materials to this specification and the content of this specification, the descriptions, definitions, and / or terminology used in this specification shall prevail.

[0146] Finally, it should be understood that the embodiments described in this specification are merely illustrative of the principles of the embodiments described herein. Other variations may also fall within the scope of this specification. Therefore, alternative configurations of the embodiments described herein are intended to be illustrative rather than limiting, and are considered consistent with the teachings of this specification. Accordingly, the embodiments described herein are not limited to those explicitly introduced and described herein.

Claims

1. A method for guiding coronary artery interventional procedures, characterized in that, The method includes: The left and right coronary angiography image sequences of the target object are acquired. The left and right coronary angiography image sequences are acquired by a digital subtraction angiography device in the first stage of the interventional procedure of the target object and include the interventional device. Multiple angiography image pairs corresponding to multiple heartbeat phases are determined from the left coronary angiography image sequence and the right coronary angiography image sequence. Each angiography image pair contains a left coronary angiography image and a right coronary angiography image corresponding to a heartbeat phase. For each cardiac phase, a coronary artery segmentation image is generated based on the corresponding angiographic image pair; Acquire a real-time fluoroscopic image of the target object, wherein the real-time fluoroscopic image is acquired by the digital subtraction angiography device in the second stage of the interventional procedure and includes the interventional device. Based on the real-time fluoroscopic image and the coronary artery segmentation images corresponding to the multiple heartbeat phases, a coronary artery path map is generated to guide the interventional procedure.

2. The method as described in claim 1, characterized in that, The step of determining multiple angiographic image pairs corresponding to multiple cardiac phases from the left coronary angiography image sequence and the right coronary angiography image sequence includes: Extract the phase feature information of each frame of the angiography image sequence from the left coronary angiography image sequence and the right coronary angiography image sequence; Based on the phase feature information, the left coronary angiography image sequence and the right coronary angiography image sequence are matched to determine the plurality of angiography image pairs.

3. The method as described in claim 1, characterized in that, The step of determining multiple angiographic image pairs corresponding to multiple cardiac phases from the left coronary angiography image sequence and the right coronary angiography image sequence includes: For each frame of angiography in the left coronary angiography image sequence and the right coronary angiography image sequence, the cardiac phase determination model is used to process the frame of angiography to determine the cardiac phase corresponding to the frame of angiography and its confidence level. The cardiac phase determination model is a machine learning model. For each heartbeat phase, Based on the cardiac phase corresponding to each angiographic image, at least one left coronary angiographic image and at least one right coronary angiographic image corresponding to the cardiac phase are determined. Select the left coronary angiography image with the highest confidence from the at least one left coronary angiography image; Select the right coronary angiography image with the highest confidence from the at least one frame of right coronary angiography image; The selected left and right coronary angiography images are used as the angiography image pair corresponding to the cardiac cycle.

4. The method as described in claim 1, characterized in that, For each cardiac phase, based on its corresponding angiographic image pair, generating a segmented coronary artery image includes: Obtain the left coronary artery segmentation image and the right coronary artery segmentation image corresponding to the left coronary artery angiography image and the right coronary artery angiography image in the angiography image pair; The centerlines of the left coronary artery segmentation image and the right coronary artery segmentation image are extracted respectively to obtain the centerline of the left coronary artery and the centerline of the right coronary artery. Based on the center line of the left coronary artery and the center line of the right coronary artery, the angiographic images of the left and right coronary arteries are registered to determine the deformation field; The left coronary artery segmentation image and the right coronary artery segmentation image are fused based on the deformation field to generate the cardiac coronary artery segmentation image.

5. The method as described in claim 1, characterized in that, The interventional device includes a catheter, and the generation of a coronary artery path map for guiding the interventional procedure based on the real-time fluoroscopic image and the coronary artery segmentation images corresponding to the multiple cardiac phases includes: A target angiography image matching the real-time fluoroscopic image is determined from the left coronary angiography image sequence and the right coronary angiography image sequence, wherein the real-time fluoroscopic image corresponds to the left or right coronary artery; The catheter tip in the target angiographic image is segmented to determine the first position of the catheter tip; The catheter tip in the real-time fluoroscopic image is segmented to determine the second position of the catheter tip; Based on the first and second positions of the catheter tip, the coronary artery segmentation image corresponding to the target angiography image is converted to obtain a real-time coronary artery segmentation image. The real-time coronary artery segmentation image is superimposed onto the real-time fluoroscopic image to obtain the coronary artery path map.

6. The method as described in claim 5, characterized in that, The step of segmenting the catheter tip in the target angiographic image and determining the first position of the catheter tip includes: Obtain a reference contrast image from the image sequence to which the target contrast image belongs, whose acquisition time is earlier than that of the target contrast image; The reference angiography image and the target angiography image are respectively segmented into catheter tips to obtain corresponding reference catheter tip segmented images and target catheter tip segmented images; Based on the segmented image of the target catheter tip, the position coordinates of the catheter tip are determined; Based on the segmented image of the reference catheter tip, determine whether the position coordinates are correct; In response to the correct position coordinates of the catheter tip, the position coordinates are taken as the first position of the catheter tip.

7. The method as described in claim 5, characterized in that, The real-time fluoroscopic images include multiple frames of real-time fluoroscopic images acquired sequentially. Determining the target angiography image matching the real-time fluoroscopic images from the left coronary angiography image sequence and the right coronary angiography image sequence includes: For the multiple first real-time fluoroscopic images in the multiple real-time fluoroscopic images, multiple first target angiography images matching the first real-time fluoroscopic images are determined from the left coronary angiography image sequence and the right coronary angiography image sequence, wherein the acquisition time of the multiple first real-time fluoroscopic images meets a preset condition. Based on the heartbeat phases corresponding to multiple frames of the first target imaging images, the heartbeat movement pattern of the target object is determined; For the second real-time fluoroscopic image in the multi-frame real-time fluoroscopic images, based on the heartbeat movement pattern and the multi-frame first target angiography images, a second target angiography image matching the second real-time fluoroscopic image is determined, wherein the acquisition time of the second real-time fluoroscopic image is later than that of the multi-frame first real-time fluoroscopic images.

8. The method as described in claim 7, characterized in that, The determination of the second target angiographic image that matches the second real-time fluoroscopic image further includes: Extract the phase feature information from the multiple frames of the first real-time perspective image and the second real-time perspective image; Based on the phase characteristic information, determine whether the heartbeat pattern of the target object changes during the acquisition of the second real-time perspective image; In response to determining that the heartbeat pattern has not changed, a second target imaging image matching the second real-time fluoroscopic image is determined based on the heartbeat pattern and the multiple frames of the first target imaging image.

9. A system for guiding coronary artery interventional procedures, characterized in that, The system includes: The first acquisition module is used to acquire the left coronary angiography image sequence and the right coronary angiography image sequence of the target object. The left coronary angiography image sequence and the right coronary angiography image sequence are acquired by a digital subtraction angiography device in the first stage of the interventional surgery of the target object and include the interventional device. The determination module is used to determine multiple angiography image pairs corresponding to multiple heartbeat periods from the left coronary angiography image sequence and the right coronary angiography image sequence, wherein each angiography image pair includes a left coronary angiography image and a right coronary angiography image corresponding to a heartbeat period. The segmentation module is used to generate a coronary artery segmentation image for each cardiac phase based on its corresponding angiographic image pair; The second acquisition module is used to acquire a real-time fluoroscopic image of the target object, wherein the real-time fluoroscopic image is acquired by the digital subtraction angiography device in the second stage of the interventional surgery and includes the interventional device. The generation module is used to generate a coronary artery path map to guide the interventional procedure based on the real-time fluoroscopic image and the coronary artery segmentation images corresponding to the multiple heartbeat phases.

10. A computer-readable storage medium storing computer instructions that, when read by a computer, execute a method for guiding coronary interventional procedures as described in any one of claims 1 to 8.

Images