Method and system for augmenting real-time intraoperative X-ray video feeds with anatomical and device-related overlays and metrics.

Abstract

An exemplary method includes receiving in real time a video stream of intervention procedures of body lumens and devices deployed within body lumens, captured by an image acquisition device; identifying body lumens and devices deployed within body lumens in the video stream; visually presenting body lumen markers indicating the characteristics of the body lumen, including curvature markings and start-restriction zones to be avoided when deploying the device; visually presenting a display of device markers indicating the position of the device in deployment in real time; and providing visual indicators of device parameters on the video stream in real time.

Description

Technical Field

[0001] Cross - Reference to Related Applications This application claims priority to U.S. Provisional Patent Application No. 63 / 488,655, filed on March 6, 2023, and incorporates by reference in its entirety the content of that provisional patent application as if fully set forth herein.

Background Art

[0002] X - ray angiography helps visualize the path of a body lumen (e.g., a blood vessel) along with the body lumen region to be considered during body lumen (e.g., intravascular) surgery. Since X - ray angiography continues to be one of the major elements of the success of body lumen interventions, real - time clinical decision - making largely depends on the surgeon's perception of the angiographic feed and its interpretation. Such decision - making involves visual augmentation for the body lumen and the deployment sites of devices such as other embolization materials like blood flow diversion stents, adjunct stents, in - sac devices, coils, beads, liquids, particles, etc., and requires a real - time multi - dimensional approach using both two - dimensional (2D) and three - dimensional (3D) methods to reliably identify the multifaceted aspects of such intervention procedures in a multi - dimensional process with time constraints.

[0003] In view of these and other considerations, the disclosure made herein is presented.

Summary of the Invention

[0004] In an example, what is described herein are methods and systems for augmenting real - time intraoperative X - ray video feeds using anatomical information, 3D rendering, and device - related overlays and metrics.

[0005] Further examples described herein disclose systems and methods relating to methods and systems for enhancing real-time intraoperative X-ray video feeds using several anatomical information, catheters, 3D renderings, and device-related overlays, while simultaneously adding predictive deployment-related guidance or metrics to the surgeon or robotic system performing the surgery.

[0006] The aforementioned features, functions, and advantages can be achieved independently in various examples, or in combination in further embodiments. Further details of the examples can be understood by referring to the following description and drawings.

[0007] Novel features described in the Explanatory Examples are shown in the attached claims. However, the Explanatory Examples, as well as preferred uses, further purposes, and descriptions thereof, will be best understood by referring to the following detailed description of the Explanatory Examples of this Disclosure, in conjunction with the attached drawings. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram of a system including an image capture device, a device deployment module, and a display device, according to an exemplary embodiment. [Figure 1A] This block diagram shows the operation performed by the device deployment module according to an exemplary embodiment. [Figure 2] A block diagram representing the system in Figure 1, according to an exemplary embodiment, is shown. [Figure 3] Figure 1 shows the system according to an exemplary embodiment, and another block diagram is shown illustrating the details of body lumen identification. [Figure 3A] This illustrates an apposition mismatch of a device deployed within a body lumen, according to an exemplary embodiment. [Figure 4]A block diagram representing the system of Figure 1, with details associated with identified parameters of the device, is shown according to an exemplary embodiment. [Figure 4A] This example demonstrates the determination of the cornering angle, one of the many features of the device. [Figure 5] Figure 1 shows the system according to an exemplary embodiment, and another block diagram is shown illustrating the details of body lumen identification. [Figure 5A] An exemplary embodiment shows the superposition of a central line and an annular ring on the wall of the body lumen within the body lumen. [Figure 5B] An exemplary embodiment shows a preferred catheter track guideline offset from the centerline of the body lumen. [Figure 5C] An exemplary embodiment shows a device deployed within a body lumen. [Figure 5D] An example of registration of a body lumen model between 2D and 3D is shown according to an exemplary embodiment. [Figure 6] A block diagram representing the system shown in Figure 1 illustrates the generation of a probability graph of the device's landing zone according to an exemplary embodiment. [Figure 7] Figure 1 shows linearized graphical information that can be generated by the device deployment module of the system to provide key information for body lumen intervention via a real-time video stream of X-ray angiography, according to an exemplary embodiment. [Figure 7A] An exemplary embodiment shows the radius vector of the tactile circle at a given point and the outward deviation vector pointing outward from the center of the tactile circle. [Figure 8] An image enlarged with various markers, according to an exemplary embodiment, is shown. [Figure 9] This example demonstrates the generation of wire and deployed device masks using a trained model. [Figure 10] A block diagram illustrating a generative modeling workflow in an exemplary embodiment is shown. [Figure 11] Shows low-dose X-ray images and corresponding high-dose X-ray images according to an exemplary embodiment. [Figure 12] Is a block diagram of a computing device according to an exemplary embodiment. [Figure 13] Is a flowchart of a method for augmenting a real-time intraoperative X-ray video feed with anatomical and device-related overlays and metrics according to an exemplary embodiment. [Figure 14] Is a flowchart of additional operations executable with the method of FIG. 13 according to an exemplary embodiment. [Figure 15] Is a flowchart of additional operations executable with the method of FIG. 13 according to an exemplary embodiment. [Figure 16] Is a flowchart of additional operations executable with the method of FIG. 13 according to an exemplary embodiment. [Figure 17] Is a flowchart of additional operations executable with the method of FIG. 13 according to an exemplary embodiment. [Figure 18] Is a flowchart of additional operations executable with the method of FIG. 13 according to an exemplary embodiment. [Figure 19] Is a flowchart of additional operations executable with the method of FIG. 13 according to an exemplary embodiment. [Figure 20] Is a flowchart of additional operations executable with the method of FIG. 13 according to an exemplary embodiment. [Figure 21] Is a flowchart of additional operations executable with the method of FIG. 13 according to an exemplary embodiment. [Figure 22] [[ID=​​​​​​This specification discloses a system and method comprising capturing a real-time angiographic video feed during a body lumen (e.g., intravascular) intervention, such as the placement of a device (e.g., a blood flow redirection device); analyzing the real-time video feed; performing analysis on the real-time video and imaging feeds to determine the location of catheters, wires (e.g., guidewires), and devices during deployment; visually presenting in real time on a display (e.g., superimposed on the video or on a separate display) body lumen markers, device (e.g., stent) boundaries and characteristics, deployment guides, and predictions regarding device deployment; providing deployment-related quality evaluation metrics; and guiding communication with a surgeon or robotic system during the intervention.

[0010] The term "body lumen" is used herein to refer to blood vessels, lymphatic vessels, bile ducts, esophagus, trachea, or other body lumen. Furthermore, the term "device" is generally used to refer to other embolic materials such as blood flow diversion stents, auxiliary stents, intracapsular devices, coils, beads, liquids, particles, etc.

[0011] Figure 1 is a block diagram of a system 100, including an image capture device 102, a device deployment module 104, and a display device 106, according to an exemplary embodiment. The components of system 100 may be configured to function in a manner interconnected with each other and / or with other components coupled to each system. One or more of the described operations or components of system 100 may be divided into additional operational or physical components, or combined into fewer operational or physical components. In some further examples, additional operational and / or physical components may be added to system 100. Furthermore, any component or module of system 100 may include, or be provided in such form, a processor (e.g., a microprocessor, a digital signal processor, etc.) configured to execute program code including one or more instructions for performing the logical operations described herein.

[0012] System 100 may further include any type of computer-readable medium (non-temporary computer medium) or memory, such as a storage device including a disk or hard drive, which stores program code that, when executed by one or more processors, causes System 100 to perform the operations described above. In one example, System 100 may be included in another system.

[0013] The image acquisition device 102 is configured to directly capture images and videos of body lumens or read them from available data streams during an intervention procedure (e.g., deployment of a stent, such as a blood flow redirection device, within a body lumen). For example, the image acquisition device 102 preferably includes a bidirectional angiography X-ray system (e.g., Philips Healthcare's Azurion system or Siemens Healthineers' Artis system) or a computed tomography (CT) scanning device that combines a series of X-ray images taken at different angles around the patient's body and uses computer processing to generate tomographic images (slices) of the body lumen.

[0014] Alternatively, the image acquisition device may be a data acquisition device that extracts available imaging information from a separate angiography or CT scanner.

[0015] In one example, the image acquisition device 102 may include a micro-CT scan device that uses X-ray-based three-dimensional (3D) imaging techniques to observe the inside of a patient's body slice by slice. Micro-CT scans are similar to CT scans but are smaller in scale and offer improved resolution. For example, body lumens can be imaged with pixel sizes as small as 100 nanometers, while objects up to 200 millimeters in diameter can be scanned.

[0016] Therefore, in one example, the image acquisition device 102 may include an X-ray source that generates X-rays, which then pass through a part of the patient having a body lumen of interest. The image acquisition device 102 also includes an X-ray detector that records the X-rays as a two-dimensional projection image. The X-ray source can then be rotated by a small angle on a rotating platform to acquire another X-ray projection image. This step can be repeated through 180 or 360 degrees, thereby capturing images of the body lumen from different angles.

[0017] In one example, the image acquisition device 102 can also generate real-time video of a body lumen. For example, the image acquisition device 102 may include at least one camera (e.g., one camera, or two cameras in a bidirectional arrangement) deployed in the patient's body lumen along with a body lumen device, which generates a real-time feed of the body lumen and its deployment. In another example, the image acquisition device 102 may include an external imaging device (CT scan device) that generates a real-time video feed of the body lumen and a device deployed in the body lumen (e.g., a catheter and a stent deployed via wires). The image acquisition device 102 may be configured to extract still images from the video. The image acquisition device 102 is then configured to provide such video feeds and / or images to a display device 106 for display on the display device 106. An example of such a device may be found in a catheterization laboratory used for nervous system, cardiac system, and peripheral body lumen interventions.

[0018] The device deployment module 104 is configured to receive such video, analyze the video, and perform analysis on the video to determine the real-time location of the catheter, wires, and device during device deployment. The device deployment module 104 then communicates such information to the display device 106, which visually presents the information overlaid on the video or image to the healthcare professional. In particular, the device deployment module 104 can visually overlay device (e.g., stent) and body lumen markers, guides and guidelines, as well as predictions regarding device deployment, onto the images and video on the display device 106, and can provide deployment-related quality evaluation metrics (e.g., scores) on the display device 106. Such visual information and metrics can provide guidance to the surgeon or instruct the robotic system to adjust the deployment technique and improve device deployment to achieve the desired outcome.

[0019] Accordingly, the device deployment module 104 is configured to receive a digital video feed from an imaging system such as an X-ray angiography system or a CT scan device during an interventional procedure, and to display information to the physician or operator in real time during surgery to help the physician or operator position the body lumen device in an optimal configuration. In this specification, the term "intraoperative" is used to indicate occurrence or execution during the course of a surgical procedure. In one example, the device deployment module 104 augments the information on the video feed and displays the augmented video feed or images extracted therefrom on the display device 106.

[0020] Throughout this specification, blood flow redirection stents are used as exemplary devices. However, it should be understood that the techniques, methods, and systems disclosed herein can also be used with other intervention devices (e.g., embolic coils, intracapsular device deployment, peripheral body lumen stent deployment, or other lumen stent deployment such as carotid, biliary, or femoral artery stents).

[0021] In one example, the stent can be placed in or sheathed within a catheter and advanced along a wire to a position in the body lumen where the stent is to be deployed or positioned. Exemplary augmented information generated by the device deployment module 104 includes (i) identification and marking of the locations of the stent, catheter, and wire, and (ii) anatomical identification and catheter landmarks to add visual context for the physician or operator and enhance scene understanding, particularly when using low-dose X-ray systems that produce low-contrast X-ray images. By enhancing the clarity of the stent, catheter, and guidewire and superimposing relevant features (anatomical, physiological, or other), the device deployment module 104 can reduce the need for high-dose X-ray imaging and the need for several intravascular contrast injections to generate a “roadmap” image for digital subtraction angiography, thereby reducing exposure to contrast agents, decreasing the required computational power, and increasing the efficiency and precision of the surgical procedure.

[0022] In one example, the information may include surgical scoring metrics. For instance, the device deployment module 104 may provide a measure of how well the surgical procedure / deployment is performed in relation to the preoperative surgical plan or expected deployment outcome, or a predictive deployment state that helps support correct deployment.

[0023] In one example, video augmentation can be performed directly on a display device 106 that can be placed in a neurovascular interventional catheterization laboratory. In another example, the display device 106 may include an augmented reality or virtual reality headset that can overlay information on an existing display or a virtual display, thereby reducing the need for additional hardware in the catheterization laboratory.

[0024] In another example, video and data augmentation can be presented using a hologram as the display device 106. This hologram projects 2D information and 3D models into the space near the user without requiring the user to wear additional visual devices. The holographic visual information can be interactively manipulated to switch and change information and models at the user's command before and after surgery.

[0025] Therefore, the device deployment module 104 is configured to perform operations including feature recognition of vascular structures and deployed devices, real-time analysis and scoring, and to provide various outputs and predictions to the user or robotic system.

[0026] Figure 1A is a block diagram showing the operations performed by the device deployment module 104 according to an exemplary embodiment. As shown, the analysis of device deployment, such as a stent, begins with feature recognition of wires, markers, body lumen, device, curvature, diameter, length, size, reference point, centerline, and wire / strut braiding angle.

[0027] Multiple inputs can be derived using recognized or identified features. These include coning angle, inter-marker distance, curvature of the body lumen, curvature of the stent, appointment of the stent to the body lumen, stent position, stent shape, clinically undesirable irregular shapes (e.g., ribboning, twisting, fishmouth), deviation from the centerline, deviation from the ideal or defined deployment path, braiding angle, or braiding density. Inter-marker distance may also include landmarks such as the distance from a specific marker to a reference point on the body lumen, aneurysm orifice, start and end points of the curve, ideal landing zone, ideal or defined path, and deviation from the centerline.

[0028] The coning angle is not a fixed angle but fluctuates during the deployment procedure. The ideal coning angle is a function of the tortuosity and curvature of the body lumen, the amount of tension or compression applied to the catheter, the original diameter of the body lumen, the dimensions of the aneurysm, and the resultant force between the microcatheter and the stent delivery wire. An example of how the coning angle is determined is described below in relation to Figure 4A.

[0029] The curvature of the body lumen can also be determined, providing input for ideal deployment calculations. More pronounced curvature presents additional challenges to deployment, likely requiring more push-pull movements and catheter manipulation to achieve the desired results. Curvature is also an important lateral input for analyzing cone shape, coning angle, deviation from the centerline, deployment path, etc.

[0030] Braid angle and braid density can be partially recognized as stent characteristics or applied to the stent placement area through known and recognized information such as stent design, curvature, diameter, and appointment. Braid angle and braid density are factors in predicting blood flow redirection.

[0031] The appointment between the stent and the body lumen is also useful for evaluating deployment techniques, positioning, landing zone, shortening, avoidance of end-leaks, and stent placement.

[0032] In addition, feature recognition can identify actions and events during interventional cases. These actions and events can be detected and used to improve stent deployment scoring algorithms. For example, contrast injection / flush can be used to identify any area of ​​improper stent appointment, stent resheathing can be used to improve technique or detect any slippage from the stent resheathing pad, and the intervener can be alerted if the stent delivery wire moves to a smaller arterial branch.

[0033] Using the input from feature recognition to analysis and scoring, the outputs and predictions generated by system 100 (for example, as shown in Figure 1A) can be presented, notified, displayed, or messaged to the user as results.

[0034] Figure 2 shows a block diagram representing system 100 according to an exemplary embodiment. Block 200 represents the real-time acquisition of a data stream containing video and other information related to a body lumen (angiography) in which a body lumen device such as a stent is positioned. The data stream is shown on display device 106 (see extended image 210). The data stream containing video and other information may be, for example, an X-ray angiography data stream captured by a CT scanning device (image acquisition device 102 described above). The video stream may also be acquired from the imaging system's digital video feed (e.g., High Definition Multimedia Interface (HDMI®), Digital Visual Interface (DVI), serial port, etc.) or from an external screen capture of X-ray angiography (e.g., an external camera facing a screen displaying the video feed in a catheterization laboratory).

[0035] The video feed is considered an input stream to block 202. Block 202 can be implemented, for example, by a device deployment module 104. In block 202, the device deployment module 104 can identify a body lumen device (e.g., a stent) traversing the body lumen via a catheter and wire using image recognition techniques.

[0036] For example, the device deployment module 104 can have access to the 3D geometric registration of the device in 3D space (e.g., a 3D model of the stent) in block 204. Thus, the device deployment module 104 can identify the stent's position, stent markers, stent position and orientation, catheter, and wire. In particular, the device deployment module 104 can identify the stent's characteristics (e.g., proximal end, distal end, wire / strut angle, orientation, etc.).

[0037] Such information (stent location, markers, features, indicators) is generated as graphical annotations in block 206. These graphical annotations are then extended to the video feed on display device 106 in block 208. Both the video stream acquired in block 200 and the graphical annotations in block 206 can be displayed on display device 106. In other words, the graphical annotations are superimposed on or extended to the video stream while they are displayed on display device 106.

[0038] The expanded image 210 shows information related to the stent, catheter, and wire expanded into the video feed. This image shows the stent 212 and wire 214, which can be distinguished or clarified in the expanded image 210 by color coding or contrast enhancement.

[0039] In addition to block 202, which is associated with real-time recognition of the deployed device (stent), the device deployment module 104 can detect the characteristics of the body lumen into which the device is deployed in block 218. Based on such detection, the device deployment module 104 can recognize the characteristics of the body lumen and identify different regions of the body lumen and their suitability for stent deployment. Next, as will be described in more detail with respect to Figure 3, the device deployment module 104 can also generate and / or point out indicators related to the body lumen, such as a tortuosity indicator and a no-start zone indicator.

[0040] Figure 3 shows another block diagram representing the system 100 according to an exemplary embodiment, illustrating details of body lumen identification. Block 218 can be implemented by a device deployment module 104. The device deployment module 104 may have access to a 3D body lumen model in block 220. For example, the device deployment module 104 may have access to a 3D model of the body lumen generated from images acquired before the interventional procedure via, for example, an image acquisition device 102 (e.g., a rotational scan angiography system or a CT scan device).

[0041] Furthermore, the contrast agent can be injected into the body lumen while capturing a video feed. The device deployment module 104 can access images via the video feed when the contrast agent is injected in block 222, and then the device deployment module 104 can map the body lumen pathway (e.g., details of the body lumen including branches). The device deployment module 104 can also compare the 3D body lumen model of block 220 with the body lumen pathway mapped when the contrast agent is injected into the body lumen in block 222, visually clearly identifying different regions of interest within the body lumen. Such comparisons may also be useful for spatially registering (e.g., aligning) the 3D body lumen model of block 220 to the live image.

[0042] For example, as shown in Figure 3, the device deployment module 104 can identify the paths of catheters, wires, and stents within a body lumen (pathfinding), identify geometric parameters of the body lumen such as the diameter of various parts of the body lumen, determine the curvature / tortuosity of various parts of the body lumen, and identify zones of the body lumen where a stent should not be deployed (warning zones). Next, the device deployment module 104 can visually superimpose body lumen markers (e.g., markers indicating lumen width, centerline, boundary, curvature, etc.) onto the video feed image.

[0043] Figure 3 shows an extended image 224, which displays an image from a video feed depicting a body lumen 226. The device deployment module 104 can accurately determine the degree of meandering or curvature of the body lumen 226. Furthermore, the device deployment module 104 can identify and label the most meandering body lumen regions and their characteristics, including curvature in 3D, body lumen loops, or branching in 3D space, in the extended image 224. For example, the device deployment module 104 can superimpose a meandering indicator on the extended image 224. In particular, the device deployment module 104 can mark the body lumen 226 with curvature markings such as curvature marking 228, curvature marking 229, and curvature marking 230, which indicate regions of the body lumen 226 that have curvature exceeding a certain threshold curvature. In this context, curvature can be determined, for example, as the reciprocal of the radius of a portion of the body lumen 226, and serves as an indicator of how tortuous the body lumen 226 is. Curvature markings 228-230 are marked and displayed, for example, in a specific color (e.g., red), to visually clarify such markings for the surgeon. These markings can help the surgeon determine where to deploy the stent and which areas to avoid during deployment.

[0044] Furthermore, the device deployment module 104 can also identify start-stop and end-stop zones within the body lumen 226. Start-stop zones are areas where tortuosity is unfavorable, hidden branches are present, or junctions exist, and if the distal end of the stent is placed in such an area, the performance of the stent may be impaired. In other words, when deployment is initiating, the surgeon should avoid placing the distal end of the stent in such a start-stop zone, and when deployment is ending, the surgeon should avoid placing the proximal end of the stent in an end-stop zone. Start and end-stop zones are determined by the surgeon before stent deployment and can also be entered manually.

[0045] As shown in Figure 3, the device deployment module 104 marks a no-start zone 232 where the body lumen 226 branches. On the expanded image 224, the device deployment module 104 places polygons around the no-start zone 232 to indicate such zones to the surgeon during stent deployment. Alternatively, the no-start / no-end zone or optimal start / end zone is indicated by a periphery indicator (ellipse) of the body lumen and color-coded to show good and bad areas at the stent location. Alternatively, optimal start and end ("start here" and "end here") zones may be employed, in which the markings are oriented towards the locations where the stent will optimally start and / or end. These locations may be determined by the device deployment module 104, manually entered by the surgeon, or determined by separate planning software and entered into the device deployment module 104.

[0046] In addition, in the example, the device deployment module 104 can provide warnings, numerical scores, and other such indicators on the augmented image 224 to assist in real-time stent deployment. Recognition of the tortuosity of the body lumen 226 and the no-start zone can help the surgeon achieve better deployment outcomes.

[0047] For example, it is desirable that the stent has a sufficient appointment with respect to the wall of the body lumen 226. Stent appointment refers to how closely the outer surface of the stent is in contact with the inner wall of the body lumen 226. If the outer diameter of the stent is smaller than the inner diameter of the body lumen, the stent can be characterized as having a loose appointment (unsuitable joint) with respect to the wall of the body lumen 226. Such a loose appointment may be undesirable because it can lead to displacement or movement of the stent when deployed in the body lumen 226. Rather, it is desirable to have a stent with a high appointment that is as close as possible to the wall of the body lumen, ensuring a stable position within the body lumen and providing effective blood flow redirection.

[0048] Aposition can be expressed by the coverage percentage in a specific cross-section of the body lumen 226. For example, in a given cross-section of the body lumen 226 based on a 3D-registered body lumen model, the appointment mismatch can be determined as follows:

number

[0049] In another example, the appointment mismatch % can also be determined as follows:

number

[0050] Other appointment metrics can also be used. For example, wall appointment mismatch can be calculated based on stent edge detection and compared to the body lumen wall obtained from digital subtraction angiography images.

[0051] Figure 3A illustrates (in a linearized form) appointment mismatch of a stent deployed in a body lumen in an exemplary embodiment. Figure 3A shows potential scenarios that may occur in different underlying disease states, such as moyamoya disease. The wall appointment score depends on the degree to which the stent is in visual / perceptual contact with the body lumen wall. Risks associated with high mismatch include potential endoleak, recanalization of an aneurysm, or other complications related to poor wall appointment of the stent.

[0052] As another example, the overall appointment score of the stent can be determined. In Figure 3A, L M L is the local mismatch length. T This is the total length of the stent. Overall appointment score: A Total This can be determined as follows:

number

[0053] The overall appointment score and appointment mismatch can be highlighted on the augmented image, providing indications of potential stent placement problems such as appointment warnings, suggestions for deployment techniques, or recanalization, stent deployment end-leaks, or other concerns.

[0054] Returning to Figure 3, in tortuosic areas such as the zone marked by curvature markings 228–230, stent appointment may be inadequate. Such areas should ideally be avoided, and therefore such markings assist the surgeon during deployment. In some cases, it is necessary to traverse high-curvature areas during stent placement, and the markings can serve as a warning to the surgeon to pay more attention to stent appointment in those areas.

[0055] Figure 4 shows a block diagram representing system 100 with details associated with identified parameters of the stent, according to an exemplary embodiment. As shown in Figure 4, the device deployment module 104 may have access to 3D rendering data of the stent (e.g., a 3D model of the stent supplied by the stent manufacturer) in block 204. Thus, the device deployment module 104 can superimpose the stent model into a body lumen and determine the characteristics of the stent (e.g., appointment).

[0056] For example, as shown in Figure 4, the device deployment module 104 can determine information including localization within the body lumen (stent positioning), appointment, the centerline of the body lumen and the stent, and, if the stent is a braided stent, the wire orientation of the stent, the coning angle, and the braiding angle (e.g., half the angle formed by the intersecting filaments of the braid in a braided stent). The device deployment module 104 can then superimpose indicators of such information onto the video feed image.

[0057] For example, as shown in Figure 4, the augmented image 234 shows recognized features such as the contour of the body lumen 236 superimposed on the video feed. The device deployment module 104 can further superimpose a predictive rendering 237 of the stent 238 and show the centerline 240 of the stent 238.

[0058] Furthermore, in one example, as shown in Figure 4, the device deployment module 104 can visually superimpose an appointment indicator on the expanded image 242, showing how well the stent 238 is in contact with the wall of the body lumen 236. For example, as shown in Figure 4, the device deployment module 104 can generate elliptical displays such as ellipses 244 at different points along the length of the stent 238. These ellipses are intended to show the relationship between the diameter of the stent 238 and the respective diameters / circumferences of the body lumen 236, but appear elliptical due to the angularity of the body lumen 236 and the stent 238.

[0059] The ellipse 244 serves as an aposition indicator at a specific location or cross-section of the stent 238 and the body lumen 236 (e.g., how well the stent 238 is in contact with the wall of the body lumen 236). The ellipse can be color-coded. For example, a green ellipse may indicate an acceptable aposition, while a red portion may indicate a poor or unacceptable aposition. For example, a portion of the ellipse 244 may be green, while a portion 246 may be red, indicating a poor aposition or mismatch between the body lumen 236 and the stent 238 (e.g., the diameter of the stent 238 may be larger than the diameter of the body lumen 236 at a given site).

[0060] Figure 4A illustrates the determination of the coning angle, one of the many features of a stent, in an exemplary embodiment. In particular, Figure 4A provides an example of identifying the coning angle, the cone shape during deployment, the inner curvature side, and the calculated distance from the last recognized appointment position to the catheter marker.

[0061] As output, for example, the coning angle can be color-coded in image 247 to show the analysis results of the deployment status. Scoring and suggested deployment procedures are also displayed to guide the deployment process in real time. In one example, the coning angle can be determined based on the angle formed between the marker and a defined distance. In another example, the coning angle can be calculated based on the segmentation of the stent identified at a defined distance D from the microcatheter marker. If "r" is the radius of the stent opening at distance D, then these can be combined and calculated, for example, in a straight body lumen, as 2 × arctan(r / D).

[0062] If the stent is open at a curved section, the coning angle can be defined by the hydraulic mean angle, a normalization function, or a symmetric equivalent angle from the identified stent segmentation. Alternatively, the cone shape of the tapered section of the stent can be identified and matched to a known stent response profile from the force interaction. The output from the analysis can be displayed as an deployment force indicator for deployment technique suggestions. The cone angle, internal curvature, and centerline distance from the last appointment position are also analyzed and can indicate the quality of the technique in the deployment procedure for both opening and re-sheathing. Technique suggestions can also be provided to the user as an indicator or message, including visual, auditory, or audiovisual outputs. Suggestions and scoring can be displayed to guide and assist the procedure.

[0063] Figure 5 shows another block diagram illustrating the system 100 according to an exemplary embodiment, detailing body lumen identification. Figure 5 is similar to Figure 3, and in block 218, the device deployment module 104 determines several parameters of the body lumen, such as diameter, centerline, curvature, and stenosis indicator. The device deployment module 104 then visually superimposes the body lumen parameter information or indicator onto an image in a video feed, such as an expanded image 248.

[0064] As shown in the extended image 248, the device deployment module 104 identifies the body lumen 250 and marks its various parameters. For example, the device deployment module 104 indicates the centerline 252 of the body lumen 250 with a dotted line, following the curvature and diameter changes of the body lumen 250. The device deployment module 104 also superimposes meandering indicators or curvature markings such as curvature markings 254 and 256, which indicate areas of the body 250 that have curvature exceeding a certain threshold curvature.

[0065] The device deployment module 104 also identifies and marks the no-start zone 258 within the body lumen 250. As described above, the no-start zone is an area where tortuosity is unfavorable, where hidden branches are present, or where fractions exist, and where the distal end of the stent may be positioned in such an area, potentially impairing the performance of the stent. In addition or alternatively, the device deployment module 104 can identify "start here" and "end here" zones within the body lumen 250 that may be optimal for starting and ending the stent.

[0066] Therefore, the device deployment module 104 performs real-time angiographic registration of various features of the body lumen 250 to support optimal stent deployment results. The start prohibition zone, centerline, tortuosity, and other information are superimposed on the real-time angiographic screen (display device 106) to provide visual identification of various features of the body lumen and stent.

[0067] Figures 2–5 show examples of real-time deployment analysis performed by the device deployment module 104, which can provide animated or augmented graphical features and indicators that highlight prominent features of the body lumen and stent on radiographic angiography. Once the device deployment module 104 identifies and registers (determines position and orientation) the stent, it labels various stent features. The output (augmented image) displayed on the display device 106 may include real-time position, centerline, annular ring or loop (which may appear as a circle, ellipse, intersection of two or more circles or ellipses, or a distorted shape thereof) indicating or defining the body lumen wall, stent features, stent aposition, and important angles of the stent.

[0068] Figure 5A shows the superposition of a central line and an annular ring of the body lumen wall within a body lumen according to an exemplary embodiment. As shown, the central line of the body lumen is marked with a dotted or dashed line. The drawn annular loop or ring of the body lumen wall is defined by the intersection of the body lumen wall and a normal plane perpendicular to the central line at a given point along the central line. The base point of this ring is a point of interest, preferably determined by specific points of interest that are equally spaced along the central line within the region of interest of stent deployment and / or optimal start and end positions of the stent, the distal and nearest ends of the aneurysm opening, the centroid of the aneurysm opening, etc. The ring can also be used as a marker. For example, the ring can use color coding, spacing density, line thickness, opacity or transparency to indicate various markings such as curvature, start / end no-go zones, expected end position, landing probability, stent wall appointment, performance metrics or scores.

[0069] In one example, the displayed indicators can be toggled as desired by the surgeon to assist with deployment. The determined stent characteristics are superimposed on the specific anatomical, physiological, and physical conditions of the body lumen, assisting the surgeon in real time.

[0070] Furthermore, in the example, the device deployment module 104 can provide a quality assessment of stent deployment and determine a score or heatmap-based identifier that may trigger a warning. An exemplary score may include a weighted average of several deployment parameters in real time. For example, the score may include a weighted average of parameters indicating deployment status such as coning angle, deviation from the centerline, proximity of the stent end to a no-start or no-end zone, stent appointment, braiding angle, undeployed, partially deployed, insufficiently deployed, or optimally deployed.

[0071] For example, the expansion score can be calculated as follows:

number

[0072] Therefore, the device deployment module 104 is configured to provide real-time quality assessment of the deployment. The output of the device deployment module 104 may include an overall quality score, along with a visual overlay of the expected final position of the stent. The score changes in real time as the surgeon adjusts the positioning and deployment of the stent during the procedure.

[0073] Another example of feedback that can be provided to the user is that the device deployment module 104 can provide tracking of the catheter tip during deployment. In particular, the device deployment module 104 can track instantaneous deviations of the catheter tip from predefined preferred catheter tip tracking guidelines.

[0074] Figure 5B shows a preferred catheter track guideline 257 offset from the body lumen centerline 259 in an exemplary embodiment. The preferred catheter track guideline 257 defines the preferred path that the stent deployment catheter should follow when the stent is deployed in a manner consistent with the optimal deployment plan generated by the device deployment module 104.

[0075] If the catheter is generally following the preferred catheter track guideline 257, the user may be provided with visual and / or auditory feedback indicating that it is running on the trajectory. Conversely, if the catheter is generally deviating from the preferred catheter track guideline 257, the user may be provided with visual and / or auditory feedback indicating the deviation to suggest or signal to the user or robot that potential modifications should be made to change the deployment technique and / or update the deployment plan.

[0076] Figure 5C shows a stent 249 deployed within a body lumen 251 according to an exemplary embodiment. The illustration in Figure 5C shows an acceptable stent wall appointment, indicated by the contour of the stent 249 matching the extent of the body lumen ring of the body lumen 251. Figure 5C also shows an acceptable wire catheter deployment ratio, indicated by the catheter tip being aligned with the preferred catheter track guideline 255.

[0077] Figure 5D shows an example of registration of a body lumen model between 2D and 3D according to an exemplary embodiment. Figure 5D shows an X-ray image 261 on the left and the generated 3D body lumen model 263 on the right. This information can be processed, for example, in block 218 of Figure 3 (e.g., a real-time body lumen detection system). The 3D body lumen model 263 can be used to accurately measure the diameter and meandering position of the body lumen, as well as the position and orientation of the stent and wire in real time. This information can then be supplied to the device deployment module 104.

[0078] When using a two-plane image capture system, the location of interest identified in the video feed (e.g., catheter tip) can be back-projected into 3D space by determining the position and directional vector of the X-ray source and tracing the path of the X-ray from the X-ray source to the detector. The intersection of two such X-rays from the two-plane video feed gives coordinates in 3D space, which can be associated with a 3D body lumen model 263 by image registration (e.g., aligning images from different sources or time points by minimizing a cost function representing the similarity between images).

[0079] Furthermore, geometric changes in body lumens resulting from interventional interactions can be observed from 2D angiographic images. These changes can be superimposed onto a 3D body lumen model 263 for real-time reference. In addition, using a new dataset from multiple 2D views (e.g., X-ray images 261) acquired from angularly separated imaging vectors, the changes can be modeled based on reconstruction from the new dataset and learning from a pre-reconstructed 3D model. Updates and modifications to the body lumen model can be generated and fed into the latest 3D model to improve the accuracy of output predictions or measurement accuracy. The updated 3D model can then be used to update data used for marking, feedback, and display, such as curvature, centerlines, catheter track lines, rings, start / end no-go zones, landing zones, landing probabilities, and appointment. The 3D body lumen model 263 can be presented in an additional window or panel to enable 3D perception and real-time 3D identification during the intervention. For example, such changes can be visualized directly in 3D using devices such as visual displays, hologram devices, virtual / augmented reality wearables or headsets, and on-screen visual dashboards.

[0080] On the display channel of the 3D body lumen model 263, independent movements selected by the user or dependent rotations related to the two-plane view can be applied. For example, movements such as roll, yaw, pitch, and other viewpoint changes can be performed on the 3D body lumen model 263 (the displayed model) based on user preference and / or based on the movement of the two-plane gantry of the imaging system. Interactive movements on the 3D model can further aid in understanding cases in interventional and deployment procedures.

[0081] Figure 6 shows a block diagram representing system 100, illustrating the generation of a probability graph of the stent landing zone according to an exemplary embodiment. The device deployment module 104 may include a real-time calculation and comparison engine in block 260, configured to determine and provide graphical indicators and animations of stent deployment parameters. This helps in understanding the stent implantation effect or deployment results.

[0082] In the example shown, extended image 262 displays a real-time landing probability chart 264 (e.g., a distribution probability curve) displayed to help predict the landing zone 266 (where the proximal end of the stent will be positioned when the deployment is complete). This can be important for the outcome of the deployment.

[0083] When determining the landing zone 266, the device deployment module 104 can take into account the effects of shortening, etc. A technical problem that arises when using stents in intraluminal procedures is that it is difficult to predict the final position of the stent after deployment into the body lumen because the change in stent length depends on the patient's anatomical structure and the placement of the stent in the body lumen. In other words, the problem with stent deployment is the change in overall length (shortening) that the stent experiences when it is released from the catheter into the body lumen. The device deployment module 104 takes the shortening effect into account when determining the final length of the stent after deployment and the landing zone.

[0084] While a particular stent length may appear sufficient on a 2D image, it may be insufficient when actually deployed in the body lumen due to shortening. In particular, due to the 3D configuration of the body lumen, it may be longer than it appears in the 2D image. For example, the body lumen may meander, twist, and bend in 3D space, thereby making its actual length longer than it appears from the viewpoint of the 2D image. Furthermore, the stent may be compressed, stretched, or otherwise change in length during deployment due to the deployment technique and / or its interaction with the body lumen. The device deployment module 104 estimates such shortening effects and determines the final length of the stent and the probability of the landing zone, as shown by the real-time landing probability chart 264.

[0085] In the example, based on the determination of the landing zone 266 and other characteristics of the stent, the device deployment module 104 may propose catheter movements that yield favorable results. Proposed catheter movements include adherence to preferred catheter tip tracking guidelines, rotation, translation, and other similar manipulations of the catheter, microcatheter, and wire. These may also include repeated small movements or vibrations, such as linear back-and-forth motion, as well as repeated clockwise and counterclockwise rotations. Such proposals may be provided to a human surgeon to adjust the deployment technique, or they may be provided as algorithms to a robot deploying the stent.

[0086] Figure 7 shows linearized graphical information that can be generated by the device deployment module 104 to provide key information for body lumen intervention via a real-time video stream of X-ray angiography, according to an exemplary embodiment. By extracting geometric information from the video feed, the device deployment module 104 can label and register body lumen pathways and determine information such as curvature, diameter, and centerline of body lumen, as described above. A 3D body lumen model from the patient's medical data can be co-registered with 2D image frames extracted from the video stream to improve the accuracy of geometric parameters. Upon receiving a stent deployment video stream, the device deployment module 104 determines geometric comparisons and derives information such as deployment ratio, appointment information, and catheter track line, enabling the provision of intraoperative procedural suggestions, scoring, or warnings.

[0087] Graph 300, shown at the top of Figure 7, shows a curve 302 illustrating the variation in curvature of the body lumen along its length. Graph 304 shows the planned or predicted stent deployment position and diameter. In particular, in Graph 304, bars such as bar 306 represent the diameter of the body lumen along its length and correspond to curve 302 in Graph 300.

[0088] Furthermore, in Graph 304, the device deployment module 104 provides the predicted or proposed position and diameter of the stent in region 308, indicated by the intersecting lines. Region 308 is the space occupied by the stent. As shown, the planned position of the stent avoids starting and ending in areas of large curvature in the body lumen, as indicated by the curve 302.

[0089] Graph 310 further provides stent deployment data that can help surgeons (or robots) achieve the proposed stent position. For example, Graph 310 may include a graphical indicator of the deployment ratio at specific locations along the body lumen, as represented by the dashed line 312.

[0090] The deployment ratio refers to the wire push-to-catheter pull-back ratio for achieving an optimally deployed state. In particular, during deployment, as described above, the surgeon or robot can deploy the stent in a desired configuration (desired position, desired diameter, desired angle, desired body lumen wall appointment, etc.) by performing reciprocal movements between the wire and catheter to exsheathe and reinsert the stent in anterior-posterior movements. The device deployment module 104 can provide such information to the surgeon or robot to achieve the desired deployment of the stent.

[0091] Additional indicators for improving and / or monitoring deployment accuracy can take the form of preferred catheter tip track guidelines, as described above. A preferred catheter tip track guideline is a 3D curve that follows a preferred intraluminal path traversing a body lumen segment where the stent should be deployed, and further indicates the radial deviation from the body lumen centerline that the catheter should preferably follow during stent deployment. For example, this track guideline can correspond to the path the catheter would follow if the catheter-wire manipulation strictly adhered to the aforementioned deployment ratio. Conversely, the catheter-wire deployment ratio can be derived from the anticipated device manipulation required to follow the track guideline.

[0092] Generally, the deviation of a track guideline from the centerline at a given point along the centerline lies on or near the radius vector of the tactile circle associated with that point. Figure 7A shows, in an exemplary embodiment, the radius vector 322 of the tactile circle 324 at a given point 326, and the outward deviation vector 328 pointing outward from the center of the tactile circle 324.

[0093] The degree and direction of radial deviation are determined, but are not limited to, by analytical / geometric algorithms or machine learning models that incorporate geometric and clinical factors such as stent type, body lumen diameter, centerline curvature at that point, location and size of aneurysm orifice, clinical preference for increased or decreased stent mesh density at a particular location, and axial proximity to the presence and / or start and end of the side branch body lumen.

[0094] Furthermore, the device deployment module 104 can update its optimal stent deployment plan and therefore modify the track guidelines during stent deployment to accommodate changes in the calculated landing zone location and its proximity to or overlap with the no-termination zone. If the device deployment module 104 cannot find a deployment plan that avoids the no-termination zone, it can alert the surgeon that the landing zone and the no-termination zone coincide, and that stent resheathing and repositioning are recommended.

[0095] Before final deployment or after complete deployment, the device deployment module 104 can generate a graph 314 for post-deployment analysis. Graph 314 is similar to graph 304 but shows the actual placement of the stent rather than the predicted placement. As shown, graph 314 can point out, highlight, or emphasize areas where the appointment is not optimal (e.g., where the diameter of the body lumen is larger than the respective diameters of the stent in such areas, or vice versa), such as region 316. Region 316 may correspond, for example, to a portion 246 of the ellipse 244 shown in Figure 4 above.

[0096] Graph 314 also points out that in the final deployment of the stent, the stent is shorter than expected, as indicated by the unoccupied regions such as the proximal region 318 and the distal region 320 of the stent. The surgeon can then determine whether such deployment is acceptable or whether redeployment is necessary if final deployment has not yet been performed.

[0097] In this example, the device deployment module 104 may include using a neural network model that can be trained to accurately locate various markers observed during stent deployment, such as distal, proximal, and resheathing markers on the delivery wire, microcatheter, and intermediate catheter markers.

[0098] Figure 8 shows images augmented with various markers according to an exemplary embodiment. In Figure 8, the image 400 on the left shows a raw image from a video feed, which is then input to the device deployment module 104. Thus, the image represents the input frame to the device deployment module 104. Image 400 shows the body lumen 402 and wire 404.

[0099] The device deployment module 104 is configured to generate an extended image 406 with markers overlaid on image 400 (input frame). For example, as shown by legend 408, the device deployment module 104 marks the re-sheathing position 410, the distal marker 412 of the delivery wire, the position of the microcatheter marker 414, the intermediate catheter marker 416, and the proximal marker 418 of the delivery wire.

[0100] Knowledge of the positions of these markers helps in the localization and orientation recognition / registration of the entire catheter device, as well as in evaluating any inconsistencies in a given deployment. The positions of the markers can be displayed in an enlarged image 406 to help the surgeon understand the location of the main components of the catheter.

[0101] The machine learning model (such as a neural network model) of the device deployment module 104 can be directly trained using pre-recorded videos of body lumen procedures in a supervised or semi-supervised manner, in which case markers and other key points can be labeled by a human expert to train the model and predict them. The neural network model can accurately predict marker positions using a sequence of past frames rather than a single (most recent) frame, helping to overcome information loss from occluded fields of view in a single frame.

[0102] Furthermore, post-processing of model predictions, such as utilizing a fixed relative order between subsets of markers, can enhance localization and help avoid false positives and duplicates. Several types of neural network models, such as convolutional neural networks with regression heads, can be used to predict target locations. For example, models similar to CenterNet or models from the Transformer architecture family can be used.

[0103] Another neural network model can be trained to predict accurate pixel-level masks of various components of a deployment system, such as flow diverter devices (e.g., stents) and guide / delivery wires. A "mask" outlines the shape of an object identified in an image. This model can also take a sequence of past frames and predict the mask for the most recent frame. Since accurate prediction along the boundaries of a deployed stent is important, this model can be trained with a specific loss, such as a weighted loss, to better learn the stent boundaries. Such a weighted loss imposes a greater penalty on the network for mispredictions around body lumen boundaries during training, for example.

[0104] Figure 9 shows the generation of wire and deployed stent masks using a trained model in an exemplary embodiment. Figure 9 shows image 500, representing the input frames to the model. Image 502 in the center shows the model prediction, showing the delivery wire mask 504 and deployed stent mask 506 identified by the trained model. Image 508 on the right shows the ground truth (actual masks provided by direct observation or identified by X-ray images) of the delivery wire and deployed stent. As shown, the model output shown in image 502 is substantially accurate compared to the ground truth in image 508.

[0105] In one example, the device deployment module 104 can be trained on a dataset of low-resolution and high-resolution images including deployed stents, and a model can be further implemented to learn the structural characteristics of stents depending on the type of stent. Then, using such a model, the resolution and / or dynamic range of the X-ray images can be improved to predict a more accurate and visually enhanced version of the stent captured using a low-dose X-ray setting.

[0106] When trained to high accuracy, this model can also help the stent segmentation model output precise masks, particularly at the object / stent boundary, and can be useful in deriving secondary stent deployment characteristics, such as the distance between the deployed stent and the body lumen wall (i.e., appointment determination) and the stent opening angle, among many other properties. Image segmentation refers to, for example, annotating or assigning each pixel in an image to a single class or object. The output is a mask that outlines the shape of the object in the image.

[0107] The model can be enhanced by conditioning it with known (physical) stent properties, which helps the model better represent high-resolution output. Stent properties can be obtained from design files (including stent wire material properties, number of wires, and wire angles), 3D computer-aided design (CAD) models, or 3D (micro) CT scans of the stent.

[0108] Figure 10 shows a block diagram 600 representing a generative modeling workflow according to an exemplary embodiment. The block diagram can be implemented by or within the device deployment module 104.

[0109] The workflow is divided into a training phase and a test phase. In the training phase, in block 602, low-dose X-ray (low-resolution) images are fed into the model. In block 604, the model is also fed with generated models of devices (e.g., stents) and catheters (e.g., CAD models or 3D micro-CT scans provided by the manufacturer). Next, in block 606, high-dose X-ray (high-resolution) images are fed into the model.

[0110] Figure 11 shows an example of a low-dose X-ray image 700 on the left and a corresponding high-dose X-ray image 702 on the right. Thus, this model can be trained to identify / learn stents and their characteristics in low-dose images from catheter and stent models as well as high-resolution images.

[0111] High-dose X-rays relative to low-dose X-rays can be the relative energy levels in the X-ray tube settings. For example, in a radiation setting produced by an X-ray tube at 120 kVp, the low dose could be 50 milliampere-seconds (mAs) and the high dose 150 mAs. Another example of low-dose versus high-dose is 40 mAs versus 180 mAs, with 100 mAs as the standard dose. Thus, low-dose versus high-dose can be two relative settings in an imaging device. In another example, low-dose and high-dose X-rays can be accompanied by high and low imaging frame rates, respectively, when acquiring X-ray images.

[0112] Returning to Figure 10, during the test or inference phase, in block 608, low-dose images are fed into the model, and in block 610, devices (e.g., stents) and catheters are identified. In block 612, images corresponding to high doses are acquired, the model results are scored, and any necessary adjustments are made.

[0113] In this way, the model is trained to identify body lumens and device parameters using low-dose X-ray images (low-resolution images), thereby eliminating or reducing the need for high-dose X-ray imaging, which can be harmful to patients due to radiation exposure. Furthermore, using low-resolution images can improve computational efficiency and real-time processing of video streams.

[0114] Based on the extracted segmentation masks and marker positions identified by the model, the device deployment module 104 is configured to suggest various relevant indicators, such as the proximity (apposition) of the (deployed) device to the body lumen wall, candidate locations for initiating device deployment, and other risk scores relevant to clinical decision support.

[0115] Figure 12 is a block diagram of a computing device 800 according to an exemplary embodiment. The computing device 800 may represent or include any of the devices described above (e.g., the image capture device 102, the device deployment module 104, the display device 106, etc.).

[0116] The computing device 800 may have a processor 802, a communication interface 804, and data storage 806, each connected to a communication bus 812. The computing device 800 may also include hardware that enables communication within the computing device 800 and between the computing device 800 and other devices. The hardware may include, for example, a transmitter, a receiver, and an antenna.

[0117] The communication interface 804 may be a wireless interface and / or one or more wired interfaces that enable both short-range and long-range communication to one or more networks or one or more remote devices (for example, to enable communication with the communication bus 812). Such a wireless interface may provide communication via one or more wireless communication protocols, Bluetooth, Wi-Fi (e.g., the IEEE 802.11 protocol), Long-Term Evolution (LTE), cellular communication, Near Field Communication (NFC), and / or other wireless communication protocols. The wired interface may include an Ethernet interface, a CAN network interface, a USB interface, or a similar interface for communicating with a wired network via wire, twisted pair, coaxial cable, optical link, fiber optic link, or other physical connection.

[0118] The data storage 806 may include, or take the form of, one or more computer-readable storage media that can be read or accessed by the processor 802. The computer-readable storage media may include volatile and / or non-volatile storage components such as optical, magnetic, organic, or other memory or disk storage, which may be integrated in whole or in part with the processor 802. The data storage 806 is considered a non-temporary computer-readable medium. In some examples, the data storage 806 may be implemented using a single physical device (e.g., one optical, magnetic, organic, or other memory or disk storage unit), while in other examples, the data storage 806 may be implemented using two or more physical devices.

[0119] Therefore, the data storage 806 is a non-temporary computer-readable storage medium in which executable instructions 814 are stored. Executable instructions 814 contain computer-executable code. When executable instructions 814 are executed by the processor 802, the processor 802 is caused to perform operations on the computing device 800 (for example, operations performed by the image capture device 102, the device deployment module 104, or the display device 106).

[0120] The processor 802 may be a general-purpose processor or an application-specific processor (e.g., a digital signal processor, an application-specific integrated circuit (ASIC), etc.). The processor 802 can receive input from the communication interface 804, process the input, and generate output to be stored in the data storage 806. The processor 802 can be configured to execute executable instructions 814 (e.g., computer-readable program instructions) stored in the data storage 806 and can be executed to provide the functions of the computing device 800 described herein.

[0121] The computing device 800 may further include an output interface 808 for outputting information to other devices. If the computing device 800 represents a display device 106, the computing device 800 further includes a display 810. The output interface 808 also outputs information to the display 810 or other components. For this reason, the output interface 808 may be a wireless interface (e.g., a transmitter) or a wired interface. The processor 802 can receive input from the communication interface 804, process that input, and generate output to the display 810.

[0122] In another example, the output interface 808 can output information in electronic form to provide feedback or commands to the robot interface 816, which controls the delivery of the device via various control means, enabling the robot to change and manipulate the linear or rotational position of the device.

[0123] The output interface 808 can also simultaneously provide relevant information and feedback to face-to-face and remote proctors via a feedback mechanism 818 (e.g., a visual or audiovisual feedback mechanism). The feedback mechanism 818 may include a display, an augmentation device, a hologram device, a virtual / augmented reality wearable or headset, an on-screen visual dashboard, voice, a wearable for voice feedback, a haptic sensing device, and similar sensory feedback mechanisms.

[0124] Figure 13 is a flowchart of Method 900 for extending a real-time intraoperative radiographic video feed with anatomical and device-related overlays and metrics, according to an exemplary embodiment. Method 900 can be performed, for example, by a device deployment module 104.

[0125] Method 900 may include one or more operations or actions represented by one or more of blocks 902-910, 1000, 1100, 1200, 1300, 1400, 1500, 1600, 1700, and 1800-1806. Although the blocks are illustrated in a sequential order, these blocks may be executed in parallel in some cases and / or in an order different from the order described herein. Furthermore, the various blocks may be combined into fewer blocks, divided into additional blocks, and / or removed, based on a preferred embodiment.

[0126] In addition, flowcharts illustrating the operation of one possible embodiment of Method 900 and other processes and operations disclosed herein illustrate the operation of Method 900 and other processes and operations disclosed herein. In this regard, each block may represent a module, segment, or portion of program code containing one or more instructions executable by a processor for performing a particular logical operation or step in a process. The program code may be stored in any type of computer-readable medium or memory, such as a storage device including a disk or hard drive. The computer-readable medium may include non-temporary computer-readable medium or memory, such as computer-readable medium for storing short-term data, such as register memory, processor cache, and random access memory (RAM). The computer-readable medium may also include non-temporary medium or memory such as read-only memory (ROM), optical or magnetic disks, and secondary or persistent long-term storage such as compact disk read-only memory (CD-ROM). The computer-readable medium may also be any other volatile or non-volatile storage system. The computer-readable medium may be considered, for example, a computer-readable storage medium, a tangible storage device, or other manufactured product. In addition, for Method 900 and other processes and operations disclosed herein, one or more blocks in Figure 12 may represent circuits or digital logic configured to perform specific logical operations within the process.

[0127] In block 902, method 900 includes receiving in real time a video stream of an intervention procedure captured by an image acquisition device 102 of a body lumen and a device deployed within that body lumen in a processor (e.g., a processor 802 of a device deployment module 104).

[0128] In block 904, method 900 includes the processor identifying a body lumen and a device deployed within the body lumen in a video stream.

[0129] For example, as shown in Figures 3 and 5, in block 906, method 900 includes the processor visually overlaying body lumen markers, which represent the characteristics of body lumen, onto the video stream in real time. In one example, the user may have the ability to turn on and off the overlaying of body lumen markers or any other markers on the display at the user's discretion via a user interface (e.g., using graphical user interface items such as menus and buttons).

[0130] For example, as shown in Figures 2, 4, 6, and 8, in block 908, method 900 includes the processor visually presenting, in real time on the video stream, a display of device markers indicating the location of the device in real time during deployment.

[0131] For example, as shown in Figures 4 and 7, in block 910, method 900 includes the processor providing a visual indicator of device parameters, including the device appointment, on a video stream in real time.

[0132] Figure 14 is a flowchart of additional actions that can be performed with Method 900 according to an exemplary embodiment. Body lumen markers can include curvature markings. For example, as shown in curvature markings 228-230 in Figure 3 and curvature markings 254, 256 in Figure 5, in block 1000, the action includes visually marking one or more locations of a body lumen where the curvature of the body lumen exceeds a threshold curvature to indicate the degree of tortuosity of the body lumen to the surgeon performing the interventional procedure.

[0133] Figure 15 is a flowchart of additional actions that can be performed with Method 900 according to an exemplary embodiment. The body lumen markers may include start-do not and / or end-do not zones to be avoided during device deployment. In block 1110, the action includes visually marking one or more sites having bifurcations or hidden branches of a body lumen to inform the surgeon that one or more sites should be avoided as locations where the ends of the device should be deployed during device deployment.

[0134] Figure 16 is a flowchart of additional operations that can be performed with Method 900 according to an exemplary embodiment. For example, as shown in Figures 2 and 8, in block 1200, the operation includes the processor visually marking in real time on the video stream markers indicating the catheter and wires used to deploy the device.

[0135] Figure 17 is a flowchart of additional operations that can be performed with Method 900 according to an exemplary embodiment. For example, in block 1300, as shown in Figure 6, the operation includes visually presenting indicators for the proximal and distal ends of the device.

[0136] Figure 18 is a flowchart of additional operations that can be performed with Method 900 according to an exemplary embodiment. For example, as shown in Figure 6, in block 1400, the operation includes the processor visually displaying in real time a graphical representation of landing probabilities on the video stream, indicating the probability that the proximal end of the device will be positioned at a particular location upon completion of deployment. As described above, in one example, the processor determines the probabilities taking into account the shortening effect to determine the final length of the device.

[0137] Figure 19 is a flowchart of additional operations that can be performed with Method 900 according to an exemplary embodiment. The processor has access to a three-dimensional (3D) model of the device in 3D space (for example, as described above with respect to block 204). For example, in block 1500, as shown in Figures 2, 4, and 6, the operation includes visually presenting a display of device markers that indicate the position of the device in deployment in real time, based on the 3D model of the device.

[0138] Figure 20 is a flowchart of additional operations that can be performed with method 900 according to an exemplary embodiment. The processor has access to a 3D body lumen model of the body lumen generated from images acquired before the interventional procedure via an image acquisition device (for example, as described above with respect to block 220). For example, in block 1600, as shown in Figures 3 and 5, the operation includes visually superimposing body lumen markers that indicate the characteristics of the body lumen based on the 3D body lumen model.

[0139] Figure 21 is a flowchart of additional actions that can be performed with Method 900 according to an exemplary embodiment. In block 1700, the action includes the processor visually providing the surgeon performing the intervention with a score indicating the quality of device deployment, the score taking into account one or more of the following: device appointment, proximity or overlap of the distal end of the device to the no-start zone or "start here" zone, recognition and / or prediction of coning angle, cone shape, braid angle, and deviation from the centerline of the body lumen.

[0140] Figure 22 is a flowchart of additional operations that can be performed with Method 900 according to an exemplary embodiment. In one example, as described above with respect to Figures 10-11, the video stream includes a feed of low-dose X-ray images, and the processor includes a neural network model trained to identify devices and body lumens using the low-dose X-ray images.

[0141] In particular, in block 1800, the operation includes receiving a low-dose X-ray image (block 602 and low-dose X-ray image 700) showing a given device and a given body lumen. In block 1802, the operation includes receiving a high-dose X-ray image (block 606 and high-dose X-ray image 702) corresponding to the low-dose X-ray image. In block 1804, the operation includes receiving a 3D model of the device (block 604). In block 1806, the operation includes identifying the device in the low-dose X-ray image using the high-dose X-ray image and the 3D model of the device.

[0142] The above detailed description refers to various features and operations of the disclosed system with reference to the accompanying drawings. The exemplary embodiments described herein are not intended to be limiting. Certain aspects of the disclosed system can be arranged and combined in a wide variety of different configurations, all of which are contemplated herein.

[0143] Furthermore, unless the context suggests otherwise, the features shown in each figure can be used in combination with each other. Therefore, it should be understood that the figures should be viewed generally as constituent aspects of one or more overall embodiments, and not all illustrated features are necessarily required for each embodiment.

[0144] In addition, any enumeration of elements, blocks, or steps in this specification or claims is for clarity only. Therefore, such enumeration should not be construed as requiring or implying that these elements, blocks, or steps follow a particular sequence or are performed in a particular order.

[0145] Furthermore, a device or system may be used or configured to perform the functions shown in the diagram. In some cases, components of a device and / or system may be configured to perform a function in such a way that those components are actually configured and structured (using hardware and / or software) to enable such a function. In other cases, components of a device and / or system may be arranged in such a way that they are adapted, executable, or suitable to perform that function when operated in a particular manner.

[0146] The terms "substantially" or "about" mean that the characteristic, parameter, or value mentioned does not need to be exactly realized, and that deviations or variations, including, for example, tolerances, measurement errors, limitations of measurement accuracy, and other factors known to those skilled in the art, may occur to the extent that they do not preclude the effect that the characteristic is intended to produce.

[0147] The configurations described herein are for illustrative purposes only. Those skilled in the art will understand that other configurations and elements (e.g., machines, interfaces, actions, sequences of actions, and groupings) can be used instead, and that some elements may be omitted entirely depending on the desired outcome. Furthermore, many of the elements described herein are functional entities and can be implemented as discrete or distributed components, or in combination with other components, in any suitable combination and location.

[0148] While various aspects and embodiments are disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for illustrative purposes only and are not intended to limit, and the true scope is indicated by the following claims, along with the full scope of equivalents of such claims. Furthermore, the terms used herein are used solely for the purpose of describing specific embodiments and are not intended to limit.

[0149] Therefore, embodiments of this disclosure may relate to one of the exemplary embodiments (EEEs) listed below.

[0150] EEE1 is a method comprising: a processor receiving in real time a video stream of intervention procedures of body lumens and devices deployed within body lumens, captured by an image acquisition device; the processor identifying body lumens and devices deployed within body lumens in the video stream; the processor visually superimposing body lumen markers indicating the characteristics of body lumens onto the video stream in real time; the processor visually presenting in real time a display of device markers indicating the position of the deployed device on the video stream; and the processor providing in real time visual indicators of device parameters on the video stream.

[0151] EEE2 is a method of EEE1, wherein the body lumen marker includes curvature marking, size and location of the body lumen, and visually superimposing the body lumen marker including curvature marking includes visually marking one or more locations of the body lumen where the curvature of the body lumen exceeds a threshold curvature in order to indicate the degree of tortuosity of the body lumen to the surgeon performing the interventional procedure.

[0152] EEE3 is a method of EEE1 or 2, wherein the body lumen marker includes a start no-go zone and / or end no-go zone to be avoided when deploying the device, and visually superimposing the body lumen marker includes visually marking one or more sites having bifurcations or hidden branches of the body lumen to inform the surgeon that one or more sites should be avoided as locations where the ends of the device should be deployed during device deployment.

[0153] EEE4 is one of the methods EEE1-3 and further includes the processor visually marking the video stream in real time with markers indicating the catheter and wires used to deploy the device.

[0154] EEE5 is the same as EEE4, but further includes visually presenting information indicating whether the catheter is following preferred catheter track guidelines.

[0155] EEE6 is one of the methods of EEE1-5, in which the processor visually presents the display of device markers on the video stream in real time, including visually presenting indicators for the proximal and distal ends of the device.

[0156] EEE7 is a method of EEE6 that further includes the processor visually displaying in real time a graphical representation of landing probabilities on the video stream, indicating the probability that the proximal end of the device will be positioned at a specific location upon completion of deployment.

[0157] EEE8 is a method of EEE7 that takes into account the shortening effect to determine the final length of the device, in order to determine the probability.

[0158] EEE9 is one of the methods from EEE1 to EEE8, and the processor has access to a 3D model of the device in three-dimensional (3D) space and visually presents a display of device markers that indicate the position of the device in deployment in real time, based on the 3D model of the device.

[0159] EEE10 is one of the methods EEE1-9, in which the processor has access to a 3D body lumen model of the body lumen generated from images previously acquired via an image acquisition device, and visually superimposes body lumen markers that indicate the characteristics of the body lumen based on the 3D body lumen model.

[0160] EEE11 is one of the methods of EEE1-10, wherein a processor generates a score indicating the quality of device deployment, the score taking into account one or more of the following: the appointment of the device, the proximity or overlap of the distal end of the device to a no-start zone or "start here" zone, the recognition and / or prediction of the coning angle, cone shape, braid angle and braid density, and the deviation of the body lumen from a defined path including the centerline of the ideal deployment path; and further comprising providing feedback indicating the score to (i) a user via audio / video feedback, a virtual reality or augmented reality display, or (ii) a robot interface controlling the delivery of the device, enabling the robot to change the linear or rotational position of the device.

[0161] EEE12 is one of the methods EEE1-11, where the video stream includes a feed of low-dose X-ray images, and the processor includes a neural network model trained to identify devices and body lumens using the low-dose X-ray images.

[0162] EEE13 is a method of EEE12 in which a neural network model is trained by receiving low-dose X-ray images showing a given device and a given body lumen, receiving high-dose X-ray images corresponding to the low-dose X-ray images, receiving a 3D model of the device, and identifying the device in the low-dose X-ray images using the high-dose X-ray images and the 3D model of the device.

[0163] EEE14 is a system comprising: an image capture device configured to capture a video stream of an intervention procedure, including body lumens and devices deployed within them, in real time; a display device configured to communicate with the image capture device and display the video stream; and a device deployment module that communicates with the image capture device and the display device. The device deployment module comprises a processor and a non-temporary computer-readable medium storing a plurality of executable instructions. When an instruction is executed by the processor, it causes the device deployment module to perform an operation, which includes any of the operations of EEE1 to EEE13. For example, an operation may include: receiving a video stream; identifying body lumens and devices deployed within them in the video stream; visually superimposing body lumen markers indicating the characteristics of the body lumens onto the video stream displayed in real time on the display device; visually presenting a display of device markers indicating the position of the deployed device in real time on the video stream displayed on the display device; and providing visual indicators of device parameters on the video stream displayed on the display device.

[0164] EEE15 is a system of EEE14, and the body lumen markers include curvature markings, start no-go zones and / or end no-go zones to be avoided when deploying the device, and visually superimposing the body lumen markers including curvature markings to visually mark one or more locations of the body lumen where the curvature of the body lumen exceeds a threshold curvature in order to indicate the degree of tortuosity of the body lumen to the surgeon performing the intervention, and visually marking each of the one or more locations of the body lumen that have bifurcations or hidden branches to inform the surgeon that each of the one or more locations should be avoided as a location where the proximal end of the device should be deployed during device deployment.

[0165] EEE16 is a system of EEE14 or 15 that visually presents a real-time display of device markers on a video stream, visually presenting indicators of the proximal and distal ends of the device, and visually presenting a real-time graphical display of landing probabilities on the video stream, indicating the probability that the proximal end of the device will be positioned at a particular location upon completion of deployment, wherein determining the probabilities takes into account shortening effects to determine the final length of the device.

[0166] EEE17 is one of the systems of EEE14-16, and the device deployment module has access to (i) a 3D model of the device in three-dimensional (3D) space, and (ii) a 3D body lumen model of the body lumen generated from images previously acquired via an image acquisition device, and visually presents a display of device markers that show the position of the device during deployment in real time, based on the 3D model of the device, and visually superimposes body lumen markers that show the characteristics of the body lumen, based on the 3D body lumen model.

[0167] EEE18 is a system of any of EEE14-17, and its operation is to generate a score indicating the quality of device deployment, the score taking into account one or more of the following: the appointment of the device, the proximity or overlap of the distal end of the device to a no-start zone or "start here" zone, the recognition and / or prediction of the coning angle, cone shape, braid angle and braid density, and the deviation of the body lumen from a defined path including the centerline of the ideal deployment path, and further comprising providing feedback indicating the score to (i) a user via audio / video feedback, virtual reality or augmented reality display, or (ii) a robotic interface controlling the delivery of the device, enabling the robot to change the linear or rotational position of the device.

[0168] EEE19 is one of the EEE14-18 systems, where the video stream includes a feed of low-dose X-ray images, and the device deployment module includes a neural network model trained to identify devices and body lumens using low-dose X-ray images.

[0169] EEE20 is a system based on EEE19, and its neural network model is trained by receiving low-dose X-ray images showing a given device and a given body lumen, receiving high-dose X-ray images corresponding to the low-dose X-ray images, receiving a 3D model of the device, and identifying the device in the low-dose X-ray images using the high-dose X-ray images and the 3D model of the device.

[0170] EEE21 is a method comprising: a processor receiving in real time a video stream of intervention procedures of body lumens and devices deployed within body lumens, captured by an image acquisition device; the processor identifying body lumens and devices deployed within body lumens in the video stream; the processor visually presenting body lumen markers indicating the characteristics of the body lumens; the processor visually presenting a display of device markers indicating the location of the deployed device in real time; and the processor providing visual indicators of device parameters on the video stream in real time.

[0171] EEE22 is a method of EEE21, further comprising generating a representation of an annular ring indicating the wall of a body lumen.

[0172] The method of EEE21 may also include any of the other operations or steps of EEE1-13.

[0173] EEE23 is a method comprising: a processor receiving in real time a video stream of intervention procedures of body lumens and devices deployed within body lumens, captured by an image acquisition device; the processor identifying body lumens and devices deployed within body lumens in the video stream; the processor visually presenting a display of device markers indicating the location of the device in the body lumen during deployment in real time; and the processor providing visual indicators of device parameters on the video stream in real time.

[0174] EEE24 is a method of EEE23, further comprising a processor visually presenting body lumen markers that represent the characteristics of body lumen.

[0175] EEE25 is a method of EEE24 in which the visual presentation of body lumen markers indicating the characteristics of body lumen involves a processor visually overlaying body lumen markers indicating the characteristics of body lumen onto a video stream in real time.

[0176] The method of EEE23 may also include any of the other operations or steps of EEE1-13.

[0177] An EEE14 system can also perform any of the operations from EEE21-22 and E23-25.

Claims

1. It is a method, The processor receives in real time a video stream of the body lumen and the intervention procedures of the device deployed within the body lumen, which have been captured by the image acquisition device. The processor identifies the body cavity and the device deployed within the body cavity in the video stream, The processor visually superimposes body lumen markers, which represent the characteristics of the body lumen, onto the video stream in real time. The processor provides a real-time visual display of a device marker indicating the position of the device being deployed on the video stream, The processor provides a real-time visual indicator of the device's parameters on the video stream, Methods that include...

2. The body lumen marker includes curvature markings, size, and position of the body lumen, and visually superimposing the body lumen marker including the curvature markings is To indicate the degree of tortuosity of the body lumen to the surgeon performing the intervention procedure, one or more locations in the body lumen where the curvature exceeds the threshold curvature shall be visually marked. The method according to claim 1, including the method described in claim 1.

3. The body lumen marker includes a start-prevention zone and / or end-prevention zone to be avoided when deploying the device, and visually superimposing the body lumen marker is Visually mark one or more locations in the body lumen that have bifurcations or hidden branches, to inform the surgeon that these locations should be avoided during the deployment of the device. The method according to claim 1, including the method described in claim 1.

4. The method according to claim 1, further comprising the processor visually marking the video stream in real time markers indicating catheters and wires used to deploy the device.

5. The method according to claim 4, further comprising visually presenting information indicating whether the catheter conforms to preferred catheter track guidelines.

6. The processor visually presents the display of the device marker on the video stream in real time. Visually display the indicators at the proximal and distal ends of the device. The method according to claim 1, including the method described in claim 1.

7. The method according to claim 6, further comprising the processor visually displaying in real time on the video stream a graphical representation of landing probabilities indicating the probability that the proximal end of the device will be positioned at a specific location upon completion of deployment.

8. The method according to claim 7, wherein determining the probability takes into account shortening effects to determine the final length of the device.

9. The method according to claim 1, wherein the processor has access to a 3D model of the device in a three-dimensional (3D) space and visually presents the display of the device marker indicating the position of the device in deployment in real time, based on the 3D model of the device.

10. The method according to claim 1, wherein the processor has access to a 3D body lumen model of the body lumen generated from images previously acquired via the image acquisition device, and visually superimposes the body lumen markers indicating the characteristics of the body lumen based on the 3D body lumen model.

11. The processor generates a score indicating the quality of the deployment of the device, the score taking into account one or more of the following: the appointment of the device, the proximity or overlap of the distal end of the device to a no-start zone or "start here" zone, the recognition and / or prediction of the coning angle, cone shape, braiding angle and braiding density, and the deviation of the body lumen from a defined path including the centerline of the ideal deployment path. The feedback indicating the score is provided (i) to the user via audio / video feedback, virtual reality or augmented reality display, or (ii) to a robot interface that controls the delivery of the device, enabling the robot to change the linear or rotational position of the device. The method according to claim 1, further comprising:

12. The method according to claim 1, wherein the video stream includes a feed of low-dose X-ray images, and the processor includes a neural network model trained to identify the device and the body lumen using the low-dose X-ray images.

13. The aforementioned neural network model is Receiving low-dose X-ray images showing a given device and a given body lumen, Receiving a high-dose X-ray image corresponding to the low-dose X-ray image, Receiving a 3D model of the aforementioned device, Using the high-dose X-ray image and the 3D model of the device, the device is identified in the low-dose X-ray image. The method according to claim 12, which is trained by

14. It is a system, An image capture device configured to capture in real time a video stream of an intervention procedure including a body lumen and a device deployed within the body lumen, A display device configured to communicate with the image capture device and display the video stream, A device deployment module that communicates with the image capture device and the display device, wherein the device deployment module comprises a processor and a non-temporary computer-readable medium storing a plurality of executable instructions, and when the instructions are executed by the processor, they are sent to the device deployment module, Receiving the aforementioned video stream, In the video stream, the body lumen and the devices deployed within the body lumen are identified, Visually superimposing body lumen markers indicating the characteristics of the body lumen onto the video stream displayed in real time on the display device, On the video stream displayed on the display device, a device marker indicating the position of the device being deployed in real time is visually presented; To provide a visual indicator of the device's parameters on the video stream displayed on the display device, A device deployment module that performs operations including, A system equipped with these features.

15. The body lumen marker includes curvature markings, a start-restricted zone and / or end-restricted zone to be avoided when deploying the device, and visually superimposing the body lumen marker including the curvature markings is To indicate the degree of tortuosity of the body lumen to the surgeon performing the intervention, one or more locations of the body lumen where the curvature exceeds the threshold curvature are visually marked. Visually marking one or more locations of the body lumen that have bifurcations or hidden branches to inform the surgeon that, during deployment of the device, each of these one or more locations should be avoided as a site where the proximal end of the device should be deployed. The method according to claim 14, including the method described in claim 14.

16. To visually present the display of the device marker on the video stream in real time is to Visually displaying indicators at the proximal and distal ends of the device, The method involves visually displaying, in real time, a graphical representation of the landing probability on the video stream, indicating the probability that the proximal end of the device will be positioned at a specific location upon completion of deployment, wherein determining the probability involves taking into account the shortening effect in determining the final length of the device. The system according to claim 14, including the system described in claim 14.

17. The device deployment module has access to (i) a 3D model of the device in three-dimensional (3D) space, and (ii) a 3D body lumen model of the body lumen generated from images previously collected via the image acquisition device. Visually presenting the display of a device marker that indicates the position of the device in deployment in real time is based on the 3D model of the device, The system according to claim 14, which visually superimposes the body lumen markers that indicate the characteristics of the body lumen, based on the 3D body lumen model.

18. The aforementioned operation is, To generate a score indicating the quality of the deployment of the device, the score taking into account one or more of the appointment of the device, the proximity or overlap of the distal end of the device to a no-start zone or "start here" zone, the recognition and / or prediction of the coning angle, cone shape, braiding angle and braiding density, and the deviation of the body lumen from a defined path including the centerline of the ideal deployment path, The feedback indicating the score is provided (i) to the user via audio / video feedback, virtual reality or augmented reality display, or (ii) to a robot interface that controls the delivery of the device, enabling the robot to change the linear or rotational position of the device. The system according to claim 14, further comprising:

19. The system according to claim 14, wherein the video stream includes a feed of low-dose X-ray images, and the device deployment module includes a neural network model trained to identify the device and the body lumen using the low-dose X-ray images.

20. The aforementioned neural network model is Receiving low-dose X-ray images showing a given device and a given body lumen, Receiving a high-dose X-ray image corresponding to the low-dose X-ray image, Receiving a 3D model of the aforementioned device, Using the high-dose X-ray image and the 3D model of the device, the device is identified in the low-dose X-ray image. The system according to claim 19, which is trained by

21. It is a method, The processor receives in real time a video stream of the body lumen and the intervention procedures of the device deployed within the body lumen, which have been captured by the image acquisition device. The processor identifies the body cavity and the device deployed within the body cavity in the video stream, The processor visually displays body lumen markers that indicate the characteristics of the body lumen, The processor visually displays a device marker indicating the position of the device being deployed in real time, The processor provides a real-time visual indicator of the device's parameters on the video stream, Methods that include...

22. The method according to claim 21, further comprising generating a representation of an annular ring indicating the wall of the body lumen.

23. It is a method, The processor receives in real time a video stream of the body lumen and the intervention procedures of the device deployed within the body lumen, which have been captured by the image acquisition device. The processor identifies the body cavity and the device deployed within the body cavity in the video stream, The processor visually displays a device marker indicating the position of the device within the body lumen during deployment in real time. The processor provides a real-time visual indicator of the device's parameters on the video stream, Methods that include...

24. The method according to claim 23, further comprising visually presenting a body lumen marker indicating the characteristics of the body lumen using the processor.

25. Visually presenting the body lumen markers that indicate the characteristics of the body lumen is, The method according to claim 24, comprising the processor visually superimposing the body lumen markers, which represent the characteristics of the body lumen, onto the video stream in real time.

Images