Method and system for enhancing real-time intraoperative x-ray video feed with anatomical and device-related overlay and metrics

Abstract

An example method includes receiving, in real time, a body cavity captured by an image capture device and a video stream of an interventional procedure of a device being deployed within the body cavity; identifying in the video stream a body cavity and a device being deployed within the body cavity; visually presenting a body cavity marker indicative of a body cavity feature, the body cavity feature including a curvature marker and a non-starting region to be avoided when deploying the device; visually presenting a display of device indicia indicative of the location of the device in real time during deployment; and providing a visual indicator of the device parameter on the video stream in real time.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 488,655, filed on March 6, 2023, the entire contents of which are incorporated herein by reference as if fully set forth in this specification. Background Art

[0003] X-ray angiography facilitates visualization of the path of a body lumen (e.g., a blood vessel) and the region of interest during body lumen (e.g., intravascular) procedures. As X-ray angiography remains one of the primary components of successful body lumen interventions, real-time clinical decision making relies heavily on the surgeon's perception and interpretation of the angiographic feed. This decision making is a time-sensitive, multidimensional process that requires real-time, multidimensional approaches that utilize two-dimensional (2D) and three-dimensional (3D) modalities to reliably discern the multiple aspects of such interventional procedures and to provide complementary visualization of the body lumen and the deployment site of devices such as low-conductivity stents, auxiliary stents, intrasaccular devices, coils, other embolic materials (e.g., beads, liquids, particles, etc.).

[0004] It is with respect to these and other considerations that the disclosure herein is made. Summary of the Invention

[0005] In various examples, described herein are methods and systems for enhancing real-time intraoperative X-ray video feeds with anatomy, 3D rendering, and device-related overlays and indicators.

[0006] In additional examples described herein, methods and systems are disclosed involving enhancing a real-time intraoperative X-ray video feed with several anatomical, catheter, 3D rendering, and device-related overlays while adding predictive deployment-related guidance or indicators for the operating surgeon or robotic system.

[0007] The features, functions, and advantages discussed above can be achieved independently in various examples or in combination in yet other examples. More details of the examples can be found in the following description and accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The novel features believed characteristic of the illustrative examples are set forth in the appended claims.However, the exemplary examples together with the preferred mode of use, further objects and description thereof will be best understood from the following detailed description of exemplary examples of the present disclosure read in conjunction with the accompanying drawings.

[0009] Figure 1 is a block diagram of a system including an image capture device, a device deployment module, and a display device according to an example embodiment.

[0010] Figure 1Ais a block diagram illustrating operations performed by a device deployment module according to an example embodiment.

[0011] Figure 2 A representative embodiment according to an example embodiment is shown Figure 1 Block diagram of the system.

[0012] Figure 3 A representative embodiment according to an example embodiment is shown Figure 1 Another block diagram of the system and showing details of body cavity recognition.

[0013] Figure 3A An adherence mismatch of a device deployed within a body lumen is shown according to an example embodiment.

[0014] Figure 4 A representative embodiment according to an example embodiment is shown Figure 1 A block diagram of a system showing details associated with identified parameters of a device.

[0015] Figure 4A Determining the cone angle among other features of the device according to example embodiments is shown.

[0016] Figure 5 A representative embodiment according to an example embodiment is shown Figure 1 Another block diagram of the system and showing details of body cavity identification.

[0017] Figure 5A Shown is a superimposition of a centerline and lumen wall circumferential rings within a body lumen according to an example embodiment.

[0018] Figure 5B A preferred catheter trajectory guideline is shown offset from a body lumen centerline according to an example embodiment.

[0019] Figure 5C A device is shown deployed within a body cavity according to an example embodiment.

[0020] Figure 5D An example of body cavity model registration between 2D and 3D according to an example embodiment is shown.

[0021] Figure 6 A representative embodiment according to an example embodiment is shown Figure 1 A block diagram of the system showing a probability graph of the device landing area.

[0022] Figure 7 It is shown that according to an example embodiment, Figure 1 The linearized graphical information generated by the system's device deployment module is used to provide key information for body cavity intervention through real-time video streaming of X-ray angiography.

[0023] Figure 7A The radial vector of the osculating circle at a given point and the outward deviation vector pointing away from the center of the osculating circle are shown according to an example embodiment.

[0024] Figure 8 An image enhanced with various markers is shown according to an example embodiment.

[0025] Figure 9 The generation of masks of wires and deployed devices using a trained model is shown according to an example embodiment.

[0026] Figure 10 Shown is a block diagram representing a generative modeling workflow according to an example implementation.

[0027] Figure 11 A low-dose X-ray image and a corresponding high-dose X-ray image according to an example are shown.

[0028] Figure 12 is a block diagram of a computing device according to an example embodiment.

[0029] Figure 13 is a flow chart of a method for enhancing a real-time intraoperative X-ray video feed with anatomical and device-related overlays and indicators, according to an example embodiment.

[0030] Figure 14 According to an example embodiment, Figure 13 A flowchart of additional operations that may be performed by the method.

[0031] Figure 15 According to an example embodiment, Figure 13 A flowchart of additional operations that may be performed by the method.

[0032] Figure 16 According to an example embodiment, Figure 13 A flowchart of additional operations that may be performed by the method.

[0033] Figure 17 According to an example embodiment, Figure 13 A flowchart of additional operations that may be performed by the method.

[0034] Figure 18 According to an example embodiment, Figure 13 A flowchart of additional operations that may be performed by the method.

[0035] Figure 19 According to an example embodiment, Figure 13 A flowchart of additional operations that may be performed by the method.

[0036] Figure 20 According to an example embodiment, Figure 13A flowchart of additional operations that may be performed by the method.

[0037] Figure 21 According to an example embodiment, Figure 13 A flowchart of additional operations that may be performed by the method.

[0038] Figure 22 According to an example embodiment, Figure 13 A flowchart of additional operations that may be performed by the method. DETAILED DESCRIPTION

[0039] The systems and methods disclosed herein relate to: capturing a real-time angiographic video feed during a body lumen (e.g., intravascular) interventional procedure (e.g., placement of a device (e.g., a flow diversion device)); analyzing the real-time video feed; analyzing the real-time video and imaging feeds to determine the positioning of a catheter, wire (e.g., a guidewire), and device during deployment; visually presenting in real time (e.g., superimposed on the video or on a separate display) body lumen markers, device (e.g., stent) boundaries and features, deployment guidelines, and predictions regarding deployment of the device; providing quality assessment metrics related to deployment; and guiding the surgeon or communicating with a robotic system during the intervention.

[0040] As used herein, the term "body cavity" refers to a blood vessel, lymphatic vessel, bile duct, esophagus, trachea, or other body cavity. In addition, the term "device" is generally used to indicate a diverting stent, auxiliary stent, intracystic device, coil, or other embolic material (e.g., beads, liquids, particles, etc.).

[0041] Figure 1 is a block diagram of a system 100 including an image capture device 102, a device deployment module 104, and a display device 106, according to an example embodiment. The components of system 100 may be configured to operate in a manner interconnected with each other and / or with other components coupled to the corresponding system. One or more of the operations or components described in system 100 may be divided into additional operations or physical components, or combined into fewer operations or physical components. In some other examples, additional operations and / or physical components may be added to system 100. In addition, any component or module of system 100 may include or be provided in the form of a processor (e.g., a microprocessor, a digital signal processor, etc.) that is configured to execute program code including one or more instructions for implementing the logical operations described herein.

[0042] The system 100 may also include any type of computer-readable medium (non-transitory computer-readable medium) or memory, such as a storage device including a magnetic disk or hard disk, for storing program code that, when executed by one or more processors, causes the system 100 to perform the operations described herein. In examples, the system 100 may be included within other systems.

[0043] The image capture device 102 is configured to directly capture or read from an available data stream images and videos of a body cavity during an interventional procedure (e.g., deploying a stent such as a flow-directing device in a body cavity). For example, the image capture device 102 preferably includes a biplane angiographic X-ray system (e.g., the Azurion system from Philips Healthcare or the Artis system from Siemens Healthineers) or a computed tomography (CT) scanning device that combines a series of X-ray images taken from different angles around the patient's body and uses computer processing to generate cross-sectional images (slices) of the body cavity.

[0044] Alternatively, the image capture device may be a data collection device that extracts available imaging information from a separate angiography or CT device.

[0045] In an example, image capture device 102 may include a micro-CT scanner, which uses X-rays to view the patient's body slice by slice using three-dimensional (3D) imaging technology. Micro-CT scanning is similar to CT scanning imaging, but on a smaller scale and with higher resolution. For example, body cavities can be imaged with pixel sizes as small as 100 nanometers, and objects as large as 200 mm in diameter can be scanned.

[0046] Thus, in an example, image capture device 102 may include an X-ray source that generates X-rays that are then transmitted through a portion of a patient's body cavity of interest. Image capture device 102 may also include an X-ray detector that records the X-rays as a 2D projection image. The X-ray source may then be rotated a fraction of a degree on a rotating platform, and another X-ray projection image captured. This process may be repeated over 180 or 360 degrees to capture images of the body cavity from different angles.

[0047] In an example, the image capture device 102 can also generate a real-time video of the body cavity. For example, the image capture device 102 may include at least one camera (e.g., one camera or two cameras in a dual-plane setup) that is deployed in the patient's body cavity along with the body cavity device, and the camera generates real-time feedback of the body cavity and deployment. In another example, the image capture device 102 includes an external imaging device (CT scanning device) that generates a real-time video feed of the body cavity and the device being deployed in the body cavity (e.g., a stent being deployed via a catheter and wire). The image capture device 102 can be configured to extract still images from the video. The image capture device 102 is then configured to provide such video feed and / or images to a display device 106 for display thereon. An example of such a device is a catheterization laboratory for neurological, cardiac, and peripheral body cavity interventions.

[0048] The device deployment module 104 is configured to receive such video and is configured to analyze the video and analyze the video to determine the location of the catheter, wire and device in real time during device deployment. The device deployment module 104 then transmits this information to a display device 106, which visually presents the information superimposed on the video or image to the healthcare professional. Specifically, the device deployment module 104 can visually superimpose device (e.g., stent) and body cavity markings, guidance and instructions, and predictions about device deployment on the images and videos on the display device 106, and can provide quality assessment indicators (e.g., scores) related to deployment on the display device 106. This visual information and indicators can provide guidance to the surgeon or instruct the robotic system to adjust the deployment technology to enhance the deployment of the device and achieve the desired results.

[0049] Thus, the device deployment module 104 is configured to receive a digital video feed from an imaging system (e.g., an X-ray angiography and CT scanning device) during an interventional procedure in order to display information to a physician or operator in real time during the procedure to assist the physician or operator in positioning the body cavity device in an optimal configuration. The term "intraoperative" as used herein refers to a situation occurring or being performed during a surgical procedure. In the example, the device deployment module 104 enhances the information on the video feed and displays the enhanced video feed or an image extracted therefrom on the display device 106.

[0050] Throughout this document, flow-diverting stents are used as example devices. However, it should be understood that the techniques, methods, and systems disclosed herein can be used with other interventional devices (e.g., embolic coils, intrasaccular device deployment, peripheral body lumens or other lumens, such as carotid, biliary, or femoral stent deployment, etc.).

[0051] In an example, a stent may be mounted or sheathed within a catheter that is advanced over a wire to a location within a body lumen where the stent is to be deployed or positioned. Example enhanced information generated by the device deployment module 104 includes: (i) identification and marking of the location of the stent, catheter, and guidewire, and (ii) anatomical landmarks and catheter markers to add visual context and enhance the physician or operator's understanding of the scene, particularly when using a low-dose X-ray system that produces lower contrast X-ray images. By improving the clarity of the stent, catheter, and guidewire and superimposing relevant features (anatomical, physiological, or other), the device deployment module 104 can reduce the need for high-dose X-ray imaging and reduce the need for multiple injections of intra-arterial contrast agent to generate digital subtraction angiography "roadmap" images, thereby reducing contrast agent exposure and reducing the required computing power while improving surgical efficiency and accuracy.

[0052] In an example, the information may also include surgical scoring indicators. For example, the device deployment module 104 may provide metrics indicating the progress of the surgical procedure / deployment relative to the preoperative surgical plan or expected deployment results, or provide a predicted deployment status to help support correct deployment.

[0053] In an example, video augmentation can be performed directly on the display device 106, which can be placed in a neurointerventional catheterization lab. In another example, the display device 106 can include an augmented reality or virtual reality headset that can overlay information on top of an existing display or on a virtual display, thereby reducing the need for additional hardware in the catheterization lab.

[0054] In another example, a hologram can be used as a display device 106 to present video and data augmentation, projecting 2D information and 3D models in a space close to the user without requiring the user to wear additional visual equipment. The holographic visual information can be interactive, switching and changing information and models based on user commands before and after surgery.

[0055] Thus, the device deployment module 104 is configured to perform operations including characterization, real-time analysis, and scoring of the vasculature and the device being deployed, and provide various outputs and predictions to a user or robotic system.

[0056] Figure 1A is a block diagram illustrating operations performed by the device deployment module 104 according to an example embodiment. As shown, analysis of deployment of a device (e.g., a stent) begins with identification of features, such as wires, markers, lumens, devices, curves, diameters, lengths, dimensions, references, centerlines, wire / strut braid angles, and the like.

[0057] The identified or recognized features can be used to derive a number of inputs, including taper angle, distance between markers, curvature of the lumen, curvature of the stent, adherence of the stent to the lumen, positioning of the stent, shape of the stent, non-clinically desired irregular shapes (e.g., banding, twisting, fish mouth), deviation from the centerline, deviation from an ideal or defined deployment path, braid angle or braid density. The distance between markers can also include the distance from a specific marker to a reference point on the lumen, a landmark such as an aneurysm ostium, the start and end points of a curve, an ideal landing zone, an ideal or defined path, deviation from the centerline, etc.

[0058] The cone angle is not a fixed angle, but varies during deployment. The ideal cone angle is a function of the curvature of the body lumen, the magnitude of the tensile or compressive forces applied to the catheter, the diameter of the native body lumen, the size of the aneurysm, and the net force between the microcatheter and the stent delivery wire. Figure 4A An example describing how to determine the cone angle.

[0059] The curvature of the body lumen can also be determined and used as an input for ideal deployment calculations. More curved body lumens present additional deployment challenges and may require more push-pull maneuvers and catheter manipulation to achieve the desired effect. Curvature is also an important lateral input for analyzing taper shape, taper angle, centerline deviation, deployment path, and more.

[0060] Braid angle and braid density can be partially identified as stent characteristics or can be applied to the stent region through known and identified information such as stent design, curvature, diameter, adherence, etc. Braid angle and braid density are factors in flow steering prediction.

[0061] The adherence of the stent to the body lumen is also valuable for evaluating deployment technique, positioning, landing zone, foreshortening, avoidance of endoleaks, and stent packing.

[0062] Furthermore, feature validation can identify actions and events during interventional cases. These actions and events can be detected and used to improve stent deployment scoring algorithms. For example, contrast injection / flushing can be used to identify any areas of poor stent apposition, stent re-sheathing can be captured to improve technique or detect any slippage of the stent from the sheath cushion, and movement of the stent delivery wire into a smaller branch of the artery can alert the interventionalist.

[0063] With inputs ranging from feature validation to analysis and scoring, the outputs and predictions generated by the system 100 (e.g., Figure 1A as shown) can be presented, notified, displayed or sent to the user as a result.

[0064] Figure 2 A block diagram representing a system 100 according to an example embodiment is shown. Block 200 represents the real-time acquisition of a data stream comprising video and other information relating to a body cavity (angiogram) in which a body cavity device such as a stent is being positioned. The data stream is displayed on a display device 106 (see enhanced image 210). For example, the data stream comprising video and other information can be an X-ray angiogram data stream captured by a CT scanning device (the image capture device 102 described above). The video stream can also be obtained from a digital video feed of an imaging system (e.g., a High Definition Multimedia Interface (HDMI) or Digital Video Interface (DVI), a serial interface, etc.) or from an external screen capture of the X-ray angiogram (e.g., an external camera pointed at a screen in a catheterization lab displaying the video feed).

[0065] The video feed is considered an input stream to block 202. For example, block 202 may be implemented by the device deployment module 104. At block 202, the device deployment module 104 may identify a body cavity device (e.g., a stent) being passed through the body cavity via a catheter and wire through image confirmation techniques.

[0066] For example, the device deployment module 104 can access the device 3D geometric registration in 3D space (e.g., a 3D model of the stent) at block 204. Thus, the device deployment module 104 can identify the location of the stent, stent markers, the position and orientation of the stent, the catheter, and the wire. In particular, the device deployment module 104 can identify features of the stent (e.g., proximal end, distal end, wire / strut angle, orientation, etc.).

[0067] This information (positioning of the stent, markings, features, indicators) is generated in the form of a graphical annotation at block 206. The graphical annotation is then enhanced onto the video feed on the display device 106 at block 208. Both the video stream acquired at block 200 and the graphical annotation of block 206 can be displayed on the display device 106. In other words, the graphical annotation is superimposed or enhanced onto the video stream when displayed on the display device 106.

[0068] Enhanced image 210 shows information associated with stents, catheters, and wires enhanced on the video feed. The image shows stent 212 and wire 214, which may be distinguished or clarified in enhanced image 210 by color coding or contrast enhancement.

[0069] In addition to block 202 associated with real-time confirmation of the device (stent) being deployed, the device deployment module 104 may also detect characteristics of the body cavity in which the device is being deployed at block 218. Based on this detection, the device deployment module 104 may confirm the characteristics of the body cavity and may identify different areas of the body cavity and whether they are suitable for stent deployment. The device deployment module 104 may also generate and / or specify indicators associated with the body cavity, such as a tortuosity indicator, a non-starting area indicator, etc., as described below with respect to Figure 3 As described in more detail.

[0070] Figure 3 Another block diagram representing system 100 according to an example embodiment is shown, and details of body lumen identification are shown. Block 218 may be implemented by device deployment module 104, which may access a 3D body lumen model at block 220. For example, device deployment module 104 may access a 3D model of the body lumen generated from images collected prior to the interventional procedure, such as collected by image capture device 102 (e.g., a rotational scanning angiography system or a CT scanning device).

[0071] In addition, the contrast agent can be flushed through the body cavity while capturing the video feed. The device deployment module 104 can access the image through the video feed at block 222 when the contrast agent is flushed, and then the device deployment module 104 can map the body cavity path (e.g., details of the body cavity, including branches, etc.). The device deployment module 104 can also compare the 3D body cavity model of block 220 with the body cavity path mapped at block 222 when the contrast agent is flushed through the body cavity to clearly visually identify different parts of interest in the body cavity. This comparison is also useful for spatial registration (e.g., alignment) of the 3D body cavity model of block 220 with the real-time image.

[0072] For example, Figure 3 As shown, the device deployment module 104 can identify the path of the catheter, wire, and stent within the body cavity (path finder), can identify geometric parameters of the body cavity, such as the diameter of the body cavity at various portions, can determine the curvature / tortuosity of the body cavity at different portions, and can also identify areas in the body cavity where stents should not be deployed (warning areas). The device deployment module 104 can then visually overlay body cavity markers on the video feed image (e.g., identifying markers showing lumen extent, centerline, boundaries, curvature, etc.).

[0073] Figure 3 An enhanced image 224 is shown in which an image depicting a body cavity 226 from a video feed is displayed. The device deployment module 104 can accurately determine the curvature or curvature of the body cavity 226. The device deployment module 104 can further identify and mark the most curved body cavity areas and their characteristics in the enhanced image 224, including curvature in 3D, body cavity rings, or branches in 3D space. For example, the device deployment module 104 can overlay a tortuosity indicator in the enhanced image 224. Specifically, the device deployment module 104 can mark the body cavity 226 with curvature markers, such as curvature marker 228, curvature marker 229, and curvature marker 230 to indicate areas of the body cavity 226 whose curvature exceeds a specific threshold curvature. In the present specification, the curvature can be determined, for example, as the inverse of the radius of a portion of the body cavity 226 and as an indicator of the degree of curvature of the body cavity 226. The curvature markers 228-230 can be marked and indicated, for example, with a unique color (e.g., red) so that the surgeon can clearly see these markers visually. These markings help surgeons determine where to deploy the stent and which areas to avoid during deployment.

[0074] Additionally, the device deployment module 104 can identify non-start and non-end regions within the body lumen 226. Non-start regions are regions with unfavorable curvature, hidden branches, or bifurcations, where the distal end of the stent may compromise stent performance if located within such regions. In other words, the surgeon should avoid placing the distal end of the stent within such non-start regions at the start of deployment and avoid placing the proximal end of the stent within such non-end regions at the completion of deployment. The non-start and non-end regions can also be determined and manually entered by the surgeon prior to stent deployment.

[0075] like Figure 3 As shown, the device deployment module 104 marks a non-starting area 232 at the bifurcation of the body cavity 226. On the enhanced image 224, the device deployment module 104 places a polygonal shape around the non-starting area 232 to point out this area to the surgeon during the stent deployment process. Alternatively, the non-starting / ending areas or optimal starting / ending areas can be indicated by body cavity circumferential indicators (ovals) that are color-coded to indicate good and bad areas for stent placement. Alternatively, optimal starting and ending ("start here" and "end here") areas can also be used, where the marks point to the locations where the stent optimally starts and / or ends. These locations can be determined by the device deployment module 104, manually entered by the surgeon, or determined by separate planning software and entered into the device deployment module 104.

[0076] Additionally, in an example, the device deployment module 104 can provide warnings, numerical scores, and other such indicators on the enhanced image 224 to assist in real-time deployment of the stent. Understanding the tortuosity of the body lumen 226 and non-starting areas can help the surgeon achieve enhanced deployment results.

[0077] For example, it is desirable that the stent adheres well to the wall of the body lumen 226. The adherence of the stent refers to the proximity of the outer peripheral surface of the stent to the inner wall of the body lumen 226. If the outer diameter of the stent is smaller than the inner diameter of the body lumen, the stent may be characterized as having loose adherence (poor adherence) to the wall of the body lumen 226. This loose adherence may be undesirable because it may cause the stent to migrate or move once deployed in the body lumen 226. It is more desirable to have a high degree of adherence of the stent so that the stent is as close to the body lumen wall as possible to stabilize its position in the body lumen and provide effective blood flow diversion.

[0078] Adhesion can be expressed as a coverage percentage at a specific cross-section of the body cavity 226. For example, based on the 3D registered body cavity model at a given cross-sectional area of ​​the body cavity 226, the adhesion mismatch percentage can be determined as follows:

[0079]

[0080] In another example, the adhesion mismatch percentage may also be determined as:

[0081]

[0082] Other apposition metrics can be used. In one example, wall apposition mismatch can be calculated based on edge detection of the stent and compared to the body lumen wall from a digital subtraction angiography image.

[0083] Figure 3A The apposition mismatch of a stent deployed within a body lumen according to an example embodiment is shown (in a linearized manner). Figure 3A Displays potential scenarios that could occur under different underlying disease states (e.g., Moya Moya and others). The wall apposition score is based on the percentage of visual / perceived contact between the stent and the lumen wall. Risks associated with a higher mismatch include potential endoleaks, aneurysm recanalization, or other complications related to stent-wall apposition.

[0084] As another example, an overall apposition score for the stent can be determined. Figure 3A In, L M is the local mismatch length, L T is the total length of the stent. Total adhesion score A Total It can be determined as:

[0085]

[0086] The total apposition score and apposition mismatch may also be highlighted on the enhanced image to provide apposition warnings, deployment technique recommendations, or indicators of potential stent problems, such as recanalization, stent deployment endoleaks, or other issues.

[0087] Reference again Figure 3 In curved areas (e.g., areas marked with curvature markers 228-230), the stent may not adhere well. Ideally, these areas should be avoided, so such markings aid the surgeon during deployment. In some cases, stent implantation must traverse areas of high curvature, and markings can alert the surgeon to pay more attention to stent adherence in these areas.

[0088] Figure 4 A block diagram representing a system 100 is shown with detailed information associated with identified parameters of a stent, according to an example embodiment. Figure 4 As shown, the device deployment module 104 can access 3D rendering data of the stent (e.g., a 3D model of the stent provided by the stent manufacturer) at block 204. In this way, the device deployment module 104 can overlay the model of the stent within the body cavity and determine the characteristics of the stent (e.g., adherence).

[0089] For example, Figure 4 As shown, the device deployment module 104 can determine information including: placement location within the body lumen (positioning of the stent), adherence, the centerline of the body lumen and the stent, the orientation of the stent filaments (e.g., if the stent is a braided stent), the taper angle, and the braid angle (e.g., half the angle formed by the intersecting filaments in the braid of the braided stent). The device deployment module 104 can then overlay indicators of such information on the image in the video feed.

[0090] For example, Figure 4 As shown, the augmented image 234 depicts identified features, such as the outline of a body cavity 236 superimposed on the video feed. The device deployment module 104 may also superimpose a predicted rendering 237 of a stent 238 showing a centerline 240 of the stent 238.

[0091] Furthermore, in the example, Figure 4 As shown, the device deployment module 104 visually superimposes an adherence indicator on the enhanced image 242 that indicates the relative degree of adherence of the stent 238 to the wall of the body cavity 236. Figure 4 As shown, device deployment module 104 can generate elliptical displays, such as ellipse 244, at different portions along the length of stent 238. The ellipse is intended to indicate the relationship between the diameter of stent 238 and the corresponding diameter / circumference of body lumen 236, but is displayed as an ellipse due to the angularity of body lumen 236 and stent 238.

[0092] Oval 244 serves as an indicator of the adherence of stent 238 to body lumen 236 at a particular location or cross-section (e.g., the relative degree of stent 238 to the wall of body lumen 236). The oval can be color-coded. For example, a green oval can indicate acceptable adherence, while a red portion can indicate poor or unacceptable adherence. For example, a portion of oval 244 can be green, while a portion 246 can be red, indicating poor adherence or mismatch between body lumen 236 and stent 238 (e.g., the diameter of stent 238 at a given portion may be greater than the diameter of body lumen 236).

[0093] Figure 4A Determining the taper angle and other features of a stent according to example embodiments is shown. In particular, Figure 4A Examples are provided for identifying the taper angle, deployment taper shape, inner curvature side, and calculated distance from the last confirmed adherence position to the catheter marker.

[0094] For example, as an output, the cone angle can be color-coded in image 247 to display the analysis results of the deployment status. Scoring and recommended deployment techniques can also be displayed to guide the real-time deployment process. In one example, the cone angle can be confirmed based on the angle formed between the marker and a defined distance. In another example, the cone angle can be calculated based on the segmentation of the identified stent at a defined distance D from the microcatheter marker. If "r" is the radius of the opened stent at distance D, the cone angle can be calculated as: for example, 2×arctan(r / D) on a straight body lumen.

[0095] When the stent is opened at a bend, the cone angle can be defined by a hydraulic mean angle, a normalized function, or a symmetrical equivalent angle based on an identified segment of the stent. For another example, the conical shape of the tapered portion of the stent can be identified and referenced to the known stent response curve in the force interaction. The output of the analysis can be displayed as a deployment force indicator to provide deployment technique recommendations. The cone angle, inner curvature, and centerline distance from the last attached position can also be analyzed to indicate whether the technique of opening and re-sheathing during deployment is good. Technical recommendations can also be provided through indicators or messages, including visual, audio, or audio-visual as output to the user. Recommendations and scores can be displayed to guide and assist the process.

[0096] Figure 5 Another block diagram representing the system 100 is shown, showing details of body cavity identification, according to an example embodiment. Figure 5 and Figure 3 Similarly, where the device deployment module 104 determines several parameters of the body lumen, such as diameter, centerline, curvature, and narrowing indicators, at block 218 , the device deployment module 104 then visually overlays the information or indicators of the body lumen parameters onto an image of the video feed, such as the enhanced image 248 .

[0097] As shown in enhanced image 248, device deployment module 104 identifies body lumen 250 and labels various parameters thereof. For example, device deployment module 104 indicates a centerline 252 of body lumen 250 as a dashed line that follows body lumen 250 as it curves and changes its diameter. Device deployment module 104 also superimposes tortuosity indicators or curvature markers, such as curvature marker 254 and curvature marker 256, to indicate areas in body lumen 250 where the curvature exceeds a certain threshold curvature.

[0098] The device deployment module 104 also identifies and marks non-start regions 258 in the body lumen 250. As described above, non-start regions are regions with unfavorable tortuosity, hidden branches, or bifurcations, which may compromise the performance of the stent if the distal end of the stent is positioned in such regions. Additionally or alternatively, the device deployment module 104 may identify "start here" and "end here" regions that may be optimal for starting and ending stent deployment within the body lumen 250.

[0099] Therefore, the device deployment module 104 performs real-time angiographic registration of various features of the body lumen 250 to assist in achieving optimal stent deployment results. Non-starting areas, centerlines, tortuosity, and other information are superimposed on the real-time angiographic screen (display device 106) to provide visual identification of various features of the body lumen and stent.

[0100] Figure 2-5 An example of real-time deployment analysis performed by the device deployment module 104 is shown, which can provide animated or enhanced graphical features and indicators to highlight prominent features of the body cavity and stent on an X-ray angiogram. When the device deployment module 104 identifies and registers the stent (determines the position and orientation of the stent), the device deployment module 104 marks various stent features. The output (enhanced image) displayed on the display device 106 can include real-time positioning, center lines, circumferential rings or circles (which may be displayed as a circle, an ellipse, two or more circles intersecting, or an ellipse or a variation thereof) that refer to or define the body cavity wall, stent features, stent adhesion, and the critical angle of the stent.

[0101] Figure 5A The centerline and circumferential rings of the body cavity are superimposed in a body cavity according to an example embodiment. As shown, the centerline of the body cavity is marked with a dotted or dashed line. The depicted circumferential rings or rings of the body cavity wall are defined by the intersection of the body cavity wall and the normal plane of the centerline at a given point along the centerline, wherein the starting point of the ring is a point of interest and is preferably spaced equidistantly along the centerline within the region of interest for stent deployment and / or determined by points of particular interest, such as the optimal stent start and end placement positions, the most distal and most proximal extents of the aneurysm ostium, the center of mass of the aneurysm ostium, etc. These rings can also be used as markers. For example, these rings can use color coding, spacing density, line thickness, opacity or transparency to indicate various markers, such as curvature, non-start / end areas, expected end location, landing probability, stent wall adhesion, performance index or score, etc.

[0102] In one example, the surgeon can toggle displayed indicators as needed to assist with deployment, overlaying determined stent features onto the specific anatomy, physiology, and physics of the body lumen to assist the surgeon in real time.

[0103] In addition, in an example, the device deployment module 104 can determine a score or heat map-based identifier to provide a quality assessment of the stent deployment and possibly provide a warning. An example score may include a weighted average of several parameters of real-time deployment. For example, the score may include a weighted average of parameters indicating the following: taper angle, centerline deviation, proximity of the stent end to non-start or end areas, stent adherence, braid angle, deployment status (e.g., not deployed, partially deployed, under-deployed, or optimally deployed), etc.

[0104] For example, the deployment score can be calculated as follows:

[0105]

[0106] Where w1-w6 are weights assigned to specific parameters or variables during deployment, C is the taper angle, A is the wall adherence score, F is the fishmouth score of the distal end of the stent, L is the landing zone score based on the proximity or overlap of the stent end with the non-start / non-end region or the start / end region, D is the deployment based on the percentage of stents deployed, and M is the deviation from the centerline of the body lumen (e.g., measured as a fraction). These factors are for illustrative purposes only. More or fewer factors can be used to determine the deployment score.

[0107] Therefore, the device deployment module 104 is configured to provide a real-time quality assessment of the deployment. The output of the device deployment module 104 may include an overall quality score and a visual overlay of the desired final position of the stent. The score changes in real time as the surgeon adjusts the positioning and deployment of the stent during the procedure.

[0108] As another example of feedback that can be provided to the user, the device deployment module 104 can provide tracking of the catheter tip during deployment. Specifically, the device deployment module 104 can track the instantaneous deviation of the catheter tip from a predefined preferred catheter tip trajectory guideline.

[0109] Figure 5B A preferred catheter trajectory guideline 257 according to an example embodiment is shown offset from a body lumen centerline 259. This preferred catheter trajectory guideline 257 defines a path that a stent deployment catheter preferably follows when the stent is deployed in a manner consistent with the optimal deployment plan generated by the device deployment module 104.

[0110] If the catheter generally follows the preferred catheter trajectory guideline 257, visual and / or audio feedback can be provided to the user to indicate on-track performance. Conversely, if the catheter generally deviates from the preferred catheter trajectory guideline 257, visual and / or audio feedback can be provided to the user to indicate this deviation and suggest or prompt the user or robot for possible corrections to change the deployment technique and / or update the deployment plan.

[0111] Figure 5C Stent 249 is shown being deployed within body lumen 251 according to an example embodiment. Figure 5C The illustration depicts acceptable stent-wall adhesion, as shown by the contours of stent 249 conforming to the confines of the lumen annulus 251. Figure 5C An acceptable wire-to-catheter deployment ratio is also shown, as indicated by alignment of the catheter tip with the preferred catheter trajectory guide wire 255 .

[0112] Figure 5D An example of body cavity model registration between 2D and 3D according to an example embodiment is shown. Figure 5D The left side shows an X-ray image 261, and the right side shows a generated 3D body cavity model 263. Figure 3 This information is processed at block 218 (e.g., a real-time body cavity detection system) in the 3D body cavity model 263. This 3D body cavity model 263 can be used in real time to accurately measure the diameter and curvature of the body cavity, the position and orientation of the stent and wire. This information can then be fed to the device deployment module 104.

[0113] Using a biplane image capture system, locations of interest identified in the video feed (e.g., catheter tip, etc.) can be projected back into 3D space by knowing the X-ray source positioning and pointing vector and then tracing the path of the X-rays from the X-ray source to the detector. The intersection of two such rays from the biplane video feed generates coordinates in 3D space, which can be associated with the 3D body cavity model 263 through image registration (aligning images from different sources or time points, for example, by minimizing a cost function representing image similarity).

[0114] In addition, the geometric changes of the body cavity caused by interventional interactions can be observed from the 2D angiographic view. These changes can be superimposed on the 3D body cavity model 263 for real-time reference. In addition, a new data set of multiple 2D views (e.g., X-ray images 261) acquired using imaging vectors separated from each other in angle can be used to model the changes based on the reconstruction of the new data set and the learning of the previously reconstructed 3D model. Updates and modifications to the body cavity model can be generated and fed into the latest 3D model to increase the accuracy of output predictions or improve the accuracy of measurements. The updated 3D model can then be used to update data for marking, feedback, and display, such as curvature, centerline, catheter trajectory, rings, non-start / non-end areas, landing areas, landing probability, adherence, etc. The 3D body cavity model 263 can be presented in an additional window or panel to enable 3D perception and real-time 3D recognition during intervention. For example, such changes can be visualized directly in 3D using devices such as visualization displays, hologram devices, virtual / augmented reality wearable devices or head-mounted devices, visualization dashboards on screens, and the like.

[0115] User-selected independent motion or dependent rotation associated with the biplane views can be applied to the display channel of the 3D body cavity model 263. For example, the 3D body cavity model 263 (the displayed model) can be moved, such as roll, tilt, pitch, and other combinations of view changes, based on user preferences and / or based on the motion of the biplane gantry of the imaging system. Interactive motion on the 3D model can further aid understanding during intervention and deployment procedures.

[0116] Figure 6 A block diagram representing system 100 is shown illustrating generation of a probability map of a stent landing zone according to an example embodiment. The device deployment module 104 may include a real-time calculation and comparison engine at block 260 configured to determine and provide graphical indicators and animations of stent deployment parameters that facilitate understanding of stent placement effectiveness or deployment results.

[0117] In the example shown, the enhanced image 262 displays a real-time landing probability graph 264 (e.g., a distribution probability curve) to help predict the landing zone 266 (where the proximal end of the stent will be placed at the end of deployment), which may be very important to the outcome of the deployment.

[0118] In determining the landing zone 266, the device deployment module 104 can account for effects such as foreshortening. A technical problem that arises when using a stent in an intraluminal procedure is that it is difficult to predict the final positioning of the stent after deployment within the body cavity due to variations in the stent length, which depends on the patient's anatomy and the positioning of the stent within the body cavity. In other words, a problem with stent deployment is that the total length of the stent changes (foreshortening) as the stent is released from the catheter into the body cavity. The device deployment module 104 accounts for foreshortening when determining the final length and landing zone of the stent after deployment.

[0119] Although a particular length of stent may appear adequate on a 2D image, when the stent is actually deployed in a body lumen, that length may not be adequate due to perspective foreshortening effects. In particular, due to the 3D configuration of the body lumen, the body lumen may be longer than what appears in a 2D image. For example, a body lumen may be tortuous and may have twists and bends in 3D space, so that the actual length of the body lumen is longer than it appears from the perspective of a 2D image. Furthermore, due to the deployment technique and / or interaction with the body lumen, the stent may be compressed, stretched, or otherwise change its length during deployment. The device deployment module 104 estimates this perspective foreshortening effect and determines the final length of the stent and the probability of the landing area, as shown in the real-time landing probability chart 264.

[0120] In an example, based on the determination of the landing zone 266 and other characteristics of the stent, the device deployment module 104 can recommend catheter motions that lead to favorable outcomes. Recommended catheter motions include following preferred catheter tip trajectory guidewires, rotations, translations, and other similar manipulations of catheters, microcatheters, and wires. They can also include repetitive small motions or oscillations, such as linear back-and-forth motions and repeated clockwise and counterclockwise rotations. Such recommendations can be provided to a human surgeon to adjust deployment techniques, or can be provided as an algorithm to a robot deploying the stent.

[0121] Figure 7 The figure shows linearized graphical information that can be generated by the device deployment module 104 according to an example embodiment, which is used to provide key information for body cavity intervention through a real-time video stream of X-ray angiography. By extracting geometric information from the video feed, the device deployment module 104 can mark and align the body cavity path, and can determine information such as the body cavity curvature, diameter and centerline as described above. The 3D body cavity model from the patient's medical data can be co-registered with the 2D image frames extracted from the video stream to improve the accuracy of the geometric parameters. When the stent deployment video stream is received, the device deployment module 104 determines a geometric comparison to derive information such as deployment ratio, adhesion information, catheter trajectory line, etc., so that intraoperative procedural recommendations, scores or warnings can be provided.

[0122] Figure 7 Graph 300 shown above shows curve 302, which shows the change in body lumen curvature along the length of the body lumen. Graph 304 shows the planned or predicted stent deployment position and diameter. Specifically, in graph 304, bars such as bar 306 represent the body lumen diameter along the length of the body lumen, corresponding to curve 302 of graph 300.

[0123] Additionally, in graph 304, device deployment module 104 provides a predicted or suggested location and diameter for the stent at region 308, which is depicted by the cross-hatching. Region 308 represents the space that the stent will occupy. As shown, the planned location of the stent avoids starting and ending in areas of significant curvature within the body lumen, as indicated by curve 302.

[0124] Graph 310 also provides stent deployment data, which may help the surgeon (or robot) achieve the proposed stent position.For example, graph 310 may include a graphical indicator of the deployment ratio at a particular location along the body lumen, as shown by dashed line 312.

[0125] The deployment ratio refers to the ratio of wire push to catheter pull to achieve optimal deployment. Specifically, during deployment, as described above, the surgeon or robot can perform an oscillating movement between the wire and the catheter to remove the stent from the catheter and re-sheath the stent in a back-and-forth motion, thereby deploying the stent to the desired configuration (at the desired location, with the desired diameter, desired angle, desired adhesion to the body cavity wall, etc.). The device deployment module 104 can provide such information to the surgeon or robot to achieve the desired deployment of the stent.

[0126] Another indicator for improving and / or monitoring deployment accuracy can take the form of a preferred catheter tip trajectory guideline as described above. The preferred catheter tip trajectory guideline is a 3D curve that tracks the preferred intraluminal path through the body cavity segment where the stent is to be deployed, which further indicates the radial deviation from the body cavity centerline that the catheter should preferably follow during stent deployment. For example, if the catheter-wire manipulation fully follows the above-mentioned deployment ratio, the trajectory guideline may correspond to the path that the catheter will follow. Conversely, the catheter-wire deployment ratio can be derived based on the expected device manipulation required to follow the trajectory guideline.

[0127] Generally speaking, at a given point along the centerline, the deviation of the trajectory guide line from the centerline will fall on or near the radial vector of the osculating circle associated with that point. Figure 7A A radial vector 322 of an osculating circle 324 at a given point 326 and an outward deviation vector 328 pointing away from the center of the osculating circle 324 are shown in accordance with an example embodiment.

[0128] The extent and direction of the radial deviation will be determined by an analytical / geometric algorithm or machine learning derived model that incorporates geometric and clinical factors such as, but not limited to, stent type, lumen diameter, centerline curvature at that point, location and size of the aneurysm ostium, clinical preference for increasing or decreasing stent lattice density in a particular location, the presence of side branch lumens, and / or axial proximity to the start and end points of the deployed stent.

[0129] Additionally, the device deployment module 104 can update its optimal stent deployment plan, thereby changing the trajectory guideline during stent deployment to accommodate the calculated changed landing zone location and its proximity or overlap with the non-end zone. If the device deployment module 104 cannot find a deployment plan that avoids the non-end zone, it can alert the surgeon that the landing zone and the non-end zone overlap and recommend re-sheathing and repositioning the stent.

[0130] Before final deployment or after deployment is complete, the device deployment module 104 can generate a graph 314 for post-deployment analysis. Graph 314 is similar to graph 304, except that graph 314 displays the actual position of the stent rather than the predicted position of the stent. As shown, graph 314 can indicate, emphasize, or highlight areas where adhesion is not optimal (e.g., the diameter of the body cavity is larger than the corresponding diameter of the stent at that area, or vice versa), such as area 316. For example, area 316 can correspond to the above-mentioned Figure 4 Portion 246 of ellipse 244 is shown.

[0131] Graph 314 also indicates that in the final deployment of the stent, the stent is shorter than predicted, as indicated by the unoccupied areas, such as area 318 at the proximal end of the stent and area 320 at the distal end of the stent. The surgeon can then determine whether this deployment is acceptable or whether redeployment is necessary if final deployment has not yet been performed.

[0132] In an example, the device deployment module 104 may involve the use of a neural network model that may be trained to accurately locate various landmarks seen during stent deployment, such as distal, proximal, and resheathing markers on the delivery wire, microcatheter, and mid-catheter markers.

[0133] Figure 8 An image enhanced with various markers according to an example embodiment is shown. Figure 8 , image 400 on the left shows a raw image from a video feed, which is then input to device deployment module 104. Thus, this image represents an input frame to device deployment module 104. Image 400 shows body cavity 402 and wire 404.

[0134] The device deployment module 104 is configured to generate an enhanced image 406 with markers superimposed on the image 400 (input frame). For example, as shown in legend 408, the device deployment module 104 marks the re-sheathing location 410, the distal marker 412 of the delivery wire, the location of the microcatheter marker 414, the mid-catheter marker 416, and the proximal marker 418 of the delivery wire.

[0135] Knowing the location of these markers can help with positioning and posture confirmation / registration of the entire catheter device while also assessing any inconsistencies in a given deployment. The locations of these markers can be displayed in the enhanced image 406 so that the surgeon can understand the location of key catheter components.

[0136] The machine learning model (e.g., neural network model) of the device deployment module 104 can be trained directly using pre-recorded body cavity procedure videos in a supervised or semi-supervised manner, where landmarks and other key points can be labeled by human experts to train the model to predict them. The neural network model can potentially use past frame sequences rather than just a single (most recent) frame to accurately predict landmark locations and help overcome any information loss caused by occlusions in a single frame.

[0137] In addition, post-processing the model predictions (e.g., using a fixed relative order of the labeled subsets relative to each other) can enhance localization and help avoid false positives and duplicates. Several types of neural network models can be used, such as convolutional neural networks with regression heads for predicting object locations. For example, models like CenterNet or models from the Transformer architecture family can be used.

[0138] Another neural network model can be trained to predict accurate pixel-level masks of various components of the deployment system, such as the deflector device (e.g., stent) and the guide / delivery wire. The "mask" outlines the shape of the object identified in the image. The model can also take a series of previous frames and predict the mask for the latest frame. Given the importance of accurately predicting the boundaries of the deployed stent, a specific loss (e.g., a weighted loss) can be used to train the model to better learn the boundaries of the stent. For example, during training, this weighted loss will impose a greater penalty on the network's incorrect predictions around the boundaries of the body cavity.

[0139] Figure 9 Generating masks of filaments and deployed stents using a trained model is shown according to an example embodiment. Figure 9 An image 500 representing an input frame to the model is depicted. The middle image 502 shows the model predictions, depicting a delivery wire mask 504 and a deployed stent mask 506 identified by the trained model. The right image 508 represents the ground truth for the delivery wire and deployed stent (ground truth masks provided by direct observation or identified from an X-ray image). As shown, the model output shown in image 502 is substantially accurate compared to the ground truth in image 508.

[0140] In one example, the device deployment module 104 may also implement a model that is trained on low- and high-resolution image datasets using deployed stents to learn the structural characteristics of the stents depending on the type of stent. This model may then be used to increase the resolution and / or dynamic range of X-ray images to predict a more accurate and visually enhanced version of the stent captured using a low-dose X-ray setting.

[0141] When trained to a high degree of accuracy, this model can also help stent segmentation models output accurate masks, particularly at the object / stent boundary, to help derive auxiliary stent deployment characteristics, such as the distance of the deployed stent to the body cavity wall (i.e., determining adhesion) and the angle of stent deployment. For example, image segmentation refers to annotating or assigning each pixel in an image to a single class or object. The output is a mask that outlines the shape of the object in the image.

[0142] The model can be enhanced by conditioning on known (physical) properties of the stent, which will help it better represent the high-resolution output. The properties of the stent can be obtained from the design files (including the material properties of the stent's filaments, the number of filaments, and the angles of the filaments), a 3D computer-aided design (CAD) model of the stent, or a 3D (micro)CT scan.

[0143] Figure 10 A block diagram 600 representing a generative modeling workflow according to an example embodiment is shown. The block diagram may be implemented by or within the device deployment module 104.

[0144] The workflow is divided into a training phase and a testing phase. During the training phase, at block 602, the model is fed with low-dose X-ray (low-resolution) images. At block 604, the model is also fed with device (e.g., stent) and catheter generation models (e.g., CAD models provided by the manufacturer or 3D micro-CT scans). The model is then fed with high-dose X-rays (high-resolution images) at block 606.

[0145] According to the example, Figure 11 A low dose X-ray image 700 is shown on the left and a corresponding high dose X-ray image 702 is shown on the right. The model can thus be trained to recognize / learn the stent and its characteristics in low dose images as well as high resolution images of the catheter and stent model.

[0146] Low-dose X-rays and high-dose X-rays can refer to the relative energy levels of an X-ray tube setting. For example, a low dose can be 50 milliampere-seconds (mAs) and a high dose can be 150 mAs, with the radiation produced by the X-ray tube set to 120 kVp. Another example of low-dose and high-dose could be 40 mAs and 180 mAs, with 100 mAs being the standard dose. Thus, low-dose and high-dose could be two relative settings in an imaging machine. In another example, low-dose X-rays and high-dose X-rays could involve a high imaging frame rate and a low imaging frame rate, respectively, when capturing an X-ray image.

[0147] Back to Figure 10In the testing or inference phase, the model is fed low-dose images at block 608 and identifies the devices (e.g., stents) and catheters at block 610. At block 612, high-dose equivalent images are acquired to score the model's results and make any necessary adjustments.

[0148] In this way, the model is trained to identify body cavities and device parameters using low-dose X-ray images (low-resolution images), thereby eliminating or reducing the need for high-dose X-ray imaging that could harm patients due to radiation exposure. In addition, the use of low-resolution images can improve the computational efficiency and real-time processing of video streams.

[0149] From the extracted segmentation masks and the landmark locations identified by the model, the device deployment module 104 is configured to propose various relevant indicators, such as the proximity (adhesion) of the (deployed) device to the body cavity wall, candidate locations for starting device deployment, and other risk scores relevant to clinical decision support.

[0150] Figure 12 is a block diagram of a computing device 800 according to an example embodiment. The computing device 800 may represent or be included in any of the devices described above (eg, image capture device 102, device deployment module 104, display device 106, etc.).

[0151] The computing device 800 may have a processor 802, a communication interface 804, and a data memory 806, each connected to a communication bus 812. The computing device 800 may also include hardware for enabling communication within the computing device 800 and between the computing device 800 and other devices. For example, the hardware may include a transmitter, a receiver, an antenna, and the like.

[0152] The communication interface 804 can be a wireless interface and / or one or more wired interfaces that allow short-range and long-range communication with one or more networks or one or more remote devices (e.g., allowing communication with the communication bus 812). Such a wireless interface can provide communication through one or more wireless communication protocols, Bluetooth, Wi-Fi (e.g., Institute of Electrical and Electronics Engineers (IEEE) 802.11 protocol), Long Term Evolution (LTE), cellular communication, near field communication (NFC), and / or other wireless communication protocols. The wired interface can include an Ethernet interface, a CAN network interface, a USB interface, or the like to communicate via wires, twisted pair cables, coaxial cables, optical links, fiber optic links, or other physical connections to a wired network.

[0153] The data storage 806 may include or take the form of one or more computer-readable storage media that can be read or accessed by the processor 802. The computer-readable storage media may include volatile and / or non-volatile storage components, such as optical, magnetic, organic, or other memory or disk storage, which may be integrated in whole or in part with the processor 802. The data storage 806 is considered to be a non-transitory computer-readable medium. In some examples, the data storage 806 may be implemented using a single physical device (e.g., one optical, magnetic, organic, or other memory or disk storage unit), while in other examples, the data storage 806 may be implemented using two or more physical devices.

[0154] Thus, data storage 806 is a non-transitory computer-readable storage medium having stored thereon executable instructions 814. Executable instructions 814 include computer-executable code. When executable instructions 814 are executed by processor 802, processor 802 is caused to perform operations of computing device 800 (e.g., operations performed by image capture device 102, device deployment module 104, or display device 106).

[0155] The processor 802 can be a general-purpose processor or a special-purpose processor (e.g., a digital signal processor, an application-specific integrated circuit (ASIC), etc.). The processor 802 can receive input from the communication interface 804 and process the input to generate output, which is stored in the data memory 806. The processor 802 can be configured to execute executable instructions 814 (e.g., computer-readable program instructions) that are stored in the data memory 806 and are executable to provide the functionality of the computing device 800 described herein.

[0156] The computing device 800 may also include an output interface 808 to output information to other devices. If the computing device 800 represents the display device 106, it also includes a display 810. The output interface 808 outputs information to the display 810 or to other components. Therefore, the output interface 808 can be a wireless interface (e.g., a transmitter) or a wired interface. The processor 802 can receive input from the communication interface 804 and process the input to generate output to the display 810.

[0157] In another example, the output interface 808 can output information in electronic form to provide feedback or commands to the robotic interface 816 to control the transport of the device through various control methods to allow the robot to change and manipulate the linear or rotational position of the device.

[0158] The output interface 808 can also provide relevant information and feedback to both on-site and remote supervisory personnel simultaneously through a feedback mechanism 818 (e.g., a visual or audio-visual feedback mechanism). The feedback mechanism 818 can include a display, an expansion device, a hologram device, a virtual / augmented reality wearable device or head-mounted device, an on-screen visual dashboard, audio sounds, an audio feedback wearable device, a tactile sensing device, and similar sensory feedback mechanisms.

[0159] Figure 13 is a flow chart of a method 900 for enhancing a real-time intraoperative X-ray video feed with anatomical and device-related overlays and indicators according to an example embodiment. The method 900 may be performed, for example, by the device deployment module 104.

[0160] Method 900 may include one or more operations or actions as shown in one or more of blocks 902-910, 1000, 1100, 1200, 1300, 1400, 1500, 1600, 1700, and 1800-1806. Although these blocks are shown in sequence, in some cases, these blocks may be performed in parallel and / or in a different order than described herein. Additionally, the various blocks may be combined into fewer blocks, divided into additional blocks, and / or deleted, depending on the desired implementation.

[0161] In addition, for method 900 and other processes and operations disclosed herein, the flowchart illustrates the operation of one possible implementation of the example given. In this regard, each block can represent a module, segment, or portion of program code, which includes one or more instructions that can be executed by a processor to implement a specific logical operation or step in the process. The program code can be stored in any type of computer-readable medium or memory, for example, a storage device including a disk or hard disk. The computer-readable medium can include non-transitory computer-readable medium or memory. For example, a computer-readable medium that stores data for a short time such as register memory, processor cache, and random access memory (RAM). The computer-readable medium can also include non-transitory media or memory, such as secondary or persistent long-term memory, such as read-only memory (ROM), optical disk or disk, compact disk read-only memory (CD-ROM). The computer-readable medium can also be any other volatile or non-volatile storage system. For example, the computer-readable medium can be considered to be a computer-readable storage medium, a tangible storage device, or other article of manufacture. In addition, for method 900 and other processes and operations disclosed herein, Figure 12 One or more blocks in the process may represent circuits or digital logic arranged to perform specific logical operations in the process.

[0162] At block 902 , the method 900 includes receiving, in real-time, at a processor (eg, the processor 802 of the device deployment module 104 ), a video stream of a body cavity captured by an image capture device 102 and an interventional procedure of a device being deployed within the body cavity.

[0163] At block 904 , method 900 includes identifying, by a processor, in the video stream, a body cavity and a device being deployed within the body cavity.

[0164] At block 906, method 900 includes visually superimposing, by a processor, body cavity markings indicating body cavity features on the video stream in real time, such as Figure 3 、 5 In an example, the user may be able to turn on and off the overlay of the body cavity marker or any other marker on the display as desired through the user interface (eg, using graphical user interface items such as menus, buttons, etc.).

[0165] At block 908, method 900 includes visually presenting, by the processor, a display of a device marker on the video stream in real time, the device marker indicating the location of the device in real time during deployment, e.g., Figure 2 、 4 , 6, and 8.

[0166] At block 910, method 900 includes providing, by a processor, a visual indication of device parameters in real time over a video stream, the device parameters including the fit of the device, such as Figure 4 、 7 shown.

[0167] Figure 14 is a flow chart of additional operations that may be performed using method 900 according to an example embodiment. The body cavity marking may include a curvature marking. At block 1000, operations include visually marking one or more portions of the body cavity where the curvature of the body cavity exceeds a threshold curvature to indicate the curvature of the body cavity to a surgeon performing an interventional procedure, e.g., Figure 3 The curvature marks 228-230 and Figure 5 The curvature marks 254 and 256 are shown in FIG.

[0168] Figure 15 is a flow chart of additional operations that may be performed using method 900 according to an example embodiment. The body cavity marking may include non-start and / or non-end areas that should be avoided when deploying the device. At block 1100, operations include visually marking one or more portions of the body cavity having bifurcated or hidden branches so as to inform the surgeon during deployment of the device that the one or more portions should be avoided as locations where the device end should be deployed, such as in Figure 3 As shown in the non-starting area 232, Figure 5As shown in the figure by non-starting area 258, and as shown in the figure by non-starting area 268.

[0169] Figure 16 is a flow chart of additional operations that may be performed using method 900 according to an example embodiment. At block 1200, operations include visually marking, by a processor, on a video stream in real time, indicia indicating the catheter and wire used to deploy the device, such as Figure 2 、 8 shown.

[0170] Figure 17 is a flow diagram of additional operations that may be performed using method 900 according to an example embodiment. At block 1300, operations include visually presenting indicators of the near and far ends of the device, such as Figure 6 shown.

[0171] Figure 18 is a flow chart of additional operations that may be performed using method 900 according to an example embodiment. In block 1400, operations include visually displaying, by a processor, a graphical representation of a landing probability in real time on a video stream, the landing probability indicating the probability that the proximal end of the device will be placed in a particular location at the end of deployment, e.g., Figure 6 In an example, the processor determines the probabilities taking into account foreshortening effects to determine the final length of the device as described above.

[0172] Figure 19 is a flow chart of additional operations that may be performed using method 900 according to an example embodiment. The processor may access a three-dimensional (3D) model of the device in 3D space (e.g., as described above with respect to block 204). At block 1500, operations include visually presenting a display of device markers that indicate the location of the device in real time during deployment based on the 3D model of the device, e.g., Figure 2 、 4 , as shown in 6.

[0173] Figure 20 is a flow chart of additional operations that may be performed using method 900 according to an example embodiment. The processor may access a 3D body cavity model of a body cavity generated from images collected by an image capture device (e.g., as described above with respect to block 220) prior to an interventional procedure. At block 1600, operations include visually superimposing body cavity markers indicating body cavity features based on the 3D body cavity model, such as Figure 3 、 5 shown.

[0174] Figure 21is a flow diagram of additional operations that may be performed according to an example embodiment using method 900. At block 1700, operations include visually providing, by a processor, to a surgeon performing an interventional procedure, a score indicative of device deployment quality, wherein the score considers one or more of: device adherence, proximity or overlap of the distal end of the device with a non-starting region or a "start here" region, taper angle, taper shape, braid angle confirmation and / or prediction, and deviation from a body lumen centerline.

[0175] Figure 22 is a flow chart of additional operations that may be performed using method 900 according to an example embodiment. In the example, as described above with respect to Figure 10-11 As described, the video stream includes a feed of low-dose X-ray images, and the processor includes a neural network model trained to identify devices and body cavities using the low-dose X-ray images.

[0176] Specifically, at block 1800, operations include receiving a low-dose X-ray image depicting a given device and a given body cavity (block 602 and low-dose X-ray image 700). At block 1802, operations include receiving a high-dose X-ray image corresponding to the low-dose X-ray image (block 606 and high-dose X-ray image 702). At block 1804, operations include receiving a 3D model of the device (block 604). At block 1806, operations include identifying the device in the low-dose X-ray image using the high-dose X-ray image and the 3D model of the device.

[0177] The above detailed description describes various features and operations of the disclosed system with reference to the accompanying drawings. The illustrative embodiments described herein are not intended to be limiting. Certain aspects of the disclosed system can be arranged and combined in a variety of different configurations, all of which are contemplated herein.

[0178] In addition, unless the context indicates otherwise, the features shown in each figure can be used in combination with each other. Therefore, the drawings should generally be viewed as forming aspects of one or more overall embodiments, but it should be understood that not all of the features shown are required for each embodiment.

[0179] In addition, any enumeration of elements, blocks or steps in this specification or claims is for clarity purposes. Therefore, such enumeration should not be interpreted as requiring or implying that these elements, blocks or steps follow a specific arrangement or are performed in a specific order.

[0180] In addition, devices or systems can be used or configured to perform the functions shown in the figures. In some cases, components of the devices and / or systems can be configured to perform the functions, such that the components are actually configured and constructed (using hardware and / or software) to achieve such performance. In other examples, components of the devices and / or systems can be arranged to be suitable, capable, or applicable to perform the functions, such as when operated in a particular manner.

[0181] The terms "substantially" or "approximately" indicate that the features, parameters or values ​​described need not be achieved precisely, but may exhibit deviations or changes, including, for example, tolerances, measurement errors, measurement precision limitations and other factors known to those skilled in the art, but the magnitude of these deviations or changes will not hinder the effect that the feature is intended to provide.

[0182] The arrangements described herein are for illustrative purposes only. Therefore, those skilled in the art will appreciate that other arrangements and other elements (e.g., machines, interfaces, operations, sequences, and groupings of operations, etc.) may be used instead, and that some elements may be omitted entirely, depending on the desired results. Furthermore, many of the elements described are functional entities that may be implemented as discrete or distributed components, or in combination with other components, in any suitable combination and location.

[0183] While various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for illustrative purposes only and are not intended to be limiting, with the true scope being indicated by the following claims and the full range of equivalents to which such claims are entitled. Furthermore, the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting.

[0184] Accordingly, embodiments of the present disclosure may be directed to one of the Enumerated Example Embodiments (EEE) listed below.

[0185] EEE1 is a method comprising: receiving, at a processor in real time, a video stream of an interventional procedure of a body cavity captured by an image capture device and a device being deployed within the body cavity; identifying, by the processor, the body cavity and the device being deployed within the body cavity in the video stream; visually superimposing, by the processor, body cavity markers indicating characteristics of the body cavity on the video stream in real time; visually presenting, by the processor, a display of device markers indicating a position of the device in real time during deployment on the video stream in real time; and providing, by the processor, visual indicators of device parameters on the video stream in real time.

[0186] EEE2 is the method of EEE1, wherein the body cavity marker includes a curvature marker, a size, and a positioning of the body cavity, and wherein visually superimposing the body cavity marker including the curvature marker includes: visually marking one or more portions of the body cavity where the curvature of the body cavity exceeds a threshold curvature to indicate the curvature of the body cavity to a surgeon performing an interventional procedure.

[0187] EEE3 is the method of any one of EEE1-2, wherein the body cavity marking includes non-starting areas and / or non-ending areas that should be avoided when deploying the device, and wherein visually superimposing the body cavity marking includes: visually marking one or more portions of the body cavity having bifurcations or hidden branches so as to inform the surgeon during deployment of the device that the one or more portions should be avoided as locations where the end of the device should be deployed.

[0188] EEE4 is the method of any one of EEE1-3, further comprising: visually marking, by a processor, on the video stream in real time, markings indicating a catheter and a wire used to deploy the device.

[0189] EEE5 is the method of EEE4, further comprising: visually presenting information indicating whether the catheter follows a preferred catheter trajectory guideline.

[0190] EEE6 is the method of any one of EEE1-5, wherein visually presenting, by the processor, a display of the device indicia in real time on the video stream includes visually presenting indicators of a near end and a far end of the device.

[0191] EEE7 is the method of EEE6, further comprising: visually displaying, by the processor, in real time on the video stream, a graphical representation of a landing probability, the landing probability indicating a probability that the proximal end of the device will be placed at a particular location when deployment is complete.

[0192] EEE8 is the method of EEE7, wherein determining the probability takes into account a foreshortening effect to determine a final length of the device.

[0193] EEE9 is a method of any one of EEE1-8, wherein the processor accesses a three-dimensional (3D) model of the device in a 3D space, and wherein display of a device marker visually presenting a position of the device in real time during deployment is based on the 3D model of the device.

[0194] EEE10 is the method of any one of EEE1-9, wherein the processor accesses a 3D body cavity model of the body cavity, which is generated from images previously collected via an image capture device, and wherein visually superimposing body cavity markings indicating body cavity features is performed based on the 3D body cavity model.

[0195] EEE11 is a method of any one of EEE1-10, further comprising: generating, by a processor, a score indicating the quality of device deployment, wherein the score takes into account one or more of: adherence of the device, proximity or overlap of the distal end of the device to a non-starting area or a "start here" area, taper angle, taper shape, braid angle and braid density confirmation and / or prediction, deviation from a defined path of a body cavity including a centerline of an ideal deployment path; and (i) providing feedback indicating the score to a user via audio-visual feedback, virtual reality, or augmented reality display, or (ii) providing feedback indicating the score to a robotic interface delivered to a control device to allow the robot to change the linear or rotational position of the device.

[0196] EEE12 is the method of any one of EEE1-11, wherein the video stream includes a feed of low-dose X-ray images, wherein the processor includes a neural network model trained to identify the device and the body cavity using the low-dose X-ray images.

[0197] EEE13 is the method of EEE12, wherein the neural network model is trained by: receiving a low-dose X-ray image depicting a given device and a given body cavity; receiving a high-dose X-ray image corresponding to the low-dose X-ray image; receiving a 3D model of the device; and identifying the device in the low-dose X-ray image using the high-dose X-ray image and the 3D model of the device.

[0198] EEE14 is a system comprising: an image capture device configured to capture, in real time, a video stream of an interventional procedure involving a body cavity and a device being deployed within the body cavity; a display device in communication with the image capture device and configured to display the video stream; and a device deployment module in communication with the image capture device and the display device, wherein the device deployment module comprises a processor and a non-transitory computer-readable medium having a plurality of executable instructions stored therein, the executable instructions, when executed by the processor, causing the device deployment module to perform operations including those of EEEs 1-13. For example, the operations may include: receiving a video stream; identifying a body cavity and a device being deployed within the body cavity in the video stream; visually superimposing a body cavity marker indicating features of the body cavity on the video stream displayed on the display device in real time; visually presenting a display of the device marker on the video stream displayed on the display device indicating a position of the device in real time during deployment; and providing a visual indicator of parameters of the device on the video stream displayed on the display device.

[0199] EEE15 is the system of EEE14, wherein the body cavity marker includes a curvature marker, a non-start and / or non-end area to be avoided when deploying the device, wherein visually superimposing the body cavity marker including the curvature marker includes: visually marking one or more portions of the body cavity where the curvature of the body cavity exceeds a threshold curvature to indicate the curvature of the body cavity to a surgeon performing an interventional procedure; and visually marking corresponding one or more portions of the body cavity having bifurcations or hidden branches to inform the surgeon during deployment of the device that the corresponding one or more portions should be avoided as the proximal location where the device should be deployed.

[0200] EEE16 is the system of any of EEE14-15, wherein the display of the device marker visually presented in real time on the video stream includes: visually presenting indicators of the proximal and distal ends of the device; and visually displaying in real time on the video stream a graphical representation of a landing probability, the landing probability indicating a probability that the proximal end of the device will be placed in a particular location at the end of deployment, wherein the probability is determined to take into account a perspective foreshortening effect to determine a final length of the device.

[0201] EEE17 is a system of any one of EEE14-16, wherein the device deployment module accesses (i) a three-dimensional (3D) model of the device in 3D space, and (ii) a 3D body cavity model of the body cavity generated from images previously collected via an image capture device, and wherein: the display of device markers that visually present the position of the device in real time during deployment is based on the 3D model of the device, and the visual superposition of body cavity markers that indicate body cavity features is based on the 3D body cavity model.

[0202] EEE18 is a system of any of EEE14-17, wherein the operation further comprises: generating a score indicating the quality of device deployment, wherein the score takes into account one or more of: adherence of the device, proximity or overlap of the distal end of the device to a non-starting area or a "start here" area, taper angle, taper shape, braid angle and braid density confirmation and / or prediction, deviation from a defined path of a body cavity including a centerline of an ideal deployment path; and (i) providing feedback indicating the score to a user via audio-visual feedback, virtual reality, or augmented reality display, or (ii) providing feedback indicating the score to a robotic interface delivered to a control device to allow the robot to change the linear or rotational position of the device.

[0203] EEE19 is the system of any one of EEE14-18, wherein the video stream comprises a feed of low-dose X-ray images, wherein the device deployment module comprises a neural network model trained to identify the device and the body cavity using the low-dose X-ray images.

[0204] EEE20 is the system of EEE19, wherein the neural network model is trained by: receiving a low-dose X-ray image depicting a given device and a given body cavity; receiving a high-dose X-ray image corresponding to the low-dose X-ray image; receiving a 3D model of the device; and identifying the device in the low-dose X-ray image using the high-dose X-ray image and the 3D model of the device.

[0205] EEE21 is a method comprising: receiving, at a processor, in real time, a video stream of an interventional procedure of a body cavity captured by an image capture device and a device being deployed within the body cavity; identifying, by the processor, the body cavity and the device being deployed within the body cavity in the video stream; visually presenting, by the processor, a body cavity marker indicating characteristics of the body cavity; visually presenting, by the processor, a display of a device marker indicating, in real time, the position of the device during deployment; and providing, by the processor, a visual indicator of device parameters on the video stream in real time.

[0206] EEE22 is the method of EEE21, further comprising: generating a display of a circumferential ring representing a wall of the body cavity.

[0207] The method of EEE21 may also include any other operations or steps of EEE1-13.

[0208] EEE23 is a method comprising: receiving, at a processor, in real time, a video stream of an interventional procedure of a body cavity captured by an image capture device and a device being deployed within the body cavity; identifying, by the processor, the body cavity and the device being deployed within the body cavity in the video stream; visually presenting, by the processor, a display of device markers indicating, in real time, the position of the device in the body cavity during deployment; and providing, by the processor, a visual indicator of device parameters on the video stream in real time.

[0209] EEE24 is the method of EEE23, further comprising: visually presenting, by the processor, a body cavity marking indicating a feature of the body cavity.

[0210] EEE25 is the method of EEE24, wherein visually presenting the body cavity marker indicating the body cavity feature includes: visually superimposing, by a processor, the body cavity marker indicating the body cavity feature on the video stream in real time.

[0211] The method of EEE23 may also include any other operations or steps of EEE1-13.

[0212] The system of EEE14 can also perform any operations of EEE21-22 and EEE23-25.

Claims

1. A method comprising: receiving, in real time at a processor, a video stream of the interventional procedure of the body cavity and the device being deployed within the body cavity as captured by the image capture device; identifying, by a processor, in the video stream, a body cavity and a device being deployed within the body cavity; visually superimposing, via a processor, body cavity markers indicating body cavity features on the video stream in real time; visually rendering, by the processor, a display of device markers indicating in real time the location of the device during deployment on the video stream; as well as Visual indicators of device parameters are provided in real time over the video stream by the processor.

2. The method according to claim 1, wherein The body cavity marker includes a curvature marker, a size, and a position of the body cavity, wherein visually superimposing the body cavity marker including the curvature marker includes: One or more portions of the body lumen where the curvature of the body lumen exceeds a threshold curvature are visually marked to indicate the curvature of the body lumen to a surgeon performing an interventional procedure.

3. The method according to claim 1, wherein The body cavity marker includes a non-starting region and / or a non-ending region to be avoided when deploying the device, and wherein visually superimposing the body cavity marker includes: One or more portions of the body lumen having bifurcations or hidden branches are visually marked to inform the surgeon during deployment of the device that the one or more portions should be avoided as locations for deploying the end of the device.

4. The method according to claim 1, further comprising: Markings indicating the catheter and wire used to deploy the device are visually marked in real time on the video stream by a processor.

5. The method according to claim 4, further comprising: Information indicating whether the catheter is following a preferred catheter trajectory guideline is visually presented.

6. The method according to claim 1, wherein Displays that visually render device markers in real time on a video stream via a processor include: Indicators of the proximal and distal ends of the device are visually presented.

7. The method according to claim 6, further comprising: A graphical representation of landing probability is visually displayed in real time by a processor over the video stream, the landing probability indicating the probability that the proximal end of the device will be placed in a particular location when deployment is complete.

8. The method according to claim 7, wherein Determining the probability takes into account foreshortening effects to determine the final length of the device.

9. The method according to claim 1, wherein The processor accesses a three-dimensional (3D) model of the device in 3D space, and wherein displaying of a device marker visually presenting a real-time indication of a position of the device during deployment is based on the 3D model of the device.

10. The method according to claim 1, wherein The processor accesses a 3D body cavity model of the body cavity, which is generated from images previously collected via an image capture device, and wherein visually superimposing body cavity markings indicative of body cavity features is performed based on the 3D body cavity model.

11. The method according to claim 1 , further comprising: generating, by a processor, a score indicative of device deployment quality, wherein the score considers one or more of: device adherence, proximity or overlap of the distal end of the device with a non-starting zone or "start here" region, taper angle, taper shape, braid angle and braid density confirmation and / or prediction, deviation from a defined path of the body lumen including a centerline of an ideal deployment path; and (i) providing feedback indicative of the score to a user via audio-visual feedback, a virtual reality, or an augmented reality display, or (ii) providing feedback indicative of the score to a robotic interface that controls the device to allow the robot to change the linear or rotational position of the device.

12. The method according to claim 1, wherein The video stream includes a feed of low-dose X-ray images, wherein the processor includes a neural network model trained to identify the device and the body cavity using the low-dose X-ray images.

13. The method according to claim 12, wherein: The neural network model is trained in the following way: receiving a low-dose X-ray image depicting a given device and a given body cavity; receiving a high-dose X-ray image corresponding to the low-dose X-ray image; receiving a 3D model of the device; and The device in the low-dose X-ray image is identified using the high-dose X-ray image and the 3D model of the device.

14. A system comprising: an image capture device configured to capture, in real time, a video stream of an interventional procedure involving the body cavity and a device being deployed within the body cavity; a display device in communication with the image capture device and configured to display the video stream; as well as A device deployment module in communication with an image capture device and a display device, wherein the device deployment module comprises a processor and a non-transitory computer-readable medium having a plurality of executable instructions stored therein, the executable instructions, when executed by the processor, causing the device deployment module to perform operations comprising: Receive video stream; identifying a body cavity and a device being deployed within the body cavity in a video stream; visually superimposing a body cavity marker indicating a feature of the body cavity in real time on a video stream displayed on a display device; visually presenting a display of device indicia indicating a location of the device in real time during deployment on a video stream displayed on a display device; as well as Visual indicators of parameters of the device are provided over a video stream displayed on a display device.

15. The system according to claim 14, wherein: The body cavity marker includes a curvature marker, and a non-starting and / or non-ending area to be avoided when deploying the device, wherein visually superimposing the body cavity marker including the curvature marker comprises: visually marking one or more portions of the body lumen where the curvature exceeds a threshold curvature to indicate the curvature of the body lumen to a surgeon performing an interventional procedure; and The corresponding one or more portions of the body lumen having bifurcations or hidden branches are visually marked to inform a surgeon during deployment of the device that the corresponding one or more portions should be avoided as locations where the proximal end of the device should be deployed.

16. The system of claim 14, wherein: Displays that visually present device markers in real-time on a video stream include: visually presenting indicators of the proximal and distal ends of the device; and A graphical representation of landing probability is visually displayed in real time on the video stream, the landing probability indicating the probability that the proximal end of the device will be placed in a particular location at the end of deployment, wherein the probability is determined to account for foreshortening effects to determine a final length of the device.

17. The system of claim 14, wherein: The device deployment module accesses (i) a three-dimensional (3D) model of the device in 3D space, and (ii) a 3D body lumen model of the body lumen generated from images previously collected via an image capture device, and wherein: The display of a device marker that visually presents a real-time indication of the location of the device during deployment is based on the 3D model of the device, and Visually superimposing body cavity marks indicating body cavity features is performed based on the 3D body cavity model.

18. The system according to claim 14, wherein: The operations further include: generating a score indicative of device deployment quality, wherein the score considers one or more of: device adherence, proximity or overlap of the distal end of the device with a non-starting zone or "start here" region, taper angle, taper shape, braid angle and braid density confirmation and / or prediction, deviation from a defined path of the body lumen including a centerline of an ideal deployment path; and (i) providing feedback indicative of the score to a user via audio-visual feedback, a virtual reality, or an augmented reality display, or (ii) providing feedback indicative of the score to a robotic interface that controls the device to allow the robot to change the linear or rotational position of the device.

19. The system of claim 14, wherein: The video stream includes a feed of low-dose X-ray images, wherein the device deployment module includes a neural network model trained to identify the device and the body cavity using the low-dose X-ray images.

20. The system of claim 19, wherein: The neural network model is trained in the following way: receiving a low-dose X-ray image depicting a given device and a given body cavity; receiving a high-dose X-ray image corresponding to the low-dose X-ray image; receiving a 3D model of the device; and The device in the low-dose X-ray image is identified using the high-dose X-ray image and the 3D model of the device.

21. A method comprising: receiving, at a processor, in real time, a video stream of the body cavity captured by the image capture device and the interventional procedure of the device being deployed within the body cavity; identifying, by a processor, in the video stream, a body cavity and a device being deployed within the body cavity; visually presenting, by a processor, body cavity markings indicative of characteristics of the body cavity; visually presenting, by a processor, a display of device indicia indicating, in real time, the location of the device during deployment; as well as Visual indicators of device parameters are provided in real time over the video stream by the processor.

22. The method according to claim 21, further comprising: Generates a display of circumferential rings representing the body cavity wall.

23. A method comprising: receiving, at a processor, in real time, a video stream of the body cavity captured by the image capture device and the interventional procedure of the device being deployed within the body cavity; identifying, by a processor, in the video stream, a body cavity and a device being deployed within the body cavity; visually presenting, by the processor, a display of device markings indicating in real time the position of the device in the body cavity during deployment; as well as Visual indicators of device parameters are provided in real time over the video stream by the processor.

24. The method according to claim 23, further comprising: Body lumen markings indicative of characteristics of the body lumen are visually presented by the processor.

25. The method according to claim 24, wherein Body cavity markings that visually indicate body cavity characteristics include: Body cavity markers indicating features of the body cavity are visually superimposed on the video stream in real time by a processor.

Images