Navigation of a slender robotic drive

Abstract

To improve workflow and provide further assistance in manipulating an interventional device with a robotic configuration, a navigation system is provided that supports robotic anatomical path navigation of an elongated, robotically driven device, including a data input, a data processor, and an output interface. The data input receives and provides to the data processor robot-related data of a designated robot provided to perform an endoluminal-related task, device-related data related to intrinsic or mechanical properties of the elongated device driven by the designated robot, and subject-related anatomical path image data of a region of interest through which the device will be moved by the designated robot. The data processor calculates an estimate of a range of motion for the device within the anatomical path based on the robot-related data, the device-related data, and the subject-related anatomical path image data. The output interface provides the estimated range of motion.

Description

[Technical Field]

[0001] The present invention relates to navigation of elongated robotically-driven devices, and more particularly to a navigation system for assisting robotic anatomical path navigation of elongated robotically-driven devices, a system for robotic anatomical path navigation, and a method for assisting robotic anatomical path navigation of elongated robotically-driven devices. [Background technology]

[0002] Robotic solutions for treating endovascular diseases have been increasing in recent years. Applications of robotic-assisted procedures span multiple interventions, including coronary, peripheral, and neurovascular interventions. Various designs have been used for robotic systems, including articulated arms or motorized robotic modules that can translate and rotate vascular devices. Depending on the intervention and robot design, different devices with different ranges of motion and characteristics are required. For example, robotic-assisted neuroendovascular procedures (e.g., stroke, aneurysm, etc.) require a wider range of motion for the device to reach targets within the carotid artery and brain. Robotic endovascular navigation provides different endovascular devices that can be exchanged. However, procedure interruptions for device changes, which require finding a new device, unloading the current device, and reloading the new device in a specific manner, can be cumbersome. Most importantly, exchanges have been shown to significantly increase the risk of complications, such as dissection and thrombosis, and waste valuable time. Summary of the Invention [Problem to be solved by the invention]

[0003] Therefore, there may be a need to improve workflow and further assist in the manipulation of interventional devices with robotic configurations. [Means for solving the problem]

[0004] The object of the present invention is solved by the subject matter of the independent claims, further embodiments are incorporated in the dependent claims. It is noted that the below described aspects of the invention also apply to a navigation system for assisting robotic anatomical path navigation of an elongated robotic driven device, a system for robotic anatomical path navigation and a method for assisting robotic anatomical path navigation of an elongated robotic driven device.

[0005] According to the present invention, there is provided a navigation system for assisting robotic anatomical path navigation of an elongated robotically driven device. The system includes a data input, a data processor, and an output interface. The data input is configured or enabled to receive robot-related data for a designated robot configured to perform at least one endoluminal-related task and / or drive the elongated drive. Such robot-related data may include data related to driving the elongated device by the robot. The data input is configured or enabled to receive device-related data related to intrinsic and / or mechanical properties of at least one elongated device driven by the designated robot. The data input is further configured or enabled to receive subject-related anatomical path image data of a region of interest through which the device is moved by the designated robot. The data input is configured to provide data to the data processor. A non-transitory machine-readable storage medium or memory may be provided and encoded with instructions for execution by the data processor. The data processor is configured to calculate an estimate of a range of motion for at least one device within the anatomical path based on the robot-related data, the device-related data, and the subject-related anatomical path image data (which is an input for the estimation calculation). The output interface is configured to provide an estimated operating range.

[0006] The effect is that providing information about the travel limitations of devices used in robotic-assisted interventions allows for optimized use of the devices. For example, being shown that the range of motion is sufficient for a particular task provides the user with confidence that a sudden need for device replacement will be avoided. Alternatively, if the range of motion is shown to be insufficient, the user can consider replanning or changing the device at an appropriate time, for example, before navigating further into the vasculature. Unwanted procedure interruptions are either avoided or at least better integrated into the workflow. The provided knowledge about the (estimated) range of motion allows the user to smooth the workflow. Thus, knowledge about the range of motion supports successful navigation.

[0007] For example, the operating range of a device changes dynamically as the user navigates. This means that the maximum reach of the robotic controller changes depending on sag, twisting, device buckling, and energy storage along the way. Thus, a device that was initially expected to have sufficient length to reach a goal may, midway through navigation, appear to have insufficient length, suggesting that the user replace the device early or remove energy storage from the system.

[0008] In one example, at least one iteration of updating the estimated operating range is provided.

[0009] As an advantage, knowledge of i) the specific device, ii) the specific current anatomical situation, and iii) the range of motion of the current robotic configuration assists the user in manipulating the interventional device(s) with the robotic configuration, thus improving the workflow.

[0010] In one example, the term robot configuration relates to the placement and state of the robot.

[0011] According to one example, the data processor is configured to calculate, for the at least one device, estimates of ranges of motion along different possible paths, and the output interface is configured to output the ranges of motion along the different possible paths.

[0012] Optionally, a rating is calculated for different paths based on factors such as the amount of sag and energy accumulation, which can be pre-defined weighting factors and other inputs, such as path tortuosity, number of branching passages, narrowing passages, width of vessels, risk significance of vessel passages, etc. The determined rating value is displayed to the user.

[0013] According to one example, the data input is configured to provide device-related data including a plurality of device-related data for a plurality of different devices, the data processor is configured to calculate a plurality of estimates for the operating range based on the plurality of device-related data, and the output interface is configured to output the operating range along different possible paths.

[0014] Optionally, a rating is calculated for different devices based on predetermined weighting factors such as reach, maneuverability, etc. The determined rating value is displayed to the user.

[0015] According to one example, the range of motion is provided as graphical information overlaid on anatomical image data of the region of interest. Additionally or alternatively, as an option provided, the output interface is configured to provide the estimated range of motion as an image matrix in which the maximum range of motion of each device in each path is uniquely marked in the image.

[0016] According to one example, the data input is configured to provide workflow data, and the data processor is configured to calculate at least one estimate of the operating range also based on the workflow data.

[0017] According to one example, a neural network-based controller is provided having a convolution filter configured to capture contextual patterns in the image data, and optionally, a data processor is provided, optionally configured to calculate estimates based on training of the neural network.

[0018] According to the present invention, there is also provided a system for robotic anatomical path navigation, the system comprising a robotic arrangement according to one of the previous examples and a navigation system, the robotic arrangement configured to control and drive at least one device for insertion into and movement along an anatomical path of a region of interest of a subject, the navigation system configured to provide an estimate of a range of motion of the at least one device within the anatomical path of the region of interest of the subject.

[0019] According to one example, an imaging arrangement configured to provide subject-related anatomical path image data is provided, where the subject-related anatomical path image data is additionally or alternatively optionally provided as a 2D X-ray image.

[0020] This allows the use of live images to estimate the range of motion, which on the one hand simplifies the procedure since no further data on the current / actual anatomy is required, and on the other hand provides the most accurate information on the current anatomy through which the device needs to be navigated.

[0021] In accordance with the present invention, there is also provided a method for assisting robotic anatomical path navigation of an elongated robotic drive, the method comprising: receiving robot-related data for a designated robot configured to perform at least one endoluminal-related task; receiving device-related data relating to intrinsic or mechanical characteristics of at least one elongate device driven by a designated robot; receiving subject-related anatomical path image data of a region of interest through which a device will be moved by a designated robot; calculating an estimate of a range of motion for at least one device within the anatomical path based on the robot-related data, the device-related data, and the subject-related anatomical path image data; outputting the estimated operating range; It has.

[0022] According to one aspect, each device's travel limit depends on the relationship between the current device, the robotic system, and the patient's anatomy as observed in the interventional images. This is provided to dynamically calculate and display the maximum reach of a robotically controlled intravascular device by combining imaging interpretation, device intelligence, and robot-related data (which may include, for example, robot configuration data such as the robot's configuration (e.g., geometry, joint configuration, other design data, robot intrinsic parameters, robot state, robot settings, alignment and / or calibration data, etc.)) with drive-related feedback data (e.g., kinematics, encoder data from robot motors, motion data (rotational, translational, and / or rolling, and / or positioning data, etc.)). The maximum reach of an intravascular device can also be referred to as its range of motion. Factors that determine the device's maximum reach in the anatomy depend on the robotic system design, the placement of the intravascular device on the robot, the length of the device, the location of the access site (e.g., radius or femur), and the patient's anatomy. Information regarding the range of motion can be presented as a graphical overlay, text, audio, or haptic feedback.

[0023] In one example, a procedure may use multiple devices simultaneously, each with its own range of motion and optimal position for supporting distal devices, if any, within it.

[0024] The present invention focuses on combining new robotic technologies with interventional imaging platforms. The motion range estimation system can be used, for example, with fixed C-arm systems and mobile fluoroscopy systems. As an example, the motion range estimation system is designed to work with intervention-guided therapy devices.

[0025] As an advantage, the usability and interface of the image-guided robotic system is enhanced, which can be made available as part of the robot core interface or in a software as a service solution.

[0026] According to one aspect, the range of motion of a device driven by a robotic device is determined based on the current anatomical situation. The process of calculating the range of motion is based, inter alia, on image data representative of the current situation. Image data is acquired, and possible paths within a given anatomical structure are determined for a given device or selection of possible devices.

[0027] In a further example, multiple ways of using the image data are provided. In one example, only the last image of the patient is used, capturing the device in its most recent position within the anatomy and vasculature, for example if the device is already in view.

[0028] Another approach uses a sequence of images from the past to the present. In this case, for example, the last 100 x-ray images from the same patient showing how the device has moved through the vasculature so far are used during training / inference. These images can be from the same anatomical region or from various anatomical regions and different views stitched together.

[0029] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter.

[0030] Exemplary embodiments of the present invention are described below with reference to the following drawings: [Brief explanation of the drawings]

[0031] [Figure 1] 1 illustrates a schematic diagram of an example of a navigation system for assisting robotic anatomical path navigation of an elongated robotic drive. [Figure 2] 1 illustrates an example of a system for robotic anatomical path navigation. [Figure 3] 1 illustrates the basic steps of an example method for assisting robotic anatomical path navigation of an elongated robotic drive. [Figure 4] 1 shows another example of a schematic setup for assisting robotic anatomical path navigation. [Figure 5] 4 shows a further example of an operating scheme. [Figure 6] 1 shows an example of a presentation shown on a display. [Figure 7] 1 shows an example of an uncertainty indicator. DETAILED DESCRIPTION OF THE INVENTION

[0032] Specific embodiments will now be described in detail with reference to the accompanying drawings. In the following description, like drawing reference numbers are used for like elements, even in different drawings. Matters defined herein, such as detailed configurations and elements, are provided to facilitate a comprehensive understanding of the exemplary embodiments. Additionally, well-known functions or configurations will not be described in detail since they would obscure the embodiments in unnecessary detail. Furthermore, phrases such as "at least one of," when preceding a list of elements, modify the entire list of elements, and not individual elements of the list.

[0033] FIG. 1 schematically illustrates an example of a navigation system 10 for assisting robotic anatomical path navigation of an elongated, robotically driven device. The system 10 includes a data input 12, a data processor 14, and an output interface 16. The data input 12 is configured to receive robot-related data for a designated robot configured to perform at least one endoluminal task. The data input 12 is also configured to receive device-related data related to intrinsic or mechanical properties of at least one elongated device driven by the designated robot. The data input 12 is further configured to receive subject-related anatomical path image data of a region of interest through which the device will be moved by the designated robot. The data input 12 is also configured to provide this data to the data processor 14. The data processor 14 includes a memory (not shown in detail) for storing instructions and a processor for executing the instructions. The data processor 14 is configured to calculate an estimate of a range of motion for at least one device within an anatomical path based on the robot-related data, device-related data, and subject-related anatomical path image data that are input to the estimation calculation. The output interface 16 is configured to provide the estimated range of motion.

[0034] The three dashed arrows indicate input of different types of data, with the first arrow 18 indicating a feed of robot-related data. The second arrow 20 indicating a feed of device-related data. The third arrow 22 indicating a feed of subject-related anatomical path image data. The fourth dashed arrow 22 indicates an output of data, i.e., providing a calculated estimate of the range of motion. Optionally, the estimate is displayed on a display 26.

[0035] Frame 28 shows that in one option, data input 12, data processor 14 and output interface 16 are provided within a common housing. In another option, data input 12, data processor 14 and output interface 16 are provided as individual components.

[0036] In an example not shown in further detail in the drawings, the operating range includes at least one parameter from the group including travel limit, working length, and maximum reach.

[0037] FIG. 2 illustrates an example system 100 for robotic anatomical path navigation. Note that the system is shown in the context of an operating room, optionally with other equipment such as an imaging system. The system 100 can also be provided as a standalone solution. The system 100 includes a robotic configuration 102 and an example navigation system 104 according to one of the preceding examples. The robotic configuration 102 is configured to control and drive, e.g., manipulate, at least one device 106 that is inserted into and moves along an anatomical path of a region of interest of a subject 108. The navigation system 10 is configured to provide an estimate of the range of motion of the at least one device 106 within the anatomical path of the region of interest of the subject 108.

[0038] The robotic configuration 102 is shown only diagrammatically. The robotic configuration 102 can be a standalone robot with a stand and one or more manipulators. The robotic configuration 102 can also be integrated into other mobile equipment in the operating room, such as an operating table. The robotic configuration 102 can be supported on the floor, attached to a wall structure, or suspended from the ceiling. The robotic configuration 102 can have one or several "arms" as manipulators. The manipulators can have multiple degrees of freedom of movement, such as three degrees of freedom, or four, five, or six degrees of freedom.

[0039] The robotic system can optionally include a tracking system, such as those used in surgical navigation systems based on IR light or EM tracking or (optical) shape sensing technology. Such a tracking system can track the position of the robotic configuration relative to the patient and / or to equipment in the operating room, such as an X-ray system. In addition to internal encoding methods (e.g., motor encoders, linear encoders, etc.), the tracking system can provide information about the proximity of the robotic components to the patient access site and their relative position within the configuration. This is one way to calibrate and / or track the relative displacement of the device relative to the patient and the device itself. Alternatively, the tracking system can track the device directly using EM markers or optically tracked fiducials attached to the device, or using general computer vision methods with a stereo camera.

[0040] Optionally, the subject 108 is shown positioned on a subject support 110. A bedside controller 112 may be located nearby. A suspended monitor arrangement 114 is also shown. Also shown is a console 116 for operating equipment in the operating room. The console 116 can be located in the same room or in a separate room.

[0041] Optionally, an imaging arrangement 118 configured to provide subject-related anatomical pathway image data is provided. By way of example, the imaging arrangement 118 is an X-ray imaging system having a movably mounted C-arm 120 with an X-ray source 122 and a detector 124 mounted at opposite ends of the C-arm 120.

[0042] A first communication line 126 represents a data connection between the robot configuration 102 and the instance 104 of the navigation system 10. A second communication line 128 represents a data connection between the imaging configuration 118 and the instance 104 of the navigation system 10. A third communication line 130 represents a data connection between the instance 104 of the navigation system 10 and the console 116.

[0043] The data connection can be provided as a wired or wireless communication.

[0044] Optionally, subject-related anatomical pathway image data is provided as 2D X-ray images.

[0045] FIG. 3 shows the basic steps of an example method 200 for assisting robotic anatomical path navigation of an elongated robotically driven device. The method 200 comprises the following steps: In a first sub-step 202, robot-related data is received for a designated robot provided to perform at least one endoluminal-related task; in a second sub-step 204, device-related data related to intrinsic or mechanical properties of at least one elongated device driven by the designated robot is received; in a third sub-step 206, subject-related anatomical path image data is received for a region of interest in which the device is to be moved by the designated robot; the three sub-steps can be performed simultaneously or sequentially in any order. In a further step 208, an estimate of the range of motion is calculated for at least one device within the anatomical path based on the robot-related data, the device-related data, and the subject-related anatomical path image data. In a next step 210, the estimated range of motion is provided, for example, to an operator.

[0046] Returning to Figures 1 and 2, a device for intraluminal related tasks, ie, a device 106 that is inserted into and moves along an anatomical path in a region of interest of a subject, may be referred to as an interventional device.

[0047] In one example, the elongated robotic drive 106 is configured to be navigated within an anatomical structure. The anatomical structure may refer to any type of structure in a subject having different characteristics for navigating the device. In one example, the anatomical structure may refer to a vessel or lumen, such as a vasculature or organ.

[0048] The navigation system 10 for robotic anatomical path navigation may also be referred to as a navigation device (for robotic anatomical path navigation) or a navigation arrangement (for robotic anatomical path navigation).

[0049] In a first option, the navigation system 10 is provided as a data processing configuration. In one example, the navigation system is provided as a kit for upgrading an existing system. In another example, the navigation system is provided as a separate part of a system.

[0050] In a second option, the navigation system 10 also comprises at least one device 106. As an example, the navigation system 10 is provided as a set of devices 10, for example a navigation set with several different interventional devices.

[0051] In one example, when executed by the data processor 14, the instructions cause the system to receive at least one of the following group: robot-related data, positioning or motion data, equipment-related data, and image data.

[0052] The term "anatomical pathway navigation" refers to navigating, i.e., guided movement along a given anatomical lumen suitable for insertion of a device, such as a guidewire or catheter. The anatomical pathway or lumen can be any hollow anatomical structure, such as the vasculature, the airway, or the intestinal or gastrointestinal tract. In one example, the anatomical pathway or lumen is a naturally occurring pathway.

[0053] The term "data input 12" refers to providing or supplying data for a data processing step. The data input 12 can also be referred to as an image data input 12. The data input 12 can also be referred to as a data supply, image data supply, image input, input unit, or simply input. In one example, the data input 12 is data connectable to an imaging source arrangement. The data input 12 is configured to receive data, for example from a respective data source, and forward it to the data processor 14.

[0054] The term "data processor 14" refers to a processor or part of a processor configuration provided to perform calculation steps using data provided by a data input. The data processor 14 may also be referred to as a data processor, a processor unit, or a processor. In one example, the data processor 14 is data-connected to a data input and output interface. Optionally, the data processor 14 is configured to calculate the range of a device when manipulated and moved by a robot. Thus, the data processor 14 may be referred to as a "range calculation unit." The range calculation unit receives data collected before, during, or after a robotic-assisted surgery and calculates the possible range of motion of each device, e.g., particularly a flexible device, relative to a patient, imaging, or robotic reference frame.

[0055] The term "output interface 16" refers to an interface for providing processed or calculated data for further purposes. The output interface 16 may also be referred to as an output or an output unit. In one example, the output interface 16 may be data-connectable to a display arrangement or display device. In another example, the output interface 16 is data-connected to a display. As an example, a signal from a controller may be provided by the output interface 16. The output interface 16 is configured to receive data from the data processor 14 and forward it to, for example, a display.

[0056] The term "robot-related data" refers to data from the use of a robot. The term "robot-related data" refers to at least one of the group including, for example, coding data, kinematics, end-effector pose, robot tracker data, and robot user control inputs for moving the robot. In one example, the robot-related data includes robot movement-related data, such as movement capabilities, options, and limitations. In another example, the robot-related data includes robot state-related data, such as robot status data. As another option, the robot-related data also includes robot calibration data. The robot-related data may also be referred to as robot data.

[0057] The term "device-related data" refers to data of the mechanical characteristics of a device, such as total length, working length, width, X-ray opacity, stiffness, shape, joint model (for maneuverable devices), attachment method to a robot, etc. Device-related data includes data related to the device. As an example, device-related data includes the intrinsic characteristics of the device and parameters of its interface with a robot. Device-related data can also be referred to as device data. The working length may be the length of a section that can fit inside another device.

[0058] The term "subject-related anatomical path image data" refers to navigation image data from an imaging system. In one example, the anatomical path image data is provided as vasculature image data. The subject-related anatomical path image data may also be referred to as subject anatomical path image data or subject path image data.

[0059] The term "endoluminal related tasks" refers to tasks within a lumen, such as within a blood vessel, the respiratory system, or other hollow section within the anatomy. The term "endoluminal related tasks" can also be referred to as "endovascular related tasks."

[0060] The term "operating range" refers to at least one of the group consisting of travel limit, working length, and maximum reach.

[0061] The imaging data can be two-dimensional (2D) or three-dimensional (3D). Preferably, 2D image data, such as live fluoroscopic images, is provided.

[0062] Optionally, anatomical path navigation refers to intravascular navigation, and device-related data is data related to an intravascular device.

[0063] In one example, the output interface 16 is configured to provide an estimated operating range to assist the operator.

[0064] In one example, the data processor 14 is configured to calculate, based on the robot-related data, an estimate of a range of motion for at least one device within the anatomical path relative to the target data.

[0065] According to the present invention, the "estimation", or better the corresponding model, is based on multiple movement probabilities of, for example, an elongated device type with kinematics within the constraints of the anatomical environment.

[0066] For example, the model is obtained by training a neural network using retrospective robot data and imaging data. The neural network can be designed with multiple channels in the output layer, each channel corresponding to a distinct possible estimate of the device constraint for a different movement probability.

[0067] The term "travel limit" relates to the distance that a device can travel when moved by a robot, for example, the distance along a particular travel path.

[0068] The term "working length" refers to the length of the vessel section over which the interventional device is moved, along which the interventional device is capable of operating. Thus, the working length may be a portion of the travel distance. In one example, the working length is the amount that the device (tip) can reach into the vasculature given constraints.

[0069] The term "maximum reach" relates to the maximum possible distance that the device can reach. For example, the maximum reach is the actual position within a blood vessel. The maximum reach may be the distance at which the working function is still possible. In another option, the maximum reach may be longer than the working length.

[0070] In an example not shown in further detail in the drawings, the data processor 14 is configured to calculate estimates of the range of motion along different possible paths for at least one device, and the output interface 16 is further configured to output the range of motion along the different possible paths.

[0071] In one example, not shown in further detail in the drawings, data input 12 is configured to provide device-related data including a plurality of device-related data for a plurality of different devices. Data processor 14 is configured to calculate a plurality of estimates for the operating range based on the plurality of device-related data. Furthermore, output interface 16 is configured to output the operating range along different possible paths.

[0072] Optionally, estimates of the operating range along different possible paths are provided for a number of different devices.

[0073] As an example, the maximum range of motion of each device within each blood vessel or other anatomical pathway is uniquely marked within the displayed image. An example output of the system is an image matrix in which the maximum range of motion of each device within each blood vessel is uniquely marked within the image. For example, the coordinate corresponding to the maximum reach is marked with a different intensity compared to other pixels, or the entire trajectory within each blood vessel up to the coordinate of the maximum reach is marked with a different intensity.

[0074] As an effect, visualization of device limitations allows a user, e.g., an interventionist, to observe in real time the range of motion that each robotic controller can reach with the current settings and imaging feedback, and thus adjust catheterization strategies and devices accordingly. As another exemplary effect, visualization of device limitations allows a user to adjust the relative placement of the robot as needed. The term "relative" refers to position in relation to the subject. Optionally, additionally or alternatively, the relative position of the subject can be adjusted as needed.

[0075] Optionally, the processor is configured to compare the operating ranges and provide a ranked suggestion of at least two operating ranges.

[0076] In one example, the device-related data includes a plurality of device-related data for a plurality of different devices.

[0077] In one example, not shown in further detail in the drawings, the range of motion is provided as graphical information superimposed on anatomical image data of the region of interest. Additionally or alternatively provided options include the output interface being configured to provide the estimated range of motion as an image matrix in which the maximum range of motion of each device in each path is uniquely marked in the image.

[0078] As an example, the graphical information may be provided as an indicator of the catheter's travel limit, working length, or maximum reach.

[0079] In one example, the travel limit refers to the path length that the device 106 can move or travel.

[0080] In another example, working length refers to the possible path length that the device 106 can travel while still being able to provide the designated motion task. Such a task can be a forward motion, such as an imaging or ablation procedure, a backward motion procedure, such as a pullback, or even a repetitive back and forth motion procedure.

[0081] In one example, maximum reach refers to the possible length along a vessel that still allows for adequate operational procedures, such as the distal imaging reach of an imaging catheter.

[0082] For example, the coordinate corresponding to the maximum reach is marked with a different intensity compared to other pixels. In another example, the entire trajectory of each path segment, e.g., a blood vessel, up to the coordinate of the maximum reach is marked with a different intensity.

[0083] In one example, not shown in further detail in the drawings, the robot-related data includes at least one of the group of data obtained from the robot such as robot encoder information, robot travel constraints, CAD design, forward kinematics, inverse kinematics, velocity, acceleration, end effector pose, user controller input, and robot positioning relative to the patient.

[0084] In one example, not shown in further detail in the drawings, the device related data includes at least one of device specific insights, device type, device length, device stiffness, data regarding device-robot relationships such as distance of the device's tip or edge from a robot actuation unit, rollers or grippers, maneuverability information, and manufacturer grouping.

[0085] In another example, the device-related data relates to robotic data, and by way of example, rollers and grippers, as are known in the art, may be robotic drive elements of a robot for rotating or gripping, which may contact the elongated device or directly actuate a movement on the elongated device, such as translation or rotation. The robot may be further configured to rotate, translate and / or roll the elongated device, separate from the rollers and grippers.

[0086] Thus, "rollers" and "grippers" are actuators that relate to robot data. As an example, calibration is provided to provide the robot data that relates to the actuators. Calibration data provides knowledge of the relative positions of devices within a device stack, such as 0 position + length + robot position.

[0087] In one example, steerability information relates to a particular device that can be articulated. Optionally, this is provided under the umbrella of a device model that includes shape behavior and controllability, where specific joints for a steerable device are added to the standard translation-rotation control.

[0088] In one example, not shown in detail in the figures, the subject-related vasculature image data comprises at least one of data obtained from medical imaging, such as fluoroscopy or ultrasound imaging, and pre- or intra-operative medical imaging data, such as 3D rotational angiography, CT, CBCT, and MRI images. Further, the data processor 14 is configured to determine anatomical structures within the image data and identify anatomical paths suitable for navigation of at least one device within the anatomical path. Additionally or alternatively provided, optionally, the subject-related anatomical path image data includes target data including at least one of a group comprising a target and a path segment.

[0089] Optionally, the subject-related anatomical pathway image data of the region of interest includes at least one of the following group: vasculature, respiratory passageway, or intestinal tract.

[0090] Optionally, the image data is provided as 2D image data, for example live or current fluoroscopic image data.

[0091] The data processor is configured to provide a segmentation to identify an anatomical pathway.

[0092] In one example, not shown in further detail in the drawings, the data input 12 is configured to provide workflow data, and the data processor 14 is further configured to calculate at least one estimate of the operating range also based on the workflow data.

[0093] For example, the workflow data includes information related to the access site, e.g., depending on the access point (femur or radius), the distance between the device's starting point (device base) and the target anatomy changes, thus directly affecting the working length.

[0094] For example, the workflow data may include information related to the target anatomy. As an example, information from the target anatomy may determine en route meandering, which directly impacts the estimation of the working length.

[0095] In one example, the workflow data comprises information related to the type of procedure being applied. As an example, data regarding the procedure type, e.g., coil embolization of an aneurysm in the brain, embolization for a stroke, etc., implicitly informs the system regarding some of the intricacies regarding device operation, navigation paths, etc. Optionally, if a neural network is applied, the neural network learns from the input data.

[0096] In one example, the workflow data includes information about kinks / buckling / sagging, etc. As an example, these components reduce the effective working length of the device. For example, if the system recognizes a kink or significant sagging in the system, it can adjust its predictions to output a shorter working length.

[0097] Optionally, the above inputs are placed into descriptor vectors that are later digitized and fed to the neural network during both training and inference.

[0098] By way of example, the access site includes a (right / left) femoral and / or radial access point. By way of example, the procedure type includes mechanical thrombectomy or coil embolization.

[0099] As an advantage, for example, in robotic endovascular interventions, the user's correct device selection is facilitated, already taking into account the maximum reach within the anatomical structure. Optionally, a robot is positioned near the access area, and multiple candidate devices, e.g., catheters, guidewires, guide catheters, microwires, etc., are selected for possible use, i.e., deployment during the intervention. By determining, i.e., estimating, the range of motion, candidate devices with lengths that are unsuitable for a particular task can be deselected, which avoids situations where the user must replace the selected device due to a lack of range of motion. Thus, by providing an estimate of the range of motion, the efficiency of the procedure is significantly supported and improved.

[0100] Estimating the range of motion allows for the use of standard intravascular devices for robotic navigation, where standard devices may be designed primarily for manual navigation. Such devices may have lengths that are insufficient for certain long-distance robotic-assisted procedures, but may be suitable for multiple robotic-assisted procedures. Estimating the range of motion also accommodates robots with limited ranges of motion, which directly impacts the working length of the robotic controller.

[0101] By facilitating the selection of devices with the correct length, the need for manual steps such as multiple device exchanges or robotic displacement to obtain additional range of motion, which interrupt the procedure, require significant staff involvement, and ultimately prolong the procedure, is avoided or at least minimized.

[0102] This estimation can be implemented in the robotic assistance system for performing the respective task. When providing image data that reflects the current anatomical situation, real-time information regarding the maximum range of motion of each device within the anatomy is provided to the user. The estimation may take into account multiple parameters such as device shape, sag, anatomy, robot type, and device type. The effect is to beneficially improve medical workflow and provide a new level of confidence to, for example, interventionalists.

[0103] The assistive device is designed to use some or all of the above data to estimate an output that encodes the range of each device. The device output can be numerical, tabular, image, or other format.

[0104] FIG. 4 shows another example schematic setup. A range estimator 300 (in the center) is connected to a data feed (on the left). As an example of live or current image data, a 2D X-ray image 302 representing an anatomical image of a subject's region of interest is shown as an input. A first arrow 304 indicates the data feed or input to the range estimator 300. Additionally, a number of parameters 306 are provided and fed to the range estimator 300 as further inputs, indicated by a second arrow 308. The parameters relate to both the device and the robot used to operate the device. Examples of parameters are the device type, device entry point, specific device, device mount on the robot, and robot encoder parameters. The range estimator 300 calculates the device's range of motion for a given anatomical structure and provides this as an output, indicated by a third arrow 310. FIG. 312 shows the anatomical structure as an output, e.g., an X-ray image, such as an angiogram, overlaid with a graphical representation 314 of the range of motion. 4 shows a motion range estimator module that uses two sets of input data: first, the robot / device state, including the intrinsic parameters of the robot and device and their relative relationships, and second, imaging feedback, which indicates the current configuration of the device within the vasculature. The output of the "Range Estimator" module 300 is then overlaid as the maximum motion range on the intervention image.

[0105] 5 shows a further example of an operation scheme. Range calculation unit 350 is data-connected to medical imaging 352, i.e., an imaging device, which provides image data 354 to range calculation unit 350. Range calculation unit 350 is further data-connected to a robotics system 356, which provides robotics data 354 to range calculation unit 350. Optionally, a workflow source 360, which provides, for example, an event log, or audio data of the current scene in the operating room, or video of user activity in the operating room, is further provided and provided to range calculation unit 350 as robotics data 358. Robotics system 356 may also provide data to device data 362.

[0106] Range calculation unit 350 is data connected to one or more displays 364. Optionally, range calculation unit 350 is data connected to an operating room intelligence unit 366, which further utilizes the generated operating range data.

[0107] In certain embodiments, the range calculation unit 350 or distance estimator 300 or controller or resulting model is developed based on parameters defined from the movement probability of an elongated device type (type defined in the device data) with potential kinematics (contained in the robot data) in a determined anatomical environment (e.g., vascular or respiratory or other intraluminal structures contained in the anatomical path image data), and may include parameters related to a target region or location (which may be contained in the anatomical path image data).

[0108] These parameters may be entered manually or may be generated based on a set of data.

[0109] In more specific embodiments, the range calculation unit 350 or range estimator 300 or controller or resulting model may include robot data, device data, anatomical path image data, and may also include determined limits of the range of motion (see more exemplary details in subsequent sections).

[0110] In one example, not shown in further detail in the drawings, the range calculation unit 350 or range estimator 300 or controller or resulting model comprises a neural network-based controller including a convolution filter. The filter may be configured to capture contextual patterns within the image data. Additionally or alternatively provided, an option is provided in which the data processor is configured to calculate the estimate based on training of the neural network.

[0111] In one example, context patterns are learned as weights of convolution kernels and extracted as feature maps from input data. For example, in supervised training, weights are estimated based on minimizing the distance between the estimated operating range and ground truth operating range labels. Examples of low-level context features include landmarks on the device and anatomical structures and device boundaries in the image. High-level context patterns include the overall structure of the device relative to vascular structures.

[0112] In a further example, the context pattern is a spatial context pattern. In one example, the spatial context pattern is a pattern that captures 2D or 3D spatial relationships between various anatomical structures and the device based on the input data. These can be low-level patterns, such as the location of various edges in the image, or high-level patterns, such as the overall location of the device within the vasculature, or the alignment between the robot data / state and the device location within the 3D vasculature. Convolution kernels are used in the neural network to capture spatial context as described above.

[0113] In another example, the context pattern is a non-spatial context pattern.

[0114] In one example, the neural network also uses fully connected layers to capture vectorized and numerical patterns and embeddings from robot data, machine-related data, or workflow data. The neural network may use recurrent layers such as RNNs, LSTMs, and transformers to capture temporal dependencies when using time-series data.

[0115] Optionally, for training purposes of the network-based controller, various input data are provided as synthetic data obtained in a computer simulation environment, including different robot configurations, synthetic data from device models, and the use of target anatomical structures, as further described in subsequent sections of this disclosure.

[0116] Optionally, a range estimator training phase is provided. The weights used in the neural network for range estimation are learned and stored during the training phase. To create training data, various data are collected from the manual navigation of the robot. All relevant data, such as imaging, robot, device, workflow data, etc., are stored during the data acquisition step. Every time the robot reaches a limit during manual navigation, the image coordinates and corresponding intervention data (image, robot, device, workflow data) are stored. The coordinates of the device limit are then used as ground truth labels. Finally, the training data and corresponding labels are used to train the neural network.

[0117] During training, the neural network weights are optimized using a backpropagation process. As an example, at each iteration, the neural network predictions are compared to the ground truth labels using a distance function. Relevant distance functions for training neural networks may include, but are not limited to, L-2 distance (Euclidean), L-1 distance, binary cross-entropy, and Dice loss.

[0118] As a further option, a range estimator inference phase is provided. During the inference phase, the weights obtained during training are stored and used to calculate the expected maximum range of motion. In this step, real-time intervention data (images, robot, device, workflow, etc.) is fed to the neural network controller in the same format used during training. Finally, forward propagation of the input data through the neural network model produces the output.

[0119] As a further option, learning in simulation is provided. For example, in a computer simulation environment, simulated intervention sessions are generated from various configurations of an endoluminal (e.g., endovascular) robot, various endoluminal device models, and target anatomical structures. Each device is synthetically advanced through all branches relevant to the target procedure, and device limitations are acquired and stored for each new configuration. The synthetically generated [robot, device, imaging] data is then used as input signals to the controller introduced in the main claim. The device limitations are used as ground truth labels corresponding to the input data. The set of input labels and ground truth labels generated here is used to train the neural network controller introduced above.

[0120] Optionally, graphical element assignment is provided, for example, to assign or modify single or multiple visual, audio, or text elements on the graphical display based on the range of motion of the robot controller. An example of this embodiment increases the range of motion on a fluoro, contrast, or roadmap image, as shown in FIG. 6.

[0121] In Figure 6, an example of a presentation shown on a display is provided. An image 400 represents a region of interest of a subject 402, showing respective views of vascular structures 404. An overlaid indicator 406, e.g., a highlight, indicates the calculated operating range. Figure 6 shows the operating range superimposed on a display interface. The overlay can be extended onto a 2D or 3D acquired (or simulated) image. Optionally, the image is fluoro, DRR, DSA, roadmap, CBCT, CT, etc.

[0122] In one example, not shown in further detail in the drawings, data processor 14 is configured to determine an operating range uncertainty, and output interface 16 is configured to provide an indication of the operating range uncertainty.

[0123] 7 shows an example of a probability or uncertainty indicator. In the lower right portion, a subject 450 is depicted in simplified form. Also shown is a robotic drive 452 driving a device 454 partially inserted into the subject 452. In enlarged portion 456, the distal end 458 of device 454 is shown. Multiple circles 460 of different grey values, colors, or patterns indicate different degrees of certainty, i.e., uncertainty indicators, for the calculated range of (the distal tip of) device 454.

[0124] Optionally, the uncertainty of the device range is estimated using Monte Carlo dropout, which approximates Bayesian inference for deep Gaussian processes. To calculate the uncertainty, a subset of neurons in the neural network controller presented in the main embodiment is turned off during forward propagation to trigger dropout. Next, every incoming batch of data is passed through the model multiple times, for example, 10 times. Each time, the dropout mechanism produces a slightly different form of the model, which can result in a different operating range estimate by the neural network. The results of all these processes are aggregated to calculate the upper and lower limits of the device's operating range. Finally, these uncertainty boundaries are visualized on a display. An exemplary form of this visualization could be using different colors or dotted lines for the uncertainty boundaries overlaid on the display image.

[0125] In a first example, a system is provided that includes a robotic configuration and a navigation system.

[0126] In a second example, a system is provided that includes a robotic arrangement and navigation system and at least one device: The interventional device is configured for insertion into a pathway of interest, such as a vasculature.

[0127] Optionally, a system is provided that includes several devices, i.e. at least two or more devices, for example a set of devices is provided.

[0128] In one example, the robotic configuration is configured to be directly aligned with the subject to calculate an estimate of the range of motion of the robotically controlled, driven or manipulated device.

[0129] Optionally, the device's range of motion is estimated based on the registration. The robotic system is configured to be directly registered to the patient's anatomy to calculate the range of motion of the robotic controller. This registration loop is closed by finding the relationship between the robotic system and the anatomy visualized in the X-ray system. One approach is to calculate the registration transformation via pre- or intra-operative 3D imaging and 2D / 3D registration techniques.

[0130] Optionally, once a predetermined amount of the range of motion has been reached, user feedback is provided, the user feedback being provided as at least one of the group comprising visual feedback, audible feedback, or tactile feedback.

[0131] For example, the device may assign haptic feedback, such as vibration, to the physical controller when the device is expected to reach a limit calculated in the previous embodiment.

[0132] In one example, the predetermined amount is provided as half the range of motion, for example, about 50%, or as less than half the range of motion, such as 75%, 80%, 85%, 90%, 95%, or 95% or more. In another example, the predetermined amount is provided as the complete range of motion. As an example, visual feedback is provided as an overlay on an anatomical image or as a separate optical signal. In an additional or alternative option, vibration or other tactile feedback is provided to the user via a control handle against the user's foot or hand or as a vibration in a floor area when a travel limit is reached or is about to be reached.

[0133] Optionally, the system control is modified based on the operating range, for example the robot controller modifies the gain or speed of the system based on the distance of the device tip from the device operating limit calculated in the manner described in the previous embodiment.

[0134] Optionally, system control changes include changes to the imaging system when approaching the limits of the operating range, such as changing the frame rate, adapting the resolution, zooming in, etc.

[0135] The term "subject" may also refer to an individual. A "subject" may also be referred to as a patient, although it should be noted that this term does not imply whether the subject actually has a disease or disorder.

[0136] In one example, a computer program is provided that includes instructions that, when the program is executed by a computer, cause the computer to perform the method of the preceding example.

[0137] In one example, a computer program or program element for controlling an apparatus according to one of the above examples is provided, which program or program element is configured to perform the method steps of one of the above method examples when executed by a processing unit. Optionally, a computer readable medium having stored thereon the computer program of the previous example is provided.

[0138] In another exemplary embodiment of the invention, a computer program or a computer program element is provided, characterized in that it is configured to perform, on a suitable system, the method steps of the method according to one of the previous embodiments.

[0139] Thus, a computer program element may be stored in a computing unit or distributed across more than one computing unit that may be part of an embodiment of the present invention. This computing unit may be configured to perform or direct the execution of the steps of the above-mentioned method. Furthermore, it may be configured to operate the components of the above-mentioned apparatus. The computing unit can be configured to operate automatically and / or to execute a user's order. The computer program may be loaded into the working memory of a data processor. The data processor may thus be equipped to perform the method of the present invention.

[0140] Aspects of the present invention may be embodied in a computer program product, which may be a collection of computer program instructions stored on a computer-readable storage device that can be executed by a computer. The instructions of the present invention may be any interpretable or executable code mechanism, including, but not limited to, a script, an interpretable program, a dynamic link library (DLL), or a Java class. The instructions may be provided as a complete executable program, a partial executable program, a modification (e.g., an update) to an existing program, or an extension (e.g., a plug-in) to an existing program. Furthermore, portions of the processing of the present invention may be distributed across multiple computers or processors.

[0141] As described above, a processing unit, e.g., a controller, implements the control method. This controller can be implemented in a variety of ways using software and / or hardware to perform the various functions required. A processor is one example of a controller that uses one or more microprocessors that can be programmed using software (e.g., microcode) to perform the required functions. However, a controller may be implemented with or without a processor, or as a combination of dedicated hardware to perform some functions and a processor (one or more programmed microprocessors and associated circuitry) to perform other functions.

[0142] Examples of controller components used in various embodiments of the present disclosure include, but are not limited to, conventional microprocessors, application specific integrated circuits (ASICs), and field programmable gate arrays (FPGAs).

[0143] This exemplary embodiment of the present invention encompasses both computer programs that use the present invention from the beginning, and computer programs that convert existing programs into programs that use the present invention by means of an update.

[0144] Furthermore, the computer program element may be capable of providing all the steps necessary to fulfill the procedures of the exemplary embodiments of the methods described above.

[0145] According to a further exemplary embodiment of the present invention, a computer readable medium, such as a CD-ROM, is presented, having stored thereon a computer program element, which computer program element is described by the previous section. The computer program may be stored and / or distributed on a suitable medium, such as an optical storage medium or a solid-state medium, supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless communication systems.

[0146] However, the computer program may also be presented over a network such as the World Wide Web and can be downloaded into the working memory of a data processor from such a network. According to a further exemplary embodiment of the present invention, a medium for making a computer program element available for downloading is provided, the computer program element being configured to perform a method according to one of the aforementioned embodiments of the present invention.

[0147] It should be noted that the embodiments of the present invention are described with reference to different subject matters. In particular, some embodiments are described with reference to method-type claims, and other embodiments are described with reference to apparatus-type claims. However, those skilled in the art will understand from the above and below description that, unless otherwise specified, any combination of features belonging to one type of subject matter, as well as any combination between features relating to different subject matters, is considered to be disclosed in the present application. However, all features can be combined to provide a synergistic effect greater than the simple sum of the features.

[0148] While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered exemplary or explanatory and not restrictive. The invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure and the dependent claims.

[0149] In the claims, the word "comprise" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be interpreted as limiting the scope.

Claims

1. 1. A controller configured to provide assistance for driving an elongated robotic drive in robotic anatomical path navigation, the controller comprising: A data input section; A data processor; an output interface; and the data input unit is enabled to receive and provide to the data processor robot-related data of a designated robot provided for driving the elongated drive device to perform at least one endoluminal-related task, such as robot-related data including robot configuration data and / or data related to the driving, device-related data related to intrinsic or mechanical properties of at least one elongated device driven by the designated robot, and subject-related anatomical path image data of a region of interest through which the device is moved by the designated robot; the data processor, optionally together with computer program instructions, is configured to calculate an estimate of a range of motion for the at least one elongate device within the anatomical path based on the robot-related data, the device-related data, and the subject-related anatomical path image data as inputs for calculation of the estimate; the data processor is further configured to output the estimated operating range via the output interface. controller.

2. 1. A navigation system to assist robotic anatomical path navigation of an elongated robotic drive device, the system comprising: an input unit; A data processor; an output interface; and the data input unit is enabled to receive and provide to the data processor robot-related data of a designated robot provided for driving the elongated drive device to perform at least one endoluminal-related task, such as robot-related data including robot configuration data and / or data related to the driving, device-related data related to intrinsic or mechanical properties of at least one elongated device driven by the designated robot, and subject-related anatomical path image data of a region of interest through which the device is moved by the designated robot; the data processor is configured to calculate an estimate of a range of motion for the at least one elongate device within the anatomical path based on the robot-related data, the device-related data, and the subject-related anatomical path image data as inputs for calculation of the estimate; the output interface is configured to provide the estimated operating range. Navigation system.

3. The controller of claim 1 , wherein the operating range includes at least one parameter from the group including travel limit, working length, and maximum reach.

4. the data processor is configured to calculate estimates of operating ranges along different possible paths for the at least one device; The controller of claim 1 or 2, wherein the output interface is configured to output the ranges of motion along the different possible paths.

5. the data input is configured to provide device-related data comprising a plurality of device-related data for a plurality of different devices; the data processor is configured to calculate a plurality of estimates of operating ranges based on the plurality of device-related data; The controller of claim 1 , 2 or 3 , wherein the output interface is configured to output the ranges of motion along the different possible paths.

6. the range of motion is provided as graphical information overlaid on anatomical image data of the region of interest; 6. The controller of claim 1, wherein the output interface is configured to provide the estimated operating ranges as an image matrix in which the maximum operating range of each device in each path is uniquely marked in the image.

7. 7. The controller of claim 1, wherein the robot-related data includes at least one of data obtained from the robot such as robot encoder information, robot travel constraints, CAD design, forward kinematics, inverse kinematics, velocity, acceleration, end effector pose, user controller input, and robot positioning relative to a patient.

8. The controller of any one of claims 1 to 7, wherein the equipment related data includes at least one of equipment specific insights, equipment type, equipment length, equipment working length, equipment stiffness, equipment shape, equipment-robot relationship data such as distance between the tip or end of the equipment from the robot actuation unit, forces, torques, speeds, robot type, equipment manipulation mechanism (roller, gripper, belt, fixed), maneuverability information and manufacturer group.

9. the subject-related endoluminal structural image data comprises at least one of the group of data obtained from medical imaging, such as fluoroscopy or ultrasound imaging, and pre- or intra-operative medical imaging data, such as 3D rotational angiography, CT, CBCT, and MRI images; the data processor is configured to determine anatomical structures within the image data to identify anatomical paths suitable for navigation of the at least one device within the anatomical paths; The controller of claim 1 , wherein the subject-related anatomical path image data includes target data including at least one of the group including targets and path segments.

10. the data input component is configured to provide workflow data; The controller of claim 1 , wherein the data processor is configured to calculate at least one estimate for an operating range based on the workflow data.

11. a neural network-based controller having a convolution filter configured to capture contextual patterns within the image data; A controller according to claim 1 , wherein the data processor is configured to calculate the estimate based on training of the neural network.

12. the calculated estimate includes a determination of an operating range uncertainty; A controller according to claim 1 , wherein the output interface is configured to provide an indication of the operating range uncertainty.

13. A navigation system for assisting robotic anatomical path navigation of an elongated robotic drive, the system comprising a controller according to any one of claims 1 to 12.

14. Robot configuration, and 14. The navigation system according to claim 13. In a system having the robotic arrangement is configured to control and drive navigation of at least one elongated device along an anatomical path of at least one region of interest of a subject, the robotic arrangement further configured to provide robot-related data; the navigation system is configured to provide an estimate of a range of motion of the at least one elongate device within the anatomical path of the region of interest of the subject. system.

15. an imaging arrangement configured to provide the subject-related anatomical pathway image data; The system of claim 13 , wherein the subject-related anatomical pathway image data is preferably provided as a 2D X-ray image.

16. 10. A computer program comprising instructions stored and encoded in a non-transitory processor-readable medium, the computer program being executed by the controller of claim 1.

Images